Technology 3 min read

Facebook's AI Habitat Uses Photorealistic Homes to Train AI Models

kirill_makarov /

kirill_makarov /

It’s not an easy task for a domestic robot if you ask it to bring you a cup of water or to check on the kids in their room. Besides the mechanical, physical skills needed to perform such a task, the robot will have to rely on its AI spirit to know where is ‘what’ and how to get from point A to point B within the house.

Westworld-like robots may never come, but household robots and also robots operating in the workplace, like in the office, in hospitals, and industry are becoming a real thing.

Other than the hardware side of things, the work on robots involves great emphasis on the software, and this where Facebook‘s new AI Habitat simulation platform comes in.

Facebook’s “AI Habitat” For Embodied AI

Per Facebook, it’s AI Habitat “enables training of embodied AI agents (virtual robots) in a highly photorealistic & efficient 3D simulator, before transferring the learned skills to reality. This empowers a paradigm shift from ‘internet AI’ based on static datasets (e.g., ImageNet, COCO, VQA) to embodied AI where agents act within realistic environments, bringing to the fore active perception, long-term planning, learning from interaction, and holding a dialog grounded in an environment.”

Training AI agents how to navigate their surroundings and to recognize objects using static datasets for long hours is a resource-intensive task, which also happens to be a very polluting practice. A recent study revealed that training a single AI releases as much carbon into the atmosphere as five cars over their lifetime.

On the other hand, Facebook’s Habitat provides a virtual home with forms, objects, and obstacles, just like in real houses. Training AI models in a cloud-based 3D platform would dramatically cut training time, and domestic robots powered by such software won’t have to bump into walls to find their way and learn about their surroundings.

Facebook’s AI habitat is optimized for fast AI training. Habitat-Sim (Github repository), the 3D simulator within Habitat, comes with “configurable agents, multiple sensors, and generic 3D dataset handling… and reaches over 10,000 FPS multi-process on a single GPU”.

Besides Habitat-Sim, the Habitat platform also includes Habitat-API “a modular high-level library for end-to-end development in embodied AI, and Habitat-Challenge “an autonomous navigation challenge that aims to benchmark and accelerate the progress in embodied AI.”

Compatible with MatterPort3D, Gibson, and other 3D datasets, Habitat comes with its own dataset called Replica. Created by  Facebook Reality Labs (FRL) using real photos and depth mapping, Replica has a variety of hyperrealistic indoor spaces with high-quality, dense geometry and textures.

Facebook here is putting an emphasis on flexibility and the power of open sourcing to accelerate progress into the field of embodied AI which would benefit the next generation of AI-powered assistants “from a robot asked to “grab my phone from the desk upstairs” to a device that helps its visually impaired wearer navigate an unfamiliar subway system”.

Read More: Do You Know? There’s A Subreddit Only AI Chatbots Can Post

First AI Web Content Optimization Platform Just for Writers

Found this article interesting?

Let Zayan Guedim know how much you appreciate this article by clicking the heart icon and by sharing this article on social media.

Profile Image

Zayan Guedim

Trilingual poet, investigative journalist, and novelist. Zed loves tackling the big existential questions and all-things quantum.

Comments (0)
Least Recent least recent
share Scroll to top

Link Copied Successfully

Sign in

Sign in to access your personalized homepage, follow authors and topics you love, and clap for stories that matter to you.

Sign in with Google Sign in with Facebook

By using our site you agree to our privacy policy.