What did you need? Robots can now ask you for Clarification

Researchers at Brown University developed an algorithm that enables robots to ask questions to aid problem solving. In the future of AI and robotic assistants, not every person will understand how to control a given machine. Machine learning systems will have to be able to adapt and ask questions.

To integrate into the social life of humans, robots face a challenge not as simple as it sounds: be able to identify objects in a domestic environment and fetch them.

If you say, for example, “Could you pass me the bottle?” A human child, barely able to decipher speech, might know what to do immediately. If not, the kid can easily ask, “which bottle?”

For a robot, however, this is a complex task that requires powerful speech analysis software and detection systems so then it embarks on exploring the environment to locate a specific bottle.

Compute Again?

Led by computer science professor Stefanie Tellex, researchers at Brown University’s Humans to Robots Lab developed an algorithm that helps robots get social feedback in order to perform a task, namely fetching objects.

When we ask for an item, we usually point to it at the same time. Tellex’s lab had previously developed an algorithm that enables robots to respond to verbal commands as well as gestures. By making the algorithm combine verbal and physical cues, Tellex and her team showed that robots could interpret commands even better.

Still, the task gets even more complicated for the robot when there are many similar objects next to each other.

Back to our example, if you weren’t specific enough and ordered, “pass me the bottle.” A confused robot would not normally be able to ask for clarification.

This new algorithm enables them to do just that. The Brown University robot assess its own certainty regarding the user’s command. Depending on the level of certainty, the robot would then hand over the object or, after making its best guess, it hovers its gripper over the object and asks “this one?”

Robots Akin to Akinator

As it stands, the system can only ask for confirmation with a single question, but it can already draw intelligent conclusions from the human feedback and make smart inferences, laying the groundwork for future developments.

The work of Tellex and her team was funded in part in part by grants from NASA and the Defense Advanced Research Projects Agency, hinting at the possible applications. Brown’s Humans to Robots Lab will present the research (PDF) this spring at the International Conference on Robotics and Automation in Singapore.

Tellex’s lab system refers to another popular program: Akinator, the Web Genie. The app that can guess which character you pick based on Twenty Questions game. Magic? No, statistics!

Machine learning that can seek out clarification for its solutions is a testament to the growth of AI systems. Shout out to Redditor @losthoneyb for sharing.