HumAnS Lab

 

Multi-Modal Interaction for Assistive Robotics in the Home

Multimodal communication channels are defined as any channel that allows
humans to provide control signals to the robot – via voice, gestures, body
movement, haptics, EMG measurements, etc. In these efforts, we seek to
evaluate the different types of inputs provided by the diverse set of
communication channels and understand how to map human interaction data to a
robot’s available command set.

Grasping objects in a normal home space using gestural and textual commands

To enable human to robot interaction in an assistive living environment,
multiple modes of communication must be utilized. In this work, we
investigate the use of gestural pointing and interactive feedback, to
provide directions to a robotic assistant. The process involves a human
directing the robot’s attention to a general direction of interest using a
gestural interface. Utilizing interactive voice response technology, the
robot asks the human questions if additional information is required to
disambiguate the human’s intent. In this iterative fashion, the robot is
able to gather understanding of the best way to get the desired task
completed.

Haptic-Based Interaction

The use of haptic feedback has been shown to enable ‘untrained’ users access
to robotic devices by 1) allowing users command of a robotic platform to
accomplish necessary tasks while 2) enabling the robot to provide guiding
forces based on knowledge of its own internal characteristics. We accomplish
this hybrid interaction loop by translating available robot sensory inputs
into direct haptic feedback values.