HumAnS Lab

Understanding a Child’s play for Robot Interaction by Sequencing Play Primitives

In this project, we focus to build a system for a robot playmate that extracts and sequences the low-level play primitives during child play. If a robot can understand how to manipulate a toy in such a way that the human does, it will gain the ability to interact with numerous kinds of toys. The extraction of play primitives is based on the observation of motion gradient vectors computed from the image sequence. The Hidden Markov Models (HMMs) are used to recognize 14 different play primitives during play. Experimental results from a data set of 30 play scenarios demonstrate 94.51% accuracy recognizing and sequencing the play primitives.

Why do children need playmates? Interactive play in childhood closely links to children’s social, physical, and cognitive development. Simple toys such as those chosen in this research accelerate child’s imagination as they build their own scenes, knock them down, and start over. Along with the toys, playmates are an important source for building collaboration, cooperation, empathy and impulse control, reduced aggression, and better overall emotional and social adjustments. Based on this theory, the challenge that must first be addressed is whether a robot can become an effective playmate. We believe that by observing others play, the robot can effectively learn acceptable play behaviors. Most research that addresses learning manipulation tasks from human-robot interaction derived from observation and/or demonstration that tries to address the problem associated with general manipulation. In contrast, the intent of the approach proposed here is the use of dynamic pattern recognition methods, using only visual information without further aids.

The training and experiment result show that most of the behaviors during a play can be understood by sequencing
low-level play primitives. This fact is very encouraging in a sense that when applied to a robot playmate, the robot can
observe and take turns with children while engaging in a daily basis and therapeutic play.