In just a short time, mobile devices have revolutionized the way we access information and interact with each other. With an ever expanding list of sensors and features, these devices are also capable of reshaping our experiences with physical systems. Prior efforts to explore this potential of mobile devices have often considered traditional user…

Modern intelligent agents will need to learn the actions that humans perform. They will need to recognize these actions when they see them and they will need to perform these actions themselves. We want to propose a cognitive system that interprets human manipulation actions from perceptual information (image and depth data) and consists of…