Abstract
Human tool use prowess distinguishes us from other animals. In many scenarios, a human is able to recognize objects, seen for the first time, as potential tools for a task, and use them without requiring any learning. Here we propose a framework to enable similar abilities in robots. We first characterized human tools to identify a special category of tools that humans are able to use immediately through a process of skill transfer from their limbs, rather than tool learning. Motivated by the tool characterization and our neuroscientific studies on human tool use and embodiment, we then developed a tool cognition framework that enables a robot to recognize a previously unseen object as a tool for a task, plan how to grasp and wield the tool in the face of constraints and obstacles, before finally performing the task with the tool. Furthermore, the framework allows for flexibility in tool use, where the same tool can be adapted for different tasks, and different tools for the same task, all without any prior learning or observation of tool use. We demonstrate the possibilities offered by our tool cognition framework in several robot experiments with both toy and real objects as tools.