Saito, Namiko and Ogata, Tetsuya and Mori, Hiroki and Murata, Shingo and Sugano, Shigeki (2021) Tool-Use Model to Reproduce the Goal Situations Considering Relationship Among Tools, Objects, Actions and Effects Using Multimodal Deep Neural Networks. Frontiers in Robotics and AI, 8. ISSN 2296-9144
![[thumbnail of pubmed-zip/versions/2/package-entries/frobt-08-748716-r1/frobt-08-748716.pdf]](http://publish7promo.com/style/images/fileicons/text.png)
pubmed-zip/versions/2/package-entries/frobt-08-748716-r1/frobt-08-748716.pdf - Published Version
Download (2MB)
Abstract
We propose a tool-use model that enables a robot to act toward a provided goal. It is important to consider features of the four factors; tools, objects actions, and effects at the same time because they are related to each other and one factor can influence the others. The tool-use model is constructed with deep neural networks (DNNs) using multimodal sensorimotor data; image, force, and joint angle information. To allow the robot to learn tool-use, we collect training data by controlling the robot to perform various object operations using several tools with multiple actions that leads different effects. Then the tool-use model is thereby trained and learns sensorimotor coordination and acquires relationships among tools, objects, actions and effects in its latent space. We can give the robot a task goal by providing an image showing the target placement and orientation of the object. Using the goal image with the tool-use model, the robot detects the features of tools and objects, and determines how to act to reproduce the target effects automatically. Then the robot generates actions adjusting to the real time situations even though the tools and objects are unknown and more complicated than trained ones.
Item Type: | Article |
---|---|
Subjects: | Euro Archives > Mathematical Science |
Depositing User: | Managing Editor |
Date Deposited: | 29 Jun 2023 03:27 |
Last Modified: | 21 Aug 2023 09:05 |
URI: | http://publish7promo.com/id/eprint/2855 |