Enhanced Visual Scene Understanding through Human-Robot Dialog
2011 (English)In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE , 2011, 3342-3348 p.Conference paper (Refereed)
We propose a novel human-robot-interaction framework for robust visual scene understanding. Without any a-priori knowledge about the objects, the task of the robot is to correctly enumerate how many of them are in the scene and segment them from the background. Our approach builds on top of state-of-the-art computer vision methods, generating object hypotheses through segmentation. This process is combined with a natural dialog system, thus including a ‘human in the loop’ where, by exploiting the natural conversation of an advanced dialog system, the robot gains knowledge about ambiguous situations. We present an entropy-based system allowing the robot to detect the poorest object hypotheses and query the user for arbitration. Based on the information obtained from the human-robot dialog, the scene segmentation can be re-seeded and thereby improved. We present experimental results on real data that show an improved segmentation performance compared to segmentation without interaction.
Place, publisher, year, edition, pages
IEEE , 2011. 3342-3348 p.
, Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, ISSN 2153-0858
Service Robotics, Human-Robot Dialog, Machine Learning, Segmentation
IdentifiersURN: urn:nbn:se:kth:diva-46701DOI: 10.1109/IROS.2011.6048219ISI: 000297477503104ScopusID: 2-s2.0-84455206614ISBN: 978-1-61284-454-1OAI: oai:DiVA.org:kth-46701DiVA: diva2:453961
International Conference on Intelligent Robots and Systems (IROS '11). San Francisco, CA, USA. 25 Sep - 30 Sep 2011
FunderEU, FP7, Seventh Framework Programme, IST-FP7- IP-215821ICT - The Next Generation
QC 201111182011-11-042011-11-042012-04-03Bibliographically approved