| Title | Talking with Robots about Objects: A system-level evaluation in HRI |
| Publication Type | Conference Paper |
| Year of Publication | 2012 |
| Authors | Peltason, J., N. Riether, B. Wrede, and I. Lütkebohle |
| Refereed Designation | Refereed |
| Conference Name | 7th ACM/IEEE Conference on Human-Robot-Interaction |
| Conference Start Date | 05-03-2012 |
| Conference Location | Boston, Masschusetts, USA |
| Abstract | We present the design process, realization and evaluation of a robot system for interactive object learning. The system-oriented evaluation, in particular, addresses an open problem for the evaluation of systems, where overall user satisfcation depends not only on the performance of the parts, but also on their combination, and on user behavior. Based on the PARADISE method known from spoken dialog systems, we have defined and applied internal and external metrics for fine-grained and largely automatable identification of such relationships. Through evaluation with n=28 subjects, indicator functions explaining up to 55% of variation in several satisfaction metrics were found. Furthermore, we demonstrate that the system’s interaction style reduces the need for instruction and successfully recovers partial failures. |
| Refereed Designation | Refereed |
