tutorial_ROMAN'09

ROMAN 2009 Tutorial on How to analyze HRI Data

In this tutorial, we want to provide participants with methodological insights and practical experiences on acquiring and analyzing data in human-robot interaction.
We will address the following questions viewed from different disciplinary angles:  how to code, annotate, transcribe, and analyze data both manually and (semi-) automatically. We will introduce tools that can be used in the process. In the tutorial, participants will be invited to do a range of practical exercises with authentic data since data analysis can only be practiced by doing it. By providing these examples we want to encourage fruitful interdisciplinary discussion.  

Motivation and objectives

More and more researchers in HRI conduct user studies with inexperienced users. These studies are highly interdisciplinary endeavors since they include the technical development of the robot and the social component of the user. Different types of data are acquired in the studies, e.g. a) system log files, b) questionnaires, c) video taped recordings, and d) manual annotations based on the above. This leaves the researcher with much information but also with many questions on how to deal with it. Therefore, to fully exploit the data we propose that also the analysis requires interdisciplinary methods and tools.

In a full-day tutorial, we want to introduce participants to our approach of analyzing data in HRI by offering the possibility to practice all steps which are included in the process with authentic data that we have acquired in our own studies. Next to manual data analysis we want to introduce tools that, based on our interdisciplinary competency, use computer science approaches to (semi-) automatically annotate certain aspects of the data (e.g., hand trajectories, speech), which should be very helpful for researchers who are looking for ways to analyze their data more effectively. Moreover, we want to keep the tutorial open for problems that the participants have encountered while analyzing their own data.

List of topics

•    Introduction on how to acquire data in user studies
•    Preparation of data for analysis (annotation, transcription, coding)
•    Linking different types of data for analysis and visualization
•    Analysis tools (comparison between different tools, i.e., ELAN, Praat, Interact)
•    Manual annotation of data (coding schemes, inter- and intra-rater reliability)
•    (Semi-) Automatic analysis methods (e.g., 2D hand-trajectory annotation, forced alignment based on speech)

List of presenters

Katrin Lohan, Computer Science, Bielefeld University            
Manja Lohse, Communication Science, Bielefeld University       
Karola Pitsch, Interaction Analysis, Linguistics, Bielefeld University   
Lars Schillingmann, Computer Science, Bielefeld University        
Anna-Lisa Vollmer, Mathematics, Bielefeld University          

Intended audience

The intended audience is researchers in HRI that face the challenge of effectively analyzing the data acquired in user studies. We would like to welcome an interdisciplinary audience (e.g. computer science, robotics, social sciences), as our approach to analyzing data is also highly interdisciplinary.
 

Schedule

Time    Topic
09:00 - 09:15    Welcome by the organizers
09:15 - 09:45    Getting to know the audience
09:45 - 10:00    Overview
10:00 - 10:30    Motiontracking on videos (tools: Interact, Icewing, OpenGazer)
10:30 - 11:00    Break
11:00 - 11:30    Exercise on offline motion tracking
11:30 - 12:30    Qualitative analysis - linking hand trajectories and gaze (tools: Elan, Matlab)
12:30 - 13:30    Lunch
13:30 - 15:00    Quantitative analysis (tools: Elan, SALEM, Matlab)
15:00 - 15:30    Break
15:30 - 16:30    Automated Segmentation of Speech (tools: Praat, Esmeralda)
16:30 - 17:00    Discussion and Closing

Organizers: 

        Manja Lohse
        Applied Informatics, CoR-Lab, Bielefeld University
        Universitätsstraße 25, 33615 Bielefeld, Germany
        phone: +49 521 106 12209; email: mlohse 'at' techfak.uni-bielefeld.de

        Katrin Lohan
        Applied Informatics, CoR-Lab, ITALK, Bielefeld University
        Universitätsstraße 25, 33615 Bielefeld, Germany
        phone: +49 521 106 12230; email: klohan 'at' techfak.uni-bielefeld.de

        Dr. Karola Pitsch
        Applied Informatics, CoR-Lab, ITALK, Bielefeld University
        Universitätsstraße 25, 33615 Bielefeld, Germany
        phone: +49 521 106 12240; email: kpitsch 'at' techfak.uni-bielefeld.de

        Anna-Lisa Vollmer
        Applied Informatics, CoR-Lab Bielefeld University
        Universitätsstraße 25, 33615 Bielefeld, Germany
        phone: +49 521 106 12230; email: kpitsch 'at' techfak.uni-bielefeld.de

        Dr. Katharina Rohlfing
        Emergentist Semantics Group, CI-TEC, Bielefeld University
        Universitätsstraße 25, 33615 Bielefeld, Germany
        phone: +49 521 106 67127; email: rohlfing 'at' techfak.uni-bielefeld.de

        Dr. Britta Wrede
        Hybrid Society Group, CoR-Lab, Bielefeld University
        Universitätsstraße 25, 33615 Bielefeld, Germany
        phone: +49 521 106 2934; email: bwrede 'at' techfak.uni-bielefeld.de

References
Fink, G.A. (1999): Developing HMM-based Recognizers with ESMERALDA. In Lecture Notes in Artificial Intelligence, Vol. 1692, pp. 229-234

Lohan, K.; Vollmer, A.-L.; Fritsch, J.; Rohlfing, K.; and Wrede, B. (2009): Which ostensive stimuli can be used for a robot to detect and maintain tutoring situations?. In International Workshop on Social Signal Processing 2009, Amsterdam, Netherlands.

Lohse, M.; Rohlfing, K.; Wrede, B.; Sagerer, G. (2008): Try Something Else! When Users Change Their Discursive Behavior in Human-Robot Interaction. In Proceedings of 2008 IEEE International Conference on Robotics and Automation (ICRA). Pasadena, CA, USA

M. Lohse, M. Hanheide, K. Pitsch, K.J. Rohlfing, and G. Sagerer (to appear): Improving HRI design applying Systemic Interaction Analysis (SInA). To appear in Interaction Studies 10:3 (2009)

Pitsch, K.; Vollmer, A.-L.; Fritsch, J.; Wrede, B.; Rohlfing, K.; and Sagerer, G. (2009): On the loop of action modification and the recipient's gaze in adult-child interaction. In GESPIN (Gesture and Speech in Interaction) 2009, Poznan, Poland.

Pitsch, K.; Kuzuoka, H.; Suzuki, Y.; Süssenbach, L.; Luff, P.; and Heath, C. (2009): "The first five seconds": Contingent stepwise entry into an interaction as a means to secure sustained engagement. In: ROMAN (IEEE International Symposium on Robot and Human Interactive Communication) 2009, Toyama, Japan.

Rohlfing, K.J.; Loehr, D.; Duncan, S.; Brown, A.; Franklin, A.; Kimbara, I; Milde, J.-T.; Parrill, F.; Rose, T.; Schmidt, T.; Sloetjes, H.; Thies, A.; Wellinghoff, S. (2006). Comparison of multimodal annotation tools -- workshop report. Gesprächsforschung - Online Zeitschrift zur verbalen Interaktion, vol. 7, pp. 99-123.

Vollmer, A.-L.; Lohan, K.; Fischer, K.; Nagai, Y.; Pitsch, K.; Fritsch, J.; Rohlfing, K.; and Wrede, B. (2009): People Modify Their Tutoring Behavior in Robot-Directed Interaction for Action Learning. In International Conference on Development and Learning 2009, Shanghai, China.

Links
Annotation Tools:
Elan: http://www.lat-mpi.eu/tools/elan/
(download and different manuals in English and a forum)

Praat: http://www.fon.hum.uva.nl/praat/
(download and different manuals in English)

Anvil: http://www.anvil-software.de/   
(download and manuals)

Interact: http://www.mangold-international.com
(information about the software which is not open source)

Intercoder Reliability:
http://astro.temple.edu/~lombard/reliability/
(website that introduces intercoder reliability and compares different approaches)

Transcription Symbols for Conversation Analysis
http://www.sscnet.ucla.edu/soc/faculty/schegloff/TranscriptionProject/pa...

Esmeralda:
http://sourceforge.net/projects/esmeralda/
(download)

Opengazer:
http://www.inference.phy.cam.ac.uk/opengazer/
(download and manual)

Icewing:
http://icewing.sourceforge.net/
(download)

Software for statistical analysis:
Matlab: http://www.matlab.com
SPSS: http://www. spss.com

SALEM (Statistical Analysis of Elan files in Matlab):
http://aiweb.techfak.uni-bielefeld.de/content/salem-statistical-analysis...

Attachment	Size
1_Overview.pdf	155.19 KB