Research Projects

Interactional Coordination and Incrementality in HRI - A Museum Guide Robot (2013 - 2016)

Principal Investigator - with Sebastian Wrede (Cognitive Systems Engineering)
Funded by CITEC


Staff: Raphaela Gehle, NN

Interaction & Space. From Conversation Analysis to Human-Robot-Interaction (2012 - 2016)

Dilthey Fellowship
Funded by the Volkswagen Foundation

Ziel des Projekts ist die empirisch fundierte Entwicklung von dynamischen Interaktionsmodellen, die - exemplarisch - ein humanoides Roboter-System dazu befähigen sollen, die Rolle eines TourGuides im Museum zu übernehmen. Die besondere Herausforderung für technische Systeme besteht darin, dynamisch und kontingent auf das variable Verhalten der Besucher zu reagieren und angesichts der Mobilität der Teilnehmer einen gemeinsamen 'Interaktionsraum' herzustellen. Den Ausgangspunkt der Interaktionsmodelle bilden detaillierte Mikro-Analysen der sozialen Praktiken und multimodalen Koordinierung (Sprache, Blick, Gestik, Körperorientierung, Bewegung im Raum) zwischen einem menschlichen TourGuide und Besuchern während eines geführten Ausstellungsbesuchs: Wie organisieren die Teilnehmer den Einstieg in eine fokussierte Interaktion? Wie orientieren sie sich in einer dynamischen Mehr-Personen-Konstellation? Wie erfolgt der gemeinsame Übergang von einem Exponat zum nächsten?
In methodischer Hinsicht zielt das Projekt darauf, eine traditionelle geisteswissenschaftlich-qualitative Forschungsmethode anschlussfähig zu machen für die Entwicklung neuer Mensch-Maschine-Schnittstellen: Die videobasierte Ethnomethodologische Konversationsanalyse wird mit neuartigen MotionCapture-Daten und quantifizierenden Verfahren verbunden und in einen interdisziplinären iterativen ‘Research Loop’ aus empirischer Analyse von Mensch-Mensch-Interaktion, Interaktionsmodellierung, technischer Implementierung und Mensch-Roboter-Experimenten integriert.

Cooperation: Applied Informatics, Sebastian Wrede, EU-project HumaVips, Kunsthalle Bielefeld
Student Assistant: Raphaela Gehle, Lukas Rix, Antje Amrhein

VASA: Virtual Assistants and their Social Acceptability (2012 - 2014)


VERSTANDEN: Verständigungssicherung in der Sprachdialog-basierten Mensch-Technik-Interaktion mit einem Assistenzsystem für Menschen mit altersbedingten Einschränkungen (2013 - 2014)

Principal Investigator - with Stefan Kopp (Sociable Agents)
Funded by CITEC & BMBF

The project aims at investigating the possibilities of cognitive interaction technologies for assisting people with special needs in managing routine activities in their everyday lives. In particular, the project develops a virtual agent and explores its social acceptability as an intuitive communication interface that provides personalized video communication and supports the scheduling of daily activities. - In close collaboration with the technical project part, the empirical project part aims at investigating how the social practices of participants and the ecology of the setting change when a new technological tool - such as a virtual assistant - is introduced into their homes/workplace. Thus, we not only evaluate the functionality and usability of the new technology, but also how the participants integrate it into their daily routines and how these might be shaped and changed by it. A special focus lies on the question how such technologies can be designed to be ‘socially acceptable’ both at first contact and during long-term interaction. Research starts with ethnographic field work ‘on site’ to explore current interactional processes in order to derive empirically based suggestions for designing the system’s interface and communicational skills, which will subsequently be tested and evaluated in field trials.

Cooperation: v. Bodelschwingh Foundation Bethel
Marcel Kramer, Ramin Yaghoubzadeh (Sociable Agents)
Student Assistant: Kirsten Kästel, Paul Diestelhorst

Alignment in Augmented Reality based Cooperation (2010 - 2014)

Principal Investigator - with Thomas Hermann (Ambient Intelligence)
Funded by DFG. Project C5 in the SFB 673 ‘Alignment in Communication’

The project aims at investigating interactional processes of establishing, sustaining and manipulating “joint attention” (co-orientation, co-referencing) in collaborative task-oriented interaction. To do so, we have developed an Augmented Reality setup as an interception and manipulation method, which allows to record and to manipulate the co-participants’ audio-visual perceptions and the material surrounding: Participants are equipped with head-mounted displays, auditory headsets and motion sensors, which allow both to record the participants’ perspective (“from within”, Goffman) and to ‘disturb’ resp. ‘enhance’ specific aspects of interactional conduct. We conduct a series of semi-experimental studies, in which two co-participants are asked to collaboratively envision and discuss a local recreation area using a map and a set of related objects. Analysis focuses (i) on revealing interactional procedures and patterns using video-based Ethnomethodological Conversation Analysis, and (ii) links this qualitative approach with quantitative methods, such as data mining.

Christian Schnier, Alex Neumann (Ambient Intelligence)
Student Assistant: Katharina Geretzky, Dominik Weissgerber, Clemens Westrup
Project Webpage

SoziRob. A humanoid robot as fitness partner (2010 - 2013)

Associated supervisor. - Project headed by Britta Wrede & Franz Kummert (Applied Informatics, Bielefeld University)
Funded by the German Areospace Association (DLR)

The project aims at exploring ways in which a robot system could provide support and motivation for people in undertaking their daily physical exercises. Such an application is particularly interesting for astronauts in space, for whom it is vital during any long-term mission in zero or low gravity to maintain physical fitness and psychological well-being. In our scenario, a humanoid robot is supposed to assume autonomously the role of a fitness instructor and support a structured bike training which is inspired by indoor cycling (‘spinning’). To do so, the robot not only needs to announce next training actions, but importantly, needs to observe and evaluate the trainees actions and provide positive or corrective feedback. Within this interdisciplinary realm, our project part investigates - on the basis of video-data acquired in fitness studios - the interactional practices of micro-coordination and ‘motivation’ between human instructors and trainees involved in spinning classes. Results of the empirical analysis are used to build interactional models for the robot system, which are, in turn, evaluated in both short- and long-term studies of human-robot-interation.

Staff: Luise Süssenbach
Student Assistant: Lisa Skutella, Katarina Schoppmann
Project Webpage

iTalk. Integration and Transfer of Action and Language Knowledge in Robots (2008 - 2012)

Research staff. - Project headed by Katharina Rohlfing, Britta Wrede & Gerhard Sagerer (Applied Informatics & Emergentist Semantics, Bielefeld University)
Funded by the European Union (FP7)

The EU-project “iTalk” aims at developing artificial embodied agents that will be able to acquire complex behavioral, cognitive, and linguistic skills through individual and social learning. Within this frame, the Bielefeld team investigates social aspects of learning and focuses on a scenario, in which a human tutor presents and explains some task to a learner, who/which observes the action and, in turn, attempts to understand its structure in view of eventually imitating it. We use ‘tutoring’ in parent-infant-interaction as an empirical model to explore the topics of (i) action structuring / acoustic packaging (-> Lars Schillingmann), (ii) tutoring spotter (-> Katrin Lohan), and (iii) variability (-> Karola Pitsch). Karola’s research focuses on the interplay between the tutor’s and learner’s actions. She accesses the topic of ‘variability of tutor’s actions’ from an interactional perspective detailing the sources and consequences of observable actions in the unfolding course of action. This leads to an interactional account of “motionese” behavior and the investigation of feedback strategies in both parent-infant- and human-robot-tutoring. Methodologically, Karola uses Conversation Analysis to reveal interactional patterns and attempts - in interdisciplinary collaboration - to link qualitative research with quantitative approaches and formalization. Results of her empirical research are used as basis for the design for human-robot-experiments, which she evaluates with regard to the systems’ functioning and to understand the ways in which the users’ reactions and expectations are shaped in situ.

Student Assistant: Raphaela Gehle, Lukas Rix
Project Webpage

PaperWorks. Interweaving Paper and Digital Documents (2005 - 2008)

Research staff. - Project headed by Christian Heath & Paul Luff (Work, Interaction & Technology, King’s College London)
Funded by the European Union (FP6)

The EU-project “PaperWorks” is concerned with developing distinctive technologies for interleaving paper documents with digital materials. The project aims to provide people with new forms of functionality in everyday environments through seemingly mundane artefacts. This involves the production of novel hardware (paper substrates, inks, and reading devices), software and information infrastructure, which are informed by empirical studies of people using paper documents and associated tools as part of their everyday activities and by studies of individuals and organizations who/which produce digital and paper content. The project examines how publishers, professionals and other ’users’ currently design material using particular media. Drawing from the findings of these studies we undertake technical interventions where we provide people with various augmented technologies and authoring tools to support the production of content and interlinking of materials. Within this framework, Karola is particularly interested in the collaborative practices of exhibit/museum designers when envisioning exhibitions. She undertakes semi-experimental user studies of participants who use novel swiping devices when attempting to control a PowerPoint presentation from paper-based slides.

Project Webpage

Multimodality in Immersive Classroom Interaction (2002 - 2006)

PhD project
Funded by the DFG, Graduate Program ‘Task-Oriented Communication’ (Bielefeld University)

Karola’s PhD work is concerned with multimodal aspects classrooms interaction. Based on video-recording from immersive history classes (in which German and Argentinian students carry out their history courses in a foreign language) three issues are investigated on the level of micro-practices: (1) Multimodal procedures of dealing with locally occurring language problems; (2) The interactional production of blackboard inscriptions and their role as material resources (“intermediary objects”) in learning interactions; (3) Practices of interactional coordination of the ‘official’ classroom discourse and the students’ ‘private’ note-taking. On the basis of reconstructing these detailed practices and patterns and their interactional ‘risks and side effects’, the study offers a set of conceptual and methodological implications for a multimodal take of Conversation Analysis (parallel activities, participants’ on-line analysis, intermediary objects).