Testing the coordinated interplay account

Temporal coordination To test the CIA, my research more closely investigated the temporal coordination between utterance comprehension, and the effects of depicted events on comprehension. The coordinated interplay account by Knoeferle and Crocker (accepted) predicts that the time course with which a depicted event influences thematic role assignment depends on when that depicted event is identified as relevant by the utterance. We monitored eye movements in a scene during the comprehension of German utterances that differed with respect to when they identified a relevant depicted event for thematic role assignment and structuring of the utterance. Findings confirmed the coordinated interplay account: Gaze patterns revealed differences in the time course with which a depicted event triggered thematic role assignment depending on whether that event was identified as relevant for comprehension early or late by the utterance.

Knoeferle, P. (2007). Comparing the time-course of processing initially ambiguous and un- ambiguous German SVO/OVS sentences in depicted events. In: R. van Gompel, M. Fischer, W. Murray, & Robin Hill (eds). Eye-movement Research. Elsevier. pdf of final draft

The priority of depicted events: what's the origin? Our research has shown that people prefer to rely on depicted events rather than their stereotypical knowledge (see here). However, the presence of depicted events in the scene may have exaggerated their importance. Two eye-tracking experiments examined this issue by varying the accessibility of scene information. When we varied the visual presence of the scene (the scene disappeared before the utterance was heard), findings confirmed a greater relative priority of depicted events (Experiment 1).

In contrast, when we altered the temporal extension of scene events (scene presentation emulated that they had been completed), people neither had a preference to rely on depicted events nor on their stereotypical knowledge (Experiment 2). These findings suggest that the visual presence of scene events cannot account for the preference to rely on depicted events.

Knoeferle, P. & Crocker, M.W. (2007). The influence of recent scene events on spoken comprehension: evidence from eye movements. Journal of Memory and Language.

Knoeferle, P. & Crocker, M.W. (2006). The coordinated interplay of scene, utterance, and world knowledge: evidence from eye tracking Cognitive Science, 30, 481-529.