Depicted events

Eye tracking A first line of research in the group examines effects of visual contexts that depict agent-action-patient events during language comprehension. Findings suggest the rapid use of depicted events for the resolution of local structural and thematic role ambiguity in German subject-verb-object (SVO)/ object-verb-subject (OVS) sentences (Knoeferle, Crocker, Scheepers, & Pickering, 2005). Subsequent studies extended this finding to another language (English) and other structures (MV/RR ambiguity) (Knoeferle & Crocker, 2006).

Event-related potentials Complementary evidence from event-related potentials has confirmed the eye-tracking findings. When the verb in the utterance mediated depicted events, we found a P600 for initially structurally ambiguous non-canonical (OVS) compared with canonical (SVO) sentences. No such verb-mediated P600 was found for unambiguous SVO/OVS sentences.

The Coordinated Interplay Account (CIA): Based on existing eye-tracking findings we have developed a processing account (Knoeferle & Crocker, 2006, 2007) of situated utterance comprehension. The account has been implemented as a connectionist model (CIANet, Mayberry et al., 2009). More recently, the CIANet has been extended to account for ERP findings on visual context effects in spoken language comprehension (Crocker et al., 2010).

For relevant publications see:

Crocker, M.W., Knoeferle, P., & Mayberry, M. (2010). Situated sentence processing: The coordinated interplay account and a neurobehavioral model. Brain and Language 112, 189-201. [pdf]

Knoeferle, P., Habets, B., Crocker, M.W., & Muente, T.F. (2008). Visual scenes trigger immediate syntactic reanalysis: evidence from ERPs during spoken comprehension. Cerebral Cortex. knoeferle-etalcc07.pdf

Knoeferle, P. & Crocker, M.W. (2007). The influence of recent scene events on spoken comprehension: evidence from eye movements. Journal of Memory and Language.

Knoeferle, P. & Crocker, M.W. (2006). The coordinated interplay of scene, utterance, and world knowledge: evidence from eye tracking Cognitive Science, 30, 481-529.

Knoeferle, P., Crocker, M.W., Scheepers, C., & Pickering, M.J. (2005). The influence of the immediate visual context on incremental thematic role-assignment: evidence from eye-movements in depicted events. Cognition, 95, 95-127 pdf of final draft

Knoeferle, P., Crocker, M.W., Scheepers, C., & Pickering M.J. (2003). Actions and roles: using depicted events for disambiguation and reinterpretation in German and English. In: Proceedings of the 25th Annual Conference of the Cognitive Science Society (Talk), (pp. 681-686), Boston, MA.