Doctoral Dissertations

Date of Award

8-2021

Degree Type

Dissertation

Degree Name

Doctor of Philosophy

Major

Psychology

Major Professor

Jessica F. Hay

Committee Members

Aaron T. Buss, Devin M. Casenhiser, Shannon Ross-Sheehy, Greg D. Reynolds

Abstract

This dissertation aims to empirically assess the complex, multileveled relationships between audiovisual speech perception and early language development. The majority of extant language development research has justifiably focused on infants’ ability to learn language from auditory input, and indeed, infants are precocious auditory learners (Saffran & Kirkham, 2018). Complementary to auditory speech, however, are the necessarily redundant facial movements used to articulate speech. Outside of language development research, multimodal processing has been theorized to facilitate perceptual learning and cognitive development (Bahrick & Lickliter, 2000), but only a small number of empirical studies have investigated how audiovisual speech perception in infancy is related to later language development. Infants demonstrate sensitivity to audiovisual speech early in development (Patterson & Werker, 2003), however, it is still an open question as to how infants may functionally use these redundancies in the service of language learning. This dissertation first examines how infants integrate auditory and visual speech at 4.5-months by measuring their perception of the McGurk effect. To augment this behavioral work, a meta-analysis assesses infants’ perception of the McGurk Effect across available published and unpublished literature. A dynamic neural field (DNF) computational model then explores the real-time perceptual and neuro-cognitive processes that lead to the perception of the McGurk Effect. Next, to explore how audiovisual speech perception in infants relates to later language outcomes, longitudinal relationships between infants’ perception of the McGurk Effect, their native and non-native phoneme discrimination at 7-months, and vocabulary outcomes at 13- and 18-months are explored. Finally, the utility of audiovisual redundancies was tested for infants in a difficult object-label association laboratory task. The culmination of this dissertation creates a body of empirical and computational work focused on infants’ audiovisual speech perception at multiple timepoints across children’s bourgeoning language ability in the first two years of life.

Download
Files over 3MB may be slow to open. For best results, right-click and select "save as..."

Share

COinS