Date of Award
Doctor of Philosophy
Jessica F. Hay
Aaron T. Buss, Devin M. Casenhiser, Shannon Ross-Sheehy, Greg D. Reynolds
This dissertation aims to empirically assess the complex, multileveled relationships between audiovisual speech perception and early language development. The majority of extant language development research has justifiably focused on infants’ ability to learn language from auditory input, and indeed, infants are precocious auditory learners (Saffran & Kirkham, 2018). Complementary to auditory speech, however, are the necessarily redundant facial movements used to articulate speech. Outside of language development research, multimodal processing has been theorized to facilitate perceptual learning and cognitive development (Bahrick & Lickliter, 2000), but only a small number of empirical studies have investigated how audiovisual speech perception in infancy is related to later language development. Infants demonstrate sensitivity to audiovisual speech early in development (Patterson & Werker, 2003), however, it is still an open question as to how infants may functionally use these redundancies in the service of language learning. This dissertation first examines how infants integrate auditory and visual speech at 4.5-months by measuring their perception of the McGurk effect. To augment this behavioral work, a meta-analysis assesses infants’ perception of the McGurk Effect across available published and unpublished literature. A dynamic neural field (DNF) computational model then explores the real-time perceptual and neuro-cognitive processes that lead to the perception of the McGurk Effect. Next, to explore how audiovisual speech perception in infants relates to later language outcomes, longitudinal relationships between infants’ perception of the McGurk Effect, their native and non-native phoneme discrimination at 7-months, and vocabulary outcomes at 13- and 18-months are explored. Finally, the utility of audiovisual redundancies was tested for infants in a difficult object-label association laboratory task. The culmination of this dissertation creates a body of empirical and computational work focused on infants’ audiovisual speech perception at multiple timepoints across children’s bourgeoning language ability in the first two years of life.
Cannistraci, Ryan Andrew, "Audiovisual speech processing: Implications for speech perception and language development. " PhD diss., University of Tennessee, 2021.