Skip to main content
Mijn bibliotheek
Shop
Nieuws Boeken Tijdschriften Toetsvragen Web-tv Video Audio
Tips voor Mijn BSL
Inloggen

Welkom bij THIM Hogeschool voor Fysiotherapie & Bohn Stafleu van Loghum

THIM Hogeschool voor Fysiotherapie heeft ervoor gezorgd dat je Mijn BSL eenvoudig en snel kunt raadplegen. Je kunt je links eenvoudig registreren. Met deze gegevens kun je thuis, of waar ook ter wereld toegang krijgen tot Mijn BSL. Heb je een vraag, neem dan contact op met helpdesk@thim.nl.

» Andere revante producten voor THIM studenten en medewerkers.

Registreer

Om ook buiten de locaties van THIM, thuis bijvoorbeeld, van Mijn BSL gebruik te kunnen maken, moet je jezelf eenmalig registreren. Dit kan alleen vanaf een computer op een van de locaties van THIM.

Eenmaal geregistreerd kun je thuis of waar ook ter wereld onbeperkt toegang krijgen tot Mijn BSL.

Login

Als u al geregistreerd bent, hoeft u alleen maar in te loggen om onbeperkt toegang te krijgen tot Mijn BSL.

Inloggen
Top
Psychological Research
Gepubliceerd in:

01-01-2007 | Original Article

Hearing lips in a second language: visual articulatory information enables the perception of second language sounds

Auteurs: Jordi Navarra, Salvador Soto-Faraco

Gepubliceerd in: Psychological Research | Uitgave 1/2007

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail
    Link delen
publish
Publiceer nu

Abstract

We investigated the effects of visual speech information (articulatory gestures) on the perception of second language (L2) sounds. Previous studies have demonstrated that listeners often fail to hear the difference between certain non-native phonemic contrasts, such as in the case of Spanish native speakers regarding the Catalan sounds /ɛ/ and /e/. Here, we tested whether adding visual information about the articulatory gestures (i.e., lip movements) could enhance this perceptual ability. We found that, for auditory-only presentations, Spanish-dominant bilinguals failed to show sensitivity to the /ɛ/–/e/ contrast, whereas Catalan-dominant bilinguals did. Yet, when the same speech events were presented audiovisually, Spanish-dominants (as well as Catalan-dominants) were sensitive to the phonemic contrast. Finally, when the stimuli were presented only visually (in the absence of sound), none of the two groups presented clear signs of discrimination. Our results suggest that visual speech gestures enhance second language perception at the level of phonological processing especially by way of multisensory integration.
vorige artikel Taking the brain serious: introduction to the special issue on integration in and across perception and action
volgende artikel Kinematic cues in perceptual weight judgement and their origins in box lifting
Literatuur
Alsius, A., Navarra, J., Campbell, R., & Soto-Faraco, S. (2005). Audiovisual integration of speech falters under high attention demands. Current Biology, 15, 839–843.CrossRefPubMed
Arnold, P., & Hill, F. (2001). Bisensory augmentation: a speechreading advantage when speech is clearly audible and intact. British Journal of Psychology, 92, 339–355.CrossRef
Auer, E. T., & Bernstein, L. (1997). Speechreading and the structure of the lexicon: computationally modeling the effects of reduced phonetic distinctiveness on lexical uniqueness. Journal of the Acoustical Society of America, 102, 3704–3710.CrossRefPubMed
Best, C. (1995). A direct realist view of cross-language speech perception. In W. Strange (Ed.). Speech perception and linguistic experience. Issues in cross-language research(pp. 171–204) Baltimore, MD: York Press.
Bosch, L., Costa, A., & Sebastian-Gallés, N. (2000). First and second language vowel perception in early bilinguals. European Journal of Cognitive Psychology, 12, 189–222.CrossRef
Burnham, D. K. (1998) Language specificity in the development of auditory-visual speech perception. In R. Campbell, B. Dodd& D.Burnham (Eds.) Hearing by eye II: advances in the psychology of speechreading and auditory–visual speech(pp. 27–60). Hove, UK: Psychology Press.
Callan, D. E., Jones, J. A., Munhall, K., Callan, A. M., Cross, C., & Vatikiotis-Bateson, E. (2003). Neural processes underlying perceptual enhancement by visual speech gestures. Neuroreport, 14, 2213–2218.CrossRefPubMed
Calvert, G. A., Brammer, M. J., Bullmore, E. T., Campbell, R., Iversen, S. D., & David, A. S. (1999). Response amplification in sensory-specific cortices during crossmodal binding. Neuroreport, 10, 2619–2623.PubMedCrossRef
Calvert, G., Spence, C.,& Stein, B. E. (Eds.). (2004). The handbook of multisensory processes. Cambridge: MIT Press.
Campbell, R. (2004). Audiovisual speech processing. In K. Brown (Ed.). The Encyclopedia of language and linguistics, 2nd edition—Editor-in-chief Keith Brown. Oxford, UK: Elsevier.
Colin, C., Radeau, M., Soquet, A., Demolin, D., Colin, F., & Deltenre, P. (2002). Mismatch negativity evoked by the McGurk–MacDonald effect: a phonetic representation within short-term memory. Clinical Neurophysiology, 113, 495–506.CrossRefPubMed
Davis, C., & Kim, J. (2004). Audio-visual interactions with intact clearly audible speech. Quarterly Journal of Experimental Psychology, 57, 1103–1121.PubMed
Ernst, M. O., & Banks, M. S. (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429–433.CrossRefPubMed
Ernst, M. O., & Bulthoff, H. H. (2004). Merging the senses into a robust percept TRENDS in Cognitive Sciences, 8, 162–169.
Fadiga, L., Craighero, L., Buccino, G., & Rizzolatti, G. (2002). Speech listening specifically modulates the excitability of tongue muscles: a TMS study. European Journal of Neuroscience, 15, 399–402.CrossRefPubMed
Flege, J. (1995). Second language speech learning: Theory, findings and problems. In W. Strange (Ed.). Speech perception and linguistic experience. Issues in cross-language research (pp. 233–272), Baltimore, MD: York Press.
Forster, K. L., & Forster, J. C. (2003). DMDX: a windows display program with millisecond accuracy. Behavioural Research Methods, Instruments,& Computers, 35, 116–124.
Garner, W. R. (1978). Selective attention to attributes and to stimuli. Journal of Experimental Psychology: General, 107, 287–308.CrossRef
Ghazanfar, A. A., & Logothetis, N. K. (2003). Facial expressions linked to monkey calls. Nature, 423, 937–938.CrossRefPubMed
Hommel, B. (in press). Feature integration across perception and action: Event files affect response choice. Psychological Research, Psychologische Forschung.
Kerzel, D. (2002). Evidence for effects of phonological correspondence between visible speech and written syllables. Psychological Research, 66, 195–200.CrossRefPubMed
Kerzel, D., & Bekkering, H. (2000). Motor activation from visible speech: evidence from stimulus response compatibility. Journal of Experimental Psychology. Human Perception and Performance, 26, 634–647.CrossRefPubMed
Klucharev, V., Möttönen R., & Sams, M. (2003). Electrophysiological indicators of phonetic and non-phonetic multisensory interactions during audiovisual speech perception. Cognitive Brain Research, 18, 65–75.CrossRefPubMed
Kuhl, P. K. (1991). Human adults and human infants show a "perceptual magnet effect" for the prototypes of speech categories, monkeys do not. Perception and Psychophysics, 50, 93–107.PubMed
Kuhl, P. K., & Meltzoff, A. N. (1982). The bimodal perception of speech in infancy. Science, 218, 1138–1141.PubMedCrossRef
Kuhl, P. K., Tsao, F. M., & Liu, H. M. (2003). Foreign-language experience in infancy: effects of short-term exposure and social interaction on phonetic learning. Proceedings of the National Academy of Sciences USA, 100, 9096–9101.CrossRef
Liberman, A. M., Cooper, F. S., Shankweiler, D. P., & Studdert-Kennedy, M. (1967). Perception of the speech code. Psychological Review, 74, 431–461.PubMedCrossRef
Massaro, D. W. (1987). Speech perception by ear and eye. In B. Dodd& R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 53–83). London, England UK: Lawrence Erlbaum Associates, Inc.
Massaro, D. W. (1998). Perceiving talking faces. Cambridge, MA: MIT Press
Massaro, D. W., Tsuzaki, M., Cohen, M., Gesi, A., & Heredia, R. (1993). Bimodal speech perception: An examination across languages. Journal of Phonetics, 21:445–478
Mattys, S. L., Bernstein, L. E., & Auer, E. T. (2002). Stimulus-based lexical distinctiveness as a general word-recognition mechanism. Perception & Psychophysics, 64, 667–679.
McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 265, 746–748.CrossRef
Mehler, J., Sebastián-Gallés, N., & Nespor, M., (2004). Biological Foundation of Language Acquisition: Evidence from Bilingualism. In M. S. Gazzaniga (Ed.), Cognitive Neurosciences III (pp. 825–836). Cambridge, MA: MIT press.
Molholm, S., Ritter, W., Javitt, D. C., & Foxe, J. J. (2004). Multisensory visual–auditory object recognition in humans: A high-density electrical mapping study. Cerebral Cortex, 14, 452–465.CrossRefPubMed
Navarra, J., Sebastián-Gallés, N., & Soto-Faraco, S. (2005). The perception of second language sounds in early bilinguals: New evidence from an implicit measure. Journal of Experimental Psychology: Human Perception and Performance, 31(5), 912.CrossRefPubMed
Pallier, C., Colomé, A., & Sebastián-Gallés, N. (2001). The influence of native-language phonology on lexical access: Exemplar-based versus abstract lexical entries. Psychological Science, 12, 445–449.CrossRefPubMed
Prinz, W. (1990). A common coding approach to perception and action. In O. Neumann & W. Prinz (Eds.), Relationships between perception and action: Current approaches (pp. 167–201). Berlin: Springer.
Reisberg, D., McLean, J., & Goldfield, A. (1987). Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.). Hearing by eye: The psychology of lip-reading. Hillsdale: LEA
Sebastián-Gallés, N., & Baus, C. (2005). On the relationship between perception and production in L2 categories. In A. Cutler (Ed.) Twenty-First Century Psycholinguistics. Hillsdale: Erlbaum.
Sebastián-Gallés, N., & Soto-Faraco, S. (1999). Online processing of native and non-native phonemic contrasts in early bilinguals. Cognition, 72, 111–123.CrossRefPubMed
Sekiyama K., & Tohkura Y. (1993) Inter-language differences in the influence of visual cues in speech perception. Journal of Phonetics, 21, 427–444.
Soto-Faraco, S., Navarra, J., & Alsius, A. (2004). Assessing automaticity in audiovisual speech integration: Evidence from the speeded classification task. Cognition, 92, B13-B23.CrossRefPubMed
Soto-Faraco, S., Navarra, J., Voloumanos, A., Sebastián-Gallés, N., Weikum, & Werker, J. F. (submitted). Discriminating languages by speechreading. Perception and Psychophysics.
Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: The MIT Press.
Strange, W. (1995). Cross-language studies of speech perception. A historical review. In W. Strange (Ed.) Speech perception and linguistic experience. Issues in cross-language research (pp. 3–45). Baltimore, MD: York Press.
Sumby, W., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212–215.CrossRef
Summerfield, Q. (1987). Some preliminaries to a comprehensive account of audio-visual speech perception. In B. Dodd, R. Campbell (Eds.). Hearing by Eye: The Psychology of Lip-Reading (pp. 3–51). London, UK: LEA.
Tomiak, G. R., Mullennix, J. W., & Sawusch, J. R. (1987). Integral processing of phonemes: Evidence for a phonetic mode of perception. Journal of the Acoustical Society of America, 81, 755–764.CrossRefPubMed
Van Wassenhove, V., Grant, K.W., & Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the Natural Academy of Sciences USA, 102(4), 1181–1186.CrossRef
Watkins, K. E., & Paus, T. (2004). Modulation of motor excitability during speech perception: The role of Broca’s area. Journal of Cognitive Neuroscience, 16(6), 978–987.CrossRefPubMed
Watkins, K. E., Strafella, A.P., & Paus, T. (2003). Seeing and hearing speech excites the motor system involved in speech production. Neuropsychologia, 41(8), 989–994.CrossRefPubMed
Werker, J. F., & Tees, R. C. (1983). Developmental changes across childhood in the perception of non-native speech sounds. Canadian Journal of Psychology, 37(2), 278–286.PubMed
Werker, J. F., Gilbert, J. H. V., Humphrey, K., & Tees, R.C. (1981). Developmental aspects of cross-language speech perception. Child Development, 52, 349–355.PubMedCrossRef
Wilson, S. M., Pinar Saygin, A., Sereno, M. I., & Iacobini, M. (2004). Listening to speech activates motor areas involved in speech production. Nature, 7(7), 701–702.
Wright, T. M., Pelphrey, K. A., Allison, T., McKeown, M. J., & McCarthy, G. (2003). Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cerebral Cortex, 13, 1034–1043.CrossRefPubMed
Metagegevens
Titel
Hearing lips in a second language: visual articulatory information enables the perception of second language sounds
Auteurs
Jordi Navarra
Salvador Soto-Faraco
Publicatiedatum
01-01-2007
Uitgeverij
Springer-Verlag
Gepubliceerd in
Psychological Research / Uitgave 1/2007
Print ISSN: 0340-0727
Elektronisch ISSN: 1430-2772
DOI
https://doi.org/10.1007/s00426-005-0031-5

Andere artikelen Uitgave 1/2007

Repetition costs in word identification: evaluating a stimulus–response integration account

  • Original Article

Feature binding and episodic retrieval in blindness for congruent stimuli: evidence from analyses of sequential congruency

  • Original Article

The manifestation of attentional capture: facilitation or IOR depending on task demands

  • Original Article

Taking the brain serious: introduction to the special issue on integration in and across perception and action

  • Editorial

Feature integration across perception and action: event files affect response choice

  • Original Article

Kinematic cues in perceptual weight judgement and their origins in box lifting

  • Original Article