Document Type
Article
Publication Date
2023
Journal Title
Frontiers in Cognition
Volume Number
2
Version
Publisher PDF: the final published version of the article, with professional formatting and typesetting
Creative Commons License
This work is licensed under a CC BY License.
Abstract
An audiovisual correspondence (AVC) refers to an observer’s seemingly arbitrary
yet consistent matching of sensory features across the two modalities; for
example, between auditory pitch height and visual height or visual size. Research
on AVCs frequently uses a speeded classification procedure in which participants
are asked to rapidly classify the pitch of a sound accompanied either by a
congruent or an incongruent visual object (e.g., high pitches are congruent with
higher/smaller visual objects and incongruent with lower/larger visual objects).
To investigate the strength of these pitch AVCs (height, size, spatial frequency,
brightness, sharpness), trials where the height AVC competed with each other AVC
in terms of pitch congruency were created. For example, when classifying pitch
height, participants were presented with trials where both visual height and size
were congruent or incongruent with pitch; additionally, there were trials where
height was congruent but size was incongruent (i.e., high pitch matched with
large object at high height) and trials where size was congruent but height was
incongruent (i.e., high pitch matched with small object at low height). Based
on previous work, congruency between pitch and height was expected to be
more important than congruency between pitch and spatial frequency, brightness,
sharpness, or size. As predicted, in all four studies, RTs when only height was
congruent were just as fast as when both dimensions were congruent. In contrast,
RTs when only spatial frequency, brightness, sharpness, or size was congruent
(and height was incongruent) were just as slow as when both dimensions were
incongruent. These results reinforce the superiority of the pitch-height AVC and
can be interpreted based on the metaphor used for pitch in English, showing the
importance of semantic/linguistic effects to understanding AVCs.
Digital USD Citation
Getz, Laura, "Competition Between Audiovisual Correspondences Aids Understanding of Interactions Between Auditory and Visual Perception" (2023). Psychological Sciences: Faculty Scholarship. 3.
https://digital.sandiego.edu/psyc-faculty/3