Vision Captures Touch in Bisensory Letter Recognition Task
^1,2Kevin Lee, ^2,3Gregory O. Gibson, ^1,2,3Krish Sathian
¹Neuroscience and Behavioral Biology Program, Emory University, Atlanta, GA;
²Department of Neurology, Emory University School of Medicine, Atlanta, GA
³Atlanta VAMC Rehabilitation Reserach and Development Center of Excellence

Abstract

The aim of the present study was to investigate the occurrence of bisensory integration between vision and touch in form perception. Subjects discriminated between letters in a two-alternative forced-choice paradigm. Letters were presented visually, tactually, or simultaneously in both modalities. In the bisensory conditions, stimuli were either congruent or incongruent between modalities, and subjects attended to either vision or touch. When attending to tactile stimuli, there was a significant congruency effect, with more accurate responses in the congruent compared to the incongruent condition. Unisensory tactile accuracy was approximately mid-way between the congruent and incongruent conditions. In contrast, while attending to visual stimuli, a concurrent tactile stimulus did not affect accuracy. This asymmetry is akin to the “visual capture” reported in prior studies. Thus, the present study, rather than demonstrating bisensory integration, showed that vision tended to dominate touch.

Introduction

Everyday life is full of competing or complementary multisensory inputs. Competing information may create sensory illusions while redundant stimuli can enable improved recognition of information. An example of complementary bisensory inputs is reading the lips of a speaker, which lets a listener understand better what a speaker says in a noisy environment. There is a famous example of the effect of competing bisensory inputs called the “McGurk effect,” where inconsistent visual and auditory inputs create an auditory illusion. Recent work in our laboratory and from other groups has shown that visual and tactile form perception activate common brain regions. The present study was designed to investigate the occurrence of bisensory integration of form. We used as visual and tactile stimuli, the letters “T” and “V”. We hypothesized that bisensory integration would occur, as manifested by more accurate and faster stimulus discrimination during bisensory congruent stimulation, but less accurate and slower discrimination during bisensory incongruent stimulation.

Methods and Materials

Subjects: n= 12 (7 female, 5 male), mean age 20.5 (SD = 1.31). All subjects were right- handed and neurologically normal.

Stimuli: There were four types of stimuli (i) visual letters alone, (ii) tactile letters alone, (iii) visual/tactile congruent letters, and (iv) visual/tactile incongruent letters.

Procedure:



Visual Letters: Subjects were asked to attend to a fixation point in the center of the screen. The fixation point randomly varied between “+” or “x”, and also randomly varied in duration (300, 400, 500, 600, or 700 ms). The fixation point offset was coincident with the onset of the visual stimulus “T” or “V”, which appeared at 5º visual angle from the fixation point in the right lower quadrant of the screen. The duration of the letter was 150 ms; a 50 ms masking stimulus followed immediately. The subject was asked to respond, as quickly and accurately as possible, whether the letter was T or V. They were then asked to indicate which fixation symbol (“+” or “x”) had been present. Responses were indicated by key-presses. Only trials on which subjects correctly identified the fixation symbol were used in the data analysis.



Tactile Letters: Letters “T” and “V”, raised in relief 3-mm from a base-plate (20 mm square), were used. Each letter was pressed into the subject’s right index finger for 150ms, using a pneumatic stimulator. Subjects were also asked to attend to a fixation symbol on the computer screen and identify it as with the visual stimuli.

Unisensory vs. Bisensory: Subjects completed two unisensory runs, one visual and one tactile; they also completed two bisensory runs. In one bisensory run (visual-tactile) subjects were asked to respond only to the visual stimulus and ignore the tactile stimulus. In the other bisensory run (tactile-visual) they were asked to respond to the tactile stimulus and ignore the visual stimulus. The order of runs was counterbalanced across subjects.

Congruent vs. Incongruent: In the bisensory runs subjects received either congruent stimuli (visual “T” & tactile “T” or visual “V” & tactile “V”) or incongruent stimuli (visual “T” & tactile “V” or visual “V” & tactile “T”). There was an equal number of all trial types, presented in pseudorandom order without replacement.

Results

Unisensory accuracy for visual and tactile letters. The task was designed such that performance on the visual and tactile tasks would be roughly around threshold (75% correct performance) making the tasks approximately the same level of difficulty. A two-tailed paired t-test indicated no significant difference between the two tasks [t(11) = 1.97, p = .08]. Error bars represent ± one standard error of the mean.



Bisensory accuracy for visual-tactile and tactile-visual letters. One-tailed paired t-tests indicated no significant difference in the visual-tactile letter accuracy between congruent and incongruent conditions [t(11) = 0.44, p = .33], but a significant difference between tactile-visual congruent and incongruent conditions [t(11) = 2.84, p < .01]. Error bars represent ± one standard error of the mean.



Unisensory reaction time for visual and tactile letters. A one-tailed paired t-test indicated a significant difference between the visual and tactile reaction times [t(11) = 3.49, p < .01]. Error bars represent ± one standard error of the mean.



Bisensory reaction time for visual-tactile and tactile-visual letters. Paired, one-tailed t-tests indicated no significant differences in reaction time between congruent and incongruent conditions for either visual-tactile [t(11) = 0.94, p = .19], or tactile-visual [t(11) = -0.65, p = .27] conditions. Error bars represent ± one standard error of the mean.

Conclusions and Future Studies

The present study, rather than demonstrating bisensory integration, showed that vision tended to dominate touch in the perception of form. When attending to tactile stimuli, there was a significant congruency effect, with more accurate responses in the congruent compared to the incongruent condition. Unisensory tactile accuracy was approximately mid-way between the congruent and incongruent conditions. In contrast, while attending to visual stimuli, a concurrent tactile stimulus did not affect accuracy. This asymmetry is akin to the “visual capture” reported in prior studies (Rock & Victor 1964; Craig, 2006). Thus, integration of visual and tactile inputs does not seem to characterize form perception, despite substantial evidence for shared neural processing of such inputs (Sathian, 2005).

Resources

This research was supported by the National Science Foundation Grant No. BCS-0519417 and by Howard Hughes Medical Institute Grant No. 52005873. I thank Dr. Krish Sathian and Dr. Greg Gibson for their help in these experiments and for their mentorship in the SURE program this summer.

References

-Craig JC (2006). Perception, 35: 351-367.
-Rock I, Victor J (1964). Science, 143: 594-596.
-Sathian K (2005). Developmental Psychobiology, 46: 279-286.