In the present research, we explore whether concurrent olfactory processing can affect the performance and response times in a visual search task. To this purpose, we exposed our participants to one of three different odorants while they performed a cued, visual search task. On any given trial, participants unknowingly breathed in either neutral room air or air mildly scented with a strawberry, apple or lemon aroma, and then they were briefly cued to search for a target image of a strawberry, apple or lemon randomly positioned among other fruit distractors. Random couplings between scents and the cued target fruit were presented over trials. Baseline visual search performance was determined by randomly interleaving trials where no odorant was presented to the participants' nostrils. In odorant trials, the fruit scent could be congruent with the target fruit or incongruent. In the latter case, it was either unrelated to any visual stimulus in the array or matched one of the distractor fruits. Although the odorant was task irrelevant, we could determine an effect of the congruency of odorant-visual object couplings on our participants' search performance. Our findings point to an odorant-visual object congruency effect that depends on the participants' baseline visual search performance in the neutral room-air condition: low performers of the visual search task benefit most when the odorant and target are congruent, whereas high performers exhibit the greatest interference when the odorant is incongruent to the target. As such, our results provide support for a role of olfaction in visual search for a food target among other food distractors.
To test the effect of ongoing odorant processing on visual-search performance, we used a four-channel olfactometer. See Supplementary Fig. 1 for an illustration of the apparatus and setup. Participants were exposed to one fruit odorant (lemon, apple, or strawberry) or neutral room air (no odorant) while they searched for an image of a cued-target fruit presented among fruit distractors (Fig. 1A; see Methods). The odorant was delivered for 5 s based on prior testing showing that participants could reliably detect the presence of an odorant if presented for at least 3 s (see Supplementary material for detection task description - Supplementary Fig. 2). Visual stimuli had a short duration (cue 50 ms, search array 100ms) and desaturated colors to make vision less reliable and allow for a possible effect of olfaction on the visual search task. Nevertheless, such short durations still allow for object recognition, also in visual search tasks. For target examples see Supplementary Fig. 3; for distractor examples see Supplementary Fig. 4.
Prior to each trial, participants did not know if an odor would be presented or if so, whether it would be congruent or incongruent to the subsequently presented visual cue. In the baseline no-odorant condition, only neutral room air was delivered to participants during the task. In the target-congruent condition, participants breathed in scented air with an aroma congruent to that of the cued target fruit -- e.g., when a lemon odorant was presented, and the target fruit was a lemon. In the target-incongruent condition, participants breathed in scented air with an aroma that was incongruent with both the cued target fruit and any of the distractor fruits -- e.g., a lemon odorant was presented while the target fruit was a strawberry, and no lemon image was present in the search array. Additionally, we included a target-incongruent, distractor-congruent condition, in which the scented air contained an aroma incongruent with that of the target but congruent with one of the fruit distractors -- e.g., a lemon odorant was presented while the target fruit was a strawberry, but a lemon image was present among the distractors. Despite the fact that the odorant was task irrelevant, we expected to find a search facilitation due to visuo-olfactory congruency and a potential disruption effect due to incongruency, which should be even more pronounced when the task-irrelevant odorant matches one of the visual distractors.
We first explored the effect of odorant-visual object congruency of visual search performance. Figure 1B depicts the proportion of correct responses for the four possible odorant-visual object couplings. The results indicate that search performance is significantly modulated by the congruency of the odorant-visual target coupling. An ANOVA analysis indeed indicated a highly significant main effect of congruency on the search performance (F = 11.88, p < 0.001; η = 0.36 - large effect size). There is a highly significant difference between the conditions where the odorant is congruent or incongruent to the searched-for fruit target (significant post-hoc pairwise comparisons are reported in the caption of Fig. 1B).
An analysis of the errors made during the visual search task indicates that the participants most likely (approx. 47% of all errors) selected a placeholder that had contained a distractor (e.g., raspberry) with a color similar to that of the visual target (e.g., strawberry), with the remaining errors evenly distributed over the other alternatives that mismatched the target on color and odorant (χ(12) = 48.07, p < 0.001). The odorant appeared to play a lesser role in determining the selection of a placeholder location which the participants erroneously thought to have contained the target. The distribution of errors did not depend on the odorant presented (χ(6) = 7.8, p = 0.25, n.s.), but it did depend on the condition of odorant-visual object congruency (i.e., target-congruent, target-incongruent, distractor-congruent; chi-test = 22.27, df = 6, p < 0.001).
Our participants were instructed to move the cursor to the placeholder containing the target, using the computer mouse. Figure 1C depicts the response times of the participants when they performed the visual search task correctly. Here, there is a significant main effect of target visual-olfactory congruency (F= 4.49, p = 0.006; η = 0.18 - large effect size) on response times: when the aroma of the scented air was congruent to that of the fruit target, participants required less time to guide the cursor to the target placeholder compared to response times in the no-odorant condition (significant post-hoc pairwise comparisons are reported in the caption of Fig. 1C). The incongruent conditions did not significantly differ from the no-odorant condition.
Importantly, additional analyses showed that visual-olfactory congruency effects on search performance and response times do not differ across different visual targets and fruit scented-odorants (strawberry, apple, lemon; see Supplementary Fig. 5).
Because visual search efficiency can vary as a function of visual field position, we examined potential location effects by testing whether the quadrant in which the target appeared (upper-left, upper-right, lower-left, lower-right; see Methods) influenced participants' performance or response times in the baseline condition. The results indicate that the probability of a correct response was independent of target position (χ(3) = 5.16, p = 0.16, n.s.). Likewise, a one-way ANOVA on response times revealed no significant effect of quadrant (F(3, 522) = 2.28, p = 0.078, n.s.). Overall, these results indicate that search efficiency in our task was largely unaffected by target spatial position and did not reflect asymmetries between upper and lower or left and right visual fields.
In a post-experimental recognition test, participants were required to name the odorants they believed they had smelled (see Methods for details), to assess whether differences in congruency effects could be related to the explicit recognition of fruit-scented odorants. Response scoring revealed that 14 out of 22 participants correctly named all three odorants, six recognized only two out of three, and two identified only one. Importantly, no significant correlations emerged between odorant recognition scores and visual search performance in the different conditions (Spearman's rank correlation coefficient; all p-values > 0.05, n.s.). These results suggest that the olfactory-visual congruency effect is likely implicit and does not depend on conscious perception of the odorants.
Our paradigm allowed for a post-hoc analysis of the correlation between search performance in the neutral (no odorant) trials to those performance levels exhibited in the trials with olfactory stimulation. Figure 1D presents a scatterplot of the individual participant's performance enhancement (positive values) or impairment (negative values) as a function of his or her respective performance in the baseline condition where neutral room air was delivered by the olfactometer during visual search. There is a highly significant negative correlation between the differential performance of the participants and their baseline (no odorant) performance (median of 0.69). For the target-congruent condition, these results indicate that low visual-search performers (baseline values below the median) benefited from the congruency of the delivered odorant, whereas high performers (baseline values above the median) showed no improvement or even lower performance (Fig. 1D -- left panel). In contrast, with incongruent odorant-visual pairings, high performers exhibited a stronger impairment relative to trials without odorants, compared to low performers (Fig. 1D -- middle and right panels).
In a similar fashion, we could explore post-hoc correlations between response times in the different olfactory conditions compared to the neutral (no odorant) condition (median of 1.33 s). Figure 1E presents a scatterplot of the individual participant's response time reduction (negative values) or elevation (positive values) as a function of his or her respective response time in the baseline condition. The negative correlation in the congruent condition indicates that the reductions in response times for congruent cross-modal odorant cueing were more pronounced in slow responders (high baseline, longer response times), while fast responders (low baseline) did not show a marked facilitation (Fig. 1E -- left panel), again linking the cross-modal congruency effect with the individual participants' search efficiency in the baseline (no odorant) condition. Note, however, that in conditions of incongruent couplings slow responders also exhibited a more pronounced reduction in response times compared to faster searchers (Fig. 1E -- middle and right panels).
Figure 2A depicts the procedure of the discrimination control task we used to determine the extent to which our participants could correctly discriminate among the different fruit aromas (see Methods). In the discrimination task, one odorant channel opens (scent of lemon, strawberry, or apple), and participants had to select the fruit matching the perceived aroma amongst eight visual stimuli. This task was conducted after the main visual search task to avoid influencing the participant's behavior in the main experiment. Figure 2B shows discrimination performance (proportion of correct responses) for the three fruit-scented odorants, while Fig. 2C presents the average response times (correct responses only). On average, participants correctly discriminate the fruit scent in 73% of trials in 5.6 s. ANOVA analyses show that there were not any significant differences between the three fruit scents with respect to the participants' odorant discrimination performance (F= 2.4, p = 0.102, n.s.) and response times (F= 2.1, p = 0.12, n.s.).
Figure 2D shows the relative effect of the congruent odorant during visual search plotted as a function of the discrimination performance in the control discrimination task. The negative correlation is highly significant (r = - 0.64, p = 0.001). Two aspects of the results need to be noted. First, the participants varied considerably in their ability to discriminate between the fruit aromas presented randomly to them via the olfactometer. While some participants performed nearly perfectly, others indicated discrimination performance levels that were above guessing levels but still low. Second, the participants who had more difficulty discriminating between the different fruit odors (value on x-axis) benefited most when these odorants were congruent to the searched-for target fruit (value on y-axis). Interestingly, participants who easily discriminated between the different fruit odorants showed little or no benefit of the congruent odor-visual couplings in the search task. Some even demonstrated lower performance when the odor they were exposed to was congruent to the cued target fruit. Note that the relative changes in visual search performance for the target incongruent and distractor congruent conditions do not correlate significantly with the odorant discrimination performance (Pearson correlation coefficients; all p-values > 0.05, n.s.; not shown in Fig. 2).