RUI: Causes and consequences of early quitting in visual search: Investigating the role of distractors

Connecticut College, New London CT

Investigators

Abstract

Distractions are everywhere in modern life, and they bring consequences ranging from the mundane to the deadly. It is well established that distracting objects can attract attention when a viewer first looks at a scene. For example, a flashing roadside billboard may take a driver’s eyes off the road for a moment. But might distractions change behavior in other, more subtle but equally critical ways? Another key aspect of visual attention arises when a person searches for something that may or may not be present and must decide whether or not they’ve found what they are looking for. An important illustration of this is that when a radiologist searches a medical image, there may or may not be an area of concern present in the scan. In these searches, there is a strategic decision component in which the searcher must decide that they’ve looked thoroughly enough to be confident that indeed no “target” is present. However, there is a gap in our knowledge – there has been little research on how distracting objects might affect this decision component. In the current project, the investigators explore the phenomenon of distractor-induced quitting, in which distracting objects alter this decision process and cause people to terminate search earlier than they otherwise would. This early quitting causes people to entirely miss targets that they would otherwise likely find. Knowledge gained from this project will advance our understanding of how distractions are processed, leading to new insights into human behavior in the fields of attention, distraction, and decision-making. Furthermore, these results have potential real-world implications for tasks that involve high-stakes searches for targets, such as medical image screening or x-ray baggage inspection. In particular, it is worth considering that the use of salient signals (e.g., from artificial intelligence) to convey information to a human observer may inadvertently trigger this exact problematic situation. For example, if a computer system is trained to scan images and highlight potential areas of interest for a radiologist (or security personnel) by using a salient signal, these quitting effects might offset any benefits the computer guidance system might otherwise afford. Finally, undergraduate students participate in the design of these experiments, collection of data, and the presentation of results at conferences including those focused on medical imaging. Some of these students are recruited from the Science Leaders program at Connecticut College, a program dedicated to providing opportunities in the sciences for students from historically excluded identities. Salient signals can alter search strategies when people are looking for targets. More precisely, in recent work, the investigators have discovered that task-irrelevant distractors can cause people to quit searching early. As a result, people more frequently miss targets when these distractors are present. In this project, the investigators use a variety of experimental protocols to explore the impact of salient distractors on visual search in tasks where targets may or may not be present. Participants search for simple targets in visual displays with multiple non-targets and press a key to indicate whether a target is present or not. Eye-tracking is employed to investigate the specific mechanisms that cause participants to quit early as a result of visual distraction – for example, when a distractor is present, does it cause participants to scan the display less exhaustively? Or does it cause participants to look at each item for a shorter time, making them more likely to look at a target but fail to process it correctly? Next, the investigators establish factors that can modulate and potentially eliminate this distractor-induced quitting, such as giving participants control over the appearance and disappearance of a salient cue that may or may not highlight the target. Finally, the investigators examine how the information content of salient signals can impact distractor-induced quitting – in other words, if salient signals sometimes draw attention to the target, do those signals cause as much (or perhaps more) disruption on the occasions where they do not draw attention to the target? Results from these studies may aid our understanding of human visual attention and visual search. Findings are shared with the scientific community at large, but also (more specifically) with colleagues in radiology in order to spark new discussion on how the investigators might apply this research to help improve communications from artificial intelligence systems to human observers in medical settings. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.