In response to our survival needs, our early-stage perceptual system has evolved neurologically to quickly detect and process patterns. In approximately 250-millisecond intervals, our eyes focus, then detect, and process patterns from an environment (Töllner et al., 2011). This process occurs repeatedly and naturally, and the patterns will consist of items that share common characteristics (Hoffman, 1975). This part of the visual system is called pre-attentive, or early-stage processing. There are both psychophysical and neurological theory components that need to be considered to fully construct a complete picture of the pre-attentive system—understanding the pre-attentive processing allow the design of information display to offer near effortless human perception and interaction. This review examines the neurological and psychological theories surrounding pre-attentive processing and evaluates a Covid-19 report page on HealthData.gov, focusing on both the conformed variables and those that violate established guidelines.
Pre-attentive Processing
In the human visual system, pre-attentive involves rapid processing of patterns based on items perceived in the outside world (Hoffman et al., 1999). Our eyes rapidly shift from view to view, and an image is formed on the retina in the back of the eye. Detected patterns (e.g., edges, colors, textures, and contrast) are fed forward through the ganglion cells, the optic nerve, lateral geniculate nucleus, and onto the visual cortex. Along the way, information is maintained in a retinotopic map, further processed, and grouped. Specialized neurons remove redundancy in perceived patterns (Airbib, 1995). This is important as to process the entirety of the visual image on the retina would overwhelm the brain (Töllner et al., 2011). In a system proposed by Anne Treisman and Garry Gelade, our visual system encodes and groups together items that share similar attributes such as color, length, contrast, or shape (Treisman & Gelade, 1980). With the grouping principle, items that share a common attribute are perceived as belonging together (Treisman, et al., 1988). Grouping can be proximity where items are located close together; alignment where items are perceived as having a common edge; similarity of color or size; common region where items are inside of a common border; continuity where items are seen as smooth and continuous (Wageman, et al., 2012). Understanding these spatial features can help designers improve design implications relating to rapid perceptual organization.
Neurologically-Based Pattern Perception
Our eyes continuously, and naturally, move from one scene to the next in what are called saccades. Each saccade takes about a quarter of a second, after which we fixate on a point for approximately 50ms, attain an image, which is focused on the retina (Hoffman et al., 1999). Although we will only be considering bottom-up, it has been suggested that the spatial attention system—the driving mechanism to determine which scene each subsequent fixation focuses on— has both a top-down (i.e., cognitive) as well as a bottom-up component (Karnath, 2015). Our eyes ingest between 1 to 10 million bits of information per second (Töllner et al., 2011) during saccades, to process this amount of information would overload our visual processing system. Human neurological systems have naturally evolved to develop a system known as pre-attentive, or early-stage processing, which allows for the quick reduction of limited sets of this information into patterns (Kamkar, et al., 2018). The beginning of pattern detection, and pre-processing starts with the retinal ganglion cells (RGCs) (Kim, et al., 2021). The more than 40 types of RGCs extract and process in parallel information such as color, textures, and contrast from each scene into RGC feature representations (Kim, et al., 2021, Kerschensteiner, 2022). Working in conjunction with horizontal and amacrine cells the RGCs produce lateral inhibitions which allow for edge detection (Cook, et al., 1998). They also produce and maintain retinotopic maps which will be transmitted through the visual processing system (Poirier, et al., 2008). This early-stage processing of edges, colors, and contrasts matches the pattern recognitions of similarity. The RGCs feed forward this processed information through to the lateral geniculate nucleus via the optic nerve.
Located in the thalamus region of the brain are two lateral geniculate nuclei (LGNs) that consist of three types of neurons spreading across six cellular layers. The first two layers (known as the ventral) contain magnocellular neurons (M) which are motion sensitive and process depth, edges, and luminance changes (Covington, et al, 2022, Ghodrati, et al, 2017); however, the M cells are not color sensitive. The parvocellular (P) are in the four outer dorsal layers and process red and green colors. The Koniocellular (K) cells are located in between the other cells and specialize in yellow and blue colors (Ghodrati, et al., 2017). In much of the literature, the LGNs are considered a "relay nucleus" (or waystation) which serve to maintain, and forward information to the visual cortex; however, there is ongoing research that seeks to determine if the LGNs may be involved in higher order signal and pattern processing (Ghodrati, et al., 2017). While color and hue can be used in similarity patterns, maintaining edge information is critical to common region, alignment, and symmetry patterns. The LGNs will feed forward visual information to the visual cortex for the next stage in pre-attentive processing. The retinotopic mapping provided by the RGCs will be maintained and fed forward to the primary visual cortex (Schneider, et al., 2004).
Receiving information transmitted from the two LGNs, the primary visual cortex consists of five separate areas, each layer does more specialized processing than the previous (Huff, et al., 2022). The first area known as V1, or the striate, is the section of the visual cortex that initially receives nearly all information transmitted via the LGNs (Tootell, et al., 1998). The V1 consists of six separate areas, each of which maintains different functionality. V1 maintains the feature map as forwarded and is responsible for low level feature processing of contrast, directional columns, edges, and lines (Rottschy, et al., 2007, Huff, et al., 2022). V1 forwards information through the ventral and dorsal pathways for further processing in the cortex. Visual information in the ventral pathway passes through the V2 and V4 layers and onto the inferior temporal visual cortex. This layer processes more complex patterns, variations in color, alignment, and spatial variations, texture, and contours (Huff, et al., 2022, Treue, 2001 & Rottschy, et al., 2007). The dorsal stream is routed through V2 and onto the posterior parietal cortex. The dorsal stream is associated with spatial processing, motion, and object recognition (Huff, et al., 2022 & Rottschy, et al., 2007).
Psychophysically-Based Pattern Perception
Feature integration, a cornerstone psychophysical theory presented in 1980 by Anne Treisman and Garry Gelade, proposes that our visual system processes feature early on, in parallel and automatically (Treisman & Gelade, 1980). Individual feature maps based on attributes such as color, texture, shape, and edge are individually and parallelly encoded, and then grouped in conjunction to generate a master map which provides spatial, regional, and other information in how individual maps are related (Treisman, et al., 1988). This is a key consideration for design as items that should be considered related but have different features, or groupings, can end up incorrectly linked in the master map, a principal known as conjunction, or feature errors (Prinzmetal, 1981, Treisman & Gelade, 1980). Feature processing aligns with the physiological system as ganglion cells and the LGN are attuned to attributes like color and luminance, while specialized neurons in V1 are attuned to features such as size, orientation, and spatial frequency (Kamkar, et al., 2018). Duncan and Humphries expanded on the work of Treisman in developing similarity theory which proposes that image components are grouped by elements that share a commonality (e.g., alignment and color) (Duncan, et al., 1989). This was again expanded on by Wagemans and others to show that similarity could be due to changes in features (e.g., changes in luminance or position) and not just the attributes themselves (Wageman, et al., 2012). This follows the same retinotopic mapping as described in neurology where maps are generated and preserved between the neurons of the various parts of the visual system in feed forward process (Wu, et al., 2012).
The Grouping of Information Display on HealthData.gov
How visual scenes are perceived are based on the different mechanisms of grouping principles. Proximity, one of the key principles of grouping, states that objects that are close in space are perceived to be in the same group (Wageman, et al., 2012). This ties into the functionality of the dorsal stream which processes information related to object location (Sheth, et al., 2016). With alignment in the grouping principle, items are assumed to be grouped together when they are perceived to be aligned in a grid or have a common edge. This correlates early in the perceptual processing system where the retinal ganglion cells detect edges (Cook, et al., 1998). This information is fed forward to the lateral geniculate nucleus where the M cells process and maintain this information in the retinotopic map for forwarding to the visual cortex (Covington, et al., 2022). Common region groupings, or groups of objects within a border, will also have their edges perceived by the RGCs and their mapping preserved in the LGN. The M cells in the LGN which detect and preserve edges, will maintain, and feed forward this information to the visual cortex (Covington, et al., 2022). With the similarity principle, items that share a common feature (e.g., size, color, or alignment) will be perceived as a group. Similar items will be further processed by the ventral stream which processes objects based on their similarities, or identities (Sheth, et al., 2016). A contour will be processed as an edge and further processed in the V1 (Wageman, et al., 2012).
HealthData.gov is a website maintained by the U.S. Department of Health and Human Services to provide families with up to date and reliable health information. The website provides health related reports, recent health news, and access to public data sets. Figure 1 shows a page on HealthData.gov with information pertinent to Covid-19. Within the nav bar are two groupings of items—the web navigation links on the left and the social media icons on the right—where each of these subgroupings will be detected as belonging together under both the powerful proximity principle, as well as the similarity principle. These nav items are also in accordance with the alignment principle as they share a common horizontal alignment left to right across the page. The items in the top nav bar (1 & 2) will be perceived as belonging together under the common border principle. Common borders and alignments have edges, and these are processed by the RGCs that mapped and fed forward to the M cells in the LGN (Cook, et al., 1998, Covington, et al., 2022), where they will be preserved. The K and P cells in the LGN specialize in colored items and can maintain portions of the nav bar, and nav bar items (Ghodrati, et al., 2017). The LGN cells will transmit the information to the visual cortex while maintaining retinotopic mapping. The neurons in V1 will maintain the mapping and process edges, columns, lines, and contrast (Huff, et al., 2022). This will go through the ventral pathway and dorsal pathways for more advanced processing. The ventral pathway is responsible for more complex patterns and alignment, while the dorsal is associated with spatial and objects (Rottschy, et al., 2007).
Figure 1. Results matching page on HealthData.gov
The search bar below the top blue navigation (3) is redundant as it is a duplicate of the search region in the top right corner. It also presents an overextended common region and uses what could be available white space; it will be processed as a common region, and due to its smooth lines and length also under the continuity principle (Wageman, et al., 2012). Each grouping in area 6 is perceived as having a common border, however, there are other borders inside of area 6 which are not needed and create additional noise. A better approach would be to use white space to segment the items inside of this area. After eliminating the borders inside of area 6, the items would still be perceived as being associated because they would share a common border and edge, hence, common region. The blue headlines of area 6 are already perceived as associated because of size and hue. The size similarity will map to the V1 tuning, and the blue color to the M LGN cells (Ghodrati, et al., 2017, Huff, et al., 2022). The horizontal line (4) is redundant and just adds noise. The border for the left navigation (5) may be eliminated as these items can be perceived and pre-attentively processed in proximity, similar, and aligned.
Conclusion
The Covid-19 reporting page on HealthData.gov shows information density where different clusters, structures, patterns, and organizations are communicating with each other. Along with the conformed variables where our neurological system is calling out the contour in areas (1, 2, 5, & 6) of good proximity, alignment, and similarity in size, and color, there are also concerning effects on visual salience in certain sections. The page has unnecessary nested common regions in the results area (6), “boxes within a box” issue. There is a redundant search bar (3) that overextends in length into the continuity principle, however, in this case is not useful. There is also a redundant horizontal line (4) that adds noise where it does not need to be. Eliminating all the adverse elements would improve the fatigue-causing aspects of the design and allow the usage of active white space for grouping or to distinguish one element from the other. The current design affects the overall performance and generates anxiety. The interfaces would benefit with a page redesign to work with the human pre-conscious visual systems. Including pre-attentive processing in interface design would help guide attention, block out noise and redundancy, to arrive at the least intrusive and most effective user experience.
References
Airbib, M. (1995). The handbook of brain theory and neural networks (2nd ed.). MIT Press. http://redwood.psych.cornell.edu/papers/field-1998.pdf
Cook, P. B. & McReynolds, J. S. (1998). Lateral inhibition in the inner retina is important for spatial tuning of ganglion cells. Nature Neuroscience 1(8), 714-9. doi: 10.1038/3714
Covington, B. P. & Khalili, Y. A. (2022). Neuroanatomy, nucleus lateral geniculate. StatPerls Publishing. https://www.ncbi.nlm.nih.gov/books/NBK541137
Huff, T., Mahabadi, N., & Tadi, P. (2022). Neuroanatomy, Visual Cortex. StatPearls Publishing. https://www.ncbi.nlm.nih.gov/books/NBK482504/
DeYoe, E. A., Carman, G. J., Bandettini, P., Glickman, S., Wieser, J., Cox, R., Miller, D., & Neitz, J. (1996). Mapping striate and extrastriate visual areas in human cerebral cortex. Proceedings of the National Academy of Sciences of the United States of America, 93(6), 2382-2386. https://doi.org/10.1073/pnas.93.6.2382
Ghodrati, M., Khaligh-Razavi, S.M., & Lehky, S. R. (2017). Towards building a more complex view of the lateral geniculate nucleus: Recent advances in understanding its role. Progress in Neurobiology, 156(Sept), 214-255. doi: 10.1016/j.pneurobio.2017.06.002
Hebart, M. N., & Hesselmann, G. (2012). What visual information is processed in the human dorsal stream? Journal of Neuroscience, 32(24), 8107-8109. https://doi.org/10.1523/JNEUROSCI.1462-12.2012
Hoffman, J.E. (1975). Hierarchical stages in the processing of visual information. Perception & Psychophysics, 18(Sep), 348-354. https://doi.org/10.3758/BF03211211
Hoffman, J.E. & Subramaniam, B. (1995). The role of visual attention in saccadic eye movements. Perception & Psychophysics, 57(Jan), 787-795. https://doi.org/10.3758/BF03206794
Karnath, H. (2015). Spatial attention systems in spatial neglect. Neuropsychologia, 75(Aug), 61-73. doi: 10.1016/j.neuropsychologia.2015.05.019.
Kerschensteiner, D. (2022). Feature Detection by Retinal Ganglion Cells. Annual review of vision science, 8(1), 135-169. https://doi.org/10.1146/annurev-vision-100419-112009
Kim, U. S., Mahroo, O. A., Mollon, J. D. & Yu-Wai-Man, P. (2021). Retinal ganglion cells—Diversity of cell types and clinical relevance. Frontiers in Neurology, Sec.Neuro-Ophthalmology, 12(May). doi: 10.3389/fneur.2021.661938
Kamkar, S., Moghaddam1, H. A., & Lashgari, R. (2018). Early visual processing of feature saliency tasks: A review of psychophysical experiments. Frontiers in Systems Neuroscience, 12(Oct). https://doi.org/10.3389/fnsys.2018.00054
Kristjánsson, Á. & Egeth, H. (2020). How feature integration theory integrated cognitive psychology, neurophysiology, and psychophysics. Attention, Perception, & Psychophysics, 82(1), 7-23. https://doi.org/10.3758/s13414-019-01803-7
Li, V. J., Chorghay, Z. & Ruthazer, E. S. (2023). A guide for the multiplexed: The development of visual feature maps in the brain. Neuroscience, 508(1), 62-75. https://doi.org/10.1016/j.neuroscience.2022.07.026
Palmer, S. & Rock, I. (1994) Rethinking perceptual organization: The role of uniform connectedness. Psychonomic Bulletin & Review, 1(1), 29–55. https://doi.org/10.3758/bf03200760
Poirier, F. J., Gosselin, F. & Arguin, M. (2008). Perceptive fields of saliency. Journal of Vision, 8(15), 1-19. https://doi.org/10.1167/8.15.14
Prinzmetal, W. (1981). Principles of feature integration in visual perception. Perception & Psychophysics, 30(4), 330-340. https://doi.org/10.3758/BF03206147
Purves D., Augustine G. J., & Fitzpatrick D. (2001). Neuroscience (2nd ed.). Sinauer Associates. https://www.ncbi.nlm.nih.gov/books/NBK10944/
Roska, B., Molnar, A., & Werblin, F. S. (2006). Parallel processing in retinal ganglion cells: How integration of space-time patterns of excitation and inhibition form the spiking output. Journal of Neurophysiology, 95(6), 3810-3822. https://doi.org/10.1152/jn.00113.2006
Rottschy, C., Eickhoff, S. B., Schleicher, A., Mohlberg, H., Kujovic, M., Zilles, K., & Amunts, K. (2007). Ventral visual cortex in humans: Cytoarchitectonic mapping of two extrastriate areas. Human Brain Mapping, 28(Jan), 1045-1059. https://doi.org/10.1002/hbm.20348
Sheth, B. R., & Young, R. (2016). Two visual pathways in primates based on sampling of space: Exploitation and exploration of visual information. Frontiers if Integrative Neuroscience. 11(Nov). doi: 10.3389/fnint.2016.00037
Schiller, P. (2010). Parallel information processing channels created in the retina. PNAS, 107(40), 17087-17094. https://doi.org/10.1073/pnas.1011782107
Wilkins, A. (2016). Intermittent illumination from visual display units and fluorescent lighting affects movements of the eyes across text. Human Factors: The Journal of the Human Factors and Ergonomics Society, 28(1), 75-81. doi: 10.1177/001872088602800108
Schneider, K. A., Richter, M. C., & Kastner, S. (2004). Retinotopic organization and functional subdivisions of the human lateral geniculate nucleus: A high-resolution functional magnetic resonance imaging study. The Journal of Neuroscience, 24(41), 8975-8985. https://doi.org/10.1523/JNEUROSCI.2413-04.2004
Schwartz, G., & Berry 2nd, M. J. (2008). Sophisticated temporal pattern recognition in retinal ganglion cells. American Physiological Society: Journal of Neurophysiology, 01(Apr), 1798-1798. https://doi.org/10.1152/jn.01025.2007
Töllner, T., Zehetleitner, M., Gramann, K., & Müller, H. J. (2011). Stimulus saliency modulates pre-attentive processing speed in human visual cortex. PLoS ONE, 6(1), e16276. https://doi.org/10.1371/journal.pone.0016276
Tootell, R. B. H., Hadjikhani, N. K., Vanduffel, W., Liu, A. K., Mendola, J. D., Sereno, M. I., & Dale, A. M. (1998). Functional analysis of primary visual cortex (V1) in humans. PNAS, 95(3), 811-817. https://doi.org/10.1073/pnas.95.3.811
Treisman, A.M. & Gelade, G.A. (1980). A feature-integration theory of attention. Cognition Psychology, 12(1), 97-136. doi: 10.1016/0010-0285(80)90005-5. PMID: 7351125
Treisman, A., Gormican, S. (1988). Feature analysis in early vision: Evidence from search asymmetries. Psychological Review, 95(1), 15-48. DOI: 10.1037/0033-295x.95.1.15
Treue, S. (2001). Neural correlates of attention in primate visual cortex. Trends in Neuroscience, 24(5), 295-300. https://doi.org/10.1016/S0166-2236(00)01814-2
Wagemans, J., Elder, J. H., Kubovy, M., Palmer, S. E., Peterson, M. A., Singh, M. & Von Der Heydt, R. (2012). A century of gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychology Bulletin. 138(6), 1172-1217. doi: 10.1037/a0029333.
