Eye-Tracking Applications in Architecture and Design

Definition

Eye-tracking is a biometrics technique that has started to find applications in research related to our interaction with the built environment. Depending on the focus of a given study, the collection of valence and arousal measurements can also be conducted to acquire emotional, cognitive, and behavioral insights and correlate them with eye-tracking data. These measurements can give architects and designers a basis for data-driven decision-making throughout the design process. In instances involving existing structures, biometric data can also be utilized for post-occupancy analysis. This entry will discuss eye-tracking and eye-tracking simulation in the context of our current understanding of the importance of our interaction with the built environment for both physical and mental well-being.

1. History of Eye Tracking

The first documented systematic interest in eye movements can be found in Aristotle [1], with the first experimental setup attributed to Prolemy, who had devised a board for examining the range of binocular single and double vision [2]. This topic was later examined in detail by Alhazen [3] and approached systematically in the modern era first by Wells [4]. During the 19th century, interest in the systematic study of eye movements increased [5,6], and in 1879 the French ophthalmologist Louis Émile Javal noticed that readers’ eyes do not scan a text with a constant speed while reading but make quick movements interrupted by short pauses instead—making the first description of the motion known as saccades [7]. In 1901, Dodge and Cline introduced a non-invasive eye tracking technique using light reflected from the cornea, recording eye position onto a photographic plate [8]. However, it only recorded eye positions on the horizontal plane and required the participant’s head to be motionless.

A few years later, motion picture photography was first used to record eye movements in two dimensions [9], utilizing a small speck of white material that had been placed on the participants’ eyes. A more intrusive approach was that of Edmund Huey in 1908, utilizing an apparatus that could be used to track eye movement during reading. Subjects had to wear a type of contact lens that had a small opening in front of the pupil and was connected to a pointer that changed its position in response to the movements of the eye [10].

The first dedicated eye-tracking laboratory was founded in 1929 by Edmund T. Rolls at the University of Cambridge, marking the recognition of eye tracking as a field of study. A number of additional advances in eye tracking systems were made during the first half of the twentieth century by combining corneal reflection and motion picture techniques (see [11] for a review). In one of them, in the late 1940s, motion picture cameras were used to study the movements of pilots’ eyes as they used cockpit controls and instruments to land an airplane, in what was the earliest application of eye tracking to usability engineering, i.e., the systematic study of users interacting with products, aimed at improving product design [12]. In the early 1950s, the development of the electrooculogram (EOG) marked a new chapter in the field of eye tracking, providing a more accurate method for tracking eye movements than earlier techniques. The EOG measures the cornea-positive standing potential relative to the back of the eye. It uses skin electrodes attached outside the eye near the lateral and medial canthus, allowing the potential to be measured when participants move their eyes a set distance in the horizontal plane [13]. In the 1950s and 1960s, Alfred Lukyanovich Yarbus pioneered the study of saccades when viewing complex images by recording the eye movements performed by observers presented with natural objects and scenes [14]. An important finding was that gaze fixations can be influenced by the instructions given to an observer, demonstrating that low-level, stimulus-driven guidance of attention can be overridden by high-level factors [15].

The pupil center corneal reflection technique (PCCR) was developed in the 1970s and became a standard method for tracking eye movements because of its accuracy and ease of application [6]. It involves using specialized glasses that shine near-infrared light directed at the pupil, creating reflections that can be detected in both the pupil and the cornea. An infrared camera incorporated in the glasses tracks the vector between the cornea and pupil, determining gaze direction. This technology allows researchers to observe the participants’ natural gaze and attention in various environments. Other, less frequently used methods have also been developed [5].

The first eye-tracking study of images, including images of architecture, was performed as early as 1935 [16], with the first study dedicated to architecture appearing in the 1970s, followed by a handful of studies in the following years (see [17]). The use of eye tracking as a tool for investigating our interactions with the built environment has grown very rapidly in recent years, and the following paragraphs discuss some of the relevant studies, especially in the context of evidence-based design for promoting psychosomatic health.

2. Eye Tracking and AI-Simulated Eye-Tracking: Applications and Findings in Architecture and Design

2.1. What Determines First Fixations

Initial gaze fixations are guided by pre-attentive processes in the brain. Our visual system has developed mechanisms to prioritize pertinent/salient information for determining suitable actions and regulating their execution. This selection process initiates as soon as an image reaches the retina, with low-level visual feature computation commencing at this stage and persisting in the lateral geniculate nucleus of the thalamus and early visual cortical areas. There are neurons in the early stages of perception that are specialized to detect specific fundamental visual attributes, such as differences in brightness and contrasting colors. As processing continues downstream through the thalamus and into the visual cortex, characteristics like orientation, direction, and speed of movement are detected (as reviewed by [18]). These features of the visual scene are computed in parallel, creating an early “saliency map” [19]. This first, pre-attentive processing, lasting about 200–250 milliseconds, creates information that guides the early deployment of selective attention and, potentially, action. Approximately 10% of the retinal output follows an alternative pathway directed towards the phylogenetically older system involving the superior colliculus as well as the pulvinar nucleus of the thalamus. This alternative route is responsible for the early response to motion detected in the peripheral visual field [20], as well as fear responses triggered directly from the pulvinar to the amygdala, such as when we perceive threats from potential predators [21]. It is interesting to note that primates seem to have a dedicated circuit for snake recognition in the pulvinar [22], a heritage of the early struggle of humans and their ancestors with serpents.

2.2. Importance

As our survival depends on our ability to quickly and accurately interpret environmental cues, any difficulties encountered in this early processing can trigger feelings of danger and stress. In other words, environments that pose challenges in their visual processing can induce anxiety, as they lack a feeling of place or “anchoring”.

So, the cohesion of visual elements plays a pivotal role in effective information processing, but whether a stimulus elicits a response of fear or attraction depends on the nature of that stimulus. For example, a building with a coherent design captures visual attention, but so does a predator, whose form also possesses geometric coherence. The form has to be quickly processed and assessed, and delays in this initial information processing can also delay the second evaluation stage, potentially causing stress and a sense of unease. In other words, coherence in the environment may prompt anxiety because it complicates the assessment of whether the surroundings are benign or threatening.

Through his pioneering work, Christopher Alexander [23] identified several parameters that create a connection between viewers and their surroundings. A number of studies have since demonstrated that exposure to certain fractal visual patterns, both in nature and in architecture, can have measurable physiological effects [24,25]. Of particular interest is the finding of the activation of the default mode network of the brain during the perception of fractals [26,27], which, together with findings on the role of fractals in stress reduction [24,25], data related to specific electroencephalogram responses to fractal patterns [28], and our own data from eye-tracking and eye-tracking simulation studies [29,30,31], supports the notion of their privileged status in terms of perceptual fluency, a key determinant of initial information processing speed.

While the benefits of exposure to natural scenes have been documented extensively, similar effects can result from artificial environments that mimic the geometrical qualities of nature [32,33]. Importantly, these qualities are not limited to fractal properties, as the pioneering work of Nikos Salingaros has shown [34,35]. Our processing system is finely tuned to the geometry of the natural environment, responding positively to specific types of organized complexity. The presence of fractals is an important aspect of this phenomenon, but it does not describe the entire range of “connecting” qualities [36]. The experience of beauty, often overlooked in modern architecture, results from a multiparametric evaluation of a visual scene, and our current understanding of these findings has started creating a new appreciation of the affective qualities of our built environment [37]. Studies have associated stress with environments that lack a specific level of organized complexity [38], an effect that has parallels with the results of sensory input deprivation [39]. There are also visual patterns that can cause discomfort directly, through mechanistic processes [40].

Eye tracking alone does not provide direct information about anything beyond visual attraction, but measurements of valence and arousal can also be collected to assess emotional, cognitive, and behavioral information [41], as discussed below.

2.3. Relevance for Architecture and Design

In recent years, numerous studies and experiments within the architecture, engineering, and construction sectors have used eye tracking for design evaluation [31,42,43,44,45]. Various companies (for example [46,47,48,49]) offer research-grade glasses for collecting real-world eye-tracking data, including gaze, fixation, and saccade information. To comprehend participant behavior in authentic/natural settings, it is essential to move beyond the lab and bring research tools directly to those settings, and portable solutions are now offered. A good compromise that is also offered is eye-tracking at home, where a webcam can be utilized, recording eye movements in front of a calibrated monitor.

In addition to actual eye-tracking using volunteers, there are now commercial artificial intelligence (AI) applications that can predict initial viewer responses to images (for example, Visual Attention Software (VAS) from 3M [50], Eyequant [51], Attention Insight [52], Neurons [53], and Expoze (version 1.01) [54]). These companies leverage artificial intelligence applications developed from extensive eye-tracking experimental data. Such applications generate maps displaying the likelihood of fixation points and estimating the temporal sequence of these fixation occurrences. This approach effectively unveils the subconscious processing of visual stimuli with remarkable accuracy. Initially designed for applications such as product design, advertising, and signage, this software has now started being used also in assessing architectural and environmental design [29,30,55]. This technology is highly suitable for performing direct comparative evaluations of different structures, facilitating the assessment of both quantitative and semi-quantitative parameters.

Various researchers, including the author of this article, have carried out eye-tracking investigations using images of architecture and constructed environments. These studies have utilized both volunteers to participate in actual eye-tracking experiments [31,42,56,57,58] as well as 3M’s VAS [29,30,50,59,60,61]. The analysis of gaze sequence and gaze can be performed in either whole images or in pre-defined areas of interest. The capacity to predict a user’s engagement with a building’s design, particularly as experienced through its facade, is important both as a study subject and because of potential practical applications.

One notable insight coming from such studies is that initial fixation points, guided by pre-attentive processing, tend to center around the presence of humans, particularly their facial features, even within depictions of architectural or urban settings. Moreover, the gaze naturally shifts towards specific elements such as contrasts, details, and structural components that play a role in establishing a comprehensive sense of geometrical coherence [29,30,31,59,60]. Certain structures instantly engage the viewers’ interest because of their cohesive design, while others actively discourage attention, diverting viewers’ gazes away from the building’s exterior. These observations indicate a strong connection between the visual processes at play and the mathematical coherence or structured complexity inherent in the design. The differences between designs that result in a fragmented gaze heatmap that seems to disintegrate upon zooming in to the image and designs that possess a coherence that is detectable in the gaze heatmap, both initially and at subsequent zooming-in levels, demonstrate heatmap coverage that can be scaled through iterative zoom levels as more detail is being “discovered” on each of them [29,30,59] (Figure 1).

Similar findings have come from a recent eye-tracking study with volunteers [31]. In one of the categories examined in this study, modern vs. traditional, a clear bias for traditional architecture over modern buildings was found. The geometric organization of traditional facades commanded the first fixation in most image pairs. The data also suggested that complexity informs the brain of stimuli worth examining more closely, hence producing significantly longer dwell times. As images in this category came from the Harris Poll of 2020 [62], where the conscious preference of Americans for traditional buildings was recorded, these results demonstrate the convergence of questionnaire and biometric data, acquired independently by different researchers in different population samples (Figure 2, more discussion on this in the next section). The same study also examined other morphological features that are thought to be inherently attractive to the gaze, such as face-like geometry. We know that infants are immediately attracted to faces and that, as early as a week after birth, they tend to look longer at faces that adults consider attractive [63], a phenomenon that has been shown to generalize across race, sex, and age by the 6th month [64]: eye-tracking results from this study showed an attraction to even the most rudimentary face-like pattern (Figure 3).

These findings carry significant implications for our perception of architecture. Contemporary buildings, particularly those featuring plain glass exteriors, often receive fleeting glances without distinct fixation points on the building itself. This stands in stark contrast to traditional structures, where the presence of organized complexity incorporating nested hierarchies appears to direct attention to the entirety of the construction. The role of pre-attentive processing becomes evident in attracting individuals to certain structures while overlooking others. For example, individuals facing old buildings can swiftly identify an entrance. To foster inviting spaces, buildings should integrate elements such as fractal scaling, organized complexity, and repeating symmetries. These features guide viewers in the right direction, fostering a sense of safety, especially in outdoor environments. Eye-tracking can be a practical guide in future building design, identifying all these issues at an early design stage.

2.4. Eye-Tracking and Stimulus Valence

The intensity and, even more so, the valence of the emotions that will follow when the unconscious gaze attraction is followed by conscious perception of a scene cannot be assessed by eye-tracking alone. Combining eye-tracking with data from questionnaires, as in [31], is a way to fill this gap. Another way is the combination of eye-tracking with other biometric techniques. One such approach is the use of electrodermal activity, also called galvanic skin response (GSR) [65]. GSR measures the skin’s conductivity and, through it, can detect changes in sympathetic nervous system activity that reflect emotional states [66]. Stimuli that evoke a sense of danger, fear, happiness, or other emotional states can result in elevation in sweat gland activity, which can be measured by GSR. Hence, GSR can provide a measure of emotional arousal, although obviously it cannot identify the valence of the emotion being experienced but only its impact on the periphery [67]. For a more complete understanding of the individual’s reaction to the environment, combinations with other techniques can be employed. For example, in a study where eye tracking and facial electromyography (fEMG) were used to collect data on the participants’ visual attention and facial expressions in response to virtual reality environments (VE) [68].

FEMG is a psychophysiological method used to detect electrical potentials generated by facial muscles [69]. This potential is in the range of microvolts and is related to the amount of muscle contraction as well as to the number of contracted muscles in a linear fashion. This method allows for a better understanding not only of muscle movements and activity themselves but, importantly, in the present context, also of their association with specific emotions and behaviors with which facial expressions correlate [70,71]. For example, parameters like the level of luminance in rooms, the presence or absence of natural lighting, wall color, and openness of spaces, as well as the presence or absence of outside landmarks and of a visible entrance, were shown to change the way people perceive a space, as assessed by the reflection of this perception on the parameters studied [68]. GSR sensor data showed that skin conductance levels were higher in a negative environment, reflecting increased stress compared to the positive environment. Moreover, heart rate variability indicated a greater emotional response in the negative space compared to baseline values and those obtained in the positive space. FEMG software can often be integrated with other data analysis tools and software platforms, including electroencephalography and eye tracking. FEMG devices can be wearable, compact, and lightweight, so they can be used outside a laboratory setting, and have already been implemented, in combination with other techniques, in neuroarchitectural investigations [72,73].

In the 1970s, Ekman and Friesen [74,75] developed a technique for quantifying facial expressions, the facial action coding system (FACS). This is a comprehensive system to distinguish all visible, anatomically-based facial movements, forming a basis for correlating muscle actions during the expression of basic emotions with those emotions. FACS dissects all observable facial movements into 44 distinct muscle action units [76] and can be used both for real-time observations and for video-recorded interactions conducted in laboratory conditions, in conjunction with automated facial coding (AFC) [77] software, which can categorize facial movements into emotional or cognitive states [78]. AFC enables the analysis of facial expressions in various contexts, where individuals are observed in their natural environment without the constraints of technical equipment, even using the participants’ computers at home, the same way as eye-tracking at home is performed through calibrated webcams. Data for both eye-tracking and facial analysis can be collected at the same time and analyzed either separately or together, with cross-correlation. The company iMotions offers such a solution, with its facial expression analysis software module integrating the facial coding engines AFFDEX by Affectiva and Realeyes. Using a webcam, one can synchronize expressed facial emotions with stimuli directly in the iMotions software, simply import videos, and carry out the relevant analysis [48].

Tracking facial expressions can be a powerful indicator of emotional experiences and complement eye-tracking, providing insights into valence and, as part of a synthesis of multiple data streams, contributing to a better understanding of our interactions with the environment.

2.5. Using Virtual Reality Environments

VEs are useful for neuroscientists and psychologists as research tools, as well as for architects, designers, and stakeholders, in applications ranging from the early design phases to the real estate sector. VEs offer the advantage of facilitating experimental research in a laboratory under controlled conditions without the compromises involved in using 2-dimensional images. VE solutions have already been used in many studies in architecture and interior and urban design, for example in manipulating ceiling height, colors, wall curvature, and surface textures [79,80,81,82,83,84] and adding virtual plants [85]. VE studies can also be combined with sensors monitoring correlates of emotional responses, such as heart rate and skin conductance [68,86,87,88]. However, the issue of the incompleteness inherent in virtual experiences presents a potential drawback [89,90]. For example, the lack of gravitational and accelerational sensations when navigating virtual spaces hinders multisensory integration. These factors make the VE experience palpably different from reality and need to be taken into account when designing experiments.

3. Conclusions

Eye-tracking, either on its own or in combination with other biometric measurements and questionnaire responses, can provide architects and designers with a basis for data-driven decision-making throughout the design process. In cases involving existing structures, biometric data can also be utilized for post-occupancy analysis.

The information obtained from these studies places the concept of biometrics in architecture and design in a more practical and also human-centered perspective, surprisingly, perhaps, to the naïve observer. By systematically exploring the workings of our subjective experiences, we can establish more objective assessments of architectural forms. This stands in contrast to the ad hoc concepts that architects and designers have commonly employed over the past century.