Review of Studies on User Research Based on EEG and Eye Tracking

Abstract

Under the development of interdisciplinary fusion, user research has been greatly influenced by technology-driven neuroscience and sensory science, in terms of thinking and methodology. The use of technical methods, such as EEG and eye-tracking, has gradually become a research trend and hotspot in this field, in order to explore the deep cognitive states behind users’ objective behaviors. This review outlines the applications of EEG and eye-tracking technology in the field of user research, with the aim of promoting future research and proposing reliable reference indicators and a research scope. It provides important reference information for other researchers in the field. The article summarizes the key reference indicators and research paradigms of EEG and eye-tracking in current user research, focusing on the user research situation in industrial products, digital interfaces and spatial environments. The limitations and research trends in current technological applications are also discussed. The feasibility of experimental equipment in outdoor environments, the long preparation time of EEG experimental equipment, and the accuracy error of physiological signal acquisition are currently existing problems. In the future, research on multi-sensory and behavioral interactions and universal studies of multiple technology fusions will be the next stage of research topics. The measurement of different user differentiation needs can be explored by integrating various physiological measurements such as EEG signals and eye-tracking signals, skin electrical signals, respiration, and heart rate.

1. Introduction

Using verbal feedback for user research is plagued by a degree of vagueness, and the results can easily be influenced by various factors. Although modern technology has yielded many efficient research methods and data analysis tools, achieving an objective assessment of user experience is still a challenge, and is one of the current hotspots of research [1]. With the rise of neuroscience and its research methods, electroencephalogram (EEG) research is increasingly being used to understand users’ true thoughts. Combining eye tracking (ET) and EEG research is a current and future trend in the field of user research. With the help of joint research between the two, we can obtain more accurate, in-depth, and comprehensive user information compared to subjective reports, making it possible to objectively understand what users see and think.

In studies related to neuroscience, the characteristic of EEG data is high temporal resolution (milliseconds) compared to other brain imaging techniques such as fMRI [2]. This feature is often used to classify techniques, and is crucial for user research because it allows for the identification of neurophysiological correlations exposed to stimuli within a functional time window. Moreover, standard EEG equipment is non-invasive, and allows participants to operate normally in the laboratory or at the test site. In Krigolson’s [3] research, computer and portable EEG systems can easily conduct ERP research. Cano et al. [4] studied human–computer interaction in gaming activities based on low-cost EEG signal technology, greatly expanding the possible use of ERP methods in various new environments. Furthermore, the cost of electroencephalography is much lower than other brain imaging techniques, and is one of the most readable and promising neuroscience tools in user research [5].

Eye trackers have become popular as user research tools for recording user gaze paths. Eye-tracking research records the movement trajectory of the eyes during specific tasks, in order to analyze the process of user cognitive behavior and psychological activity. Through eye-tracking data, we can analyze users’ gaze patterns to evaluate and optimize product designs [6]. Khalighy et al. [7] quantified the visual aesthetic quality of product design by applying eye-tracking technology. Dogan et al. [8] collected gaze data on yachts using eye-tracking devices, and then evaluated the attractiveness of design parameters using eye-tracking tools such as areas of interest (AOI), scan paths, and heat maps. Ilhan et al. [9] used eye-tracking technology to explore the relationship between product cognition and design decisions. Eye-tracking research is based on users’ actual operations, and can objectively analyze products that are used in daily life to derive reasonable rules or conclusions.

However, the various indicators of eye-tracking data themselves reflect behavioral results and cannot directly reflect cognitive and thinking processes. Behavior is the comprehensive result of various brain activities, so the referential significance of the data itself is not singular, and is difficult to explain. Eye-tracking research requires good experimental design to correctly interpret eye-tracking data, or it needs to be combined with interviews or retrospective tests to reflect cognitive and thinking processes. Kulke et al.’s [10] research showed that the combination of EEG and eye-tracking can be successfully used to study natural and salient attention shifts. Lopez-Gil’s [11] research showed that using EEG activity alone as a predictive indicator for self-regulation cannot correctly determine emotional responses to affective stimuli. However, combining different data sources with eye-tracking synchronization can overcome the limitations of single detection methods, providing added value for many different research fields. Jia et al.’s [12] research showed that EOG signals recorded through EEG networks can obtain results as accurate as typical optical eye-tracking devices, and can simultaneously evaluate neural activity during all types of eye movements. The fusion research of electroencephalography (EEG) and eye-tracking research has to some extent made up for this deficiency. This method can not only correctly interpret eye-tracking data, but also objectively and accurately reflect users’ psychological processes.

Eye-tracking (ET) and electroencephalography (EEG) are widely used in user research, and a lot of research has been conducted on product aesthetics preferences, web browsing, and interest orientation. Li et al. [13] used EEG and eye-tracking devices to record subjects’ brain activity and eye gaze data, which were quantified using neural attributes to measure the factors that affect product appearance and evaluation indicators. Guo et al. [14] distinguished and quantified the visual aesthetics of products by integrating eye-tracking indicators and EEG measurements. Slanzi et al. [5] evaluated the click intentions of network users by merging pupil dilation and EEG responses based on physiological analysis. Rodriguez et al. [15] systematically explained the brand and packaging of products from a neuroscientific perspective. We can interpret the brain and psychological activities of users when they browse specific advertisement information on a page, including attention level, memory, and emotional response. The combination of these tools and methods combines the intuitiveness of eye-tracking data, with the depth, accuracy, and comprehensiveness of EEG data, to provide almost all of the real idea information that users use when using products and demonstrating behaviors [16]. This approach can accurately locate the brain activity of users when observing and using experiences, and explore users’ attitudes, emotions, and cognitive levels deep in their hearts [17]. These methods provide us with new research ideas and directions, and can provide us with more information to understand users’ real ideas and product optimization suggestions.

By reviewing the current literature and research, our goal is to help researchers understand the research metrics and paradigms of EEG and eye-tracking technologies, as well as the current state and limitations of their applications in various research domains; this provides valuable guidance and support to advance the field of user research. Eye-tracking signals allow us to precisely locate what attracts users’ attention and observe their subconscious behavior; neuroscience explains the cognitive, emotional, and perceptual structures that support human decision making, and evaluates users’ psychological states, thus providing a more rigorous analysis process for some research questions. We provide typical indicators and theoretical knowledge of EEG and eye-tracking, as well as user research of these tools and methods in industrial products, digital interfaces, public guidance systems, transportation systems, and other spaces. Finally, the discussion section analyzes the shortcomings of this technology in current research, and provides our suggestions for future research.

2. Method

This study constituted a mixed-method review, employing a narrative-oriented approach to investigate the research topic. It employed a systematic retrieval methodology to meticulously gather data from credible and reliable sources. The extent of a systematic review depends largely on the scope and quality of the research institute. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines are a set of standard specifications established for the quality of systematic reviews [18]. It is applicable to reviews of published literature containing original data materials, and aims to improve the scientific rigor and comparability of systematic reviews. In the process of reviewing articles, it can help the review team to report clearly and transparently, and provide sufficient details [19]. We used the PRISMA 2020 flowchart for systematic review, which includes searches from the Web of Science (WoS) and China National Knowledge Infrastructure (CNKI) sources. As shown in Figure 1, We searched for terms such as “EEG”, “eye-tracking”, “ET”, “neuroscience”, “user research”, “product design”, “interface research”, and “spatial environment” as keywords, including studies that contribute to the development of this field. We expanded our search to ensure the reliability of articles through title and abstract, data identification, preliminary screening, and qualification determination. Finally, 253 articles were identified, and the main results were summarized and classified according to the research content. Among them, 115 articles were applied in this paper.

On the basis of the review, we used the bibliometric software Citespace 6.2.R3 to conduct the research analysis of keywords for the 253 screened articles. Citespace has the properties and characteristics of “graph” and “spectrum” in its literature measurement research, and is mainly used to visualize the knowledge landscape of a research field [20]. We used it to obtain the scope of application of EEG and eye-tracking technology in user research, and performed a visual analysis. During the review process, we found that articles that used EEG or eye-tracking technology with user research as the key topic were very limited. Therefore, we carefully studied the content of these articles and selected those that involved user research. Based on these literature contents, this review was refined and summarized, focusing on the main research indicators and their significance of EEG and eye-tracking technology in user research, as well as a series of applications of user research in industrial products, digital interfaces, and different spatial environments; the details of the entire research concept framework are shown in Figure 2. Finally, based on these summary conclusions, we proposed suggestions and directions for future research on the shortcomings of current EEG and eye-tracking research methods.

3. Indicators of User Research

Specific indicators of eye-tracking and EEG are of critical importance for understanding user behavior. Through complex technical processing and EEG data analysis tools such as EEGlab, Fourier transform, or wavelet analysis, we can organize complex brain waveforms into a starting point, in order extract waveforms within a specific time period to analyze brain activity during that period. Eye-tracking research can provide a complete set of interrelated eye-tracking indicators, fully recording users’ gaze trajectories, which can basically solve the problem of how users pay attention, and what they pay attention to.

3.1. Classification and Characteristics of EEG Signals

3.1.1. The Reference Value of EEG Research

An EEG is a spontaneous brain electrical signal measured by electrophysiological methods. It exists in the central nervous system, and its value lies in its ability to directly reflect the activity of neurons. The frequency domain characteristics of EEGs are very obvious, and according to different frequency bands, they can be divided into δ (1~3 Hz), θ (4~7 Hz), α (8~12 Hz), β (15~20 Hz), and γ (40 Hz or more) waves, five kinds of rhythm waves that correspond to a sleep state, fatigue state, awake and relaxed state, emotional fluctuation state, and an excited state, respectively [21]. As shown in Figure 3, different waveforms represent different psychological information. α waves are the main manifestation of cortical electrical activity when the cerebral cortex is awake, quiet and closed eyes. When people open their eyes, think about problems, or receive other stimuli, α waves disappear immediately. β waves are manifestations of cortical excitation. This wave only appears in the frontal lobe, when people are quiet with their eyes closed. If they open their eyes to see things, think about problems, or suddenly hear sounds, β waves will be triggered in other areas of the cortex. θ waves appear when subjects are tired, disappointed, or frustrated with emotional stimuli [22]. These different waveforms reflect the psychological state of the subjects. Different brain waves correspond to different frequencies, representing corresponding mental states.

In actual research, it has been found that the strength of theta waves can reflect the activity of cognitive control mechanisms [23], and the power of alpha waves in the frontal area can serve as an indicator of pleasure [24]. Rhythmic waves in various frequency bands are of great reference value for cognitive research. In research, we need to not only pay attention to the overall activity status of the brain, but also care about specific psychological components related to it, such as the attention level, memory, and emotional response related to tasks. As shown in Figure 4, the specific meanings of user research-related indicators are as follows:

(1)

Emotional indicators: The left and right frontal lobes of the brain are sensitive to positive and negative emotions, respectively. By analyzing their lateralization, we can understand the strength changes of users’ positive and negative emotions [25].

(2)

Attention indicators: The left and right parietal lobes of the brain control attention allocation, and can understand users’ attention allocation.

(3)

Memory indicators: The temporal lobe controls our memory, and can encode and recall environmental information. Through analysis of the temporal lobe, we can understand users’ memories of products [16].

3.1.2. Classic Brain EEG ERP Components and Their Meanings

The ERP, as a current focus of extensive research, is an electroencephalogram (EEG) caused by psychological activity. It reflects the physiological changes in the brain’s nerves during the cognitive process and can non-invasively, with high temporal resolution, reflect the process of brain cognition. It is an important means of observing and interpreting people’s inner thoughts and behavioral expressions. An ERP generally only has 2 to 10 microvolts, so it is necessary to filter, segment, superimpose, and extract EEG experimental data to obtain corresponding components for further research. Currently, the narrow definition of an ERP refers to classic ERP components, including P1, P2, P3 (P300), N1, and N2; meanwhile, the broad definition of an ERP includes not only narrow content, but also components such as N400 and accompanying negative reactions [26]. As shown in Figure 5, P and N represent the amplitude of positive and negative trends, respectively, and the numbers represent the time position where the amplitude appears. Among them, P1, N1, and P2 are also called the exogenous components or physiological components of an ERP. They are the most primitive components produced by the human brain when stimuli occur, and are mainly affected by the intensity and frequency of physical stimuli; N2 and P3 are the endogenous components or psychological components of an ERP. They are not affected by physical stimuli, but depend on people’s memory, perception, intelligence, and other activities. They are some of the stimulus-induced activities produced by the human brain during internal activities [27].

ERP components not only reflect physiological changes in the brain but, more importantly, can reflect psychological changes and cognitive processes at certain levels, which is of great significance for explaining user behaviors and their psychological states [28]. For example, P300 is related to the last stage of information processing and the measurement of “cognitive efficiency”, which can reflect the subject’s mental load and attention level. The amplitude of the wave generated is positively correlated with the mental state used. It also has a latency period. The more difficult the task during the latency period, the longer the waveform will be. It can be used as a marker for attention and working memory; N1 and P1 are considered to be closely related to early visual cognition, including visual attention and spatial location information capture. They can be used as objective indicators to measure the degree of visual stimulation. Negative emotional stimuli will cause larger wave amplitudes [29]; P2 and N2 reflect attention allocation to novel and potentially significant stimuli. They can be used as objective indicators to measure the intensity of emotional valence. The intensity of negative emotions will make changes in the brain more sensitive; N400 is related to semantic understanding, and can reflect the time process of semantic construction [30]. These findings have an important reference value for exploring users’ real thoughts, emotions, and evaluations in cognitive aspects.

3.2. Eye-Tracking Index

3.2.1. Physiological Indicators and Visualization

(1)

Pupil and Blink

The pupil is one of the commonly used indicators in eye tracking. Changes in the pupil include pupil dilation and pupil constriction, that is, the increase and decrease of pupil size, respectively. In experiments, the change in pupil size is generally affected by stimulus factors, thus reflecting the individual’s cognitive state. Among them, two of the more common cognitive states are emotional arousal and cognitive workload, which reflect the degree of emotional involvement of the subject when responding to stimuli and the degree of fatigue, respectively [31]. Therefore, changes in the pupil are closely related to changes in emotion, and are often used as a measurement tool for emotion induction. However, in the actual experimental process, the pupil response alone cannot directly judge the change dimension of positive or negative emotion; the level of facial expression and other physiological indicators must also be comprehensively judged.

In eye-tracking indicators, the situation of blinking can also explain information that is related to cognitive workload. Among them, attentional blinking is a common indicator in blink analysis, which means blink delay, and is closely related to the task requiring cognition [32]. In addition, the blink frequency is also related to attention. Low-frequency blinking generally means a high concentration of attention, while high-frequency blinking means the attention is dispersed.

(2)

Trajectory map and heat map

A trajectory map is a visual form of eye tracking. Its implementation principle is to generate a gaze sequence by capturing the position and duration of eye fixation, thereby reflecting the gaze position and corresponding time information from the beginning to the end of an observation. The significance of the trajectory map is that it can reflect elements with strong visual attraction in the environment, and clearly indicate the trajectory of the visual search in order [33].

The implementation principle of the heat map is to divide the scene range into uniformly distributed grids, convert the length of eye fixation time on each grid into a numerical value, and assign a color to each numerical value. The larger the numerical value, the more yellowish, reddish, and other warm tones it tends to be; the smaller the numerical value, the more greenish, bluish, and other cool tones it tends to be. Its significance lies in that it can reflect the degree of attraction to a certain stimulus presented in images, videos, objects, or environments relative to other stimuli for subjects during the dynamic gaze process, thus clarifying the degree of different stimulus processing, and providing very rich information for analyzing the subjects’ thinking.

(3)

AOI and sequence analysis

AOI statistics as a functional indicator in eye tracking systems is achieved by selecting target areas and extracting some eye tracking indicators for statistical analysis [34]. It can be used to evaluate the performance of multiple areas in the same scene by selecting one or more specific areas in the stimulus as research objects, and separately calculating and obtaining visual processing information indicators such as gaze time and frequency for each individual in each area, thus performing AOI data statistics and visualization analysis.

AOI sequence analysis mainly studies the order information of eye attention to AOI when subjects interact with stimuli (such as web pages, human-computer interaction interfaces, etc.). When studying the observation habits of consumers browsing waterfall flow websites, the page is divided into four AOIs: the image display area, text area, view details, and comments. The corresponding AOI sequence analysis can statistically analyze the data of multiple subjects’ attention order and visualize it. For example, for the order of gazing at the image display area, AOI-gazing at the text area, and AOI-viewing details, AOI has important reference value for studying user behavior.

3.2.2. Data Indicators and Their Significance

(1)

First Fixation Time (TTFF)

The TTFF represents the time required to view a specific AOI from the start of the stimulus, reflecting the attractiveness and salience of spatial elements. The TTFF can indicate stimulus-driven search with bottom-up cognitive inertia and attention-driven search with top-down information processing [35]. Although the TTFF is a basic indicator of eye tracking, it is very valuable because it can provide information about how certain aspects of the visual scene are prioritized.

(2)

First Fixation Duration (FFD)

When exploring a visual scene with our eyes, we locate a part of the image through fixation. The first fixation duration reflects the duration (seconds) of the first time that an area of interest (AOI) is fixated, and reflects the distinguishability of spatial elements in the scene [36]. When this indicator is used together with the TTFF, it enhances the value of information conveyed as an indicator that first attracts attention in a scene. If the subject’s TTFF is short and the first fixation duration is long, then that area is likely to be very eye-catching.

(3)

Average fixation duration

The average fixation duration can tell us how long an area is viewed on average. If one image leads to a much longer average fixation duration than another image, there may be something worth exploring there. In addition, by comparing AOIs, the size of this metric can determine which areas are actually more important than other areas [37].

(4)

Dwell Time

The dwell time refers to the total duration of attention to areas of interest (AOI) by subjects, which reflects the attractiveness of elements to viewers; that is, it is the total time spent focusing on an element once it is noticed.

(5)

Regression count

The regression count reflects the participant’s reprocessing of previous information. It provides information about how many times the participant returned their gaze to a specific target in a specific area of interest, which allows researchers to examine which areas repeatedly attract participants (regardless of whether they are good or bad), and which areas are seen and then moved [38]. Although eye tracking cannot indicate users’ psychological feelings when looking at something, it can provide detailed processing data for research to analyze the attractiveness of things.

3.3. Experimental Paradigm

In user research, eye tracking and electroencephalography are used to observe changes in users’ physiological data in product use or objective environments. The combination of dynamic and static images and feelings is used as the observation object to explore the top-down and bottom-up processing methods in cognitive processing, in order to establish computational models or make quantitative predictions of visual processing. Classic paradigms of eye tracking experiments involving such scenarios include eye tracking paradigms, background cue paradigms, and object perception paradigms. Among them, eye tracking paradigms generally present scene stimuli to subjects, and require subjects to freely view the scene. During the subject’s viewing process, their eye tracking information is recorded and then analyzed. The background cue paradigm is mainly based on visual search research, and aims to explore the effect of the scene background on the target search [39]. The object perception paradigm is illustrated in Figure 6. First, the name of the target object is presented to the subject, then a fixation point is presented, followed by scene stimuli; then, a mask composed of meaningless lines with a circle indicating the location where the target object appears on it is presented; and finally, the subject makes a judgment and records their reaction time [40]. Target location cues can appear before or after the scene. If the location cue appears before the scene presentation, the reaction time recording begins after the scene presentation ends.

The “Go/Nogo experiment paradigm”, “Oddball paradigm”, “Flanker paradigm”, “Stroop paradigm” and their related variations are commonly used in ERP experiments. In the Go/Nogo paradigm, LRP (lateralized readiness potential) and N200 components are mainly focused on to explore the rules of cognitive processing. The Oddball paradigm is widely used, and is an experimental paradigm that produces ERP components such as P300 and MMN related to differences in stimulus probability; they have produced many subtypes [41]. Flanker is a research paradigm used to measure target stimuli and interference stimuli, while the Stroop paradigm is used to study the interference effect of emotional stimuli on cognitive processing.

The current combination of EEG and eye-tracking research is mainly based on the Oddball paradigm. The Oddball paradigm refers to the continuous alternation of two or more visual or auditory stimuli, such as using the process of “experimental instruction language—500 ms target picture/irrelevant picture—800~1200 ms random blank screen” to study the subject’s brain activity based on differences in stimulus frequency. This experimental process has similarities and complementarities with EEG and eye-tracking experiments, in terms of experimental objects and processes [14]. Integrating multimodal physiological signals related to visual feature processes can obtain more convincing research data, which are very suitable for the quantitative analysis of combined EEG and eye-tracking methods in user research.

3.4. Preprocessing Methods and Device Connections

Before analyzing EEG data, data preprocessing is required to ensure the accuracy and reliability of the analysis results. This mainly includes noise removal and artifact removal. Noise removal methods include baseline correction, high-pass filtering, low-pass filtering, and denoising filtering, which can solve the problem of data interference caused by noise such as eye movement, muscle activity, and power interference [42]. Due to the potential differences between electrodes, some artifacts may be generated, such as eye movement artifacts and ECG artifacts. Artifact removal methods include independent component analysis (ICA), average reference, and potential difference reference [16].

The key to preprocessing eye-tracking data lies in eye movement correction and artifact removal, which effectively improve the quality and reliability of eye-tracking data, and provide strong support for the subsequent data analysis [43]. The commonly used methods for eye movement correction include linear interpolation and nonlinear interpolation, which can correct eye movement issues such as saccades and drifts [44]. The methods for artifact removal include screen brightness adjustment and glare correction. During eye-tracking experiments, artifacts may be produced due to environmental light and screen glare, and these methods can effectively remove them [45].

In EEG–ET combined experimental studies, it is necessary to connect the collection equipment, stimulus presentation equipment, behavior-recording equipment, and other hardware devices to ensure the stability and accuracy of the data. Before the experiment starts, the layout of the EEG electrodes and the position of the stimulus presentation device need to be designed to minimize signal transmission and interference between different hardware devices [46]. The electrode cap is worn on the subject’s head, and there are two types of eye-tracking instruments, mobile and fixed, which are respectively worn on the eyes and fixed above the computer monitor used for target experiments. Typically, when conducting online electroencephalogram (EEG) signal acquisition, it is customary to utilize the bilateral mastoids (M1, M2) situated posteriorly to the ears as the reference electrodes for additional channels. The electrode dedicated to capturing horizontal eye movement signals (HEO) is positioned at a distance of 1.5 cm from the lateral aspect of the right eye, while the electrode for recording vertical eye movement signals (VEO) is carefully positioned on both the upper and lower eyelids of the left eye. A high-channel-count acquisition system is used for signal acquisition, and the electrode impedance is ideally maintained below 5 KΩ throughout the experiment [47]. Task control is achieved through a synchronous platform to ultimately monitor and record data.

4. Application Scope

EEG and eye-tracking technologies are used as physiological measurement techniques that can provide visual objective evidence of user preferences. They are helpful in understanding people’s real thoughts, and are useful for user positioning, product design, and implementation and improvement of consumption strategies at the management level. They play a very important role in user research by compensating for the shortcomings of commonly used subjective measurement methods such as semantic differences (SD) and Likert scales in past research. The visualization results of Citespace on the research keywords are shown in Figure 7. The superiority of objective and visual data support provided by EEG and eye-tracking has been widely applied in measuring aesthetic preferences and emotional perception. Currently, it is mostly used for user research on industrial product appearance and usability performance, on digital interfaces as well as transportation, consumption, and other spatial environments. After further analysis, organization, and summarization, we divided the application scope into three main parts, as shown in Figure 8: industrial products, including human–computer interaction, appearance form, usability, and ease of use; digital interface, including information distribution area and interface elements; spatial environments, including public navigation systems, transportation systems, and consumer spaces.

4.1. Industrial Products

Eye tracking (ET) and electroencephalograms (EEGs) are used to evaluate the design and function of products in various fields such as transportation, furniture, clothing, and even packaging. These measurement indicators can predict quantifiable perceptual reactions, and provide a more objective and realistic understanding of users’ aesthetic preferences. Currently, they are mainly used for the evaluation of design and function, helping decision makers to analyze users’ real reflections on products, providing more intuitive and persuasive forms of expression for research results, and helping them to better understand users’ attention to product functions, styling, and structure, thereby clarifying user demands. They can help product innovation or improvement at the design level, and can help researchers to better understand user or consumer feedback, preferences, and needs in market strategies, thereby improving user experience. They have an important significance and role in consumer decision making.

4.1.1. Human–Computer Interaction

The research in human–computer interaction mainly focuses on improving the functional elements of a product to enhance the user experience. Brainwave signals and visual attention are used as feedback information to study the effects of color, speech, and other prompt features on information processing. This information is used to optimize interaction design, reduce cognitive load, and minimize usage errors. Wu et al. [47] conducted research on human–computer interactions in helicopter cockpits, and constructed a cognitive load evaluation model based on brainwave signals and other indicators. Zhang et al. [48] studied the human–computer interaction method of intelligent wheelchairs, using brainwave alpha/beta waves to control the wheelchair’s movements by identifying the alpha waves of brainwave signals when the user’s eyes are closed and relaxed. The experimental results demonstrate that this has an effective correction effect on the closed-loop control system involving human–computer interaction. Yang et al. [49] established a mapping relationship model between user perception and preference for a children’s rehabilitation pedal exerciser, using the eye-tracking method based on a BP neural network. In the study conducted by Ma et al. [50] pertaining to brain–computer interfaces (BCIs) applied in EEG hybrid systems, which incorporated a total of eight electrodes for the acquisition of EEG signals. These electrodes were strategically positioned at the Fz, Cz, P7, P3, Pz, P4, P8, and Oz locations. Notably, the ground and reference electrodes were shared with the electrooculography (EOG) electrodes, which were placed on the forehead and earlobe regions. During user interactions with the BCI, the visual stimuli presented through the flashing icon were found to elicit specific event-related potentials (ERPs), namely P300, VPP, and N170. By carefully analyzing the temporal intervals that potentially encompassed the evoked ERPs, the appropriate output commands of the ERP signals were determined. These studies can provide valuable insights for the design and evaluation of the product interaction experience.

4.1.2. Appearance Form

The significance of appearance design research is to clarify users’ aesthetic preferences and provide design information for product styling optimization. It is worth noting that in recent years, the use of brainwave and eye-tracking technology in the study of chairs has been particularly active due to the closeness of human activities. Liu et al. [51] conducted eye-tracking experiments on Chinese-style chairs, and the results showed that the styling elements that guide the cognitive recognition of styling features were in order of importance of backrest, seat cushion, and armrests, which provided key design elements for the design of new Chinese-style chairs.

Khalighy et al. [7] even elevated user preferences for products to the level of aesthetic formula principles. By analyzing output data from eye-tracking software, including gaze count, duration, and coordinates, they explored the mathematical laws of aesthetic quality in product design. Figure 9 provides examples of aesthetic calculation methods for eye-tracking data of different chairs. The experimental results in Figure 10 demonstrate that users’ predicted and actual visual performance show a close matching relationship between applicability (left) and novelty (right). In Huang et al.’s [52] experiments on multiple car seat styling samples, the eye-tracking trajectories of the subjects revealed regularity from left to right, and from top to bottom. Results showed that users had a higher aesthetic preference for car seats with horizontal and moderately complex stitching lines. These findings have strategic implications for designing user-preferred car seats. Ding et al. [53] utilized event-related potentials (ERPs) as a method to examine the neural responses linked to users’ exploration of diverse product configurations. The findings revealed that images with the ability to evoke participants’ intention for deeper engagement resulted in amplified N300 and LPP responses within the central parietal and parieto–occipital regions. This examination of the neural correlates associated with behavioral intention presents a precise measurement approach to accurately assess users’ perception. The utilization of these conceptual approaches and methodologies holds significant importance in the validation, assessment, and enhancement of product innovativeness, which provides valuable insights into the effectiveness of innovative product designs, and can facilitate informed decision-making and potential optimization strategies.

In packaging-related research, eye-tracking experiment results have shown that the scanning paths of most users are predictable [35], and users tend to associate product styling and its elements with different sensory stimuli, such as the color and flavor of food or drink. Huang et al.’s [54] eye-tracking experiments showed that participants may rely on color when searching for packaging labeled with a certain taste, and if the target cannot be found through color-based search, they switch back to a word-based search. These findings demonstrate the importance of color and taste consistency in packaging design for users.

4.1.3. Usability and Ease of Use

In studies of product usability and ease of use, surveys of attractiveness and satisfaction are key elements of user research. Eye-tracking activities and electroencephalograms (EEGs) are associated with user preferences for products, which reflect user experience-related activities at the visual (physical) level and the psychological (cognitive) level. For example, there is a positive correlation between the number of fixations and visual complexity, and a positive correlation between the average duration of fixations and user pleasure [55]. Barros et al. [56] measured the usability evaluation of satisfaction with packaging for soft drink PET bottles using eye-tracking and EEG measurements, which showed that users’ psychological cognitive and behavioral processes can be captured by EEG.

Moridis and Arapakis used EEG to explore users’ views on product usefulness, ease of use, and fun. The results showed that the EEG asymmetry in the alpha band was highly correlated with emotion, and which was significantly correlated with pleasure and interest during learning [57,58]. Beta, as an important indicator of immersive user experience [59], has been shown to be related to emotions in immersive environments [60]. Deng et al. [61] applied the theory of asymmetry of frontal a-waves to compare the energy changes in left and right brain a-waves, and screened out users’ preferred product elements combined with emotional scales. On this basis, Wang et al. [62] compared car pictures with real cars as reference groups, and investigated how people perceive car design in different scenarios. The results showed that the main differences between the most negative or positive activation electrode differences were observed in the alpha and beta bands, among which the left–right inversion in the alpha band implied a difference between user experience under photo and real car cases, and its left–right inversion can be regarded as the result of different immersive user experiences under two situations. This study is significant for exploring users’ demand for virtual and real products.

4.2. Digital Interface

In the user research of digital interfaces using EEG and eye tracking, the evaluation of interface functionality and form generally focuses on two aspects: one is the evaluation of the rationality of the interface information distribution area, and the other is the analysis of the cognition and use of interface icons. Among them, the visual performance of eye tracking technology is more intuitive than EEG, and eye tracking indicators have been widely used to study changes in user attention.

4.2.1. Information Distribution Area

The most common ways to evaluate the information distribution of interfaces in user research include usability, readability, and interface learning ability [63]. The attention and reading methods for these pieces of information are conducive to optimizing interface layout, and are used to measure usability, effectiveness, efficiency, and satisfaction during interface interaction. Wu et al. [64] rated webpage visibility based on functional information such as webpage layout, color space and distribution, and internal image position and background. Diego-Mas et al. [65] divided different functional areas of interfaces into corresponding AOIs, by collecting user eye-tracking paths and data; they obtained the best configuration for interface information distribution and used mouse trajectory lengths and numbers of mouse clicks required for users to perform tasks in interfaces as indicators of efficiency. They generated new interface distribution forms using a genetic algorithm for slice tree. Guo et al. [66] put forth suggestions aimed at improving the user experience for game navigation interface design by leveraging the association between event-related potential (ERP) components, namely P2 and N2, and perceptual experience as well as aesthetic evaluation. In a similar vein, Wang et al. [67] utilized ERPs to explore users’ aesthetic inclinations towards interface layouts. The empirical findings unveiled a heightened amplitude of ERP components P2 and N2 in interfaces that aligned with aesthetic preferences. Moreover, the late positive potential (LPP) component detected an augmented sustained attention in the response to emotionally stimulating cues, thereby elucidating the underlying mechanisms by which interface layouts exert an influence on aesthetic experiences.

Hou [68] studied user visual browsing rules for mobile phone interface layouts based on eye-tracking experimental data, and summarized three basic layout structures of mobile interfaces: grid format framework, tag-based framework, and side expansion framework. The results showed that the grid format interface structure had the shortest gaze time and scanning trajectory; the side expansion structure had the longest gaze time, more gaze points, and a scanning trajectory that was significantly longer than the other two interface structures, with the target hit rate being the lowest; the search efficiency and information processing difficulty of the tag-based structure were at an intermediate level.

An et al. [69] conducted a user interface evaluation study on the use of library programs by users, linking eye-tracking human–computer interactions with network reading to quantitatively analyze changes in attention. The research results showed that the first arrival time of articles with high-frequency clicks was the shortest, and that such articles should be placed first. The fusion arrangement of articles with medium- and low-frequency clicks can balance readers’ attention, and the distribution of information areas. In network reading, this design can bring the greatest reading benefits to guide interface design optimization.

4.2.2. Interface Elements

Interactive interface elements involve functional icons, colors, positions, and sizes of the interface. In the context of a large amount of information, well-designed icons with appropriate shapes, sizes, and colors can play a role in highlighting task-related information, and reducing conflicts between limited cognitive resources and the complexity of interface information. American cognitive psychologist Norman proposed the design concept of “emotional design”, which improves the efficiency of conveying interface information while also meeting users’ functional and psychological needs.

In the study of interface icon elements such as the type, size, and position, Hou et al. [70] found that the visual appeal of icons can effectively affect users’ visual attention during the evaluation of icon design in application programs, and that highly aesthetic icon designs have higher visual appeal. In Pan’s [71] series of Oddball experiments, it was observed that the P300 wave amplitude displayed a distinct mapping relationship with cognitive resources, whereas the P300 latency correlated with reaction time. The investigation further unveiled that rectangular-shaped icons demanded greater attentional resources, which led to improved task accuracy. Conversely, circular–linear icons demonstrated shorter response times and enhanced discriminability. The findings shed light on the relationship between the visual characteristics of icons and the allocation of cognitive resources. In the functional collection interface, whether it is text or graphic elements in the interface, functional icon features are obvious, making the interface simple and easy to search. The interface clues are organized in order to reduce the difficulty of extracting information from the page, and to quickly identify and use it [72].

There have also been studies carried out from the perspective of cognitive load. Niu’s [73] research believes that visual features are an important factor affecting icon cognition and memory. According to the behavior data and ERP data recorded by users in the experiment, the integration of icon and color features has been observed to elicit a significant increase in the amplitude of the P300 component; moreover, the load level under the semantic–color feature binding of icons under memory conditions is more significant; and the more difficult the semantic coding task of icons that is completed, the greater the cognitive load. Fang [74] started from the perspective of cognitive load, and combined this with ERP experiments to explore the impact of icons on users’ cognitive emotions. The study revealed that icons with a high aesthetic appeal and low aesthetic sensitivity elicited a significantly larger P300 amplitude compared to icons with a moderate aesthetic sensitivity. Moreover, icons with a low aesthetic sensitivity were found to induce larger LPP and P200 components in comparison to icons with a high aesthetic appeal. These findings highlight the influence of both aesthetic appeal and sensitivity on the neurophysiological responses associated with the P300, LPP, and P200 components, suggesting the importance of considering these factors in icon design and evaluation.

Color and its contrast and brightness are keys to triggering users’ visual, behavioral, and emotional harmony. In the exploratory research of visual processing, Carmel et al. [75] first verified through ERP experiments that differences in visual stimuli, such as contrast and brightness, can induce P100 components. The P100 component, which is usually the largest in the occipital area (O1, O2) and has a peak latency of about 100 ms, is significantly affected by the stimulus contrast. Wu et al.’s [76] research proved that brightness contrast seriously affects the visual effect of digital interface information elements. The results show that under high and medium loads, the higher the brightness contrast in the visual stimulus, the higher the activation level of the occipital lobe brain area, and the lower the target search error rate. They successfully induced P100 components at the O2 electrode site in the occipital area. When the brightness contrast was at an 18:1 level, the peak value and average amplitude of the P100 waveform were the highest. Under low load conditions, the activation level of brain areas tended to be consistent, and there was no significant difference in the P100 waveform peak value and average amplitude or the target search error rate. Wu et al. [77] used eye-tracking to explore the relationship between different color schemes of instrument panels and users’ harmony, happiness, and cognitive load. The experiment showed that the degree of color harmony is positively correlated with users’ experience of pleasure. These studies provide theoretical support for the design and selection of human–computer interaction color schemes.

4.3. Spatial Environment

Current life is a three-dimensional space that integrates geographical space, humanistic social space, and information space. The functional and expressive needs of space also present a precise and diversified development trend. Driven by new technologies and new demands, electroencephalography and eye-tracking technology are widely used to study the generalization characteristics of the subject, object, and expression mode of spatial cognition and behavior, in order to accurately understand users’ experience and needs. Currently, EEG and eye-tracking technology are mainly used for user research in the field of spatial environments, including public navigation systems, traffic sign systems, urban spaces, and consumption spaces. This technology plays an important role in deepening the understanding of people’s daily living space behavior and cognition, improving the efficiency of living space design and construction, and has important research value for providing satisfactory spatial experiences.

4.3.1. Public Navigation System

The public navigation systems involve the comprehensive study of elements such as landmarks, terrain, commercial space, and public facilities in public areas. Through EEG and eye-tracking technology, the spatial layouts and spatial forms that users are interested in can be observed to provide positive theoretical guidance for the development of public culture and commercial spaces. In the study of spatial display, Liu et al. [78] used eye-tracking technology to quantitatively analyze the visual attraction of different display methods in different types of spaces after the combination of cultural space and display mode. They revealed the cognitive rules of the constituent elements of cultural space. In a navigation system, Tang [79] aimed at the problems of pedestrian organization and general design in underground public space design that urgently need to be solved, taking the underground public space of the Wuhan rail transit station and commercial complex as the research object. The impression ratio of spatial cognition of subjects in different underground space forms was analyzed with synchronous monitoring of eye tracking and EEG, especially α and β waves that are highly related to wakefulness and reflect active thinking, attention, and problem-solving ability [80]. This study reasonably analyzed the system settings of underground public space planning and design, various facilities and signs, and effectively provided a reference for spatial design techniques and design theories.

Research on user cognition and behavior in public spaces has played a significant role in the tourism industry, especially in the operation of tourist attractions and commercial development. Ohkubo et al. [81] used a new method of detecting tourist impressions from electroencephalogram (EEG) information, and calculated the impression ratio of subjects during journeys to tourist attractions based on alpha and beta waves measured by a portable EEG device. The study used a portable electroencephalogram (EEG) device and a corresponding sensing system, which is distinct from conventional EEG apparatuses commonly employed in laboratory settings. This device is specifically designed for wearable applications, with the electrodes being placed on the forehead and one ear. The system encompasses a portable EEG device and a smartphone interface, serving the purpose of impression rate detection derived from the EEG signals. Notably, the system is equipped with the capability to monitor the intensity of alpha and beta waves at a frequency of one measurement per second. This novel approach offers promising prospects for real-time EEG monitoring and assessment in various applied contexts, which provides effective suggestions for improving the tourist experiences of popular attractions. Leng [82] studied the spatial and architectural elements of traditional villages and residences, collected records of the observation process of subjects using an eye tracker, and analyzed the attentional form of villages through visualized data such as gaze trajectory maps, heat maps, AOI interest maps, etc.

4.3.2. Transportation System

In the transportation system, eye-tracking technology is currently widely used to finely measure people’s perception, emotions, and behavior, which provide strong support for establishing the relationship between the characteristics of complex road environments and the psychological and physiological indicators of drivers. It has been widely used in the research of traffic systems such as signal lights, routes and signs, and isolation belts. For example, Li et al. [83] proved through experiments that tunnel brightness has a significant impact on indicators such as drivers’ gaze duration, scanning range, and blink frequency. Guo et al. [84] studied the distribution law of drivers’ gaze points under different channel width conditions. Jagerbrand et al. [85] explored the effects of three types of road environments—open, forest-covered, and diverse—on driving behavior. Qi et al. [86] started from the perspective of visual interest areas, and studied the relationship between the information volume of interest areas set by drivers for traffic engineering facilities in highways and drivers’ visual characteristics. Liu et al. [87] conducted a study to explore the cognitive implications of different background colors in traffic signs. The results showed that early-stage event-related potentials (ERPs), specifically the N1 and P2 components, exhibited similar patterns across various background colors of incorrect traffic signs, suggesting minimal influence on cognitive processing. However, blue and green background colors elicited larger P300 amplitudes during later stages, indicating heightened cognitive processing difficulty. In a separate investigation by Hou et al. [88], it was discovered that semantically incongruent traffic sign words evoked greater N400 amplitudes and theta activity, whereas semantically congruent traffic sign words evoked stronger late positive potentials, reflecting participants’ positive emotional arousal. These findings have significant implications for assessing the impact of color, semantics, and design on the comprehension of traffic signs, and on the overall understanding of environmental contexts.

Zhu [89] systematically explored the effects of a linear spatial layout of a central divider on driving behavior in urban expressways, and analyzed changes in various characteristic parameters under different visual stimuli of the central divider layout. The results show that the focus at a speed of 70 km/h was found to be more concentrated towards the middle of the road than the focus at a speed of 50 km/h. According to visual characteristics, a natural layout form can not only improve the monotony of an urban expressway landscape environment, but also provide appropriate visual stimulation. In comparison to a disorganized environment, an organized environment evokes greater amplitudes of the N2 component, which is linked to cognitive processing conflicts [90]. It provides a certain theoretical basis for improving the traffic layout of central dividers, and improving driving safety.

In addition, in the quantitative research of traffic landscape spaces, the research method of multi-source visual data fusion combined with spatiotemporal measurement has been widely used, and provides strong support for the study of the impact of road environments on drivers’ visual stimulation. For example, the concept of visual entropy was introduced to measure the size of image information and the sensitivity of the human visual system to images [91]. Devlin et al. [92] used gaze entropy as an important predictive indicator of workload. Zhao et al. [93] determined the threshold values for discriminating fatigue driving behavior indicators under different road line shapes, and established a comprehensive fatigue driving discrimination model based on various driving behavior indicators and threshold values. These application studies provide a basis for improving and ensuring the safety of urban road traffic systems.

4.3.3. Consumer Space

In consumer scenarios, electroencephalography (EEG) is one of the most commonly used neuroscience technologies in marketing research, and provides key information for studying users’ observation and consumption decisions of products. Its low cost and high temporal resolution have been widely used in consumer research, including product features, pricing, advertising attention and memory, and rational and emotional information [17]. Especially in diversified retail environments, the advancement of eye-tracking technology can play a key role in capturing attention, as well as in analyzing which elements attract consumers, to better understand their views on product displays. For instance, in situations where consumer reviews exhibit consistency, whether positive or negative, positive reviews may evoke a diminished P2 component, an amplified N2 component, and an enhanced late positive potential (LPP). These distinct components signify the allocation of attentional resources, perception of cognitive conflicts, and evaluative categorization processes linked to the LPP [94]. Researchers and practitioners can use this information to improve the sales efficiency of displays, and help promote standardized research methods for consumers in many retail market environments [95].

In the typical retail environment of a shopping mall supermarket, Bialkova [96] used ET as a method to determine the key elements that attract the attention and selection of sales points. By changing visual marketing stimuli on real supermarket shelves, the impact of brand positioning and brand strength was explored. Khushaba et al. [97] analyzed spectral activity (delta, alpha, and beta activity in the frontal lobe area) related to preferences by changing the shape and flavor of product feature stimuli and combining user behavior, electroencephalography, and eye-tracking data. Jones et al. [98] investigated consumers’ processing of pricing and discount information, and found enhanced P300 responses in individuals with high mathematical anxiety. Tasks involving pricing and discount stimuli in these individuals revealed outcomes that were indicative of heightened emotional arousal and motivational processes. Cherubino et al. [99] measured the brain activity with EEG and eye gaze of people visiting specific areas of the supermarket. The eye-tracking heat map shows that the “coldest” department (fruit and vegetable department) of the store has the potential to fully attract customers at an emotional level. In terms of neural measurement indices, fruit provides higher pleasure, while vegetables have higher attention, memory and workload, which provides the best predictive indicators for future customer purchasing behaviors. The experiment showed that innovative packaging, displays, and storytelling forms are highly attractive. Undoubtedly, these findings also provide key information for management, design, and marketing strategies in studying consumers.

5. Discussion

5.1. The Strength and Contradictions of Reviewed Findings

This review summarized the current research indicators, paradigms, and applications of EEG and eye-tracking technology in user research. The advantages of EEG and eye-tracking technology lie in their ability to provide objective, accurate, real-time biometric data. By properly connecting devices and processing relevant data, the research indicators obtained can help researchers gain insight into users’ true thoughts. The review shows that these technologies are widely used in user research in industrial products, digital interfaces, and spatial environments to better understand users’ cognition, emotions, and behavior. However, in the process of conducting this study, it was found that a user research system has not been systematically established; therefore, only a small part of the relevant literature pertains to user research, which is somewhat biased against the current development trend. Through analysis, it was discovered that EEG and eye-tracking are two commonly used biometric measurement technologies in user research. Although they can provide valuable information to researchers, they still have limitations in terms of device accuracy and experiment feasibility. However, overall, biometric technologies such as EEG and eye-tracking are in line with development trends, and in the future, these technologies will be combined with advanced technologies, such as artificial intelligence and machine learning, to overcome existing shortcomings and further develop the field of user research.

5.2. Current Limitations

On the one hand, the main driving force of EEG and ET technology in the field of user research comes from people’s growing needs for products or services; on the other hand, it comes from technological progress that promotes a deeper exploration of the relationship between vision, the human brain, cognition, perception, and behavior [22]. However, there are still limitations in the feasibility of current technology, due to its complexity and sensitivity to environmental conditions. It is crucial to be thoroughly familiar with eye-tracking protocols and equipment for successful data collection. Although portable devices have expanded the range of experimental sites, they are not suitable for outdoor environments with sunlight interference; instead, they are more suitable for indoor environments, which greatly reduces the feasibility of some experiments [95].

Neuroscientific methods are only good for studying stable and relatively common cognitive characteristics. In EEG experiments, the preparation process is relatively cumbersome. Participants need to clean the oil stains from the surfaces of their scalps in advance, and then inject conductive gel or physiological saline between the electrodes and the scalp to enhance conductivity. The preparation time is generally more than 30 min. Since the experimental results need to be averaged, the time required for ERP experiments will be longer. During experiments, due to the susceptibility of brain neural reactions to external interference, procedures need to be conducted in a relatively quiet environment away from electrical, magnetic, light and other signal interference. The participant’s limb behavior and personal conditions will also bring difficulties to subsequent data processing [100]. Therefore, most of the experimental materials that have obtained clear results so far involve still images and videos that can make participants stay still. Although emerging wearable brain state monitoring devices have provided possibilities for conducting measurements outside the laboratory in recent years, these devices present varying degrees of discomfort [101]; this, in turn, will also affect the participant’s emotions and reduce the credibility of experimental results. Therefore, it is necessary to clarify feasibility in the experimental design to ensure that experimental designs will have a higher feasibility, and effectively avoid repeated experiments and greatly reduce research time costs to improve research efficiency.

In addition, the accuracy of the device itself is also a major challenge that the scientific research field needs to face. For example, most of the stimuli used in EEG experiments may mainly reach the cortical area, while aesthetic experiences may be related to the participation of subcortical structures. This problem can only be indirectly intervened through transcranial magnetic stimulation (TMS) and transcranial direct current stimulation (TDCS) [102]. Therefore, the insufficient accuracy of physiological signal acquisition devices is also a major problem that urgently needs to be solved.

5.3. Future Research Prospects

5.3.1. Multisensory and Behavioral Interaction Research

From the literature review, it can be seen that current user research in industrial products, digital interfaces, and spatial environments mostly focuses on visual and cognitive brain research. However, since the user experience is affected by aspects such as shape, smell, and touch, multi-sensory and behavioral interaction research should be given more attention in the future. In interface interactions, the drop-down method of lists, the animation’s form, the feedback text, the color information, and the sounds should all be considered as a whole. Using EEG and eye tracking equipment, reasonable experimental designs can be used to study the interactions between the five senses and cognitive behavior, such as audio–visual, audio–taste, visual–taste, touch, smell, and other sensory interactions. For example, some studies have found that smoothness, softness, and sphericity are associated with higher brightness and chroma [103,104]; geometric shapes in vision will affect people’s taste information processing process. The influence of different shapes on taste evaluation intensity (intensity) and efficacy (potency) has been widely confirmed [105], especially the relationship between shape polysemy and perceived efficacy. In addition to shape, visual color also affects taste perception. Research has shown that compared to green and white light, consumers give higher ratings when drinking red wine under blue and red light, and are also willing to pay more. Blue and green light makes people feel that the spicy and fruity flavors of red wine are heavier, while red light makes people feel that red wine is sweeter [106]. Moreover, environmental sensory stimulation should also match the product category attributes. For example, in a dining environment, all sensory factors in the restaurant should be included in the scope of customer taste management. Not only should the shape, color, and touch clues of containers such as plates, spoons, and wine glasses be considered for their impact on consumers’ taste perception, but also the interactions between background lighting and odor with consumers’ taste perception [107].

The interaction between sensations is one of the current hot research topics. Previous multi-sensory interaction research has mostly focused on the fields of biology and psychology. In practical applications, people’s feelings about products are often integrated and semantic. Current user research ignores individual differences in multi-sensory interactions and needs to be further explored in future research. In addition, some previous research data have shown that there are differences in the performance of different subjects in multi-sensory interactions under the same cultural background [108], but there has been no in-depth discussion of this issue. For example, why some people are more likely to have multi-sensory interactions than others can be studied, and future research can also analyze the root cause of this phenomenon from the perspective of differences in thinking.

5.3.2. Universal Research on Multi-Technology Fusion

In recent years, user research has drawn a lot of experience from more mature fields such as the cognitive neuroscience of memory, language, and emotion. With clear experimental goals, matched research objects, practical experimental design and research methods, strong fault tolerance, and improvements in the level of technology, deeper content research has been conducted based on obtaining effective conclusions, gradually exploring the psychological characteristics of user aesthetics, cognition, and experience processes [109].

On the one hand, from a technological perspective, with the continuous progress of AR, VR, and artificial intelligence technologies, high-intensity blurred reality may be able to compensate for the lack of accuracy of physiological signal measurement devices to a certain extent [110], and can make physiological signal recognition and classification more accurate. For example, Yu et al. [111] proposed an emotion recognition method for EEG signals based on the integrated learning method AdaBoost, and the recognition accuracy was significantly improved. Therefore, a multidisciplinary fusion approach is essential. The introduction of statistics and computer technology will help establish more comprehensive mathematical models. The quality of quantitative models for user emotions, cognition, and experience will be much better than previous evaluation models, and the refinement level will be improved. AI (artificial intelligence), ML (machine learning), and DL (deep learning) technologies have great potential for development in the fields of driving, aviation, education, and healthcare, in predicting cognitive workload. By analyzing EEG signals and brain imaging data such as MRI, the frequency and amplitude of brainwaves can be analyzed to evaluate the structure and function of the brain for predicting and assessing cognitive workload [112]. In addition, these technologies can participate in behavior and task analysis, real-time monitoring, and feedback, using machine learning algorithms to analyze user behavior data to identify patterns and features of high cognitive workload under specific tasks, and provide personalized feedback based on the monitoring results. Moreover, training machine learning models with large amounts of EEG and brain imaging data can predict and classify cognitive workload, interpret AI models [113], implement real-time health monitoring systems, and quantitatively evaluate EEG biomarkers [114], thereby improving work efficiency and safety.

On the other hand, research has verified that the measurement results of the fusion of EEG signals and eye tracking signals are generally better than those for single signals [115]. Currently, domestic and foreign scholars are trying to collect user emotions by integrating multiple physiological quantities, such as EEG signals and eye tracking signals, skin electrical signals, breathing, and heart rate, in order to explore measurement methods that are compatible with different user-differentiated needs in ubiquitous methods. In the long run, accurate measurement is a prerequisite for quantifying emotions as reference elements to guide product and service design. The construction of this multi-channel input feedback mechanism will help predict the images and preferences of different types of users efficiently and accurately, and improve methods of multi-technology fusion in the field of user research. This is an effective way to guide design practice, provide supportive objective data, reduce decision subjectivity and one-sidedness, and promote the universality of similar and related research.

6. Conclusions

In this review, we examined the research indicators, paradigm, and scope of EEG and eye-tracking methods in the field of user research. However, the number of studies that truly combined these two technologies and fully applied them to study user characteristics, behaviors, and habits was limited. The contributions of this review lie in four aspects. First, it provided an overview of the common indicators used in eye-tracking and EEG signal acquisition in the field of user research, including pupil diameter, pupil position, eye fixation time, as well as ERP components such as δ, θ, α, β, and γ, five rhythm waves, and P1, P2, P3 (P300), N1, N2 components, and elaborated on the relevant research paradigms and the relevant information of equipment connectivity. This information can help other researchers understand and choose suitable indicators for their own research. Second, it summarizes the application areas of user research, including the human–computer interactions, appearance forms, usability and ease of use of industrial products, digital interface interactions, public navigation systems, transportation systems, and consumer spaces. This can help other researchers understand the applications of this technology in different fields. Third, it points out the shortcomings in the current research, including the limitations of current EEG and eye-tracking devices in conducting experiments in outdoor environments, the long preparation time of EEG experimental equipment, and the existence of errors and limitations in the accuracy of physiological signal acquisition. This information can provide other researchers with directions to improve research design and equipment performance. Fourth, it outlines future research directions, including multi-sensory and behavioral interaction research, and multi-technology fusion ubiquitous research. This can help other researchers understand future development directions in this field. In summary, this study provides an overview of the research indicators, research paradigms, application areas, existing shortcomings, and future research directions for eye-tracking and EEG signal acquisition in user research, providing important reference information for other researchers in this field for further research. We hope that this survey can help those who are interested in trying to apply EEG and eye-tracking technology to the field of user research to quickly browse the research status of this field.