Intelligent Eye-Tracker-Based Methods for Detection of Deception: A Survey

Abstract

Over the last few years, a large number of studies have been conducted on the monitoring of human behavior remaining beyond conscious control. One area of application for such monitoring systems is lie detection. The most popular method currently used for this purpose is polygraph examination, which has proven its usefulness in the field and in laboratories, but it is not without its drawbacks. Technological advances in data acquisition and automated analysis have ensured that contactless tools are in high demand in security fields like airport screening or pre-employment procedures. As a result, there has been a shift in interest away from traditional polygraph examinations toward the analysis of facial expressions, voice, and speech patterns, as well as eye-tracking signals to detect deceptive behavior. In this paper, we focus on the last aspect, offer a comprehensive overview of two distinct lie detection methodologies based on eye tracking, and examine the commonly used oculomotor feature analysis. Furthermore, we explore current research directions and their results within the context of their potential applications in the field of forensics. We also highlight future research prospects, suggesting the utilization of eye tracking and scan path interpretation methodologies as a potential fully functional alternative for the conventional polygraph in the future. These considerations refer to legal and ethical issues related to the use of new technology to detect lies.

1. Introduction

The complex nature of lying makes the ability to detect deception, based on subjective indicators, limited for unqualified people, allowing them to achieve the probability of correct classification only marginally better than random (54%) [1]. Even for trained investigators, this result becomes only slightly better. The absence of verbal and nonverbal indicators uniquely related to deception and the fact that liars consciously attempt not to arouse suspicion as to their truthfulness cause the detection of lies to be a challenging task. Deceivers do not want to be discovered while deceiving others. Thus, they make an effort to conceal and suppress behavioral cues that could be used to identify their lies [2].

Moreover, most lies, also known as white lies, are told for psychological reasons (to avoid conflict, to avoid hurting someone’s feelings, or to protect ourselves), and people do not experience uncomfortable emotions when telling them. On the contrary, there are black lies that can have negative effects on both individuals and entire communities, particularly in the area of security. Therefore, making crucial decisions in this field requires the ability to evaluate information using reliable tools.

Today, it is assumed that it is not possible to actually measure a lie itself because of its amorphous form. The human responses and captured behavioral data cannot be treated directly as deception or truth-telling per se. However, because many scientific fields employ the quantitative assessment of phenomena not accessible for direct physical measurement, scientific tests for lie detection and credibility evaluation are feasible.

The most widely identified scientific procedure for the detection of deception is polygraph examination. During this test, the subject is asked a structured set of questions, and his/her physiological reactions (respiratory, electrodermal activity, and blood pressure) in response to those questions are recorded. “The analytic theory of polygraph testing is that greater changes in physiological activity are loaded at different types of test stimuli as a function of deception and truth-telling in response to relevant target stimuli” [3].

In many countries such as Poland, the Czech Republic, or Japan, the results of polygraph tests are admissible in court as forensic evidence [4]. In more than 70 countries, polygraph is frequently used in police investigation work. It should be emphasized that polygraph examination can be conducted only when the person gives his/her written consent before the procedure starts. Additionally, its use in police investigations is regulated by the law of a given country and the American Polygraph Association (APA) Standards of Practice. In Poland, the Code of Criminal Procedure states that psychophysiological examination can be performed to remove a person from the suspects’ circle. The APA Standards also require that the entire examination should be audio-video recorded, which gives the judge the possibility to verify its course and compliance with the methodology.

Polygraph examinations use two main techniques: the Concealed Information Test (CIT) and the Comparison Question Test (CQT). During the CIT, examinees are asked questions with multiple answer options. Only one question contains a crime-related answer, while other questions present a few control answers not connected to the crime. The evaluation of the results is based on the hypothesis that while for the innocent subject there will be no noticeable difference in response to the relevant and neutral stimulus, for the guilty person the response to the target will be greater.

Using CQT is a common practice, especially during criminal investigations. This technique compares the arousal response of comparison questions (e.g., ‘Have you ever taken something that did not belong to you?’) vs. relevant questions (e.g., ‘Did you take money from the shop that day?’). According to test data analysis, when the examiner observes higher arousal to the relevant questions he/she can identify the examinee as deceptive, while an increased arousal caused by comparison questions suggests truthfulness. If there are no differences between the two sets of reactions, the outcome is deemed inconclusive.

However, the accuracy of both polygraph tests is questionable. For the CIT examination, although the specificity is moderately high (94–98%), the sensitivity is rather low (42–76%). This means that an innocent person is not likely to be labelled as guilty, but on the other hand there is a significant chance that this test will not be able to effectively identify guilty subjects. For the CQT, it is the opposite. The rate of correctly classified guilty subjects is fairly high (87–98%), but the rate of correctly classified innocent subjects drops dramatically (55–56%) [5]. Although the accuracy of these polygraph techniques remains debatable [6], the scientific knowledge and field experience derived from both tests can be used for developing methods utilizing other measurement tools such as eye tracking.

2. Materials and Methods

2.1. Eye Tracking Examination for Deception Detection

Eye tracking is the process of monitoring the position of the eyes, the direction of the gaze, and the size of the pupil to identify areas of visual focus at a given moment. Most widely used desktop eye tracking devices typically use infrared light sources to create reflections on the cornea, allowing precise eye tracking. However, it is worth mentioning that algorithms, such as [7], based on webcam images, are also becoming increasingly prevalent. This technology is already being used in various fields, e.g., psychology [8,9,10], human–computer interactions [11], engineering management [12], market research [13,14,15], and medicine [16]. Recent studies also provide evidence supporting the potential use of eye tracking analysis as a means of detecting deception. Changes in eye movements, such as increased fixation duration, a surge in saccade amplitude, and decreased gaze stability are thought to reflect the increased mental effort required to process information and perform a task under conditions of high cognitive load. Because cognitive load has been shown to increase in a lying human, research on lie detection based on ocular signal analysis often stems from this premise.

Eye tracking devices are advantageous in deception detection systems for numerous reasons, including:

• non-invasive data collection without physical contact (which is important during questioning especially dangerous or unpredictable persons),
• self-calibration without human assistance,
• suitability for widespread use due to mobile technology advancements,
• the ability to collect data covertly, which may prevent the subject from using countermeasures,
• shorter examination time compared to regular deception detection tests with the polygraph.

Another issue worth mentioning here is the objective manner of the measurement. Robustness to expert error or to the possibility of altering the test’s outcome is crucial from the perspective of forensic evidence theory.

Deception detection and eye tracking can be combined in two main contexts: (1) an interview scenario (similar to a police interrogation) [17] and (2) an automatic test scenario (to replace polygraph examination and minimize the examiner’s influence on the procedure). Both solutions have shown promising results. However, the first solution allows for the verification of individuals even without their awareness that they are being tested for truthfulness. This aspect may elicit objections from lawyers, psychologists, and ethicists worldwide. Hence, it is essential to focus on automatic methods that require the subject’s explicit consent to participate in the examination procedure. The classification of eye-tracking methods used for deception detection is presented in Figure 1.

When it comes to the methodology of ocular measurements for forensic purposes, the most popular methodologies in laboratory research are the Concealed Information Test and the analysis of gaze patterns while reading.

2.1.1. Concealed Information Test

The Concealed Information Test (CIT) is a type of lie-detection test that is used to assess whether a person is familiar with a particular detail of a crime or event that would only be known by someone involved in it. It is based on the comparison of physiological and ocular responses to questions concerning a crime or the details of an event, as well as to control questions [18].

Traditionally, physiological measurements during CIT are limited to the sequential presentation of each stimulus separately, as shown in Figure 2. When several stimuli are presented at the same time, it would be challenging to pinpoint exactly which stimulus caused the response. Eye tracking extends the measurement possibilities by allowing multiple stimuli to be presented at the same time, as we can clearly separate regions of interest into areas where individual stimuli are displayed. This also allows for the introduction of additional parameters that characterize the signal. It is a noteworthy enhancement, particularly in light of early eye tracking studies that have demonstrated significant variations in gaze behavior depending on the task being performed [19]. The choice of task therefore becomes crucial as it can have a major impact on the efficacy of using eye tracking measures for deception detection.

Another interesting modification of CIT is the “oddball” variant. During the procedure, a series of stimuli are shown to examinees, some of which are the oddball stimuli (objects that the participants were presented with to familiarize them before conducting the test) and others that are not-targets (both related and not-related to the crime). The participants are then instructed to classify each stimulus. For a participant who does not have any knowledge of the crime, this task is straightforward as they only have to discriminate between the known targets and unknown non-targets. However, for a guilty participant, the task becomes more challenging because they not only have to discriminate between the targets and non-targets, but they also have to recognize and avoid the crime-related probe stimuli that are presented in the series.

Among the parameters used in studies attempting to develop deception detection systems based on the eye-tracking signal for both CIT tests with sequential stimulus presentation and parallel presentation are pupil dilation, fixation duration, the number of fixations, the fixation frequency, and the number of blinks.

In addition, by separating the regions of interest in the parallel stimulus presentation scenario, investigators are able to define additional parameters describing the subject’s behavior. Such measures include: the eye-gaze dwell time on the center of the screen, the proportion of the number of fixations on neutral images and the target, and a few others. For the familiarity detection tests that involve face recognition, the focus time on individual face areas is also measured.

2.1.2. Tests Based on Reading Behavior

When reading texts, the reader’s eyes behave in a highly ordered manner. We can broadly divide the eye movements performed during this visual task into three groups: movements that follow the direction of the flow of the text, the transition of the gaze from one line of text to the next, and movements of a regressive nature performed in the direction opposite to the flow of the text [20]. A single movement cycle consists of a fast movement called a saccade and a fixation pause. The average fixation lasts about 200–250 ms, during which time the visual information is processed by cells of retina layers and sent to the brain, whereas saccades last only a few milliseconds and allow the reader to scan the text and move to another fixation. The optimal gaze pattern will vary from person to person, depending on factors such as reading speed or level of comprehension, as well as external factors, e.g., the font size or the level of difficulty of the text [21]. The foundation of analyzing and interpreting eye-tracking data obtained during reading consists of two underlying assumptions: the immediacy and eye-mind assumptions [22]. Thus, we assume that the reader interprets each word as soon as it is encountered in the text without waiting for the end of the phrase, and that the length of fixation on a word corresponds directly to the time necessary to actively process the information.

To analyze the eye-tracking signal, we can use a number of parameters. Aside from standard eye signal metrics like the length, number, and frequency of fixations; the blink rate; and saccade latency during reading, we can also measure other parameters such as regression latency, the total time for the first scan of the text that considers all forward fixations, and reread duration (time spend on re-reading). These parameters depend on both automatic behaviors, resulting from cognitive or physiological processes, and conscious behaviors of the readers used to meet specific goals. This additional task that a deceptive person has to perform makes it possible to notice changes in gaze behavior.

To detect those differences, a Relevant Comparison Test (RCT) can be used, but instead of asking participants directly, yes/no statements are displayed on the desktop in front of them while their eye movements are recorded using an eye tracker. RCT consists of two sets of statements. Set R1 refers to illicit behavior that is important for interrogators (behavior of primary interest, e.g., theft, drug selling). The second set (R2) relates to behavior whose possibility of occurrence is low (less than 3%), e.g., a terrorist attack. Examinees with a high probability did not commit the terrorist attack, thus they are truthful when responding to statements from set R2. The RCT assumption is that innocent people react with similar arousal to both sets R1 and R2, while deceptive examinees respond to R1 with differences in ocular measures. Before the test, participants are informed that to pass the test, they must answer as quickly and as accurately as possible. The methodology of this test also makes it possible to record behavioral measurements such as the response time (the time from the question display to the participant response) and the error rate (the percentage of questions answered incorrectly by the participant).

2.2. Effects of Countermeasures in Deception Detection

In situations where individuals are motivated to deceive, such as during criminal investigations, they may try to employ countermeasures (CM) to appear more convincing and avoid being caught in their deception. The literature identifies two main different types of CM techniques: physical CM, which involves actions aiming at disrupting the signal being measured (e.g., biting the tongue, discreetly moving muscles or moving gaze into specified area), and mental CM, which involves practices like visualizing past emotional events or engaging in mentally demanding activities, e.g., counting [23].

The validity of a polygraph is at major risk while countermeasures are being implemented. Studies have shown that polygraph tests are prone to error when CM are employed even after very brief preparation for the use of interference by participants [24,25], but knowledge about the polygraph technique was not enough to have a notable impact on test accuracy. These studies also indicated that truthful subjects who use CM achieved the opposite of what they intended; instead of reducing the likelihood of being incorrectly accused of lying, they increased it.

However, it is possible to develop measures to reduce the effectiveness of CM based on understanding the impact they have on physiological signals [26]. The same steps must be taken when designing an effective and reliable deception detection system using ocular features. This issue has already been addressed in several papers. In [27], the authors have shown that while classification between guilty and innocent participants was possible based on number of fixations, fixation duration was not a sufficiently robust marker. Moreover, a difference was observed between the effect of mental and physical CM on the number of blinks, indicating a more effective concealment of information using the former. The findings regarding the aforementioned fixation features in the scenario with CM were also confirmed by the authors in [28].

3. Results

An impressive amount of studies applying eye tracking to deception detection have been published in the last two decades. They present a wide range of settings, contexts (an interview or an automatic test procedure), theories, and findings. As a result, it could be challenging to learn about trends, effective approaches, and gaps in the oculomotor detection of deception. Our goal is to compare up-to-date eye tracking techniques for automatic deception detection in the context of forensic application, as well as the kind of data that have been processed so far and how they were analyzed. In addition, we are also looking for their limitations and advantages and what remains to be explored.

Our focus is on selecting research papers that describe the tests used in traditional polygraph examinations. The aim of those studies was automatic measurement with the eye tracker to obtain the most objective results. This review presents qualitative and quantitative data. We do not intend to focus solely on the test accuracy data. This review is designed to present a broad picture of research potential related to eye tracking in forensic lie detection. Research findings are discussed in Section 4 from a psychological perspective as they could contribute significantly to the development of the theory of instrumental deception detection techniques.

As described in Section 2, automatic deception detection research using the ocular signal is based on two main paradigms. The first paradigm involves CIT tests with the presentation of visual stimuli, which can be performed either sequentially or simultaneously. The second paradigm is based on analyzing visual patterns that occur during reading. Since the testing methodology and the features analyzed significantly differ between these two approaches, the considered papers have been divided into two separate sections, each corresponding to a specific approach. Several meaningful studies presented to date, along with the authors and year of publication of the paper, are summarized in

Table 1. Deception detection studies utilizing eye-movement-based CIT.

Authors	Year	CIT Variant	Diagnostic Features	Analysis Tools	Major Findings
Schwedes et al. [29]	2011	CIT with simultaneous presentation of facial images as stimuli	fixation duration	MANOVA	While concealing knowledge, fixations on the faces lasted longer than fixations on the non-selected, unfamiliar faces in the neutral display. Furthermore, the fixation durations were longer when chosen known faces were presented compared to known but not selected faces. Lying participants were correctly detected in 64.9% of cases
Peth et al. [30]	2013	Sequential CIT with questions regarding central and peripheral objects	number and duration of fixations; duration; number of blinks	ANOVA, Area Under Receiver Operating Characteristic Curve	Participants from guilty group exhibited reduced blink rates and fewer but more extended fixations on the central crime details. This pattern persisted even after the stimulus was removed. The best achieved AUC value across different arousal conditions for both the 0–5 s and 5–10 s intervals was 0.72. These results were obtained using the number of fixations on central details.
Symour et al. [31]	2013	Sequential CIT with face images as stimuli	response time, blink rate, pupil size, and pupil slope	ANOVA, Receiver Operating Characteristic Curves	The use of measurements of pupil size, pupil slope, and pre-response blink rate separately can lead to effective categorization; however, better results were achieved while combining all of the features. Incorporation of eye behavior characteristics into the classification resulted in a slightly improved result compared to an analysis based solely on reaction time. The results of the compound classification procedure used in the paper showed the highest results (100%) while using combined measures approach, namely, RT + pupil size.
Proudfoot et al. [32]	2016	CIT with simultaneous presentation of stimuli in the form of facial images	pupil dilation, eye-gaze dwell time on the center of the screen	latent growth curve modeling, Area Under Receiver Operating Characteristic Curve	Both pupil dilatation and eye-gaze dwell time change in a distinct manner during the course of an interaction, and these patterns can possibly be indicators of deception, irrespective of the presence of relevant stimuli. The classification model achieved a 73.9% true positive rate and a 13% false positive rate.
Schwedes et al. [33]	2016	Simultaneous CIT (traditional and in “oddball” version)	fixation duration	ANOVA, Area Under Receiver Operating Characteristic Curve	The second fixation proves to be an efficient marker for concealed information detection both immediately after the mock crime and (in a reduced manner) after one week has elapsed. ROC analyses on the second fixation detecting concealed knowledge showed an AUC of 0.61.
Millen et al. [34]	2017	Sequential CIT with images of faces varying in familiarity as a stimuli	number of fixations, number of regions visited, number of independent clusters of fixations on an interest area, and proportion of fixations in the inner regions of the face	RM ANOVA	Number of fixations was lower for known faces regardless of familiarity level; number of fixations was a good marker for recognition detection in case of personally familiar faces.
Lancry-Dayan et al. [35]	2018	CIT with short-term memory task (both parallel and single display)	in parallel display: gaze dwell time during the first phase (1–1000 ms) and the second phase (1000–5000 ms), number of visits, and number of fixations; in single display: mean fixation duration, response time, and accuracy in short-term memory task	ANOVA, Receiver Operating Characteristic Curves, Support Vector Machine	During short-term memory task, participants firstly fixated more on the familiar face; then, the strong tendency to avoid it was presented. Avoidance was still evident, even after participants received explicit instructions on how to perform CM. The within-subject SVM classification analysis revealed correct classification rates of 92.2%, 91.3%, and 88.7% for non-concealed, concealed, and countermeasure experiments, respectively. The intersubject analysis showed average accuracies of 93.4%, 90.8%, and 88.7%.
Millen et al. [36]	2019	Sequential CIT with face images as stimuli	number of fixations, average fixation duration, proportion of fixations in the inner part of the face, and number of visited areas of interest on the face	Area Under Receiver Operating Characteristic Curve	Longer fixation durations as well as lower number of fixations in the inner regions of the face were found for guity group regardless of conditions. During familiar face recognition, 57% of participants in the standard guilty condition and 83.5% of participants in the countermeasures condition exhibited a lower proportion of fixations on the inner face regions.
Millen et al. [28]	2020	Sequential CIT with images of faces, scenes, and objects varying in familiarity as stimuli	number of fixations, number of different interest areas of the image viewed, number of return fixations to previously viewed areas of interest, proportion of fixations made to the inner regions of the image, and average fixation duration	Area Under Receiver Operating Characteristic Curve	Deception was characterized by a lower number of fixations for all stimuli classes across all levels of familiarity with higher confidence ratings for higher familiarity levels, definitive distinction of honest answers was not possible based on other fixation measures for objects different than faces. The best AUC scores for both personally familiar (0.83) and newly learned faces (0.67) based on the full trial were achieved using the number of fixations.
Rosenzweig et al. [37]	2020	Sequential CIT with face, name, and residency used as stimuli	microsaccade rate modulation, microsaccade reaction time (msRT), and Oculomotor Modulation Function (OMF)	Paired t-test	There was a significant difference in the mean msRT between the groups. However, this measure alone was not sufficient to assess identify probes within a group. On the other hand, the deviation of the OMF was 100% successful in identifying probes in the ‘guilty’ group.
Chen et al. [38]	2022	CIT extension called rapid serial visual presentation (RSVP)	pupil size	Sample-by-sample linear mixed effects analysis on the group level and leave-one-out t-test analysis on the individual level	The pupil size observed during a RSVP task may yield valuable insights into concealed identity information. Although most of the participants qualitatively showed the desired effect on their real name, individual analysis revealed that it was not statistically significant for most of them.
klein Selle et al. [39]	2022	Sequential CIT with cards as stimuli	pupil size, number of fixations, number of blinks, fixation duration	ANOVA	Changes in fixation characteristics and the number of blinks occurred only in the concealed condition, while pupil dilation occurred in both conditions (concealing and revealing knowledge). This suggests that inhibition theory is relevant for the first two and orientation theory is relevant for the latter.

for studies based on CIT and in

Table 2. Deception detection studies utilizing other types of tests.

Authors	Year	Deception Detection Test	Diagnostic Features	Analysis Tools	Major Findings
Cook et al. [40]	2012	Comparison Question Test	Response time, response accuracy, pupil diameter, number of fixations, first-pass duration, second-pass duration	RMANOVA, Classificatory Discriminant Analysis (linear and jackknife)	Individuals who were found guilty exhibited greater pupil dilation when responding deceptively to statements. Also, fixation duration, reading, and reviewing times were shorter for those statements than for the ones they answered truthfully. The presented method allowed for the classification of 46 out of 56 guilty participants (82.2%) and 50 out of 56 innocent participants (89.3%).
Hacker et al. [41]	2014	The Relevant Comparison Test	pupil diameter, response time, response accuracy, number of fixations, first-pass duration, and second-pass duration	RMANOVA, discriminant function analysis	The distinctions between participants belonging to guilty and innocent groups can be determined by examining their pupil dilation and reading behaviors. Crime statements were associated with shorter first-pass reading times compared to neutral statements. Participants who were found guilty exhibited a lower number of fixations when reading statements related to the crime they committed. Presented method was evaluated during field studies. It resulted in the correct classification of 83. 7% of innocent participants and 72.5% of guilty participants.
Bovard et al. [42]	2019	The Relevant Comparison Test	number of fixations, first-pass duration, reread duration, pupil diameter, and blink rate	RMANOVA	Participants from guilty group showed a decrease in fixation number and spent less time reading and rereading statements related to the crime they had committed compared to the control group. Another marker indicating information concealment was increased pupil diameter. Under the distributed condition, the decision model attained an accuracy of 84%, correctly identifying 90% of innocent participants and 78% of guilty participants. In the blocked condition, the accuracy rates were 76%, comprising 74% for innocent individuals and 78% for guilty ones.

for studies utilizing other test strategies.

The experiments presented in Table 1 proved that the CIT paradigm can be applied with different experimental scenarios that engage their participants in various tasks. Examinees can learn about relevant knowledge by committing mock crime [27,30] or by committing virtual crime (e.g., via watching videos presenting details of it) [33,37]. In some studies, participants’ tasks were associated with memorizing different kinds of stimuli and selecting them according to experimental assumptions.

To better approximate real settings, the examinees in [30] had to commit a mock theft. This study evaluated the influence of emotional arousal and the 1-week interval between ‘the crime’ and the examinations on CIT validity. Those issues are important regarding field conditions. Interestingly, the authors did not confirm that peripheral information can be reliably detected based on oculomotor measures, which is opposite to studies based on autonomic responses.

A different study that employed a mock crime scenario is referenced in [32], where the authors devised a simulated smuggling scenario. In this study, variations in pupil diameter over time and the duration of fixations on the center of the screen were assessed. The results indicate that oculometric behaviors undergo changes over time. Furthermore, these patterns can discern between truth-tellers and deceivers, even in the absence of relevant stimuli, which can be an important feature for the in-field application of the system.

Another scenario used the CIT together with a card game [39], before which participants were supposed to select two cards with different meanings (a probe to conceal during CIT-stage and a target to reveal during CIT-stage). The aim of this study was to answer questions about the cognitive mechanisms that underlie the validity of CIT based on different oculomotor measures. Findings from this research are broadly discussed in Section 4.

Based on the assumption that it seems impossible to conceal one’s knowledge of a face, some authors used the protocol with face pictures to establish usefulness of oculomotor measures to detect lies. The authors of [29] tested recognition and response intention effects by using three kinds of stimuli: known and selected stimuli, known but not selected stimuli, and unknown and not selected stimuli. The examinee task was to conceal the knowledge of friend (probe), reveal the knowledge of foe (target), and select unknown stimuli (irrelevant) during the observation of the set of six photos. The fixation duration for target stimuli was longer in the revealed condition as compared to the concealed display, which indicates the response intention effect in total fixation durations. In their next experiment, these authors replicated their previous results using objects instead of faces. Knowing that facial stimuli have different and more special properties than objects, they applied the task related to discriminating between objects of different categories during the CIT-stage, which was conducted in a delay condition. This time, the scenario was based on the virtual mock crime, whereas the use of the early memory effect within the first two fixations proves to detect concealed knowledge. Similar stimuli were used by [31] in their research, which successfully combined the response time (RT) measurement with pupil size and blinking rate measurement.

We also focused on research that introduces more accurate countermeasure resistant paradigms or applies the instruction for guilty participants to look at every familiar and unfamiliar face in the same way [27,28,35,38]. The results of the countermeasure instruction studies point out that this task attenuated the initial orienting response to the familiar face but still concealed that recognition was detected by overt avoidance of the familiar face. Instead of specific instruction, a new method was used, named rapid serial visual presentation (RSVP), to present stimuli on the fringe of awareness [38].

The experimental goals of the research presented in Table 2 were associated with establishing the accuracy of the reading test under different conditions. All of the studies were based on the mock crime scenario. To evaluate these methods for detecting deception in security screening contexts [40], participants in the ’guilty’ group committed one of two mock crimes. Additionally, manipulation with the participants’ incentive to pass the test and the difficulty of statements on the test improved the group discrimination. In the next study [42], the authors compared the accuracy of the reading test that directly asks if a examinee committed crime with the accuracy of the test that indirectly asks if the examinee provided false answers on a questionnaire about those crimes. The authors of [41] discussed the importance of factors that influence RCT accuracy: the goals for reading, the standard of evaluation used to read the statements, the cognitive mechanisms associated with executing the goals, and the emotional arousal elicited by the statements.

When it comes to data analysis, our review of the literature reveals that the majority of published research results to date have predominantly relied on classical statistical analysis, particularly employing various forms of the ANOVA test and the Area Under Receiver Operating Characteristic Curve measure. Some articles have also utilized machine learning classifiers such as the Support Vector Machine (SVM) [35]. ANOVA and its variations compare mean values between groups, primarily for statistical differences. Its results can identify characteristics as potential markers of deceptions. Most of the papers directly use these features to distinguish between groups. Conversely, a machine learning algorithm such as SVM can handle data, whether it is linearly separable or not, by using different kernel functions. The recent advancement of machine learning methods and the successful outcomes demonstrated by classifiers in diverse domains provide additional motivation to explore their implementation. Nevertheless, despite their potential to unveil hidden data relationships imperceptible to humans, further efforts are needed to enhance our understanding of these algorithms’ performance, particularly when applying them in high-risk domains. Moreover, the training dataset for ML models used in those domains has to be obtained with the utmost care as an biased dataset can lead to certain demographic groups being disproportionately impacted due to an unfair and discriminatory decisions.

4. Discussion

In recent years, oculomotor measures have revealed great potential in detecting deception. This overview of cognitive and emotional processes measured by the eye tracker aims to build, in the future, an automatic, non-invasive methodology for forensic identification based on ocular measures. A handful of studies based on CIT or reading behaviors (see Table 1 and Table 2) showed findings that are especially interesting for application purposes and enable the detection of deception without attaching sensors or electrodes to an examinee.

However, the forensic use of oculomotor tests requires the establishment of their reliable scientific foundation (scientific theory, objective methods of data analysis, and a known error rate) based on the results of laboratory studies and in-the-field testing. These issues occur in courts operating under the Daubert Standard and the Federal Rules of Evidence in the US or under equivalent laws in other countries. When assessing the forensic evidence based on the new method, the judge pays attention not only to the final conclusions of the forensic opinion but also to the method itself. This means that in the courtroom, questions may arise about the development of a given method in order to assess its level of credibility. Therefore, the more reliable research, the greater the knowledge about the new tool.

Laboratory research undoubtedly makes a significant contribution to our understanding of credibility assessment tools. Its advantage over field research is its knowledge of the ground truth. The laboratory detection of deception studies shows an increase in the ability of lie detection when: (1) the mock crime scenario “better mirrors” the real life situation [43], (2) the participants’ motivation is high, and (3) the scenario concerns high-stake lies [44]. Therefore, when researchers attempt to validate a new eye-tracker technique, designed studies should reflect the settings of real-life events more. We know from research on the polygraph that the classification accuracy of laboratory participants did not differ considerably from that of field participants [45].

Scientists suggest that eye tracking and polygraph tests in the detection of deception field might share similar theoretical frameworks in CIT, which are important aspects of the construct validity. They emphasize the importance of cognitive and emotional factors influencing different arousal conditions on critical and other questions in lie-detection tests.

The cognitive approach to lie detection is a general term for many techniques constructed to evoke differences in psychophysiological and oculomotor activity between truthful and deceptive people. The idea behind this approach is that lying is more cognitively demanding than simply being truthful. The cognitive view aims to amplify this gap by using stimuli with different meanings (relevant, neutral, and target) as well as creating experimental conditions that make an already difficult task even more challenging for the liar (e.g., using the “oddball”). Ideally, these conditions are expected not to influence the ability of the truthful person to provide an answer.

During CIT with narrowly chosen targets in highly controlled conditions and a standardized set of procedures, cognitive processes play a significant role in the explanation of the scientific basis for this test. The influence of cognitive mechanisms (depending on examinees’ attention and the stimulus salience) on the organism when answering different-meaning test questions manifests in various ways (a change in autonomic arousal, eye movement signals, and EEG).

The scientific assumption of the CIT is that the differential response is caused by the relevant items. This phenomenon has so far been explained by the orienting response (OR) theory. According to this theory, rapid response is triggered by a change of the subject’s attention towards a novel or significant stimulus. It is assumed that a guilty person who is familiar with a particular element related to the crime will show a greater OR for that element than for its neutral alternatives. In contrast, an innocent examinee who is not aware of any of the circumstances of the crime will not show a distinctive response to any of the stimuli.

The latest findings [39] suggest that in CIT the different ocular signals and polygraph measures may represent different cognitive mechanisms and support the underlying theoretical constructs. As assumed, the electrodermal response is mostly related to the cognitive processing of (i.e., the orienting to) significant information, whereas the respiration and HR both reflect attempts at suppression. In particular, the reduction in HR and respiration, which is usually seen in CIT, may be explained by inhibition theory. This concept posits that an examinee’s efforts to inhibit physiological arousal, induced by relevant stimuli, may cause suppression in HR and respiration.

How do cognitive factors affect ocular measures in CIT? Studies selected for our analysis investigated whether concealing knowledge about familiar faces or items affects gaze patterns. The results described in Table 1 suggest that during deception, observed stimuli in the form of images of familiar faces caused fewer fixations, fewer returns to observed areas, and less sampling of facial areas, which was the opposite of observing unfamiliar faces [28,34,36]. To hide the knowledge about the face, the examinees applied the active avoidance strategy that caused a greater number of fixations on unfamiliar faces [32]. The CIT effect also reduces the blink rate. Taken together with the latest response fractionation approach of the CIT, the results suggest that the CIT-related modulation of pupil size is caused by orienting reaction, while the reduction in the number of blinks and fixations is caused by inhibition.

Responses associated with the CIT effect, like a reduction in blinking and fixation, as well as an increase in the pupil size, have also been linked to cognitive load. However, ref. [39] doubt that cognitive load is present during the CIT and could explain the different responses in ocular measures on relevant and neutral questions. The authors find the cognitive load during the CIT experiment to be on the low level; while applying different conditions (concealing and revealing knowledge), participants reported cognitive effort on a similar level, and the ocular measures reacted significantly in a different way (an increase in pupil size was observed more in the reveal than in the conceal condition).

To cause concealing knowledge to be more cognitively demanding, during experiments based on CIT, the researchers used a modification of the traditional CIT procedure by introducing a new type of stimulus (target stimuli) and by changing the way stimuli were presented. Ref. [37] obtained the promising results by using modified CIT procedure with “oddball”, which, according to the authors, makes this test impossible to “beat” by examinees by applying countermeasures. The oculomotor-inhibition (OMI)-based CIT showed the best results among other studies with 100% accuracy; however, the research group was only 23 participants. Other studies describe the ability to detect knowledgeable participants using mentioned features that ranged from 65% [29] to 91% [35].

Additionally, the pupillary response is related not only to cognitive factors but also to emotional stimuli. The relationship between the magnitude of the pupillary response to emotional stimuli and the intensity of the latter has been shown by several studies. Investigators have also reported that during CIT and CQT polygraph tests, the pupil dilatation is greater during examinees’ deception attempts than when they are being truthful [46].

In addition to visual tests (CIT), the second methodology is being developed to explore the relationship between reading activity and the ability to detect deception. Ref. [42] stated that the four-factor theory of deception introduced by [47] may precisely explain the effects on ocular motor and other psychophysiological measures when analyzing gaze pattern during reading. This theory emphasizes that the differential reactions observed in deceptive participants are the result of changes in arousal, emotion, control and cognitive load.

People who answer truthfully interpret statements and then give the appropriate answers. Deceivers must complete an additional task: they must make a distinction between statements answered truthfully and deceptively. They are supposed to inhibit the truthful answer to give a deceptive one. In addition, they control themselves not to reveal their involvement in crime or illicit behavior. The activation of the cognitive mechanisms could contribute to the observed effects on ocular measures. The reading test also takes into account emotional factors related to the anxiety that liars have of being identified.

During the reading of truthful and deceptive statements in the Relevant Comparison Test (RCT), three features were measured: the number of fixes, the first-pass duration, and the reread duration [40]. Additionally, behavioral measures including the response times of participants and error rates could also be evaluated to detect deception [41]. The reading test makes the identification of truthful and deceptive response pattern possible based on cognitive load while responding to true and false statements.

In summary, the results of the laboratory studies converge to the conclusion that the RCT allows for the identification of truthful and deceptive examinees with an accuracy greater than 80%. The multiple-issue screening protocol, commonly used in security recruitment procedures, proved to be able to accurately discriminate between innocent and guilty examinees and identified which issue(s) elicited a deceptive response, with an accuracy of greater than 85%. The results of this classification were obtained using a subset of five eye-tracking measures and behavioral cues: the response time, the number of fixations, the PD peak amplitude, the PD area, and the PD level. The decision to select these characteristics from the initial set of 12 measures was based on the preliminary assessment of their statistical significance [48].

The analysis of the research results presented in the article allows us to conclude that the CIT and the reading tests could be interesting solutions used under different conditions. The first test can be used to determine whether the suspect recognizes, in photos, people associated with terrorist activities, which may help in establishing an examinee’s criminal connections. Additionally, eye-tracker technology allows for the presentation of many relevant crime stimuli during one examination in a short time. It is important that CIT has a clear theoretical basis that is easily understood by judges. In Japan, this test is presented as a memory detection test.

However, it should be noted that despite numerous effective studies, the CIT cannot always reliably detect concealed information and help to identify a guilty person. The main problem is the limited availability of information about the crime that can serve as critical stimuli for the test. Details of a given crime are often described in the media (e.g., how the perpetrator was masked, what car he drove away from the scene of the incident). Such details can no longer be used in the CIT because every person who has read about the crime in the media knows its details, e.g., from the Internet.

The next CIT limitation is a high level of false negative outcomes. In real cases, the perpetrator acts under stress, which can prevent him from remembering important details of the crime. Moreover, participation in many similar crimes may cause the so-called contamination of memory traces, which means that the perpetrator will not be able to connect the particular details of a crime with the particular crime.

The problems demonstrated above can be overcome by using a reading-activity-based technique like RCT. No visual material is needed to create exam tests. This technique is also more sensitive to detecting the guilty suspects than CIT. The most obvious limitation of the second technique is reading skills. Problems with the activity of reading and text understanding can affect the effectiveness of the test or make it impossible to conduct.

When analyzing the scientific findings, it is worth noting that test accuracy results are insufficient to ensure that eye-tracker tests based on CIT or reading behaviors will perform well for all examiners, examined people, and test circumstances, including those where it has not been used. Evidence of the scientific proof of oculomotor measurements is essential to provide confidence that a test measures what it is supposed to measure. Although the results of laboratory studies show high precision in detecting deception based on eye movement measurements, their reliability may still vary in real-world situations.

Unfortunately, this review has shown that the effectiveness of eye-tracking-based methods has been tested so far only in laboratory settings. The most serious issue is that the main target populations, such as criminal offenders, cannot easily be subjected to systematic testing. Moreover, studies in laboratory conditions enable the careful control of encoding processes and items needed for CIT examinations. In real situations, the selection of adequate stimuli remains a challenge. In the field, scientists still face the same problems to develop a new lie detection technology and defendable practice standards. One of them is the difficultly of accessing the ground truth of criminal cases (like confessions or juridical outcomes). Field research is also time-consuming, and the systematical collection of consistent data is challenging. Moreover, due to ethical issues, each examinee and the person arranging the test should be informed before it that forensic tests based on eye tracker are still methods in the nascent state. This may negatively influence the willingness to participate in such an examination.

Scientific eye tracker research should also identify external factors that can affect psychophysiological mechanisms and disrupt its results. During future studies, it is necessary to identify the measurement factors or artefacts mitigating the possibility of the correct detection of the signal of the measured phenomenon. Contrary to autonomic measures (electrodermal activity and heart rate), eye behavior can be changed to some extent consciously. For this reason, researchers focused their interest on the investigation of potential accuracy decreases related to the usage of CM.

Evidence has been found that CM affects physiological and ocular responses differently [27]. Although changes in fixations and blinks could be good indicators of the detection of crime-related knowledge, studies have shown that these measures were more easily altered by the use of CM techniques than by the autonomic nervous response.

5. Conclusions

The article presents a review of the literature and an analysis of research related to the use of eye tracking methods in the context of forensic deception detection. The psychological basis of testing and various aspects of the test procedure, including scenarios, data collection procedures, and their analysis, are discussed. Eye tracking accuracy information is summarized for exams based on CITs, as well as reading tasks. The potential use of eye tracking as a tool for forensic identification should take into account years of scientific and practical experience, as well as criticism directed toward polygraph examinations. One of the main objections against the polygraph is the subjectivity of its procedure, which can be eliminated by an automatic examination based on the measurement of the eye signals.

We hope that recent developments in precise measurements and neutral networks can help measure different eye signals and analyze these data in automatic lie-detection tests with the high level of accuracy required for criminal offenders’ identification. In our future work, we plan to focus on developing an approach based on the CIT procedure to increase the potential of the eye tracker in forensic science.