A Non-Inductive Magnetic Eye-Tracker: From Dipole Tracking to Gaze Retrieval

Abstract

We analyze the information that can be retrieved from the tracking parameters produced by an innovative wearable eye tracker. The latter is based on a permanent-magnet marked corneal lens and by an array of magnetoresistive detectors that measure the magnetostatic field in several positions in the eye proximity. We demonstrate that, despite missing information due to the axial symmetry of the measured field, physiological constraints or measurement conditions make possible to infer complete eye-pose data. Angular precision and accuracy achieved with the current prototypical device are also assessed and briefly discussed. The results show that the instrumentation considered is suitable as a new, moderately invasive medical diagnostics for the characterization of ocular movements and associated disorders.

1. Introduction

The localization of magnetostatic sources on the basis of multiple measurements performed in known, pre-assigned positions is widely studied and finds application in several areas, including medical diagnostics [1,2,3]. Setups based on permanent-magnet sources enable wireless measurements, which come with low invasivity and low cost. The developed methodology produces rich tracking data that include both target position and orientation and presents the advantage of an occlusion-free detection. Modern highly magnetized materials and solid-state magnetometric devices help to reduce the invasivity level and to improve the portability/wearability of the required instrumentation. We have recently demonstrated that this methodology can achieve sufficient time and space resolution to be applicable also to eye-tracking.

Concerning the instrumentation aimed to track ocular movements, devices with diverse degrees of invasivity, accuracy, speed, wearability have been developed on the basis of several concurrent technologies. The latter include electro-oculography [4], infrared-reflection devices [5,6], video cameras [7,8], and inductive magnetic receivers (scleral search coil, SSC) [9,10,11,12]. In this list, the former items are less precise and invasive while the latter is considered the top-performance at expenses of a noticeable invasivity. It is worth noting that the need of an electrical connection constitutes the main source of the SSC invasivity, and that unwired (double induction) SSC have been proposed [13,14,15], which lead to an invasivity level comparable to that of our non-inductive approach.

Our non-inductive magnetometric instrumentation for eye-tracking is characterized by an intermediate level of invasivity (much less than wired SSC), low intrusivity (it is fully wearable), low cost (<1 k€), high speed (>100 Sa/s), high precision (<1 degree), and good robustness with respect to external disturbances, such as eye-blinking, facial muscles actuation and fluctuations of ambient illumination. After a preliminary calibration of the sensors, no patient-based calibration is required to track the bare magnet-pose and its movements, while a patient-based calibration remains necessary in applications where the eye pose must be accurately gathered from the magnet pose.

Apart from some works aimed to localize multiple [16,17,18] or distributed [19,20] magnetic sources, most of the non-inductive magnetic trackers reported in the literature use some numerical algorithms to infer the magnetic target pose (position and orientation) modeling the magnet in terms of a dipolar field source. Our device does not make an exception to this common rule, but, differing from other setups, it tracks both the target pose and the ambient field. To this end, both the magnet and the dipolar field of the magnet and the homogeneous field of the Earth are taken into account in the field modeling. Consequently the data analysis provides both the dipole pose and the ambient field, making the latter no longer a disturbance term, but a source of additional information.

Like other single-target systems, our device provides comprehensive dipole pose information, which however is not sufficient to fully characterize the target object’s pose, even if simply modeled as a rigid body. More explicitly, the information that can be extracted from dipole tracking includes three spatial and two angular co-ordinates (the system being blind to rotations around the dipole direction), thus only 5 out of the 6 degrees of freedom (DoF) of a free rigid body can be retrieved.

Wireless detection of eye movements based on magnetometric measurements for eye-gesture estimation has been proposed as well, both in experimental animals (with surgically inserted magnets) [21,22,23] and humans (with magnetized contact lenses) [20].

This paper provides an analysis of the information that can be retrieved from the 5 DoF data produced by a dipole tracker. We specifically address the case of a dipole eye-tracker, in which physiological constraints and/or predetermined maneuvers offer complementary information and make the tracking output data sufficient to fully determine the eye pose.

The paper is organized as follows: after a brief description of the developed instrumentation and of the tracking parameters obtained from the data analysis (Section 2), we describe in Section 3 several approaches that enable a complete eye-pose retrieval from the dipole pose information. In the same section some examples of the retrieved gaze trajectories are reported. In particular we discuss how the dipole-gaze misalignment may affect the estimated gaze trajectory and how physiological constraints can be taken into account to improve the accuracy. Conclusion and perspectives are drawn in Section 5.

2. Setup

2.1. Sensors and Hardware

The hardware of the eye tracker is shown in Figure 1a. Its structure is extensively described in the Ref. [24]. The core of the device is made of eight three-axial magnetoresistive sensors [25] that generate a total of 24 data per measurement (each datum corresponds to one component of the total magnetic field—Earth field plus dipole field—measured in the position where the sensor is located). A microcontroller interfaces the sensors to a personal computer. The user may set sensitivity, acquisition rate, and operation mode. Operation modes exist for sensors’ precalibration and for tracking operation, which includes data recording and/or immediate analysis and visualization.

The sensors are included in highly integrated circuits, which also contain preconditioning electronics and analog-to-digital converters (14 bit, up to 200 Sa/s). They communicate with the microcontroller via eight independent I${}^2$C buses, while the microcontroller is connected to PC via a USB interface. This architecture helps accelerate the data transfer and makes possible to acquire synchronized measurements from all the sensors.

The magnetoresistances have a quite linear response, but their gains may differ from the nominal value causing an anisotropic response for each sensor and different responses from one sensor to the other. In addition, each magnetoresistance output is affected by a non-negligible offset. Thus, a pre-calibration procedure is necessary, to equalize the gains and to identify and subtract the offsets [24].

2.2. Data Elaboration to Extract Tracking Parameters

The main scope of the described instrumentation is to retrieve the eye orientation from the pose of a small magnet embedded in a scleral lens worn by the patient (Figure 1b). The position and the orientation of the target-magnet are inferred from a set of simultaneous field measurements performed in pre-assigned positions. The measured field is modeled as a dipolar one generated by the magnet, superposed to an external one (the Earth field), which is assumed to be homogeneous over the volume occupied by the sensors.

The determination of the magnet pose (orientation and position) and intensity (dipole modulus), plus intensity and orientation of the environmental field constitute an inverse problem, which is solved by means of numeric tools.

The size of the magnet (0.5 mm thick and 2 mm in diameter) is selected in such a way as to make its field on the sensors of the same order of magnitude as the Earth field (few tens of microtesla), which creates a good condition to accurately identify both the magnet pose and the environmental field.

Having equalized the readings to convert them in calibrated magnetometric data ${\overrightarrow{B}}_k^{\left(\mathrm{meas}\right)}$, solving the inverse problem requires to minimize the quantity:

$$S=\sum _k{\left|{\overrightarrow{B}}_k^{\left(\mathrm{meas}\right)}-B_k^{\left(\mathrm{mod}\right)}\right|}^2$$

(1)

where $k=0\dots 7$ is the sensor index, and

$${\overrightarrow{B}}_k^{\left(\mathrm{mod}\right)}(\overrightarrow{r},\overrightarrow{m})=\frac{{\mu }_0}{4\pi }\left(3\frac{[\overrightarrow{m}·({\overrightarrow{r}}^{\left(k\right)}-\overrightarrow{r})]({\overrightarrow{r}}^{\left(k\right)}-\overrightarrow{r})}{{\left|{\overrightarrow{r}}^{\left(k\right)}-\overrightarrow{r}\right|}^5}-\frac{\overrightarrow{m}}{{\left|{\overrightarrow{r}}^{\left(k\right)}-\overrightarrow{r}\right|}^3}\right)+{\overrightarrow{B}}_{geo},$$

(2)

the field expected on the basis of the dipole model, in the position ${\overrightarrow{r}}^{\left(k\right)}$ of the kth sensor, being $\overrightarrow{m}$ a magnetic dipole placed in the position $\overrightarrow{r}$. The term ${\overrightarrow{B}}_{geo}$ accounts for the uniform environmental field. The positions $\left\{{\overrightarrow{r}}^{\left(k\right)}\right\}$ are referred to the sensor frame, which is rigidly connected to the patient’s head. The best fit procedure minimizes S and outputs the estimates of $\overrightarrow{r}$, $\overrightarrow{m}$, ${\overrightarrow{B}}_{geo}$, for a total of nine tracking parameters. As discussed in Ref. [26], under some conditions, the number of fitting parameters can be reduced to eight after having determined the modulus of $\overrightarrow{m}$, while it is definitely inopportune to assume a fixed value for $|{\overrightarrow{B}}_{geo}|$, because the environmental field is commonly enough homogeneous on the volume of the array, but can vary significantly when the array is moved within the room where the system operates.

We have investigated the reliability of ordinary best-fitting procedures obtaining encouraging results [26]. In particular, we verified that a Levenberg–Marquardt minimization algorithm is sufficiently fast and adequately robust with respect to the initial guess: there exists a volume larger than the region in which the magnet can move, such that any calculation starting from a dipole guess in that volume, systematically converges to the correct solution, regardless of the initially assigned orientation.

2.3. From Tracking Parameters to Eye (and Head) Pose

Once the minimization algorithm outputs the estimate of $\overrightarrow{r}$, $\overrightarrow{m}$ and ${\overrightarrow{B}}_{geo}$, the pose of eye and head can be inferred. While $\overrightarrow{r}$, $\overrightarrow{m}$ provide information of the eye pose with respect to the sensor array, the measure of the environmental field can be used as a three-dimensional compass to evaluate the absolute orientation of the sensor array, i.e., of the patient’s head. Combining relative eye-gaze and head orientations enables the determination of the absolute gaze [27] direction. Notice that the system is non-sensitive to head translation, thus the retrieval of the gaze direction is not sufficient to identify a fixation point at a finite distance.

In addition, the comparison of eye and head angular motions can be used to evaluate the eye actuation in response to head rotation, such as in measurements of vestibulo-ocular reflex (VOR) [28] (see also Section 3.3). However, this paper focuses on the analysis of the eye pose relative to the head, and we are not considering further the retrieval of head pose from ${\overrightarrow{B}}_{geo}$.

The orientation of $\overrightarrow{m}$ is related to eye rotations, but it does not provide the gaze direction directly, unless the magnetic dipole is parallel to the optical axis of the eye (let the latter be $\widehat{e}$). Eye rotations around the $\overrightarrow{m}$ direction would not be revealed. This has a twofold consequence: if $\overrightarrow{m}\Vert \widehat{e}$, torsional angular displacements are not detected; if $\overrightarrow{m}\nparallel \widehat{e}$, some extra assumptions are necessary to infer $\widehat{e}$ from $\overrightarrow{m}$, and, in particular, rotations around $\overrightarrow{m}$ would affect $\widehat{e}$ without being detected.

It is worth recalling that besides $\overrightarrow{m}$, the best-fit procedure outputs the magnet position $\overrightarrow{r}$, which provides additional information about eye motion. However, the position co-ordinates are known with respect to the sensor reference frame and not to the eye center, thus the eye gaze cannot be retrieved directly from $\overrightarrow{r}$. Despite this feature, the partially redundant information given by $\overrightarrow{r}$ could be used in conjunction with that extracted from $\overrightarrow{m}$ to improve or to complete the eye pose reconstruction, as it will be discussed in Section 3.6.

3. Gaze Retrieval

In this section, we examine some procedures that can be followed to retrieve the eye gaze with respect to the sensor frame from the tracker output, more specifically from the estimates of $\overrightarrow{m}$ and $\overrightarrow{r}$. Eye translations, that can in principle be studied with an adequate analysis of $\overrightarrow{r}$ will not be taken into consideration, and we will focus on eye-ball rotations, modeling the eye as a rigid sphere rotating about its (fixed) center. Although mathematically unnecessary, this will turn into a useful simplification of the following $\overrightarrow{m}$-related discussion with no further hypothesis or loss of information. Moreover, the fixed-center rigid sphere model applies to the $\overrightarrow{r}$ information discussion in the realistic approximation of very small displacements of the eye-ball center compared to the eye radius.

It has to be pointed out that the pose of a rigid body freely rotating about an assigned point requires three angular parameters to be fully defined. In other terms, generally speaking, the faced problem has 3 degrees of freedom (DoF). The modulus of the magnetic dipole is nominally constant, and the tracking uses its direction $\widehat{m}$ only, thus the analysis of the unitary vector $\widehat{m}$ provides only bidimensional information, e.g., the zenithal and azimuthal angles, when described in spherical co-ordinates. These 2D data are not sufficient to determine the 3 DoF orientation of the eye-ball: it is necessary to consider reduced-DoF problems to achieve complete pose determination.

3.1. Small-Angle and Near-Front Sight Approximation

A simple case is obtained in the hypothesis that torsional movements can be neglected and that the angle between the gaze and the front direction ($\widehat{z}$ in the notation of our setup, see Figure 1) is maintained small.

In this hypothesis the current gaze ${\widehat{e}}_i$ is always nearly parallel to $\widehat{z}$ and it can be expressed as the rotation of the front gaze direction ${\widehat{e}}_0$, ${\widehat{e}}_i=\mathbf{R}{\widehat{e}}_0$:

$$\mathbf{R}=\left(\begin{array}{ccc}1& 0& {\varphi }_y\\ 0& 1& {\varphi }_x\\ -{\varphi }_y& -{\varphi }_x& 1\end{array}\right)$$

(3)

the first-order approximation of a rotation matrix whose parameters ${\varphi }_x,{\varphi }_y$ represent small rotation angles around the x and the y axes, respectively. The same rotation applies to the magnetic moment ${\overrightarrow{m}}_0$, as it is rigidly connected to the eye-ball. Thus, ${\widehat{m}}_i=\mathbf{R}{\widehat{m}}_0$ and the two angles ${\varphi }_x,{\varphi }_y$ can be retrieved from this last relation.

In summary, here ${\widehat{e}}_0$ is the front sight gaze, ${\widehat{m}}_0$ the corresponding orientation of the dipole, and ${\widehat{e}}_i$ the gaze at the ith measurement, to be inferred from the estimated dipole ${\overrightarrow{m}}_i$. In this approach, ${\widehat{e}}_0$ is assumed to coincide with $\widehat{z}$, and ${\widehat{m}}_0$ should be known from an independent characterization of the magnet in the scleral lens, or tentatively assigned as it will be explained below.

3.2. Larger Rotations about the Front ($\widehat{z}$) Direction

A more complex case occurs when the first-order approximation is not applicable. If so, even with the simplifying hypothesis ${\widehat{e}}_0\Vert \widehat{z}$, the eyeball rotation cannot be, in general, resolved as a sequence of two rotations around arbitrary axes. As we will see in the following, further information is needed in this case, such as a known rotation axis or a model for the eye movements. As an intuitive proof of this, consider the two-angle rotations around the pre-set Cartesian axes x and y defined by ${\mathbf{R}}_{yx}={\mathbf{R}}_y{\mathbf{R}}_x$ or ${\mathbf{R}}_{xy}={\mathbf{R}}_x{\mathbf{R}}_y$. These have the general form

$$\begin{array}{cc}\hfill {\mathbf{R}}_{yx}& =\left(\begin{array}{ccc}cos{\varphi }_y& -sin{\varphi }_{x}sin{\varphi }_y& cos{\varphi }_{x}sin{\varphi }_y\\ 0& cos{\varphi }_x& sin{\varphi }_x\\ -sin{\varphi }_y& -sin{\varphi }_{x}cos{\varphi }_y& cos{\varphi }_{x}cos{\varphi }_y\end{array}\right),\hfill \\ \hfill {\mathbf{R}}_{xy}& =\left(\begin{array}{ccc}cos{{\varphi }_y}^{\prime }& 0& sin{{\varphi }_y}^{\prime }\\ -sin{{\varphi }_x}^{\prime }sin{{\varphi }_y}^{\prime }& cos{{\varphi }_x}^{\prime }& sin{{\varphi }_x}^{\prime }cos{{\varphi }_y}^{\prime }\\ -cos{{\varphi }_x}^{\prime }sin{{\varphi }_y}^{\prime }& -sin{{\varphi }_x}^{\prime }& cos{{\varphi }_x}^{\prime }cos{{\varphi }_y}^{\prime }\end{array}\right),\hfill \end{array}$$

(4)

and both converge to the above operator (3) in the small angle approximation. On the other hand, they represent intrinsically different operators for large angles. It is worth noting the two operators coincide if ${\varphi }_x={{\varphi }_x}^{\prime }=0$ or ${\varphi }_y={{\varphi }_y}^{\prime }=0$, that is if one of the two x or y axes happens to be the unique rotation axis. This has a practical utility that will be discussed in Section 3.3.

Indeed, the two angular data extracted from the direction of $\widehat{m}$ are not sufficient to determine uniquely the 3-DoF configuration, and the selection of a particular rotation order corresponds to arbitrarily assign the missing information. Such an approach allows for producing tentative eye gaze trajectories, but the solution is not stringent and diverse choices results in diverse and variously distorted trajectories. Nevertheless, also with this under-specified analysis, the main features of the gaze trajectories can be identified, which can be satisfactory in some eye-tracking applications.

As an example, in Figure 2 we represent tracking results obtained in an in vivo experiment, using the ${\mathbf{R}}_{xy}$ and ${\mathbf{R}}_{yx}$ operators alternatively. The subject was requested to read a short text (title, a complete line, a half-line) on a monitor and then to follow the monitor frame. The four represented trajectories are obtained from the same dataset, with an analysis of the dipole orientation based upon the two operators, respectively. In this elaboration, one selects (tentatively) the ${\widehat{m}}_0$ orientation, i.e., the trajectory point to which the $({\varphi }_x=0,{\varphi }_y=0)$ co-ordinates are being assigned in such a way to obtain as straight lines and right angles as possible. As one can see, in all cases features such as text line distribution and small displacements around the running line of sight are well tracked, while distortions cannot be canceled and affect long segments that should be straight and angles at the monitor corners that should be right.

The subject was sitting in front of the monitor so that ${\widehat{e}}_0$ is expected to coincide with (to be in the proximity of) the measurement tracked at the monitor center. Two of the trajectories—(a) and (c)—are obtained in this hypothesis, assigning ${\widehat{e}}_0$ to the central point of the figure (thus selecting the corresponding $\widehat{m}$ as ${\widehat{m}}_0$). The other trajectories—(b) and (d)—are instead obtained with a wrong assignment (${\widehat{e}}_0$ in coincidence of the upper-left corner, about 20 degree away from the nominal direction). As can be seen, the main features of the trajectory are well reproduced also in this case, which demonstrates robustness with respect to the $({\widehat{e}}_0,{\widehat{m}}_0)$ assignment.

Is worth reminding that in principle, also the ${\widehat{e}}_0$ could be at some angle from $\widehat{z}$; however, this angle amounts at less then 10° and it would cause minor changes to the plots shown in Figure 2.

3.3. Large Rotations about an Assigned Axis

The issue related to the noncommutativity of finite 3D rotation is overcome in a preeminent application of the eye-tracking systems. In VOR measurements, the head of the patient is rotated around a predetermined axis, which is usually vertical (pitch) or horizontal (yaw), the expected (ideal) response of the eye is an opposite (compensating) rotation around the same axis, and this is the quantity to be estimated and analyzed.

In this case, the data are to be interpreted in terms of a single rotation angle around one axis. That axis can be assumed to coincide with the nominal one that is approximately either the $\widehat{x}$ or the $\widehat{y}$ direction defined in Figure 1 in the cases of pitch or yaw, respectively.

Improved accuracy is achieved if the actual rotation axis is identified on the basis of the ${\overrightarrow{B}}_{geo}$ measurement set {${\overrightarrow{B}}_i$} ($i=1,\dots ,N$) recorded during the maneuver. For instance, one can find the common plane intercepting the measured ${\overrightarrow{B}}_i$ vectors (with a mean square distance minimization). The perpendicular direction to the plane, $\widehat{u}$, provides an estimate of the rotation axis direction [29,30]. Subsequently, both the head and eye rotations can be evaluated around the determined direction. The eye rotation angle can be estimated from two measured values ${\overrightarrow{m}}_i$ ($i=1,2$) as

$${\varphi }_{u,1}-{\varphi }_{u,2}=arcsin\frac{|{\overrightarrow{m}}_{1,\perp }\times {\overrightarrow{m}}_{2,\perp }|}{|m_{1,\perp }\left|\right|m_{2,\perp }|}$$

(5)

where ${\overrightarrow{m}}_{i,\perp }={\overrightarrow{m}}_i-({\overrightarrow{m}}_i·\widehat{u})\widehat{u}$ and similarly—using ${\overrightarrow{B}}_{geo}$—for the head.

This perfecting produces just a slight enhancement of the accuracy, if the misalignment between ideal and actual rotation axes is small, i.e., $\widehat{u}\approx \widehat{x}$ or $\widehat{u}\approx \widehat{y}$. Whatever the actual rotation axis is, in this application eye and head rotations never exceed $\pi /2$, which makes unnecessary to refine Equation (5) for out-of-range arcsine issues. An alternative method to determine the rotation axis is fitting a 3D circle on the ${\overrightarrow{B}}_i$, as $\overrightarrow{B}$ moves nominally on a circle at constant zenithal angle with respect to the rotation axis [31]. To this end, we have implemented a procedure that: (i) applies a rotation matrix parametrized with a couple of Euler angles $(\alpha ,\beta )$ to a set {${\overrightarrow{B}}_i$} or {${\overrightarrow{m}}_i$}; (ii) represents the data in spherical co-ordinates; (iii) determines the couple $(\alpha ,\beta )$ as that minimizing the standard deviation of the zenithal angles.

It is worth noting that assuming that the axes of rotation of the head and eye coincide is a popular but coarse approximation. Indeed, considering the case of a generic rotation, the VOR gain is not isotropic [32,33], so that—also in the simplified assumption of a linear and non-delayed response—the VOR gain should be expressed tensorially. E.g., focusing on the angular velocities, the relationship between eye and head speed would be expressed as ${\overrightarrow{\omega }}_e=\mathbf{G}{\overrightarrow{\omega }}_h$, where $\mathbf{G}$ is a $3\times 3$ matrix. $\mathbf{G}$ is expected to be a non-diagonal matrix, implying that eye and head rotation axes are generally not parallel. A correct and complete analysis of the VOR gain involves the determination of all the elements ${\mathbf{G}}_{ij}$, which (at least in principle) can be obtained from the above described data.

As an additional remark, let us point out that our prototype contains only magnetometric sensors; however, improved setups could be designed to include gyroscopic detectors. In this case, additional and complementary estimates of the head rotations would be available, with improved accuracy in VOR gain estimations.

3.4. Physiological Constraints

The arbitrariness of the rotation matrix introduced in Section 3.2 (Equation (4)) can be avoided if the DoF reduction is performed on the basis of physiological constraints of the eye. Torsional movements of the eye are strongly depressed and in a good approximation the eye movements respect a 1D constraint enunciated in the 19th century and named Donders’ law. Abundant literature is available on this subject. The reader may find a useful introduction to the matter in Ref. [34] and more mathematical details in Ref. [35]. Donders’ law states that each direction of the gaze corresponds to a single orientation of the eye, regardless of the trajectory that led to the corresponding eye pose. In other words, each direction of view corresponds to a unique eye configuration and torsional movements around the current line of sight do not occur or can be neglected.

As far as the Donders’ law is respected, the eye movements have only two DoF, hence it is possible to search for a biunique relationship between $\widehat{m}$ and $\widehat{e}$. A second phenomenological law that rules the eye movement is known as Listing’s law. It suggests a good parametrization to describe these two-DoF configurations allowed by the Donders’ law.

According to the Listing’s law, there exists a direction (primary orientation) ($\widehat{p}$) such that any eye configuration can be described as if reached moving from that primary orientation by means of a single rotation around an axis perpendicular to $\widehat{p}$, as sketched in Figure 3.

In other terms, making reference to a spherical co-ordinate system having the polar axis along $\widehat{p}$, all the eye configurations can be reached with one opportune zenithal rotation from the primary orientation: the rotation axes lay on the equatorial plane of that co-ordinate system, the so-called Listing’s plane. When the head is erect and the eye is at the primary orientation, the gaze is approximately straight ahead, thus the Listing’s plane is not far from being vertical.

Summarizing, the Listing’s law reduces the DoF of eye configuration to two: each eye configuration can be considered as the result of a rotation by a given zenithal angle around an axis $\widehat{u}$ that lays on the Listing’s plane and has a given orientation (azimuthal angle) on that plane. The data elaboration needed to retrieve ${\widehat{e}}_i$ from ${\widehat{m}}_i$ consists of the determination of those two angles from the current orientation of the dipole (${\widehat{m}}_i$) and its orientation (${\widehat{m}}_0$) when the gaze is along the primary direction. This problem is focused in next Section 3.5.

3.5. Applying the Listing’s Law to the Present Case

In our implementation, $\widehat{p}$ is approximately antiparallel to $\widehat{z}$. Let $\widehat{p}$ be known or tentatively assigned (see below). First of all we refer the tracking data to a rotated coordinate system $XYZ$ (see Figure 4) such that $\widehat{p}=-\widehat{Z}$. To this end, we determine the Rodrigues’ matrix [36,37] that leads $\widehat{z}$ to coincide with $\widehat{Z}=-\widehat{p}$ and we use this matrix to represent the measurements ${\overrightarrow{m}}_i$ in these new co-ordinates $XYZ$, where the Listing’s plane is the $XY$ plane. Note that the reference axis X can be arbitrarily chosen on the Listing’s plane, since all data of interest are related to angular displacements (and not to absolute directions).

Let ${\widehat{m}}_0$ and ${\widehat{e}}_0$ be the magnet and the eye directions corresponding to the primary orientation: by definition, ${\widehat{e}}_0=-\widehat{Z}$, while ${\widehat{m}}_0$ must be known or tentatively assigned. The task is now to determine the orientation ${\widehat{e}}_i$ corresponding to the measured ${\widehat{m}}_i$ on the basis that ${\widehat{e}}_i=R_{{\phi }_i,{\widehat{u}}_i}{\widehat{e}}_0$ and ${\widehat{m}}_i=R_{{\phi }_i,{\widehat{u}}_i}{\widehat{m}}_0$, where $R_{{\phi }_i,{\widehat{u}}_i}$ is a rotation by an angle $\theta$ around the direction $\widehat{u}$ that lies on the $XY$ plane (again, the Rodrigues’ formula can be used to determine the corresponding matrix).

$R_{{\phi }_i,{\widehat{u}}_i}$ can be uniquely determined from ${\widehat{m}}_0$, ${\widehat{m}}_i$ as follows. The rotation axis $\widehat{u}=(u_X,u_Y,0)$ is perpendicular to the projection of $\Delta \widehat{m}={\widehat{m}}_0-{\widehat{m}}_i$ on the $XY$ plane: $\widehat{u}·(m_{0X}-m_{iX},m_{0Y}-m_{iY},0)=0$. Now consider the projections of ${\widehat{m}}_0$ and ${\widehat{m}}_i$ on the plane perpendicular to $\widehat{u}$: ${\phi }_i$ the angle between those projections can be evaluated with the method described by Equation (5). The retrieved $\widehat{u}$ and ${\varphi }_i$ give the axis and the rotation angle, respectively, in the Listing’s model of the eye.

When $\widehat{p}$ and/or ${\widehat{m}}_0$ are not known a priori, they can be tentatively assigned and their appropriateness can be evaluated on the basis of the distortion of the reconstructed gaze trajectory corresponding to regular and known shapes. This is the case of the trajectory represented in Figure 5. The same recording used for Figure 2 is here processed according to the above described Listing’s model procedure. A direct comparison with the trajectories shown in Figure 2 puts in evidence a clear reduction of distortion effects. On the basis of this observation, we conclude that the Listing’s law makes it possible to develop a good procedure to accurately retrieve the eye-gaze, despite the $\widehat{m}-\widehat{e}$ misalignment.

A hardware solution based on aligning the magnetic dipole to the visual axis of the eye would greatly simplify the task of retrieving the gaze from the measured $\widehat{m}$. Unfortunately the visual axis is not known a priori, and a patient based calibration would be anyhow needed. The visual axis is generally oriented at small angle from the optical axis (the so called angle kappa amounts typically to about 5° [38]), thus a compromise solution may consist of aligning $\widehat{m}$ to the optical axis and subsequently use the Listing’s model described above for refinements. To this aim, the magnetic disc should be inserted out of the lens axis (to be non-obstructive for the view) and tilted with respect to the lens surface, an arrangement that comes with some geometrical issues, as sketched in Figure 6b,c.

In Figure 6b, for instance, ${\widehat{e}}_0$ and ${\widehat{m}}_0$ come to a complete identity and no reciprocal rotation has to be measured or estimated. Figure 6c, instead, opens new possibilities by allowing direct measurements of the torsional movement of the eye. These interesting perspectives are beyond the scope of the present work and will be subject of further investigation.

3.6. Retrieving the Eye Gaze from the Magnet Position

An alternative approach that could enable the estimation of the eye orientation with respect to the sensor array is based on the analysis of the magnet position. As said (Section 2.2), the best fit procedure provides, beside ${\overrightarrow{B}}_{geo}$ and $\overrightarrow{m}$, also the position $\overrightarrow{r}$ of the dipole. As far as the eye is adequately described in terms of a fixed-center sphere, $\overrightarrow{r}$ is expected to move on a spherical surface. The radius of that sphere is the eye radius (12 mm, as a typical value), and its center is located at the eye center ${\overrightarrow{R}}_e$, which is (at least initially) an unknown position.

In principle (see Figure 7), one could first determine ${\overrightarrow{R}}_e$ fitting a large set of {${\overrightarrow{r}}_i$} on a sphere and then use the quantities {${\overrightarrow{s}}_i={\overrightarrow{r}}_i-{\overrightarrow{R}}_e$}, similarly to what is conducted with $\overrightarrow{m}$.

More interestingly, provided that $\widehat{m}$ e $\widehat{s}$ are not parallel, it would be possible to use both to determine the eye orientation with no need for extra hypotheses. Indeed, $\widehat{m}$, $\widehat{s}$ (e.g., together with $\widehat{m}\times \widehat{s}$) would constitute a basis in the 3D space, hence the matrix R such that ${\widehat{m}}_i=R{\widehat{m}}_0$ and ${\widehat{s}}_i=R{\widehat{s}}_0$ would be uniquely determined (even over-determined) from two subsequent estimates of $\widehat{m}$ and $\widehat{s}$.

Physiological constraints prevent $\overrightarrow{r}$ from ranging over a sufficiently wide portion of the spherical surface and this results in a rather rough estimation of ${\overrightarrow{R}}_e$. As a consequence, these alternative approaches—at least with the precision achieved in current implementation—do not help improve the quality of the gaze estimations.

An additional approach based on the analysis both $\overrightarrow{r}$ and $\overrightarrow{m}$ might follow the methodology presented in Refs. [39,40] for video-oculography, where the gaze is determined on the basis of the position of the pupil and of the eye glints, after opportune calibration. In the present case, apart from offsets to be determined by an additional calibration, $\overrightarrow{r}$ and $\overrightarrow{m}$ would provide analogous information as the pupil position and glints, respectively.

4. Instrumental Uncertainties

By operating with a single rotation axis, as described in Section 3.3, facilitates the quantification of the intrinsic instrumental precision and accuracy (that will affect the tracking, regardless of the methods used to infer $\widehat{e}$ from $\overrightarrow{m}$), as set by the sensitivity of the detectors and by the operation conditions. To this end, we have performed several experiments aimed to provide estimates under typical operation conditions.

It is worth recalling that the analyses of these in-vitro measurements do not require any calibration, apart from the pre-calibration that is necessary to compensate non-ideal offsets and gains of the single sensors (see [24] for details). This equalization must be performed on each sensor array after its assembling, and is advantageously repeated if the pre-calibration is suspected to have become inaccurate, e.g., after several weeks, in case of relevant changes in temperature, in case of accidental exposure to strong magnetic fields etc.

The plots shown in Figure 8 and Figure 9 are obtained from the elaboration of data recorded with a mechanical system based on a numerically controlled motor. The magnet is rotated in steps around the z axis along a circular orbit 8 mm in radius on the $z=10$ mm plane (a region compatible with the eye positions), with the dipole radially oriented. A complete rotation (360°) is performed in 51 steps (7.06° each), and one hundred trackings are performed in static conditions after each rotation step.

The data are then analyzed as described in Section 2.3 and the rotation angle around the z axis is inferred according to Equation (5). The two plots in Figure 8 show the whole set of measurements and a zoomed portion of it, respectively.

The plot in Figure 9 reports the deviation from expected to measured angles, i.e., the difference between the average values of the angle retrieved at each step and the corresponding nominal angle (assigned by the stepped motor), as a function of this latter. The error bars represent the standard deviation of the corresponding data sub-sets, and hence the precision. The non-zero value of the deviation exceeds the error bars, pointing out a limited accuracy due to systematic errors. These latter are likely to be ascribed to inhomogeneities of the ambient field and imperfect calibration of the sensors.

The above presented measurements were performed while maintaining the system steady with respect to the ambient field. This exclude the effect of covariance between different tracking parameters, which instead can be source of additional uncertainties. Specifically, instrumental errors in the estimation of $\overrightarrow{m}$ can be evaluated when the whole system is rotated in the ambient field and, complementarily, errors in the estimation of ${\overrightarrow{B}}_{\mathrm{geo}}$ can be evaluated while moving the magnet with respect to the steady sensors. Performing these experiments in our working conditions (operating in a normal room, with some care in avoiding ferromagnetic furniture in the proximity) we obtained typical RMS fluctuations of the order of 0.3° within maximum ranges of about $\pm 1$°, for both the vectors, as detailed in

Table 1. The first two lines report the apparent rotation of $\overrightarrow{m}$ when the system is freely moved in the room and the magnet is in the $z=10$mm plane, oriented along z, in a fixed position with respect to the sensor frame. Rotations around x and y are evaluated accordingly to Equation (5) (the system is blind to rotations around z). The results are analyzed in terms of standard and maximal deviation. Conversely (last three lines), the apparent rotation of ${\overrightarrow{B}}_{\mathrm{geo}}$ is evaluated when the sensor array is fixed and the magnet (oriented along z) moves on a circle 10mm in radius on the $z=10$mm plane. These estimates are just exemplifying numbers, as the results depend on several features of the working conditions (ambient field inhomogeneities, accuracy of the sensor calibration, free motion applied to the system) that assume typical but non-reproducible values.

Quantity	St. Dev.	Range
${\varphi }_x$	0.23°	$\pm 0.98$°
${\varphi }_y$	0.28°	$\pm 0.75$°
${\theta }_x$	0.33°	$\pm 0.50$°
${\theta }_y$	0.25°	$\pm 0.85$°
${\theta }_z$	0.18°	$\pm 1.0$°

both in terms of RMS and maximal deviations.

It is important to take into account that other spurious phenomena may occur in the in vivo application and that they can relevantly affect the ultimate precision and accuracy. In particular, similarly to the case of SSC technology, slippage of the lens with respect to the eye constitute an error source. Furthermore, slippage of the sensor array with respect to the head must be carefully avoided. The latter is particularly relevant in head-impulse-test maneuvers for VOR-gain estimations. Future prototypes are being designed with a lightened sensor frame and a mechanically separated microcontroller PCB: a reduced inertia will help counteract such sensor-head slippage issue.

Furthermore, the rapid decay of the dipolar field with the distance, makes the magnet-sensor distance a crucial parameter to record data with a good signal to noise ratio. In in vivo application, physiological constraints hinder the task of maintaining the sensors at a short distance from the eye: an accurate design of the sensor frame is planned to this end, and it will greatly help improve the performance.

5. Conclusions

An innovative eye-tracker is developed, based on non-inductive magnetometric measurements simultaneously performed at 100 Sa/s rate in a set of prearranged positions with the aid of an array of magnetoresistive sensors. This kind of instrumentation extends to the eye-tracking some advantageous features of similar magnetostatic trackers developed for other applications.

Moderate invasivity, high speed, spatial and angular accuracy, robustness, simplicity, and low cost emerge as attractive characteristics in comparison with other eye-tracking technologies.

The instrumentation is in a prototypical stage and several refinements will be needed to improve its performance to the level achieved by other (mature) competing technologies and hopefully beyond. In particular, some technical issues (such as those related to scleral lens and/or sensor array slippage) should be addressed to improve precision and accuracy. There is also room for other improvements, e.g., to provide wireless data transfer or to simultaneously acquire gyroscopic data as well.

Despite position and orientation of the magnet are fully characterized, the conversion of magnet tracking data to gaze information is not immediate and may require specific procedures. In this paper, we have examined the problem of retrieving the eye gaze from the magnet orientation when optical axis and dipole are not parallel.

In particular, we have demonstrated how the combination of tracking parameters with physiological constraints of the eye motion enables an accurate reconstruction of the eye-gaze trajectories, while alternative simplified analyses can be implemented and—despite some distortion effects—they enable the extraction of basic features of the gaze trajectory.

6. Patents

A patent [41] is pending about inventions related to this research.