Binary Controller Based on the Electrical Activity Related to Head Yaw Rotation

Abstract

A human machine interface (HMI) is presented to switch on/off lights according to the head left/right yaw rotation. The HMI consists of a cap, which can acquire the brain’s electrical activity (i.e., an electroencephalogram, EEG) sampled at 500 Hz on 8 channels with electrodes that are positioned according to the standard 10–20 system. In addition, the HMI includes a controller based on an input–output function that can compute the head position (defined as left, right, and forward position with respect to yaw angle) considering short intervals (10 samples) of the signals coming from three electrodes positioned in O1, O2, and Cz. An artificial neural network (ANN) training based on a Levenberg–Marquardt backpropagation algorithm was used to identify the input–output function. The HMI controller was tested on 22 participants. The proposed classifier achieved an average accuracy of 88% with the best value of 96.85%. After calibration for each specific subject, the HMI was used as a binary controller to verify its ability to switch on/off lamps according to head turning movement. The correct prediction of the head movements was greater than 75% in 90% of the participants when performing the test with open eyes. If the subjects carried out the experiments with closed eyes, the prediction accuracy reached 75% of correctness in 11 participants out of 22. One participant controlled the light system in both experiments, open and closed eyes, with 100% success. The control results achieved in this work can be considered as an important milestone towards humanoid neck systems.

1. Introduction

Advances in cognitive engineering support the design, analysis, and development of complex human–machine–environment systems [1]. Recently, special interest has been devoted to the identification of the cognitive human reaction to an adaptive human computer interaction (HCI) [2] or to environment perception by the human. In HCI research, brain–computer interface (BCI) systems represent one of the most challenging research activities. Reviews about BCI and HMI can be found in [3,4]. BCI technologies monitoring brain activities may be classified as either invasive or noninvasive. Invasive BCI technologies generally consist of electrodes implanted in the human cortex. Noninvasive BCI technologies usually refer to electrodes positioned on the scalp to perform electroencephalography (EEG) or magnetoencephalography (MEG). EEG and MEG can, respectively, monitor the electrical and the magnetic fields generated by neuronal activities.

This work relates to the EEG used for the recognition of the head movement in a human participant. A survey on the EEG in this context can be found in [5]. The EEG is one of the most promising tools to monitor brain cognitive functionality due to its higher temporal precision with respect to other tools such as Functional Magnetic Resonance Imaging (FMRI), which presents higher spatial resolution [6]. EEG-based biometric recognition systems have been used in a wide range of clinical and research applications, among others: interpreting humans emotional states [7]; monitoring participants’ alertness or mental fatigue [8]; sleep stage identification [9]; checking memory workload [10]; finding brain area of damage in case of injury or disease [11]; automated diagnosis of patients with learning difficulties [12]; and, in general, diagnosing brain disorders [13]. EEG monitoring, analysis, and processing are useful techniques to identify intentional cognitive activities from brain electrical signals. Some studies investigated the event-related potential when the decisions of the monitored participants are affected by external stimuli during workload sessions. In fact, the workload performance does not only depend on the operator but also on the surrounding environment and on their interactions. A cognitive stimulus represents an event that involves the subject while performing different planned tasks during EEG data collection. Several recent examples may be considered. The evaluation of a driver’s behavior [14,15] or drowsiness [16,17] by analyzing the EEG signals has been conducted. Specifically, a Multitask Deep Neural Network (MTDNN) to distinguish between an alert and drowsy state of mind was set up in [18]. To prevent road accidents, the driver’s fear detected by EEG signals was evaluated when he/she was subjected to unpredictable acoustic or visual external stimuli [19]. In the automotive context, the interaction between the driver and an onboard Human Machine Interface (HMI) may support safer driving, especially in critical situations [20]. Emergency braking, a rapid change of lane, or avoiding an obstacle on the route can also be detected by the driver’s brain in advance with respect to the vehicle dynamic, and this contribution may improve the quality of the driver’s intent prediction [21] and overall system safety [22].

BCI technologies aim at converting mental activities of the human participant into electrical brain signals, producing control command feedback to external devices such as robot systems [23]. In addition, BCIs may also include paradigms that can identify patterns of brain activity for a reliable interpretation of the observed signals [24]. In a control-oriented approach, the BCI has the EEG features of the user in input and can translate the user intention into output commands to carry out an action for the peripheral devices. In this context, most of the BCI applications deal with neuro-aid devices [25]. A quadratic discriminant analysis (QDA) classifier was used to generate the control signals to start, stop, and rotate a DC motor in the reverse direction [26]. Other intelligent classification systems have recently been used: a feedforward Multilayer Neural Network [27] and two Linear Discriminant Analysis (LDA) [28] classifiers were realized to control a robot chair. Other applications relate to the control of drones [29], robot arms [30,31], virtual objects [32], or speech communication [33].

One of the main hard tasks in an EEG BCI application is the implementation of pattern recognition to identify the classes of data coming from brain activities. The pattern recognition task depends on classification algorithms, a survey of which is presented in [34]. Nonlinear classifiers, such as ANNs and support vector machines (SVM), have been demonstrated to produce slightly better classification results with respect to linear techniques (as linear discriminant analysis) in the context of mental tasks [35]. Further research has been carried out to classify neural activities when cognitive stimuli are related to olfactory [36] or visual [37] perception. The correlation between the complexity of visual stimuli and the EEG alpha power variations in the parieto–occipital, right parieto–temporal, and central–frontal regions was demonstrated in [38]; this is in agreement with the findings in [19]. In the context of mental imagery and eye state recognition, a novel learning algorithm was proposed to train neural network classifiers based on Hamilton–Jacobi–Bellman equation, obtaining better accuracy compared with other traditional approaches [39]. A Neuro-fuzzy classifier was used for decoding EEG signals during visual alertness, motor planning, and motor-execution phases by the driver [40]. Other studies demonstrated the feasibility of interpreting EEG signals correlated to drivers’ movements in the context of simulated and real car environment [21]. The cognitive activities related to motor movements have been observed in the EEG both for executed and for imagined actions [41]. A comparison between actual and imaginary movements by neural signals concluded that the brain activities are similar [42]. The classification accuracy to predict the actions of standing and sitting by motor imagery (MI) and motor execution (ME) was evaluated in [43]. The results showed that the classification of MI provides the highest mean accuracy at 82.73 ± 2.54% in the stand-to-sit transition. Another study demonstrated that ANNs perform better than SVM in motor imagery classification [44]. Different classification algorithms including LDA, QDA, k-nearest neighbor (KNN) algorithm, linear SVM, radial basis function (RBF) SVM, and naïve Bayesian were compared to classify left/right hand movement, reaching an accuracy of about 82% by the RBF SVM in [45]. A 2D study was carried out investigating the movements of the dominant hand when a participant had to track a moving cursor on a screen by an imaginary mouse on the table [46]; in this study, the prediction accuracy of horizontal movement intention was found to be higher than the vertical one. An EEG-based classifier has been implemented to identify the driver’s arms movement when he/she must rotate the steering wheel to perform a right or left turn in a virtual driving environment [47].

The classification of head movements appears more complex since, with respect to other monitored actions, EEG signals are affected by artifacts to a greater extent [48]. In this case, the EEG signals may also include noncerebral activities coming from elements such as hair, eye activity, or muscle movements [49]. Many techniques have been used to analyze such artifacts [50]. EEG artifacts were identified using the signal from a gyroscope located on the EEG device in [48]. A SVM-based automatic detection system was developed, where the artifacts were considered as a distinct class, in [51]. Artifacts have also been classified according to autoregressive models [52] and hidden Markov models [53,54], or by introducing independent component analysis (ICA)-based heuristic approaches [55]. ICA and Adaptive Filter (AF) to remove ocular artifacts included in EEG signals were used in [56]. An automatic technique to remove artifacts in head movements was proposed in [57], where the signals from an accelerometer located on the participant’s head were acquired together with EEG signals. Subsequently, EEG components correlated to accelerometer signals were removed from the EEG.

A significant relationship between the head movements and the emotions generated by visual stimuli was found in [58]. EEG spectral analysis in the context of visual-motor tasks was carried out, and the human head movements following visual stimuli were predicted with high accuracy by an ANN classifier in [59]. A significant correlation r = 0.98 was obtained between training and test trials of the same person, but relevant difficulties emerged in head movement prediction for a participant when the classifier had been trained on another subject. A positive correlation has been found between the task of drawing an image boundary on a screen by hand movement on a mouse and the brain activities in the upper alpha band from a single frontal electrode [60]. In a similar experiment, which involved the fingers of one hand, it was found that alpha band variations are correlated with visual motor performance, while the coherence in alpha and beta oscillations implies an integration among visual motor brain areas [61]. A great amount of work has also been devoted to eye or hand movements, while a small amount of work has discussed the correlation of EEG signals associated with head movement in visual-motor tasks.

2. Methods

2.1. System Architecture

The proposed EEG-based HMI consists of two main components (Figure 1). The first component is the lamp system, which has a twofold role: it generates the external visual stimuli for the participants used to train and test the ANNs, and subsequently acquires the participant’s response, which allows for the evaluation of the predictive model that has been set up. The EEG Enobio cap [62] is the second subsystem used for EEG signal acquisition from users. The two subsystems transmit their input to a common PC server through a serial port and a Bluetooth connector.

2.1.1. Lamp System

The lamp system consisted of a Raspberry pi 3 control unit and two lamps. The PC server included a python application, which randomly sends an input to the Raspberry by the serial cable, where another python application receives commands to switch on/off the two lamps. The two lamps were located at the extreme sides of a table of 1.3 × 0.6 m, and the participant could rotate their head in the yaw angle range between −45° and 45°. Figure 2 shows the experimental set environment.

2.1.2. Enobio Cap

The Enobio cap contained 8 electrodes, which can acquire the brain’s electrical activity (i.e., an electroencephalogram, EEG) sampled at 500 Hz on 8 channels with electrodes positioned according to the standard 10–20 system. So, each sample in each signal corresponds to a duration of $0.2\times {10}^{-2}$ s. Besides, 2 additional signals from electrodes positioned in the neck were considered to perform a differential filtering and to limit artifacts due to the electrical muscles’ activity. Three channels—labeled O1, O2 in the occipital lobe, and Cz located in the parietal lobe—were used (Figure 3). In fact, the occipital lobe is associated with visual processing while the parietal lobe is associated with recognition, perception of stimuli, movement, and orientation. In addition, visual motor performance may be detected in the occipital and centroparietal areas [59]. These three electrodes were used due to being located in a brain region unaffected by movement.

2.2. Experimental Set-Up

The EEG signals collected by a set of 22 participants were treated by sequential application of a Time Delay Neural Network (TDNN) and a Neural Network Pattern Recognition (PRNN) to classify the left or right head yaw rotation in response to external visual stimuli, realizing an HMI to control, in real time, the EEG user’s signals as commands to control switching on/off the lamps. All subjects gave their informed consent for inclusion before they participated in the study. The lamp system was used in two different phases of the experiments.

In the first phase (hereinafter referred to as ”ANN training“), the participant followed the visual stimulus by a yaw rotation and looked towards the switched-on lamp, while in the second phase (hereinafter referred as ”binary controller“), the participant decided on their own to rotate the head and, according to a runtime interpretation of the EEG, the lamp switched on/off.

During the ANN training, the lamps were switched on randomly, one by one, for a 5 s-period. Subsequently, they were both switched off for a period varying between 6 and 9 s. So, as soon as one lamp switched on randomly, the participant had to turn his/her head towards the light. When the lamp switched off, the participant returned to the rest position. Each ANN training experiment lasted 5 min and the participants wore an Enobio cap for the EEG signal recording with dry sensors. The EEG file was used to identify an input–output function between the EEG signal (input) and the state of the lamp system (defined by which lamp was switched on). The approach to the identification of this function is described in Section 2.5.

In the second phase of the experiment, the participant decided to turn his/her head to the left or right side, controlling the lamp system using his/her EEG and generating control through the input–output function. This control phase lasted 3 min.

2.3. Data Acquisition Protocol

The transmission protocol regarding the EEG cap to a Server PC was TCP/IP. The two systems were connected by Bluetooth and the raw data streaming was managed by the TCP/IP protocol in the EDF+ data format. For the lamp system, an owner protocol in a raw data format was used. For the artifacts, no protocol was used because the neck/head movements were collected within the EEG signal; so, the whole signal was acquired during the experiment. Subsequently, artifacts were removed in the data processing phase.

2.4. Data Processing Analysis

Preprocessing Data

The main issue in the EEG signals deployment relates to the artifacts and to the exogenous and endogenous noises that affect the acquired data. For example, the electrical fields that are present in the environment, the movements of the eyes, and muscle activity in the human participants can affect the EEG signal [50]. Muscle-derived artifacts were limited by introducing the signals coming from the differential electrodes placed in the neck. In addition, a bandstop/notch filter between 49 and 51 Hz was applied to reduce the noise produced by the electrical frequency of devices [63].

Further techniques were applied to limit the artifacts. In fact, the resulting time-series data were still contaminated with signal artifacts—such as eye blink and movements—that have the potential to considerably distort any further analysis. During data acquisition, two electrodes located in the hairless part of the head were used, CMS and DRL, similar to a dual reference electrode. The Reference and ground EEG electrodes were fixed on the mastoids of the user for noise reduction. The EEG cap is extremely lightweight to reduce artifacts deriving from movement. The cap fits perfectly to different head shapes and keeps the device in position to guarantee solid contact of the electrode to the scalp, which is a key requirement for high-quality EEG recordings. In EEG-based BCI systems, the implementation of low-pass, high-pass, or bandpass filters has been successfully used to remove physiological artifacts [34]. Independent Component Analysis (ICA) has also been applied to remove these artifacts [55].

Preliminary tests including correlation analysis were performed to reduce the number of channels due to the need to speed up computation of the controller. This analysis suggested to limit the available information only to the signals coming from three main electrodes, labeled O1, O2, and Cz. The resulting samples were normalized by ${10}^{-6}$ and limited between −1 and 1.

The signal that identified the head position was represented as −1 for left, 1 for right, and 0 for forward positions. The transition between one position and another (e.g., left to forward) was linearly smoothed using a moving average computed on a window of 300 samples (i.e., for a duration of 0.6 s). This value is the time needed to reach the final position by the participant. An empirical analysis was carried out to choose the window size. The time average of all head movement measures that were collected during the simulations is 0.6 s.

2.5. Input–Output Function Identification

The main objective in this phase was to define the best direct input–output function generated according to the relationship among a specific set of EEG samples with the related value of the head position. In this context, it was demonstrated that the variance [6] and the participants’ sensibility [1] affect the results in the experiments. Due to the complexity of this task, a two-step approach was carried out to obtain the classification results. In Figure 4, the scheme of the proposed architecture is shown, where two different ANN’s have been sequentially applied. Firstly, a Time Delay Neural Network (TDNN) was adopted to model the temporal relationships of the input EEG signals. Then, the TDNN outputs were used as input to an additional Pattern Recognition Neural Network (PRNN).

2.5.1. The Time Delay Neural Network (TDNN)

The main goal was to identify a nonlinear input–output function between 10 consecutive EEG samples (hereinafter defined as $\underline{x}\left(t\right)$) taken from O1, O2, and Cz, and the value of the head position in the sample just following the EEG recording (hereinafter defined as $y\left(t\right)$). A nonlinear function f between input pattern $\underline{x}\left(t\right)$ and output pattern $y\left(t\right)$ must be identified, such that the values $\tilde{y}\left(t\right)$ resulting from (1) minimize the mean squared error (MSE) between $y\left(t\right)$ and $\tilde{y}\left(t\right)$ values.

$$\tilde{y}\left(t\right)=f(\underline{x}(t-1),\underline{x}(t-2),...,\underline{x}(t-10))$$

(1)

The MSE computed on one prediction is given by

$$MSE=\frac{{\sum }_{t=1}^n{(y\left(t\right)-\tilde{y}\left(t\right))}^2}{n}$$

(2)

To make predictions less sensitive to input noise, the predicted values $\tilde{y}\left(t\right)$ are then averaged by a moving average of 300 preceding samples—that is,

$$\overline{y}=\frac{{\sum }_{t=0}^{299}\tilde{y}{(t-\widehat{y})}^2}{300}$$

(3)

The TDNN with 10 neurons in the hidden layer identified the nonlinear input–output function. The identification process is based on the Levenberg–Marquardt backpropagation algorithm [64,65] (Matlab^® software version 2020). The training processes obtained a solution after an average of 87 steps (about 45 s) on a common laptop (Intel i5-3360M CPU, 2.8 GHz, 8 GB).

Two key performance indexes were used to evaluate the reliability of the proposed function f generated by the TDNN: MSE and the Pearson correlation coefficient r, defined as follows:

$$MSE(y,\overline{y})=\frac{{\sum }_{t=1}^n{(y\left(t\right)-\overline{y}\left(t\right))}^2}{n}$$

(4)

$$MSE(y,\overline{y})=\frac{cov(y,\overline{y})}{{\sigma }_y{\sigma }_{\overline{y}}}$$

(5)

where

• ${\sigma }_x$${\sigma }_{\overline{y}}$ are the standard deviations of y and $\overline{y}$;
• $cov(y,\overline{y})$ is the covariance of y and $\overline{y}$.

2.5.2. The Pattern Recognition Neural Network (PRNN)

The aim of this computation was to generate the binary control. The PRNN was defined on 3 layers. The input layer was modeled in $k=98$ units related to an input pattern, where the i-th pattern $p_i$ has the values resulting from

$$p_i=\overline{y}\left(t\left(p_i\right)\right)$$

(6)

$$t\left(p_i\right)=(k+2)i+ti=0...N-1t=0...k-1$$

(7)

where N is the number of time intervals with duration $2\ast {10}^{-1}$ s contained in each resulting $\overline{y}\left(t\right)$. The hidden layer was modeled in 10 units. The output layer was modeled in three units to match the output binary target $g_{i,b}$, defined on three bits $b=$ 1..3, coded as 100 for left, 010 for right, and 001 for forward positions. For each pattern, the position was the one found at the end of each time interval (e.g., for the 0-th pattern in the time intervals 98 and 99). The identification of the PRNN has been carried out using the scaled conjugate gradient backpropagation algorithm (Matlab^® software version 2020).

In the PRNN, the results were evaluated by the cross-entropy loss H [66] given by

$$H=-\frac{{\sum }_{i=0}^{N-1}{\sum }_{b=1}^3{g}_{i,b}\ast log\left({\tilde{g}}_{i,b}\right)}{N}$$

(8)

To evaluate the accuracy of the results obtained in case of unbalanced class, the Matthews correlation coefficient was used. This index is frequently encountered in biomedical applications [67] and generally applied in machine learning problems to evaluate the quality of binary classifications. Its generalization to a multiclass case is also used as follows:

$$MCC=\frac{{\sum }_k{\sum }_l{\sum }_m{C}_{kk}{C}_{lm}-C_{kl}{C}_{mk}}{\sqrt{{\sum }_k\left({\sum }_l{C}_{kl}\right)\left({\sum }_{k^{\prime }|k^{\prime }\ne k}{\sum }_{l^{\prime }}{C}_{k^{\prime }{l}^{\prime }}\right)}\sqrt{{\sum }_k\left({\sum }_l{C}_{lk}\right)\left({\sum }_{k^{\prime }|k^{\prime }\ne k}{\sum }_{l^{\prime }}{C}_{l^{\prime }{k}^{\prime }}\right)}}$$

(9)

where $C_{kk}$ is the number correctly predicted—that is, in category k. $C_{kl}$ is the number predicted to belong to class k but belonging to class l, where $l\ne k$.

2.6. Binary Controller Testing in Real-Time

To test the reliability of the binary controller, the same participants were involved in the second phase of the experiments. In this phase, each participant, wearing the Enobio cap, decided the direction of head yaw rotations towards one of the two lamps. Consequently, the real-time binary controller, implemented in Matlab^® software version 2020, based on the classifier previously identified for each participant (Figure 5), if properly working, should switch on the related lamp.

If properly working, the binary controller could switch off the lamp once the user turned the head to the frontal position. In the testing protocol, the participant turned the head to the right or left at his/her preference for about 5 s and he/she stayed in the frontal position for about 8 s. The switching on/off control test was performed by the participants both with open and closed eyes. In this test, the action resulting from the binary controller was defined as correct: (a) if in the period with the head turned on one side (i.e., left/right) the lamp on the same side (i.e., left/right) switched on with no switches of the other lamp (i.e., right/left); (b) if in the period with the head in the forward position the lamp switched off with no other switches.

3. Results

3.1. Prediction Accuracy

The trials involved 22 participants: 5 women and 17 men, aged 18 to 62 years, including 4 left-handed. For each participant, one trial lasting 5 min was recorded. To quantify the information of the signals to give added value to the final classification, in this study, the mutual information (MI) rather than the classic statistical methods such as regression was considered more appropriate. MI is a metric that quantifies the dependence between two random variables, taking into account the linear and nonlinear information obtained about one variable observing the other one. It is often used as nonlinear counterpart of the correlation index [68]. When applied to two EEGs, MI gives a measure of the dynamical coupling between them. In comparison with other connectivity measures, mutual information is a measure of statistical dependence between signals that does not make assumptions about the nature of the system generating the signals and gives stable estimates with reasonably long data sets. MI is not limited to real values and determines whether the marginal distribution of the two variables is wider than the joint distribution of the pair.

Considering two random variables $X=x_i$ and $Y=y_i$ with probability distributions $P_X\left(x_i\right)$ and $P_Y\left(y_i\right)$, respectively, and joint probability $P_X(x_i,y_i)$, the MI quantifies the reduction in uncertainty of X by measurements made on Y and is given as

$$MI(X,Y)=H\left(X\right)-H\left(X\right|Y)=-\sum _{x_i}{P}_X\left(x_i\right)log{P}_X\left(x_i\right)+\sum _{x_i,y_i}{P}_{XY}(x_i,y_i)log\frac{P_{XY}(x_i,y_i)}{P_Y\left(y_i\right)}$$

(10)

When its value slim to none, the two variables are independents.

Table 1. Mutual Information computed from the three channels with the output features.

Part. ID	$\mathit{O}1$	$\mathit{O}2$	${\mathit{C}}_{\mathit{z}}$
P1	0.1440	0.1442	0.0475
P2	0.1488	0.1668	0.1359
P3	0.2149	0.2180	0.1725
P4	0.0975	0.0688	0.0632
P5	0.1093	0.1126	0.1049
P6	0.0951	0.1573	0.1438
P7	0.1020	0.1036	0.0496
P8	0.0953	0.1246	0.1404
P9	0.1673	0.0740	0.0833
P10	0.1164	0.1443	0.1385
P11	0.1172	0.1321	0.1065
P12	0.0775	0.1652	0.1610
P13	0.0745	0.1825	0.0660
P14	0.1314	0.1120	0.1308
P15	0.1401	0.1729	0.0888
P16	0.2235	0.1935	0.1799
P17	0.0929	0.1388	0.1003
P18	0.1014	0.1113	0.1116
P19	0.0718	0.0962	0.0901
P20	0.2314	0.2534	0.1334
P21	0.1052	0.0826	0.0649
P22	0.2754	0.1999	0.2961

reports the MI between the three channels and the output patterns. The MI values in all the cases is close to zero. This low information contribution justifies the design of a complex predictive model.

For the classification of the ANNs, each file was divided into two equal parts, with the training performed on the first half, and the testing performed on the second half.

Table 2. TDNN performance indices in the second half of the signal recorded: correlation coefficient r and MSE between the predictive and observed values.

Part. ID	r	MSE
P1	0.89	0.177
P2	0.78	0.335
P3	0.76	0.719
P4	0.90	0.468
P5	0.79	0.522
P6	0.85	0.560
P7	0.64	0.575
P8	0.65	0.636
P9	0.72	0.523
P10	0.92	0.224
P11	0.91	0.322
P12	0.91	0.225
P13	0.87	0.441
P14	0.59	1.047
P15	0.81	0.249
P16	0.78	0.411
P17	0.84	0.514
P18	0.61	0.598
P19	0.64	0.530
P20	0.90	0.451
P21	0.69	0.862
P22	0.87	0.316

shows the results related to the MSE and correlation coefficient resulting from the TDNN just for the testing. According to Cohen classification [69], the r values demonstrate a strong correlation, which guarantees that the TDNN classifier is reliable. In all experiments, the r values are greater than 0.5 and, in 15 out of 22 cases, they overcome the value r = 0.75 with a related MSE smaller than 0.55. All the results are statistically significant at a level of 0.01 (p-value $<0.01$).

Figure 6 compares the observed value and estimated value in the TDNN during the training phase for participant 1. The r value from the previous table is $0.89$, which indicates a strong correlation between these two variables.

The results related to the PRNN performances are shown in

Table 3. Results for the PRNN for the classification related to the three classes.

Part. ID	% Correct Prediction	H
P1	96.85	0.1814
P2	90.55	0.2862
P3	89.75	0.4447
P4	93.08	0.2410
P5	82.93	0.4559
P6	90.21	0.3247
P7	78.85	0.6257
P8	83.64	0.4802
P9	83.64	0.4659
P10	94.72	0.1634
P11	93.88	0.1911
P12	93.04	0.1979
P13	91.67	0.3069
P14	81.82	0.5116
P15	91.92	0.2960
P16	90.55	0.3124
P17	85.48	0.3701
P18	74.19	0.6751
P19	79.21	0.5458
P20	93.36	0.1932
P21	79.83	0.5567
P22	95.82	0.2127

. In this model, 70% of the EEG signal was adopted for the training phase while the remaining 30% was used for the test. Table 3 shows the result on the testing patterns, evaluated according to the percentage of correctly predicted patterns (referred to as accuracy) and to the cross-entropy loss H. The PRNN accuracy is greater than 74% in all cases with a mean value of 88%. The best performance is 96% for P1. The H values in Table 3 confirm that a good accuracy of the model reflected a lower entropy. In the cases where participants obtained an accuracy greater than 90%, the value H is limited to 0.3. However, the average value for H on the overall experiments is 0.67 with the minimum for P10 (accuracy = 94.72% and H = 0.16) and a maximum for P18 (accuracy = 74.19% and H = 0.6751).

The accuracy results were evaluated by the Matthews correlation coefficient, which was used for the P22 data.

Table 4. Confusion matrix for P22.

Actual Class/ Predicted Class	−1	0	1	Total
−1	38	3	0	41
0	3	163	1	167
1	0	4	51	55
Total	41	170	52	263

shows the confusion matrix; the data on the matrix diagonal show the number of times in which the classifier correctly predicted the head position associated to label 1 (right movement of the head), label 0 (forward position), and label −1 (left movements). The values out of the diagonal represent the wrong predictions between output class and related target. The MCC (9) for Table 4 is 92.04%; its value is high, confirming a good prediction, according to the previously calculated H value. In

Table 5. Confusion matrix for all participants.

Actual Class/ Predicted Class	−1	0	1	Total
−1	759	148	1	908
0	272	3200	212	3684
1	0	137	899	1036
Total	1031	3485	1112	5628

the global confusion matrix is shown for the test set of each participant. The MCC (9) for the data is 74.3%.

The Matthews index confirms the good results obtained, which are shown in Table 3. Regarding comparative performance analysis in terms of classifier accuracy, it is important to note that the literature relating to EEG signals classification for HMI applications in the context of human movements mainly focuses on hands, eyes, arms, or movement intention detection. On the contrary, the proposed classifier, and the consequent HMI controller, concern head movements and aim to evaluate the possibility to identify head turning by participants when they are subjected to visual stimuli to provide a noninvasive interface tool to control external devices.

3.2. Binary Controller Performance

Using the same trials, the same preprocessing pipelines, and the related training described in Section 3.1, the performance of the real-time binary controller based on electrical activity related to the head yaw rotation has been tested.

Table 6. HMI accuracy.

Part. ID	% of Correct Control with Open Eyes	% of Correct Control by Closed Eyes
P1	100	100
P2	86	83
P3	80	63
P4	82	80
P5	79	44
P6	91	68
P7	85	79
P8	81	79
P9	77	73
P10	100	96
P11	87	90
P12	96	96
P13	76	73
P14	54	26
P15	94	88
P16	88	86
P17	88	73
P18	81	59
P19	78	68
P20	97	87
P21	78	71
P22	81	62

shows the percentage of correct predictions for HMI used as a binary controller. The tests were performed both with open eyes and with closed eyes. In fact, this analysis in ”blind“ conditions, allows us to verify how the light stimuli affect the controller. The percentage of correct predictions is greater than 75% for 20 participants on 22 with opened eyes and 11 on 22 for closed eyes. The average difference between the performance between open eyes vs. closed eyes is always in favor of open eyes, with an average difference of 9%. The participant P1 controlled the light system without errors during both experiments with 100% success.

4. Conclusions and Future Directions

This work relates to the implementation of a HMI system to implement a binary controller; in the case study, this is applied to a switch on/off lamp system according to electrical brain activity due to head turning movements. The main purpose of the approach may be related with two applications. In the first place, it may support workers’ safety in the workplace with special reference to the driving context. Specifically, the head position and rotation are strongly associated with a driver’s attention during a driving task [70]. The recognition of head movements when the participant is subjected to visual stimuli support the development of automatic safety devices, which the next generation of vehicles will be equipped with to avoid accidents. In general, the detection of driver’s anomalous movements may prevent accident occurrence. Additionally, the HMI system described here may represent a preliminary application to support the development of assistive devices to improve the mobility of physically disabled people who are not able to use their arms or legs, for example, to turn or control motorized wheelchairs or vehicles or to use a PC. It is intended to implement a HMI system that allows interaction with the devices directly driven by the patient’s head movements. Most work that has been carried out so far adopts the EEG to monitor brain activity in cases of participant’s movements intention [71], for tracking eye movement [72], or using a gyroscope controlled to drive external components [73]. The approach that has been adopted represents a novel contribution in this field. Its main purpose is to demonstrate the feasibility of using human signals as input to the HMI to switch on/off external devices. The results show the feasibility to identify human left or right head movements by a nonlinear input–output function recording brain signals from three channels of an EEG cap. The correlation is quite relevant and, for our set of 22 participants, it is greater than 0.75 in 68% of the experiments.

The proposed approach consists of applying two consecutive NNs: firstly, a TDNN; subsequently, a PRNN, providing promising outcomes in the prediction of movements acquired by participants’ EEG signals. The accuracy of the classifier, which is computed as the rate between correct predictions out of total examples, has a mean value of 88%. In addition, the low computational time required by TDNN and PRNN allows for their application in a real time context.

The main achievement of this work is the reliability of the controller that has been realized. The tests related to the application of the classifier to classify the user movements and, consequently, to control the light system by monitoring the EEG signals show a relevant capability of the binary controller to identify the intention of the subjects to move the head on the right and left sides. On the contrary, a drawback may be recognized in the customization of the classifier. When the TDNN–PRNN system is identified on one participant, it cannot be applied on another participant. However, several issues may be questioned. The first aspect relates to the presence of artifacts in the EEG signal, which may affect the overall identification process. In fact, while classic filtering has been adopted, electrical stimuli coming from different sources (for example, the effect of the lamp light on the participant’s eyes) can be still present. However, as it relates to the effects of the light on the participant’s eyes, it is encouraging to observe that the controller works quite well with no need for stimulus, since lamps are switched on by the controller. In addition, reasonable results are also obtained with closed eyes.

The control results achieved in this work can be also considered an important milestone in the development of humanoid neck systems. Focusing on the neck element, several humanoid neck mechanisms have been developed by different researchers, specifically on soft robotic neck. Soft robotics is becoming an emerging solution to many of the problems in robotics, such as weight, cost, and human interaction. In this respect, a novel tuning method for a fractional-order proportional derivative controller has been proposed with applications to the control of the soft robotic neck [74]. A soft robotic cable-driven mechanism with the purpose of later creating softer humanoid robots that meet the characteristics of simplicity, accessibility, and safety has been proposed [75]. The complexity of the required models for control has increased dramatically, and geometrical model approaches—widely used to model rigid dynamics—are not enough to model these new hardware types [76].

Obviously, in this work, better results could have been obtained with a simple, cheaper gyroscope positioned on the head. This is why it is important to underline that the goal of our study is understanding the association between the electrical brain activity and a specific action. In fact, the electrical activity of the muscles in the neck area is 10–100 times stronger in magnitude than the electrical activity of the brain; thus, despite the techniques used to reduce artifacts, there is a high probability that the developed system takes muscle activity as a hint for the classification task.

However, as quoted before, we believe that such a hint could be useful in the first phases of machine learning in degenerative diseases that reduce human mobility when mobility is still possible. A further step that could not be obtained by an accelerometer or a gyroscope is to verify such an association when the actual movement is not performed, while the participant is ”just thinking“ to make a movement. Further work could address the contribution of the EEG versus the electromyogram (EMG) with parallel monitoring of the two systems to verify the effective information contribution of the EEG. Other studies [77] have shown that this further step is possible. A study related to post foot motor imagery has been performed [78]. This case study is focused on the study of the postimagery beta rebound to realize a brain switch using just the EEG channel. An analogous approach, involving the creation of an asynchronous BCI switch using imaginary movement, was designed in [79]; the focus of this work was to demonstrate that the 1-4Hz feature basis have the minimal power to discriminate the voluntary movement-related potentials. An issue of the EEG-based brain switches is related to the detection an asynchronous way of the control and idle states. A P300-based threshold-free brain switch to control an intelligent wheelchair was proposed in [80]; experiments on this new approach showed the goodness of the proposed methods, reporting also the possible application in participants with spinal cord injuries.

Future developments of this work will be devoted to three directions. First, further efforts should be devoted to improving the reliability of the data acquired by the EEG, enhancing filtering while retaining a quite cheap and noninvasive architecture, such that practical future implementation is feasible. While IC is one of the most widely used artifact removal techniques used in this context, recent work based on ”ad hoc“ heuristics by detecting influential independent components show better performance [55]. Secondly, the generalization of the identification of the controller on more participants rather than on one will be investigated, carrying out an additional cluster analysis on participants. Finally, and probably more important as a future goal, is to apply the method in an application context. Two applications are envisaged. The first one relates to the automotive context, as was recently investigated [21]. The second application relates to identification based on the idea of the movement rather than on the movement itself, as this final function could have important applications for severely disabled people.