Personal Identification and the Assessment of the Psychophysiological State While Writing a Signature

Abstract

This article discusses the problem of user identification and psychophysiological state assessment while writing a signature using a graphics tablet. The solution of the problem includes the creation of templates containing handwriting signature features simultaneously with the hidden registration of physiological parameters of a person being tested. Heart rate variability description in the different time points is used as a physiological parameter. As a result, a signature template is automatically generated for psychophysiological states of an identified person. The problem of user identification and psychophysiological state assessment is solved depending on the registered value of a physiological parameter.

1. Introduction

In recent years, the news about confidential information leaks regularly appeared in the world mass media. According to The Global State of Information Security Survey 2014 (global information security exploration carried out by the PricewaterhouseCoopers company and CIO, CSO journals) the main reason for security breach incidents are employees (31%) and former employees (27%) of companies. The global economy combined losses due to insiders are impressive [1]. The viewpoint consisting of the fact that further traditional means of protection improvement does not correct the situation has been formed. The appearance of this viewpoint is associated with changing the formulation of the problem: create protection from the person to whom everything is allowed in accordance with official duties. A prime example is K. Mitnick who acquired passwords to information resources by means of social engineering methods. According to his opinion the most vulnerable place in security system is “a human factor”. How can we protect information from unauthorized actions of computer system authorized users? It is possible to confirm the loyalty of employee actions by analyzing his psychoemotional state at the time of doing job tasks, and at the time of authentication. The features of writing dynamics can be used for solving this problem. In recent years, the attention of researchers was focused on improvement of this technology. One of the best results, which was declared in the late 90s, was 4% (the sum of the probabilities of False Acceptance Rate and False Rejection Rate) [2]. In 2002 it decreased to 2% [3]. The best result in 2003 was 2.78% [4], in 2008—3.3% [5], in 2012—1% [6], in 2013—2.8% [7]. The above results are illustrated in Figure 1.

Figure 1. The best results in recognizing users via signatures explored in the past years.

Figure 1. The best results in recognizing users via signatures explored in the past years.

If we look at the other results obtained at the specified time period we can see the significant differences between them even if the used identification algorithms were identical [8]. It is known that personal psychoemotional state changes are reflected in his handwriting manner [9]. There is a point of view that the difference in the reliability of personal recognition for the specified technologies was caused by the difference between the psychophysiological state of examinees at different stages: the identification procedure and the template creation. Thus a new system architecture of personal identification analyzing both the handwriting dynamics features and the signer's psychophysiological state (PPS) simultaneously becomes necessary. This is caused by at least two reasons:

(1)

Further increase in the signer's identification reliability can be achieved if the signer's psychophysiological states at the moments of template creation and identification are the same.

(2)

Results of the signer's psychophysiological state evaluation can be used to make a suggestion about the signer's admission to specific information.

The purpose of this research is to develop the architecture of such a system.

A lot of methods of person identification by handwritten passwords exist as well as those that analyze dynamics of signature reproducing or a static image of signature [2,3,4,5,6,7,8,10,11]. However not one of them allows us to take into consideration the influence of a signer's psychophysiological state on the results of the identification. The procedure of improving the parameters template after typical user login also does not solve the problem.

In Forensics letters size and position, letters direction and slope and so on are analyzed for recognition of human psychological state by handwriting [12].

Scientists have created the theoretical bases for the psychophysiological state evaluation via human physiological parameters [13,14]. Research works in the hidden noncontact assessment of physiological patameters of persons under test have been carries out [15]. The patenting activity in this field is activated [16,17]. The significant progress was noticed in hidden (remote) measuring of the current human physiological parameters: a heart rate, an arterial pressure, blood filling of the face vessels, a respiration rate and electrocardiogram [18,19].

The analysis of electrocardiograms allowed making a clear conclusion about the link between the heart rate variability (HRV) and the organism functional state [20,21]. Thus there are no arguments that deny the possibility of remote measuring of the person's physiological parameters indicated above and using them for evaluating the person's psychophysiological state. However, the question of dividing HRV parameter into range segments (psychophysiological state) still remains a problem. For example one of the HRV parameters called a stress index may vary at a speed of up to 2% per second.

This paper presents a technology of personal identification via handwriting manner that provides an assessment of a psychophysiological state via HRV.

2. Proposed Approach

2.1. Identification and Training Procedures

At the education stage the identified subjects reproduce the signature using graphic tablets herewith the physiological parameter that reflects psychophysiological state of subjects is covertly recorded. The HRV in the different time points is proposed to use as a physiological parameter. The domain of values of the physiological parameter is divided into ranges and each range is associated with signatures written in a certain psychophysiological state. The template is formed from the signatures assigned to each ranges. At the identification stage the algorithm firstly records a physiological parameter and its value defines the range and a psychophysiological state associated with this range. A comparison of values of the biometric feature for a presented signature is carried out only with template data that were obtained in current psychophysiological state. The person's template for which the compared descriptions similarity is maximum is determined. The duration of original scalable functions are proposed to be used as an additional feature in images signatures comparison.

A structural diagram that describes the proposed technology of staff identification for the admission to work with informational computing system is shown in Figure 2. The technology includes the user's signatures database 1 that is used during user's authorization. The signature generators 2 are persons that are admitted to work in informational computing system of a company. The 3 is a switch of the staff that is admitted to informational resources for signatures reproducing. A graphics tablet used as converter of signatures in the time functions is 4. Under the number 5 the scaling unit of time functions is specified. The unit number 6 is responsible for forming the informative features from the incoming input time functions. The picture 7 marks a mode switch of the identification system that works in two modes: 7.1—a template forming mode, 7.2—an identification mode. The unit 8 is intended for forming the staff templates using the signature features. The unit 9 makes a decision if an incoming signature belongs to one of employees registered in the system.

Figure 2. The structural diagram describes the identification process: 1—the signature database; 2—the signature generators; 3—an employee's commutator; 4—a graphics tablet; 5—a scaling unit; 6—a unit forming the informative features; 7—an operation mode switch: 7.1—a template forming mode, 7.2—an identification mode; 8—a templates forming unit; 9—a decisions making unit.

Figure 2. The structural diagram describes the identification process: 1—the signature database; 2—the signature generators; 3—an employee's commutator; 4—a graphics tablet; 5—a scaling unit; 6—a unit forming the informative features; 7—an operation mode switch: 7.1—a template forming mode, 7.2—an identification mode; 8—a templates forming unit; 9—a decisions making unit.

The minimum set of generated functions $f\left(t\right)$ of graphics tablets presented in the market is 3: the pen pressure on tablet surface $p\left(t\right)$, the coordinates of the pen tip $x\left(t\right)$ and $y\left(t\right)$. The duration of signature writing by one employee is a random value. This fact negatively effects the identification reliability. It is used to add the operation of normalization of original representation recognized pattern [22] to eliminate this circumstance. For the case under consideration the normalization operation is limited to the following operations: expansion of the function $f\left(t\right)$ in Fourier series; replacement of the harmonics frequency on reference values in accordance with which the first harmonic frequency is changed to $(1/{\tau }_n)$, the second is changed to $(2/{\tau }_n)$ and so on, where ${\tau }_n$ is the duration (the total writing time) of the analyzed signature fragment; the rationing of received harmonic amplitudes by the signal energy; construction of new function $f\left(t\right)$ by found amplitude and phase spectra of analyzed function. Figure 3 shows the diagrams of the pen pressure functions on a graphics tablet surface when an employee is writing a signature being in a quiet state and in a fatigued state (the subject was exposed to intensive physical effort).

These functions after transformations specified above are shown at Figure 4.

Figure 3. The pen pressure function at the output of a graphic tablet while employee reproduces a signature being in different psychophysiological states.

Figure 3. The pen pressure function at the output of a graphic tablet while employee reproduces a signature being in different psychophysiological states.

Figure 4. The pen pressure function after the normalization procedure when an employee reproduces a signature being in different psychophysiological states.

Figure 4. The pen pressure function after the normalization procedure when an employee reproduces a signature being in different psychophysiological states.

A disadvantage of the described scalar procedure is the loss of information about the differences in employee's signature writing duration. It can be noticed case when a person tries to imitate other employee's signature to get access to information resources. A peculiar feature of such imitation is the change in function durability, it is known that the registered function duration is significantly larger when the signature is imitated. Furthermore, errors of the identification by handwritten signature are often caused by the difference psychophysiological state of subject at the time of etalon creation and at the identification time.

Disregarding this fact is often the reason of low reliability of the identification results.

These disadvantages may be avoided using the following steps:

• To create signature etalons for each subject psychophysiological state separately;
• To perform the subject psychophysiological state recognition during his/her identification and to use etalon signature descriptions for corresponding psychophysiological state.

The Figure 5 shows a structural diagram of a system that allows meeting the above mentioned requirements. This diagram includes an employee signature generator that is marked with a number 1, and an employee that is marked with a number 2. The unit marked with a number 3 is a switch for connecting employee for testing. The graphics tablet is numbered 4. The unit numbered 5 is responsible for time functions scaling. The unit of feature formation is numbered 6, the number 7 marks an operation mode switch of the identification system: the number 7.1 marks a template forming mode, 7.2 numbers an identification mode. The unit numbered 8 performs employee etalon forming using signature features. The unit 9 task is making the decisions whether a signature obtained by the system belongs to one of the registered in system employee (9.1) and an evaluation of the employee's psychophysiological state. A device for determining the employee's physiological parameters during the testing and hidden etalon creation is denoted with a number 10. The display was marked with 11, question generator is marked with 12. The 13th picture is the word recognition unit by the graphic tablet signal.

Figure 5. The structural diagram of person identification: 1—a signature generator; 2—a tested employee; 3—a connection employees commutator; 4—a graphics tablet; 5—a scaling block; 6—a features formation unit; 7—an operation mode switch: 7.1—a template forming mode, 7.2—an identification mode; 8—a templates forming unit; 9—a decisions making unit; 10—a physiological parameters evaluating device; 11—a display; 12—a question generator; 13—a word recognition unit.

Figure 5. The structural diagram of person identification: 1—a signature generator; 2—a tested employee; 3—a connection employees commutator; 4—a graphics tablet; 5—a scaling block; 6—a features formation unit; 7—an operation mode switch: 7.1—a template forming mode, 7.2—an identification mode; 8—a templates forming unit; 9—a decisions making unit; 10—a physiological parameters evaluating device; 11—a display; 12—a question generator; 13—a word recognition unit.

The process described with this scheme consists in the following. The employee 2 is offered to answer the question that are shown on the monitor screen (11), then the employee should draw a signature using the graphics tablet (4) with simultaneous hidden registration of his physiological parameters using the technical device (10). Questions coming from the database (12) are to affect the psychophysiological state change. The structure and the sequence of questions are determined by one of the existing polygraph tests technologies [23]. The listed steps allow us to construct etalons of tested persons for different psychophysiological states.

The block number 5 provides scaling using direct and inverse discrete Fourier transform. For experimental purposes Wacom Bamboo tablet is used. For the given experiment the slope angle (a lateral angle and a pen elevation angle) is not taken into account. The block number 6 contains 47 informative features described in [24]. As biometrical features we recommend to use amplitude ranges of a pen velocity function on a tablet surface $Vxy\left(t\right)$ and a pen pressure function on a tablet surface $p\left(t\right)$ scaled to energy (calculated using the Fourier transform), and correlation coefficients for functions $x\left(t\right)$, $y\left(t\right)$, $p\left(t\right)$ of a handwritten password and their derivatives. Thus, the number of personal characteristics $\left(N\right)$ obtaining from a handwritten password is 47 ($N=47$, a number of amplitudes of low-frequency harmonics of the $Vxy\left(t\right)$ function is 16, a number of amplitudes of low-frequency harmonics of the $p\left(t\right)$ function is 16, a number of correlation coefficients is 15). All described features have a near-normal distribution [24]. Therefore a certain number of signatures is necessary to input and a parameter of the normal law of distribution (expectancy and root-mean-square deviation) should be calculated for every feature values to create a template in any psychophysiological state.

$$\begin{array}{c}\hfill V_{xy}\left(t\right)=\sqrt{{(x(t+\Delta t)-x\left(t\right))}^2+{(y(t+\Delta t)-y\left(t\right))}^2}\end{array}$$

(1)

where x and y are coordinates of a pen position on a tablet surface, t is a record time for coordinates of a pen position on a tablet surface, $\Delta t$ is a time interval between recording pen position coordinates. The block number 7 provides the mode change for the system work. A template formed by the block number 8 contains a set of 47 features [27] and the information about the psychophysiological state of the employee corresponding to this template. The block number 1 generates a signature that may be performed either by the employee personally or using the Monte Carlo method (only identifying features are generated).

2.2. Usage of Information about the Psychophysiological State

A Holter monitor was decided to use as a physiological record device. Physiological data obtained via Holter monitoring are presented as an electrocardiography. The electrocardiography is processed and the values of one of HRV parameters, a stress index, are defined [20].

Any psychophysiological state corresponds to a certain range of the given parameter. Questions are taken from the database (12) and used to influence the change of the employee's psychophysiological state, their structure and sequence is determined by one of the existing technologies of lie detecting examination [23].

As for the problem of dividing HRV indicators into ranges the following solution is proposed. During testing the probability density function of a physiological parameter, stress index $\left(S{I}_p\right)$, is constructed. This function is illustrated at the Figure 6. The number of significant states is selected from the parameter dispersion $S{I}_p$-s. The first range limits of state are determined through area $\left(S{I}_p\right)$ that is equal to $1/s$ near to the function maximum $\left(S{I}_p\right)$. The other areas $\left(S{I}_p\right)$ are calculated in the same way keeping their equality to $1/s$. The scaled information from graphics tablet is used for etalon construction in the corresponding psychophysiological state.

Figure 6. The formation of the probability density function of stress index P(SIp) by the example of 3 psychophysiological states.

Figure 6. The formation of the probability density function of stress index P(SIp) by the example of 3 psychophysiological states.

2.3. Decision-Making Algorithm

The unit 9 can be realized on the basis of Bayesian networks apparatus. Particularly by means of consistent application of modified Bayes hypotheses Formula (1) that shows high results while identifying the images in low informative biometric features space [25]. The essence of this method consists in calculating posterior probabilities of hypotheses $P_j(H_i/A)$ per a certain number of steps using the Formula (1). The number of steps equals to number of signature features. Each hypothesis implies that signature data belong to specific subject i.e., each hypothesis is associated with a person's specific etalon. At each step the specific identification feature data arrive in the input in the form of conditional probabilities of hypotheses and a priori probability. The conditional probabilities are calculated proceeding from the distribution law of feature values. The characteristics based on the results of spectral and correlation analysis of signature functions [26,27] are proposed to use as features. The posteriori probability that is calculated at the previously step is taken as the priori probability. At the first step all hypotheses (subjects) are considered equally probable that is $P_0(H_i/A)=1/n$, where n - number of hypotheses (subjects). In order to distinguish a known user from unknown a threshold value of posteriori probability hypotheses is set in the system.

$$\begin{array}{c}\hfill P_j\left(H_i\right|A)=P_{j-1}\left(H_i\right|A)+\left(\frac{P_{j-1}\left(H_i\right|A)P\left(A_j\right|H_i)}{\sum _{i=1}^n{P}_{j-1}\left(H_i\right|A)P\left(A_j\right|H_i)}-P_{j-1}\left(H_i\right|A)\right)\\ \hfill \times \left(1-\sum _{i=0}^n{P}_{j-1}\left(H_i\right|A)Su{m}_{ji}\right)\end{array}$$

(2)

where $P(H_i/A_j)$ - is the posteriori probability of the i hypotheses is calculated at j step at entrance of the j feature, $P(A_j/H_i)$ is the conditional probability of the i hypotheses at entrance of the $A_j$ feature. $Su{m}_{ji}$ is an area of collision of the j feature value distribution function for the i hypotheses with the j feature value distribution function for other hypotheses.

Probability densities for $Aj$ feature values obtained of a signature at the identification stage are reasonable to use as conditional probabilities. The areas $Su{m}_{ji}$ are introduced for evaluation and recording the information value of the current feature during the decision making process for all known pattern classes.

The unit 13 provides the recognition of the word written on the tablet surface. There is a lack of information about the results of the recognition if the functions $x\left(t\right),y\left(t\right),p\left(t\right)$ are used as input signals. But many papers cover the issues of handwritten text recognition and the development of recognition algorithms. The first practical results in this field are obtained in [28]. As the functions $x\left(t\right),y\left(t\right),p\left(t\right)$ are clearly derived from the handwritten text, it should be recovered for automatic or human recognition by applying the inverse transform. As the system shown at the Figure 5 is timed, it is easy to link the text and a psychophysiological state.

Thus the etalon formation unit is renewed with information about generated signatures associated with a current psychophysiological state of the tested person. The decision making unit operates with lots of etalons that characterize persons admitted to information resources as well as their psychophysiological state.

3. Results and Discussions

Twenty five test persons created signature templates, the number of psychophysiological states was 3: a usual state (a person was not affected), an emotional stress, psychic tension. The last two variants supposed a person had to answer several questions to create a template; as a result his psychophysiological state was changed that was witnessed by a change of the stress index value in a proper way. A number of signatures used to create a template in every state was 30. Thus every test person (25 in total) input 90 signatures minimum. The number of signatures that is necessary to create a template has been calculated using Chebyshev's theorem. The value sampling of the features is representative if the number of execution of signatures is more than 25 [26,27]. Likewise personal templates were calculated based on 30 signatures stored in different psychophysiological states (10 signatures were used for every psychophysiological state) that were associated with a compound psychophysiological state. A simulation experiment was carried out when for every template 1000 values of any feature were generated by a Monte Carlo method based on template feature distributions having been previously created. Using a Monte Carlo method a normal random value may be generated in the following way. If a random value R is known to be evenly distributed in the interval [0, 1] than its mathematical expectation and dispersion are correspondingly equal to:

$$\begin{array}{c}\hfill M\left(R\right)=\frac{1}{2},D\left(R\right)=\frac{1}{12}\end{array}$$

(3)

Further we generate g independent random values evenly distributed in the interval [0, 1]. The mathematical expectancy and dispersion of the sum of these values are correspondingly equal to $g/2$ and $g/12$ due to (3). Thus, root-mean-square deviation is calculated as the square root of $g/12$. Further the sum of random values R is scaled using the formula:

$$\begin{array}{c}\hfill \frac{\sum _{j=1}^g{R}_j-\left(\frac{g}{2}\right)}{\sqrt{\frac{g}{12}}}\end{array}$$

(4)

Due to the central limit theorem, if $g\to \infty$ the distribution of this scaled random value approximates normal with parameters $M=0$ and $S=1$ (mathematical expectation and root-mean-square deviation).

The preliminary testing has shown that an identification system prototype created on the basis of proposed technology (the 13 unit has not been implemented, the 9 unit has been implemented on the basis of classic Bayes hypotheses formula) gives smaller number of wrong decisions of persons' recognition than a similar identification system that does not consider a psychophysiological state.

The generated feature values were used to simulate the identification process for 25 test persons. Here 3 situations were simulated: (1) Templates of compound psychophysiological states and corresponding generated feature values were used for identification (10,000 attempts of identification were simulated); (2) Templates of a standard psychophysiological state and feature values generated in other psychophysiological states were used for identification (20,000 attempts of identification were simulated); (3) Templates of different psychophysiological states and feature values generated for corresponding states were used for identification (20,000 attempts of identification were simulated). Further probabilities of false identification conclusions for every situation were calculated as a number of false conclusions to the total number of experiments. According to the simulation experiment the number of identification errors decreased by 28% on average if the proposed approach was used when the test person's psychophysiological state at the moment of generating a template matches a psychophysiological state at the moment of identification (the 3rd situation) in comparison with the approach that does not take into account the psychophysiological state (the 1st and 2nd situations). During simulation the sum of errors of the 1st and the 2nd kind was 0.009 (ignoring psychophysiological states) and 0.007 (with regard to psychophysiological states). The error rate is expected to be high in real experiments, the probabilities rate is expected to be the same.

4. Conclusions

This article proposes the persons' identification technology and system using the signature features with psychophysiological state assessment. The proposed psychophysiological state assessment method by HRV at various time points aims at increasing the reliability of the identification procedure. This is achieved by using the etalons actual at the moment of identification that was created when the person was in the same state. In the study of the literature the previously obtained results is reviewed. From this review it is clear that in person identification studies there is a lack of experience of using a psychophysiological state. Also in the literature there is an evidence that a psychophysiological state affects handwriting including signature writing. The obtained results can be used not only in the handwriting identification but also in other biometric identification methods. The structural diagrams that explain the essence of the given technology are shown and described in detail.

The proposed technology may find be used in a human biometric identification system, in the information security system and in forensic science to search for the perpetrators.

The usage of the proposed method of personal psychophysiological portrait building for information security is possible if there is a necessary sampling of tested persons, person's psychological testing results and indexed signature samples. Here it is important that handwriting features allow decisions to be made with the required efficiency in the specified application domain.

The further research should be associated with studying the possibility of using other ways of evaluating the psychophysiological state at the time of identification. This research also should include an experimental study of characteristics of the developed technology while carrying out the simulation modeling and natural modeling of the proposed identification system when using various numbers of features and increasing the number of testing persons that are known to the system.