*3.1. Signature Data Acquisition*

For this study, we captured online signatures from 74 persons on an iPad tablet with a capacitive touch-screen of 2048 × 1536 pixels. The signatures were sampled at 63 Hz and stored as a sequence of discrete values [*xt*, *yt*], where *xt* and *yt* are the coordinate values and *t* is the time stamp.

Each person signed 25 times with their usual signatures. No instructions were given to participants when they signed, letting them acquire their signatures naturally, freely in terms of posture and position of the device, so that they would feel comfortable with the mobile device when signing. This leads to different acquisition conditions according to persons, exactly like it would be in real mobile usages.

Additionally to their usual signatures, we asked participants to append other types of signatures separately: name-surname, initials, date and place of birth. We considered these signature types because: (i) in terms of usages, they are traditionally reported by persons in legal and administrative documents; (ii) they convey complementary handwritten information on the user's identity. Each type of signature was done by the person 25 times. This dataset thus contains 9250 (74 × 25 × 5) genuine signatures of different types.

Figure 1 displays an example of one person's usual signature and the associated place of birth. We plot below in Figure 2 the velocity temporal function for both handwritten information.

**Figure 1.** Examples of (**a**) a usual signature and (**b**) the associated place information of a user who authorized their publication.

**Figure 2.** Velocity profile of (**a**) the usual signature and (**b**) the associated place information displayed in Figure 1.

In order to assess signature vulnerability to attacks, we acquired 10 skilled forgeries per signature type after displaying on the screen the shape and kinematics of the target signature. This type of forgery is considered in the literature as being the best attacks [3,43,44]. We thus obtain 3700 skilled forgeries (74 × 10 × 5) done by different forgers. Figure 3 shows an example of skilled forgeries of the usual signature and the associated place of birth displayed in Figure 1. We also display in Figure 4 the velocity temporal function for both handwritten information forgeries.

**Figure 3.** Examples of skilled forgeries of (**a**) the usual signature and (**b**) the associated place information displayed in Figure 1.

**Figure 4.** Velocity profile of (**a**) the forged usual signature and (**b**) the forged place information displayed in Figure 3.

#### *3.2. Signature Types*

We considered the five types of signatures separately: the usual signature (S), the initials (I), the name-surname (N), the date of birth (D), and the place of birth (P). From these five simple types, we constructed 7 hybrid signature types by combining:


These instances of hybrid types were constructed by concatenating the sequences of the corresponding simple signature types, resulting in a single time sequence. The identity of the user is thus expressed through several signature types of different length, which convey different complementary information to strengthen the user's identity.

## *3.3. Quantifying Quality of Signature Types*

To assess information enrichment at the enrollment phase, we quantify the information content of other simple signature types than the usual signature (initials, name-surname, date and place), and also of the 7 hybrid types mentioned in Section 3.2.

The concept of entropy is a good alternative for quantifying the information content or the disorder in signatures. In [28,40–43], we proposed the concept of personal entropy (PE), an entropy-based quality measure that quantifies simultaneously both the complexity and variability of a person's signatures. In fact, complexity and variability are related to disorder at two different levels: complexity corresponds to the intrinsic disorder in a signature sample; variability corresponds to the intra-class disorder in a set of signatures belonging to a given user.

A user's PE is measured by exploiting the local probability densities estimated when training the user's HMM on a set of 10 genuine signatures described only by *x* and *y* attributes. Indeed, the HMM automatically generates portions by the Viterbi algorithm and estimates a mixture of Gaussian densities on each portion [28]. Figure 5 illustrates how PE is computed locally, on the segments generated by the user's HMM.

**Figure 5.** Personal entropy computation on portions of a signature.

Therefore, a random variable *Z* can be associated to each stationary portion *i* of the signature, generated by the Viterbi algorithm by the user's HMM. The number of portions *N* is the number of states of the HMM. The entropy *H*(*Zi*) of a portion *i* is computed as follows:

$$H(Z\_i) = -\sum\_{z \in S\_i} p(z) \cdot \log\_2(p(z)),\tag{1}$$

where *z* corresponds to a given point in the signature described by its coordinates (*x,y*), belonging to the current portion *i*, and *p*(*z*) is the probability of observing *z*.

We studied the number of genuine samples necessary for a good HMM estimation and showed that 10 instances lead to stable PE values [28]. The local probability distribution function is estimated using all the sample points belonging to each portion, across the 10 genuine samples. After that, the entropy of each genuine signature *H*∗(*Z*) is the average of entropy values *H*(*Zi*) on all the *N* portions of the signature, divided by the signing time *T*:

$$H^\*(\mathbf{Z}) = \frac{1}{N \ast T} \sum\_{i=1}^N H(Z\_i) \, , \tag{2}$$

Finally, by averaging *H*∗(*Z*) across the 10 user's genuine samples, we obtain a user's PE for each signature type. We demonstrated that PE allows obtaining three categories of signatures, coherent across several databases, spanning from short and highly variable signatures (high PE category) to stable, longer and complex signatures (low PE category). Moreover, we showed that for different classifiers, persons with low PE are the most robust to skilled forgeries. Persons with high PE are considered being "problematic" users in the literature [28,43,45]. These results were obtained considering the usual signature of each person [28,40–43].

## *3.4. Signature Verification System*

As our aim is to assess the impact of our strategy in a mobile scenario, we used a statistical verification system that has already been evaluated on large databases acquired on mobile sensors [9,12, 15,16,27,46], and has shown to maintain good performance on well-known databases in interoperability scenarios [47], as reported in Table 1. Indeed, Table 1 presents our system's performance on several online signature databases, some acquired in an office-like scenario using a Wacom digitizer with an inking pen, and other in a mobile scenario on different touch-screen sensors (PDA, iPad, iPhone). We report the EER values on skilled forgeries only, since it is the most challenging configuration for signature verification. The system has been evaluated in BSEC'2009 and ESRA'2011 competitions on very large databases of 382 persons [15,16] that signed both on a Wacom digitizer and on a PDA device. We observe that in the mobile context, the verification performance of our system is clearly better on recent capacitive touch-screen sensors (iPad and iPhone) compared to the results obtained on the PDA device (DS3-210, PDA-64, DS3-382).

**Table 1.** Performance of our HMM-based ASV system on several online signature databases acquired in office-like (Wacom device) and mobile scenarios (touch-screen sensors), considering skilled forgeries.


Table 2 summarizes the state-of-the-art of online signature verification systems on mobile sensors, when considering skilled forgeries. We observe that in some publications, performance is not reported on skilled forgeries, which is the most challenging case for ASV systems. When comparing the results on mobile sensors in Tables 1 and 2, we note that our system shows good performance compared to the state-of-the art. Indeed, on iPad and iPhone mobile sensors, an EER of 7.04% and 4.95% respectively is reached on signatures of the same 74 users. Compared to e-Biosign database containing real signatures of 65 users acquired with a stylus on two mobile devices, we notice that our HMM-based system shows slightly better performance on the iPad device (7.04% vs. 7.9% in the best case, or vs. 10.7% on the other mobile device) and much better performance on the iPhone device (4.95% vs. 7.9% in the best case, or vs. 10.7% on the other mobile device).

Our system behaves well in mobile conditions because it is based on a statistical model, namely a continuous left-to-right HMM with four Gaussian components per state [47–49]. In other words, each writer's signature is modeled through a double stochastic process, characterized by a given number of states with an associated set of transition probabilities among them, and in each state, a continuous density, a multivariate Gaussian mixture is used to model the emission probability density. This model has the advantage of absorbing the intra-personal variability of signatures [47], which increases significantly in mobile conditions.


**Table 2.** Performance of ASV systems of the literature on several online signature databases acquired in mobile scenarios with touch-screen sensors, considering skilled forgeries.

A personalized number of states is determined according to the total number *Ttotal* of sampled points available in the genuine signatures of the HMM's training set. We consider that in average 30 sampled points are enough to estimate the mean vector and the covariance matrix of each Gaussian [47]. The number of states *N* is computed as:

$$N = \left[\frac{T\_{total}}{M \bullet 30}\right],\tag{3}$$

where *M* = 4 is the number of Gaussian densities per state and brackets denote the integer part.

Nineteen dynamic features are extracted point-wise for all signature types. These features are described in detail in the Appendix A. The usual information extracted from an HMM is the likelihood of the input signature given the user's model. We have noticed that the information coming from the segmentation of the test signature by the target user's model is complementary to that of the likelihood, especially for forgery detection. Indeed, we have shown in [47] that the segmentations made by the target model on forgeries differ from those obtained on genuine signatures. For this reason, in the verification phase, the classifier performs a score fusion combining two levels of signature analysis: one based on a local point-wise analysis of each signature by the HMM (log-Likelihood score), the other on the analysis of the signature at the level of portions, automatically segmented by the same HMM (Viterbi score) [47–49]. At the first level (log-Likelihood score), on a particular test signature, a distance is computed between its log-Likelihood and the average log-Likelihood obtained on the training signatures; then it is shifted to a similarity value—called "Log-Likelihood score"—between 0 and 1, by the use of an exponential function [47]. At the second level of analysis (Viterbi score), the user's HMM automatically performs by the Viterbi algorithm, a segmentation of each training signature into portions, according to the most likely path displayed in Figure 6. A "segmentation vector" can then be associated to each signature: the *N*-components segmentation vector, *N* being the number of states in the claimed identity's HMM has in the *i*-th position the number of points (observations) associated to state *i* by the Viterbi path, as illustrated in Figure 6. Each training signature is then characterized by a *Reference segmentation vector*. In the verification phase, on a particular test signature, a distance between its corresponding segmentation vector and each *Reference segmentation vector* is computed, and such distances are averaged to compute the final distance. It is then shifted to a similarity measure between 0 and 1 (Viterbi score) by an exponential function [47].

**Figure 6.** Computation of a signature's segmentation vector generated by the user's HMM. Feature vectors describing the signature are on the *x*-axis and the left-to-right HMM on the *y*-axis.

Finally, the similarity score for a given test signature is thus the fusion by a simple arithmetic mean of the log-Likelihood score and the Viterbi score. If the final score is higher than the value of the decision threshold the claimed identity is accepted, otherwise it is rejected.

In this work, for simple signature types, we train an HMM per person and per signature type. For hybrid types, we train an HMM for each person considering the whole time sequence constructed by concatenating the time sequences of the concerned simple types. As example, for the SDP type, we train an HMM on the complete sequence composed of the usual signature, the date and the place. Note that according to signature types, the length of the complete signature sequence will vary: for simple signature types, it will tend to increase when considering name-surname and to decrease when considering initials. For hybrid types, the length of the signature will be even higher. This will impact the number of states of the user's HMM.

#### **4. Results**

## *4.1. Quality Measure of Usual Signatures*

In a first step, we quantify the quality of usual signatures of the 74 persons available in our dataset. To this end, we trained for each person, an HMM on 10 genuine signatures to measure the user's PE. Then, a hierarchical clustering was performed on the obtained PE values, resulting in three user categories displayed in Figure 7.

**Figure 7.** Examples of signatures captured in uncontrolled mobile conditions with (**a**) high, (**b**) medium and (**c**) low PE.

In Figure 7a, we observe three examples of signatures with high PE: they are the shortest and the simplest signatures, having the aspect of a flourish, and are the most variable (see Figure 8). Such signatures are considered as being "problematic" in the literature [28,43,45]. On the other hand, Figure 7c shows three examples of signatures with low PE: they are longer, the most complex and the most stable (see Figure 8). In between, there is a transition category in terms of complexity and stability, the category of medium PE (see Figures 7b and 8).

**Figure 8.** Boxplots of PE values for all 74 persons clustered into high, medium and low PE.

#### *4.2. Quality Measure of All Signature Types*

Figure 9 displays two examples of each simple signature type, captured separately on the iPad: usual signature, initials, name-surname, date and place of birth.


**Figure 9.** Examples of the simple signature types: (**a**) usual signature; (**b**) initials; (**c**) name-surname, (**d**) date of birth, and (**e**) place of birth. These signatures belong to persons who have authorized their publication.

For each person, we compute PE values for the five simple signature types separately, and for the seven hybrid signature types: SI, SD, SP, SN, SDP, NDP and SIDP. Figure 10 presents the boxplots of the obtained PE values for the 12 signature types.

**Figure 10.** Boxplots of PE values for the 74 persons per signature type.

We first notice that initials have the highest PE values. This result is coherent since initials are the most simple and variable type of signature. We also notice a strong spread out of their boxplot in Figure 10. Moreover, we observe that some initials have a comparable PE to that of usual signatures: indeed, for initials, some entropy values are below the first quartile. This can be explained by the fact that some persons appended their initials into two, three and even four letters, sometimes linking them as usually done when producing a short signature. In this case, the initials show a higher complexity and stability.

Furthermore, we notice that the more the signature is enriched (name-surname, SI, SD, SP, SN, SDP, NDP, SIDP), the lower PE becomes: the complexity of signatures is higher and variability is lower. The hybrid types SDP, NDP and SIDP are those showing the lowest PE values and the lowest variance between persons in the boxplots displayed in Figure 10.

In the sequel, we study the relationship between information content quantified by PE and verification performance. Our objective is to identify which types of signatures are more resistant to attacks in uncontrolled mobile conditions.
