*3.2. Experiment Results*

Performance evaluations of shape context (SC)-based verification and function features (FF)-based verification method is firstly conducted. Here Skilled and Random denotes skilled forgeries and random forgeries. In the case of common threshold, the Receiver Operating Characteristic (ROC) curves are given to evaluate the performance. As for user-dependent threshold set-up, EER of every user are expressed as histograms.

The results of these two methods are shown in Figures 5 and 6. From the results, we can see that both the SC and FF method perform well, and the better results are achieved using user thresholds on random forgery verification. It is a general statement that the usage of user threshold usually can yield better performance than common threshold, as is proved by the results. For common threshold, it is difficult to use one value to cover the differences of different individuals. For user threshold, the value is user-specific, varying from one user to another.

As descried in the previous section, 20 frequently used features are categorized into 5 groups according to their properties. To achieve best performance and to investigate contributions of different features, we run a series of experiments. Since only single feature or single feature group cannot provide enough classification ability for online signature verification, we test several combinations of feature groups. For clear illustration, we use *G*1 − *G*5 to represent the 5 groups: position-related, pressure-related, velocity-related, acceleration-related, and angle-related. The symbol ∪ denotes combination of different groups. The experimental results are given in Table 4. From the results, we can see that using all 20 features performs the best. It is also shown that when velocity-related FF are removed, verification performance deteriorates a lot.

**Figure 5.** Results of shape context-based verification method (SC). (**a**) ROC curves under common threshold. (**b**) EER of each user under user threshold.

**Figure 6.** Results of function features-based verification method (FF). (**a**) ROC curves under common threshold. (**b**) EER of each user under user threshold.


**Table 4.** Comparisons between different group of function features.

To compare the performances of SC and FF method more clearly, the experimental results of two methods are shown together. Figure 7 gives the results of SC and FF method on skilled forgery while Figure 8 on random one. From the figures, it can be seen that for random forgery verification the performances of SC method and Feature Function method (FF) are similar while FF method outperforms SC method much more on skilled forgery verification. As described in the previous section, SC method is good at extracting global features from signatures with low computation cost, which are quite effective and sufficient for random forgery verification. FF method extracts more detailed features, thus achieving better performance than SC method on skilled forgery verification.

**Figure 7.** Results of SC and FF method on skilled forgery. (**a**) ROC under common threshold. (**b**) EER of each user under user threshold. In addition, those dotted lines are the average levels of corresponding methods.

In real applications, random forgeries occur much more frequently that skilled ones. Based on the experimental results, a cascade verification method is designed and tested. The shape context-based verification method is firstly used to reject obvious random forgeries quickly while the function features-based verification method is applied to re-check the signatures survived from the previous module. As illustrated in Section 2.4, FRR of SC method should be smaller than function features-based verification to achieve higher accuracy with lower computation cost. In case of common threshold, FRR of the SC method is set to be 1% and 65% skilled forgeries and 25% random forgeries can be

accepted by the second module for re-verification. Figures 9 and 10 give the detailed results on SC method, function feature method, and the two-stage method. Table 5 shows the detailed results on EERs. From the results, it can be seen that the two-stage method achieves the best performance with tolerable computation cost.

**Figure 8.** Results of SC and FF method on random forgery. (**a**) ROC under common threshold. (**b**) EER of each user under user threshold. In addition, those dotted lines are the average levels of corresponding methods.

**Figure 9.** Results of SC, FF, and two-stage method on skilled forgery. (**a**) ROC under common threshold. (**b**) EER of each user under user threshold. In addition, those dotted lines are the average levels of corresponding methods.

**Figure 10.** Results of SC, FF, and two-stage method on random forgery. (**a**) ROC under common threshold. (**b**) EER of each user under user threshold. In addition, those dotted lines are the average levels of corresponding methods.

Comparisons with the state of the art on database SVC2004 are given in Table 6. It is not easy to make fair comparisons of online signature verification methods due to different databases, training, testing, etc. We select several recently published works which use the same database (SVC2004 ) with us. The method proposed by Lai et al. [28] based on GRU network obtained slightly higher EER than our method. However, it needs more training samples and consumes more computation costs.


**Table 5.** Verification results (EER%) of different methods with common threshold and user threshold.

**Table 6.** Comparisons with the state-of-the-art works on database SVC2004.

