Biometric System Performance

For this purpose, the database generated, UC3M-CV1, was used. As is mandatory by the normative [29], the False Match Rate (FMR) and False Non-Match Rate (FNMR) were provided in a Detection Error Trade-Off (DET) plot (recommended). Failure-To-Enrol Rate (FTER) and Failure-To-Acquire Rate (FTAR) were unknown.

The first approach of the biometric performance obtained from the UC3M-CV1 database, Figure 12a–c shows, respectively, for each algorithm the error rate in % versus the threshold (number of coincident key points that determine the acceptance or rejection of the user). These graphics discussed the FMR and the FNMR according to the threshold for the 1100 intraclass or mated comparisons (50 subjects × 2 wrist patterns × 11 samples), and the 108,900 interclass or non-mated comparisons (100 wrist patterns × 99 wrist patterns × 11 samples) made.

The threshold, shown in the decision subsystem in Figure 1, was the number of coincident points to choose and accept a user. The green curve represents as a percentage the False Match Rate (FMR) or value of samples compared that should be rejected but are accepted by the algorithm. The red curve represents as a percentage the False Non-Match Rate (FNMR) or value of samples compared that should be accepted but are rejected by the algorithm.

**Figure 12.** Results: biometric system performance. It is represented the percentage of error, False Match Rate (FMR) (line-dot green curve), and False Non-Match Rate (FNMR) (continuous red curve) versus the threshold of the number of coincident key points. (**a**) SIFT®. (**b**) SURF®. (**c**) ORB.

These graphics anticipate the best biometric performance for SIFT® and the worst for ORB, just analyzing the high value for the crossing point, EER: 21.76%, 32.29%, and 39.94% for respectively, SIFT®, SURF®, and ORB, and the integer thresholds of 9, 25, and 1.

In order to verify this prediction, the Detection Error Trade-Off (DET) curves were obtained (Figure 13) for the collected UC3M-CV1 database and according to the three algorithms (SIFT®, SURF®, and ORB, respectively in Figure 13a–c) and the two capture sessions (Session 1 with green curve, Session 2 with cyan line-dot curve and the entire dataset with yellow gap-line curve).

**Figure 13.** Results: Biometric system performance. Detection Error Trade-Off curve. The False Non-Match Rate is represented versus the False Match Rate. The green (continuous), cyan (line-dot), and yellow (line-line) curves are respectively for Session 1, Session 2, and the full database. (**a**) SIFT®. (**b**) SURF®. (**c**) ORB.

The results for SIFT® were confirmed as significantly better for all the sessions. The curves were clearly closer to small values for the SIFT® case. It is interesting to point out that, for the three algorithms, the biometric performance improved in the second session, comparing it with the first, showing a more homogeneous placement of the wrist by the subjects in the capture. Otherwise, the curve performance for the entire UC3M-CV1 dataset gets worse. These results have been clearly treated in the discussion section.

In order to compare them with the results obtained with a physical contact database, the UC3M [8] and PUT [6] dataset have been processed with the same software algorithms. The compared results are shown below.

Although the EER value is deprecated according to the normative [29], Figure 14 shows this value obtained for SIFT®, SURF®, and ORB.

**Figure 14.** Results: Biometric system performance. Equal Error Rate (EER) obtained for each database using TGS-CVBR® and PIS-CVBR® with SIFT®, SURF®, and ORB algorithms.

The better performance for this working point for the UC3M-CV1 and the two other databases was obtained using, respectively, SIFT®, SURF®, and ORB algorithms.

For the UC3M-CV1 database and the three algorithms, the EER value is reduced in the second session (from 10.16 in Session 1 to 8.59 in Session 2, in SIFT® case), most probably, as it was mentioned, due to the practice of the subject using the system. However, obtaining the results for both sessions, the full database, the EER reached inadmissible values: 21.76%, 32.29%, and 39.94%. These EERs should not be compared with the results obtained and presented in the current state-of-the-art with physical contact devices.

For the UC3M database, the results were much better, probably due to three factors:


For the PUT database, the EER values were higher than in the UC3M database but lower than in the UC3M-CV1. The capture contact system [6] obtained worse quality images but homogeneous in illumination without external light influence.

In order to ratify these results, the DET graphics for the two other databases used are shown in Figure 15a (UC3M) and Figure 15b (PUT).

**Figure 15.** Results: Biometric system performance. DET curve. The FNMR percentage is represented versus the FMR percentage. The purple (continuous), blue (line-dot), and orange (line-line) curves are respectively for the SIFT®, SURF®, and ORB. (**a**) UC3M. (**b**) PUT.

The EER obtained for the UC3M dataset with SIFT®, shown in the extended purple DET curve of Figure 15a, reached 0.08%. This value was significantly lower than the obtained in all the studies of Table 1 (state-of-the-art summary). In the case of the PUT dataset, the results are clearly improvable, comparing the current values with ones of the state-of-the-art. This comparison denotes the high

correlation that exists between the design of the software algorithms and the datasets in which they are tested, in agreement with the way the datasets have been collected (mainly the capture device).

Finally, it is essential to remark that in all studied cases, SIFT® and SURF® algorithms obtained better results, but also, the computational cost was significantly higher for the key points extraction and matching, as it is analyzed in processing-time performance unit.

The results were obtained using Python™ 3.4.2. and Matlab® R2019b.

## Processing-Time Performance

Figure 16, for the three algorithms, shows the computing time spent in the completed UC3M-CV1, UC3M [8] and PUT [6] databases for the preprocessing, the feature extraction (generation of key points and its descriptors), the intraclass (mated) and interclass (non-mated) comparison (feature matching) and the total time. The hardware used for processing, Raspberry® Pi 4 Model B [32], provided with 64-bit ARM-Cortex A72 (1.5 GHz, quad-core), 4 GB RAM and 128 GB external flash memory, is compared with the Asus® K55V-SX441H [44] 64-bit laptop provided with Intel® Core™ i7 3630QM (2.4 GHz, quad-core), 8 GB RAM and 1 TB of SSD memory. As can be seen, the processing time was considerably reduced in the laptop.

**Figure 16.** Results: Processing-time performance in seconds for each database and its computing hardware using TGS-CVBR® and PIS-CVBR® with SIFT®, SURF®, and ORB algorithms. Preprocessing time is in green color due to its independence with feature extraction algorithms.

The ORB algorithm was faster in all aspects. SURF® was always slower matching features than SIFT®, that is slower extracting descriptors. The processing time of the PUT database was higher than in the UC3M-CV1 database (same number of images, subjects, and comparisons) due to the ".bmp" image non-compressed format. It has been verified that there was no loss of information with the ".jpg" format for the algorithms used. The results were obtained using Python™ 3.4.2. and Matlab® R2019b.

The processing time performance of the real-time authentication and identification system is summarized in Table 2 for the three algorithms used and the UC3M-CV1 database.

**Table 2.** Results: final system software. Processing-time performance in Frames Per Second (FPS) for the UC3M-CV1 database and its computing hardware using TGS-CVBR® and PIS-CVBR® with SIFT®, SURF®, and ORB algorithms.


\* Framerate too low to obtain a real-time processing system.

As has been indicated previously, in the processing time performance of PIS-CVBR®, SIFT® and SURF® were slow algorithms with a high computational cost. This was reflected in obtaining reduced values of Frames Per Second. As the footer of Table 2 indicates, values below 2–3 FPS were considered too low rates to obtain a real-time processing system, producing an unacceptable lag effect.

The results have been obtained using Python™ 3.4.2.
