DoubleStrokeNet: Bigram-Level Keystroke Authentication

Round 1

Reviewer 1 Report

The authors focused on DoubleStrokeNet for authenticating seven users through keystroke analysis utilizing bigram embeddings. However, the paper is currently not ready for publication, primarily due to the weak novelty of the work. Here are suggestions for improvement:

1. Increase the Number of References: The paper should incorporate a more extensive list of references to provide a stronger foundation and context for the research.
2. Expand the Introduction: It is recommended to extend the introduction to offer a more thorough overview of the research, including a detailed problem statement and clear motivation for the study.
3. Eliminate First Person Pronouns: To maintain a formal and objective tone, the authors should avoid using first-person pronouns such as "we" and "our."
4. Focus on Methodology, Minimize Repetition: The paper should emphasize the DoubleStrokeNet methodology without unnecessary repetition of content already covered in related works.
5. Clarify Batch Normalization (Batch Norm): A detailed explanation of batch normalization (Batch Norm) should be provided to ensure a clear understanding of its role and implementation within the methodology.
6. Create a Table for Keystroke Analysis: Propose the creation of a table demonstrating the effects of increasing the number of keystrokes by a specific amount (e.g., 2,4,6) to illustrate the changes in the analysis.
7. Diversify Testing Locations: Suggest conducting tests in various locations beyond Aalto University to validate the effectiveness and applicability of the proposed method in different environments.
8. Present a Comprehensive Array of Results: Encourage the authors to showcase additional experimental results and data to provide a thorough evaluation of DoubleStrokeNet.
9. Include Work Limitations in Conclusion: The conclusion should incorporate a section outlining the limitations of the research, acknowledging any constraints or areas where the methodology may fall short.

 

English is fine

Author Response

The authors focused on DoubleStrokeNet for authenticating seven users through keystroke analysis utilizing bigram embeddings. However, the paper is currently not ready for publication, primarily due to the weak novelty of the work. Here are suggestions for improvement:

Response: Thank you kindly for your thorough review.

1. Increase the Number of References: The paper should incorporate a more extensive list of references to provide a stronger foundation and context for the research.

Response: We have expanded the list of references in the paper to provide a more comprehensive foundation and context for the research. This addition aims to enhance the academic rigor and credibility of the study.

2. Expand the Introduction: It is recommended to extend introduction the to offer a more thorough overview of the research, including a detailed problem statement and clear motivation for the study.

Response: We have carefully considered your suggestion and are pleased to inform you that we have expanded the introduction section to provide a more comprehensive overview of our research. This includes a detailed problem statement and a clear motivation for the study, which we believe will enhance the overall quality and clarity of the paper.

3. Eliminate First Person Pronouns: To maintain a formal and objective tone, the authors should avoid using first-person pronouns such as "we" and "our."

Response: We have duly revised the content accordingly to align with the formal and objective tone you have suggested.

4. Focus on Methodology, Minimize Repetition: The paper should emphasize the DoubleStrokeNet methodology without unnecessary repetition of content already covered in related works.

Response: We have taken the necessary steps to refine the paper's focus on the DoubleStrokeNet methodology, ensuring that there is minimal redundancy of content already addressed in related works. The primary focus has shifted towards highlighting the distinct contributions and innovative aspects introduced by DoubleStrokeNet within the task of keystroke biometric authentication.

5. Clarify Batch Normalization (Batch Norm): A detailed explanation of batch normalization (Batch Norm) should be provided to ensure a clear understanding of its role and implementation within the methodology.

Response: We have taken this into consideration and have subsequently incorporated the necessary clarifications to ensure a clear understanding of the role of Batch Norm.

6. Create a Table for Keystroke Analysis: Propose the creation of a table demonstrating the effects of increasing the number of keystrokes by a specific amount (e.g., 2,4,6) to illustrate the changes in the analysis.

Response: It is crucial to acknowledge that as the number of keystrokes increases, there is a likelihood of encountering duplicated temporal features or highly correlated attributes. This phenomenon may potentially lead to a decrease in the overall performance of our analysis. Given this consideration, , it may be advisable to conduct this proposed analysis in future work.

7. Diversify Testing Locations: Suggest conducting tests in various locations beyond Aalto University to validate the effectiveness and applicability of the proposed method in different environments.

Response: We wholeheartedly agree with the importance of such validation. However, it is worth noting that the Aalto University dataset is currently the only publicly available typing dataset accessible to us. Regrettably, despite our best efforts, other institutions with typing datasets did not respond to our inquiries.

8. Present a Comprehensive Array of Results: Encourage the authors to showcase additional experimental results and data to provide a thorough evaluation of DoubleStrokeNet.

Response: Thank you for the suggestion to present a more comprehensive array of results. However, due to current time constraints, it may not be feasible to conduct further experiments at this point. The conducted experiments are rigorously detailed and provide a solid foundation for evaluating DoubleStrokeNet.

9. Include Work Limitations in Conclusion: The conclusion should incorporate a section outlining the limitations of the research, acknowledging any constraints or areas where the methodology may fall short.

Response: The conclusion section has been updated to include a paragraph outlining the limitations of the research. It addresses two key constraints: the need for fine-tuning when adding new users to the authentication system, and the considerable computational resources required by the DoubleStrokeNet architecture. These limitations are essential considerations in understanding the practical applicability of the proposed keystroke authentication system.

Reviewer 2 Report

The paper introduces DoubleStrokeNet, a novel approach to keystroke authentication, focusing on bigram embeddings and employing a Transformer-based neural network for analysis. In experiments, it outperforms TypeNet-based systems with EER values of 0.75% for physical keyboards and 2.35% for touchscreen keyboards using 10 enrollment typing sequences.

 

 

-  Please clearly state your contribution in the introduction.

- " The global biometric system market size was valued at 26 USD 29.09 billion in 2021 and USD 30.77 billion in 2022." Where is the reference to this claim?

- What does CAGR mean?

What does WPM mean?

- Is there a particular reason for following the ELECTRA method in your design?

- It is difficult to validate the results of the experiment without appropriate comparison with prior work. 

NA

Author Response

The paper introduces DoubleStrokeNet, a novel approach to keystroke authentication, focusing on bigram embeddings and employing a Transformer-based neural network for analysis. In experiments, it outperforms TypeNet-based systems with EER values of 0.75% for physical keyboards and 2.35% for touchscreen keyboards using 10 enrollment typing sequences.

Response: Thank you kindly for your thorough review.

-  Please clearly state your contribution in the introduction.

Response: We have carefully reviewed your suggestion and are pleased to inform you that we have made the necessary modifications to clearly state our contribution in the introduction section. In our revised introduction, we explicitly outline the contribution of our research.

- " The global biometric system market size was valued at 26 USD 29.09 billion in 2021 and USD 30.77 billion in 2022." Where is the reference to this claim?

Response: We have now included the appropriate reference for the statement regarding the global biometric system market size.

- What does CAGR mean?

What does WPM mean?

Response: We have now included the abbreviations.
CAGR - Compound Annual Growth Rate
WPM – Words Per Minute

- Is there a particular reason for following the ELECTRA method in your design?

Response: Our decision to employ the ELECTRA method in our design is motivated by its distinct advantages for our specific application. While existing deep learning approaches have made significant progress in generating biometric signatures for entire typing sequences, our approach focuses on discerning individual bigrams. This allows us to capture finer-grained details in typing patterns. Additionally, our use of self-supervised learning for acquiring embeddings of both bigrams and users enables us to leverage a vast amount of unlabeled data.

- It is difficult to validate the results of the experiment without appropriate comparison with prior work.

Response: We completely understand your concern regarding the validation of the experiment results. To address this, we have included a comparison with other architectures in the paper. This includes a detailed evaluation of DoubleStrokeNet alongside several established systems. This comprehensive comparison aims to provide a solid benchmark for evaluating the performance of our proposed approach in relation to prior work.

Reviewer 3 Report

This paper presents DoubleStrokeNet, a transformer-based keystroke authentication model designed to discriminate genuine users from others by leveraging keystroke timings and keycodes. The evaluation results on a keystroke dataset demonstrate a great performance of the proposed model in both desktop and mobile scenarios.

Strengths of this paper:

1. The paper provides a comprehensive overview of the proposed method and the evaluation dataset.

2. The diagrams and figures in this paper illustrate the DoubleStrokeNet architecture and data flows clearly.

Areas for improvement:

1. This paper lacks a comparative study of related keystroke authentication models. Only TypeNet is closely analyzed and evaluated on the Aalto dataset. The author could add more models, including those using traditional machine learning methods such as SVM, and more datasets to the evaluations.

2. DoubleStrokeNet relies heavily on keycodes. Will the performance of DoubleStrokeNet degrade significantly if the input keycodes are undefined? What's the performance of other comparative models if they can utilize keycodes as a feature?

3. It would be valuable if the impact of keyboard and touch screen dimensions could be considered in the model design and evaluations.

The paper is overall well-written and well-organized but there are some minor spelling and grammar issues to be fixed.

Author Response

This paper presents DoubleStrokeNet, a transformer-based keystroke authentication model designed to discriminate genuine users from others by leveraging keystroke timings and keycodes. The evaluation results on a keystroke dataset demonstrate a great performance of the proposed model in both desktop and mobile scenarios.

Strengths of this paper:

1. The paper provides a comprehensive overview of the proposed method and the evaluation dataset.
2. The diagrams and figures in this paper illustrate the DoubleStrokeNet architecture and data flows clearly.

Response: Thank you kindly for your thorough review.

Areas for improvement:

1. This paper lacks a comparative study of related keystroke authentication models. Only TypeNet is closely analyzed and evaluated on the Aalto dataset. The author could add more models, including those using traditional machine learning methods such as SVM, and more datasets to the evaluations.

Response: I completely understand your concern regarding the validation of the experiment results. To address this, we have included a comparison with other architectures in the paper. This includes a detailed evaluation of DoubleStrokeNet alongside several established systems. This comprehensive comparison aims to provide a solid benchmark for evaluating the performance of our proposed approach in relation to prior work.

2. DoubleStrokeNet relies heavily on keycodes. Will the performance of DoubleStrokeNet degrade significantly if the input keycodes are undefined? What's the performance of other comparative models if they can utilize keycodes as a feature?

Response: If input keycodes are undefined, the performance of DoubleStrokeNet would indeed be affected. Specifically, eliminating keycodes would result in the removal of a crucial source of information, which could significantly impact per-token performance. In the absence of keycodes, the remaining features would primarily consist of temporal differences, totaling four. This reduction in the feature set could lead to diminished discriminatory power, potentially affecting the model's performance compared to scenarios where keycodes are available. To address your second point, it is noteworthy that previous models in this domain typically leverage keycodes as an essential input feature.

3. It would be valuable if the impact of keyboard and touch screen dimensions could be considered in the model design and evaluations.

Response: While we acknowledge the potential influence of these factors on keystroke dynamics, we have deliberately designed our model to be device-agnostic. By adopting this approach, we aim to ensure the applicability and effectiveness of our proposed DoubleStrokeNet across a wide range of devices and form factors.

Reviewer 4 Report

The authors' work in this paper is entitled "DoubleStrokeNet: Bigram-Level Keystroke Authentication". The contribution of this paper is the proposal of DoubleStrokeNet, a bigram-level keystroke authentication technique that uses a transformer-based architecture to model keystroke sequences.

We have several suggestions that concern the research design and the quality of the language. However, we approve the publication of this article under minor revisions:

 

- Line 254:  After writing the Eq. (2), give the meaning of D(x,y);

- Line 317: What policy is used to select the test sample!? And why this policy?

- Line 335: How the threshold is adjusted to calculate EER !?

- Line 355: What method was used to set hyperparameter values!?

See more in the attached file.

 

Comments for author File: Comments.pdf

Several improvements must be made for an acceptable English quality of this paper(see more in the file "electronics-2630319-peer-review-v1_done.pdf") such as:

- Line 6: The definite article seems necessary before “Euclidean”.

- Line 32: Use “because” instead of “as” in this context.

- Line 38: Use “based” instead of “built”.

- Line 85: Use “focus” instead of “are centered”.

- Line 143: Insert “hand” after "right" for clarity.

- Line 207: Use “begin” instead of “start”.

- Line 241: Consider deleting “In order” here.

- Line 265: The verb “assign” appears to be in the incorrect tense; revise it to “assigned”.

 

Author Response

The authors' work in this paper is entitled "DoubleStrokeNet: Bigram-Level Keystroke Authentication". The contribution of this paper is the proposal of DoubleStrokeNet, a bigram-level keystroke authentication technique that uses a transformer-based architecture to model keystroke sequences.

We have several suggestions that concern the research design and the quality of the language. However, we approve the publication of this article under minor revisions:

- Line 254:  After writing the Eq. (2), give the meaning of D(x,y);

- Line 317: What policy is used to select the test sample!? And why this policy?

- Line 335: How the threshold is adjusted to calculate EER !?

- Line 355: What method was used to set hyperparameter values!?

See more in the attached file.

Several improvements must be made for an acceptable English quality of this paper(see more in the file "electronics-2630319-peer-review-v1_done.pdf") such as:

- Line 6: The definite article seems necessary before “Euclidean”.

- Line 32: Use “because” instead of “as” in this context.

- Line 38: Use “based” instead of “built”.

- Line 85: Use “focus” instead of “are centered”.

- Line 143: Insert “hand” after "right" for clarity.

- Line 207: Use “begin” instead of “start”.

- Line 241: Consider deleting “In order” here.

- Line 265: The verb “assign” appears to be in the incorrect tense; revise it to “assigned”.

Response: Thank you for your valuable feedback on our paper. We have addressed all your points, clarifying D(x,y) after Eq. (2), explaining the test sample selection policy, and detailing the threshold adjustment for EER. Additionally, we've improved language quality as per your suggestions.

Round 2

Reviewer 1 Report

The authors answered all my questions. 

A grammar check is needed. 

Author Response

Thank you kindly for all your valuable feedback.

Reviewer 2 Report

Thank you for addressing my previous comments.

Please make sure to add your explanation of choosing the DoubleKeyStroke approach in the manuscript itself as well. 

NA

Author Response

Thank you kindly for all your valuable feedback. We double-checked that the additional explanations are present in the manuscript, both in the introduction and conclusions.

Reviewer 3 Report

The authors have addressed my concerns from the previous review report by clarifying specific design choices and introducing evaluation results from related studies. Overall, this article offers a valuable perspective on keystroke-based biometrics that should engage Electronics readers. As a minor point, I would like to see how this method performs in real-world applications on heterogeneous devices in terms of accuracy and latency.

Author Response

Thank you kindly for all your valuable feedback. The same applies on our side as we will consider real-world applications and heterogeneous devices as future extensions.