Reviewing the Usability of Web Authentication Procedures: Comparing the Current Procedures of 20 Websites
Round 1
Reviewer 1 Report (Previous Reviewer 2)
Some minor observations are 1. Perhaps too many references. This may be limited. 2. Experimental design and methodological setup need to be justified and explained. A flow chart would be of help. 3. Please add research implications.Minor editing of English language required
Author Response
Point 1: Perhaps too many references. This may be limited.
Response 1: Thank you for this comment. I removed 13 references that could be considered less relevant to the article. The removed references are [9-11,22,26-29,31-34,38].
Point 2: Experimental design and methodological setup need to be justified and explained. A flow chart would be of help.
Response 2: Thanks for this suggestion. I added a flow chart that summarizes the steps involved to conduct the experiment.
Point 3: Please add research implications.
Response 3: Thanks for this comment. I added research implications in the first paragraph of the discussion section.
Point 4: Minor editing of English language required.
Response 4: Thanks for this comment. Another proofreading round was done to ensure that everything is correct. The English editor has done some minor English modifications.
Author Response File: 
Author Response.pdf
Reviewer 2 Report (Previous Reviewer 1)
Conclusions could be more analytical and could be a connection with the theoretical part
Author Response
Point 1: Conclusions could be more analytical and could be a connection with the theoretical part.
Response 1: Thank you for this comment. I added more information that should make the conclusion more analytical. This information is highlighted in yellow.
Author Response File: 
Author Response.docx
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
The abstract should be rewritten and focus on the present research.
The author refers to “think-aloud” theory. He should explain more this theory.
Albesher and Alhussain found that some answers could be found easily on social media p.9. The author repeat almost the same phrase in introduction
The methodology of the study should be improved. The article refers only to 20 websites in four domains. More details about the reasons that the methods selected should be provided. The sample is quite small. More details about the way that the domains and selected sites should be provided.
The author mentions that he “compared the 20 websites to find similarities and differences between their password practices”. The criteria of comparison should be mentioned.
In the conclusion author should highlight the contribution and novelty of their work.
Author Response
Response to Reviewer 1 Comments
Point 1: The abstract should be rewritten and focus on the present research.
Response 1: Thank you for your comment. You are right, I have just noticed that the beginning part is too long and not focusing on the current research. I removed the first two sentences to make the abstract more focused on the current research. I also merged the sentence of discussion and recommendations together.
Point 2: The author refers to “think-aloud” theory. He should explain more this theory.
Response 2: Thanks for this comment. I explained this theory by adding some sentences in lines 235-243.
Point 3: Albesher and Alhussain found that some answers could be found easily on social media p.9. The author repeated almost the same phrase in the introduction.
Response 3: Thank you for this comment. I agree that this sentence should not be repeated. I removed it from page 9.
Point 4: The methodology of the study should be improved. The article refers only to 20 websites in four domains. More details about the reasons that the methods selected should be provided. The sample is quite small. More details about the way that the domains and selected sites should be provided.
Response 4: Thank you for this comment. I totally agree that the methodology section needs to be improved due to the comments that you kindly mentioned. Thus, I added sentences for each comment.
- Reasons for selecting the method? Some reasons include that no usability labs, participants, or significant expenditure are required. However, the key reason is the ability to test a wide range of websites. For example, with experiments that include participants, only a few websites can be tested. Thus, a sample of the tested websites should be taken to study users’ performance and perceptions.
- Different domains and why these websites? Because responses taken by websites in one domain may not be suitable for the other domains. For instance, some social media partially freeze accounts when there is suspicious login. They still allow the user to log in but with limited activities. This practice might not be appropriate for websites in other domains such as the reservation domain. Thus, besides presenting the comparison with all websites in all domains, this research grants the opportunity to see the practices of websites in the same domain. I added this point in line 215-225. In regards to the selected websites, they were based on the two websites indicated in the research paper besides meeting the criteria of having various domains. Thus, I had to skip some of the websites. I also had to skip the sexual websites since they are prohibited in some countries. I added one sentence in line 222 that explains this point.
- The sample is small? Collecting and analyzing the data for 20 websites took more than six months. Other studies that used the same methodology such as [54,57] had smaller samples than the current study. Furthermore, the researchers of other studies focused on fewer items or one process and thus were able to increase the sample. However, the current study is comprehensive and includes many items in three processes(sing up, sing in, password reset). This response has been added to the beginning of the second paragraph of the methodology section.
Point 5: The author mentions that he “compared the 20 websites to find similarities and differences between their password practices”. The criteria of comparison should be mentioned.
Response 5: Thank your comment. The criteria of comparison are actually the tested items during each process (signing up, signing in, password reset). This point should be clear after replying to the previous comment.
Point 6: In the conclusion author should highlight the contribution and novelty of their work.
Response 5: Thank you for this comment. I added one paragraph at the end that highlights the contribution and novelty of the research.
Reviewer 2 Report
1. The title of the paper is “Evaluating the Usability of Web Authentication Procedures” which is very much relevant in today’s scenario but it is basically a review based paper which compares the different security authentication procedures followed by 20 websites.
2. The paper does not provide any insightful knowledge for which it can be considered for publication in this Journal.
3. It is simply a collection of information regarding the Usability of Web Authentication from different sources put together.
4. The overall flow of this article is not good.
Author Response
Response to Reviewer 2 Comments
Point 1: The title of the paper is “Evaluating the Usability of Web Authentication Procedures” which is very much relevant in today’s scenario but it is basically a review based paper which compares the different security authentication procedures followed by 20 websites.
Response 1: Thank you for your comment. I partially agree with the way you looked at the title. Thus, I changed the title to “Testing the Usability of Web Authentication Procedures: Comparing the Current Procedures of 20 Websites”. It is true that this study reviewed the procedures. However, it also tested different items for every website such as the number of failing attempts the website accepts, and the actions it takes to deal with this case. Another item is testing the functionality of the password meters and taking notes on that. Thus, I did review not only the current authentication procedures but also tested their usability.
Point 2: The paper does not provide any insightful knowledge for which it can be considered for publication in this Journal.
Response 2: I am sorry that it did not meet your expectations.
Point 3: It is simply a collection of information regarding the Usability of Web Authentication from different sources put together.
Response 3: Thank you for your comment. I totally do not agree with this point. I tested various items as it is described in the methodology section. Then I analyzed the results and presented the comparisons between the practices of the tested websites in an understandable manner. After that, I discussed these practices with the findings of other research papers.
Point 4: The overall flow of this article is not good.
Response 4: I am sorry that you found the overall flow of the article not good. I wish there were something specific I could fix.
Author Response File: Author Response.pdf
Reviewer 3 Report
The author/s exhibits sufficient knowledge of the research topic. The author/s gave a depth literature review. The contents are sufficiently substantial and broad-ranging to allow coverage of the field of research. Therefore I recommend acceptance of the manuscript in its present form.
Author Response
Response to Reviewer 3 Comments
Point 1: The author/s exhibits sufficient knowledge of the research topic. The author/s gave a depth literature review. The contents are sufficiently substantial and broad-ranging to allow coverage of the field of research. Therefore I recommend acceptance of the manuscript in its present form.
Response 1: Thank you for this compliment. I am glad that my thinking and working for a few years led to a mature research paper.
Author Response File: Author Response.pdf
Reviewer 4 Report
Here are the comments :
In Line 69 what are "specific issues"?
In section 1.2 "passwords" is it alphanumerical, biometrics, or hybrid?
In line 210 any reason behind choosing yahoo as an email in the experiment?
The author's research recommendations have already been implemented. Are there any additional recommendations?
In Section 7, it was stated that future work would focus on security evaluation and comparison of usability. Why wasn't this done in this research?
Author Response
Response to Reviewer 4 Comments
Point 1: In Line 69 what are "specific issues"?
Response 1: Thank you for your comment. I meant “items”. I forgot to update this word. I have updated it.
Point 2: In section 1.2 "passwords" is it alphanumerical, biometrics, or hybrid?
Response 2: Thank you for your comment. I can clearly see your valid point. Thus, adjusted the beginning of this section and added the word “alphanumeric”. The reason behind selecting “alphanumeric” is its relation with the following sections (password policies & password meters).
Point 3: In line 210 any reason behind choosing yahoo as an email in the experiment?
Response 3: Thank you for this comment. It was selected just because it is not one of the tested websites. I added this sentence in line (231)
Point 4: The author's research recommendations have already been implemented. Are there any additional recommendations?
Response 4: Thank you for your comment. It is true that some recommendations are implemented on some websites. However, there are other recommendations that are not implemented on these websites. I also added two more recommendations.
Point 5: In Section 7, it was stated that future work would focus on security evaluation and comparison of usability. Why wasn't this done in this research?
Response 5: Thank you for your comment. Testing the usability of the websites’ authentication procedures in terms of users’ performance and perception was not done in this research since it would require a new study design which would make this research paper very long. Thus, it is better to have it in another study.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
The study is topical and interesting. But it lacks rigor in analysis. No definitive model or framework has been substantiated.
Author Response
Point 1: The study is topical and interesting. But it lacks rigor in analysis. No definitive model or framework has been substantiated.
Response 1: The method used in this study is the “individual expert review.” The expert discovers the usability flaws based on the rules of thumb and his experience [97]. Thus, the evaluator does not have to follow a specific template during the analysis to capture the usability flaws and provide some recommendations. This approach was found in one series of studies [55–58] that measured the usable security for some websites. I have added this to the first methodology section.
For the previous comments, I added some sentences and references that should highlight the novelty and contribution of this study. The following list summarizes the changes:
- I added one paragraph in the introduction after the first paragraph.
- I edited the contribution paragraph and moved the definition of usable security to section 1.1.
- I changed the word principles to recommendations in the introduction.
- I edited reference 110 since I did not find the correct reference for the old reference.
- I adjusted the order of references.
- I fixed some mistakes at the end references.
Author Response File: Author Response.pdf
Round 3
Reviewer 2 Report
The article has been revised based on previous comments.
Author Response
Point 1: The article has been revised based on previous comments.
Response 1: Thank you.
Author Response File: Author Response.docx
