Joining Federated Learning to Blockchain for Digital Forensics in IoT

Round 1

Reviewer 1 Report

In this study, the authors presented a study to use federated learning to train models locally based on the data stored on the IoT devices using a dataset designed to represent the attacks on the IoT environment, and performed the aggregation via blockchain. This framework showed high  accuracy of over 98%. Overall it is an interesting work, and the  the logic behind each step and conclusion is well supported and suggests the possibility for future study. However, the following questions need to be explained before its publication in this high quality journal:

1. when splitting the date to simulate the different client, is it a random split? But in real world, there should be user bias, is there a better way for splitting the dataset? 3 or 4 clients simulated seems to be a very small number, have we tried larger number of clients?

2. When aggregation, do we need to add some differential privacy? Have we tried different way of aggregation (e,g AVG?). These technical details need to be provided.

Author Response

Dear reviewer

Thank you for your time and efforts helping us to improve the quality of this paper.

Please find the list of responses in the attached file.

Author Response File: Author Response.doc

Reviewer 2 Report

In this paper, the authors introduce a framework for implementing federated learning using data stored on IoT devices for model training. In order to simulate attacks on IoT environments, this dataset was specifically designed. In the following step, the authors used blockchain technology to aggregate parameters from the IoT gateway, maintaining a lightweight blockchain. There are several major concerns regarding the paper, even though it is well-written and organized.

 

Major comments:

 

1. The Introduction section should be strengthened. Although the importance of the topic is stated, the short related works and the contributions of the paper are not adequately mentioned.

 

2. This paper uses some terms that seem to define when the authors are speaking. For example, Merkle tree, CDW_FedAvg, and so forth.

 

3. How did you determine that reLU was the best activation function? Why didn't you try tuning hyperparameters? Can you explain why you did not choose DNN instead of the MLP with more hidden layers? Talk about it in the paper.

 

4. Combine three tables (Tabel 2, 3 and 4) and use the caption "MLP parameters for N clients where N = [2,3,4]" and change the size to 5x39xN. Tables 5,6 and 7 have the same objection.

 

5. A detailed description of the features and outputs was not provided.

 

6. The details of Algorithms 1 and 2 are not discussed very well. As an example, what are the parameters of Algorithm 1?  What is the procedure for updating the decision parameters in Algorithm 2? According to which policy? How would you define satisfying? What does the word "satisfaction" mean? It should be discussed in more detail in the paper.

 

7. In the Experiment Study section, you mentioned that “In case of an attack, smart contract will also create a report that includes the attack details.” What do you mean by attack details? How does the smart contract do that, and what parameters are reported?

 

8. Can you tell me more about your policy regarding dividing the dataset and sharing it with different clients?

 

9. Please explain more about your scenario with Ganache and Metamask at the beginning of the Experiment study section.

 

10. Can you explain how you calculated the Gwei parameter in Tables 5 to 7? Are there any formulas or anything like that?  The paper should discuss this parameter in more detail.

 

11. Using Figure 3, you concluded that “we noticed that as the clients number increases the cost of parameters aggregation increases”. So what do you suggest? Can we achieve better performance by increasing the number of clients or will this increase costs? In order to achieve the best scenario, what is the optimum number of clients?

 

12. In comparison to centralized deep learning, what are the overheads of your proposed solution? You are advised to provide quantitative comparisons within the Overall Comparison subsection in terms of performance, accuracy, overheads, security, privacy, cost, etc.

 

13. It is strongly recommended to proofread the whole paper for grammatical and typo errors.

 

14. It is recommended to provide a label for the y-axis and draw figures with more quality.

 

Minor comments:

1.       It is suggested to hyperlink the references, figures, algorithms and tables.

 

2.       There are some typos that should be corrected in the paper, such as:

·         In Figure 1 caption, [14.] -> [14].

·         Page 4, Line 163, [16]suggested -> [16] suggested

·         Page 14, Line 452, To our best knowledge, In the literature -> To the best of our knowledge, in the literature

·         Page 14, Line 462, dataset [23] Figure 7 shows -> dataset [23], Figure 7 shows

 

3.       I strongly suggest to place the whole Table 1 in one page.

 

4.       For more clarity, it is suggested to draw vertical lines of Table 1.

 

5.       Page 8, Figure 2, there are two captions for this figure, above and below!

 

It is strongly recommended to proofread the whole paper for grammatical and typo errors.

Author Response

Dear reviewer

Thank you for your time and efforts helping us to improve the quality of this paper.

Please find the list of responses in the attached file.

Author Response File: Author Response.doc

Round 2

Reviewer 2 Report

All of my concerns have been addressed by the authors now.  It is only necessary to proofread the manuscript for punctuations and presentational aspects.