A Double-Blind Trial Platform Based on Distributed Ledger Technology
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsI would like to thank the authors for this very interesting piece of research. The paper is very well written and pleasant to read. It is adequately structured and comprehensive. Two minor points one could remark:
* Please elaborate a bit more on the advantages of your approach in comparison to the ones described in section 2.2, exceeding the information in Table 1.
* Could you provide deeper insights into your code to make it easier to understand the functional principles? Is BlindBox open-source? If yes, could you provide a link to a repository or the like?
Author Response
Comments and Suggestions for Authors
I would like to thank the authors for this very interesting piece of research. The paper is very well written and pleasant to read. It is adequately structured and comprehensive. Two minor points one could remark:
- Please elaborate a bit more on the advantages of your approach in comparison to the ones described in section 2.2, exceeding the information in Table 1.
Response) Thanks for the suggestion. We have added more description about the advantages of our approach compared with the others in section 2.2, as shown on pages 8 and 9.
- Could you provide deeper insights into your code to make it easier to understand the functional principles? Is BlindBox open-source? If yes, could you provide a link to a repository or the like?
Response) Thanks for the suggestion. We have added an implementation section in the revised manuscript to provide insight into the critical codes of executing double-blinding trials in the proposed platform, as shown in Section 4. We will consider opening the source of core functions of the proposed platform in the future.
Reviewer 2 Report
Comments and Suggestions for AuthorsThis article presents a distributed ledger-based double-blind trial platform called “BlindBox.” According to the authors, this platform exploits the immutability and decentralization of distributed ledgers to improve the security of experimental data via utilizing smart contracts to restrict personnel access to experimental data and coupling them with RFID technology to encode and shuffle the experimental drugs discreetly. This use ensures that no one can ascertain the pairing between test subjects and drug groups, preventing collusion and result manipulation. When the trial period concludes, smart contracts automatically unblind the results and publish them on the IOTA platform.
Regarding the proposal itself, I have the following comments/questions.
1. How does the proposed proposal compare with other similar proposals in terms of security goals? I think the provided comparison between your proposal and others should be updated to include the pros and cons of your proposal in terms of security goals.
2. Throughout the article I only saw a performance evaluation of the proposed platform, thus I think that adding a formal analysis of the security goals provided by the proposal to the paper would enhance its value. In particular, evaluate the platform by analyzing some abuse cases that potentially could exploit it, from which you can abstract and give arguments for your claimed security goals. (e.g. To prevent collusion between researchers and participants).
Comments on the Quality of English LanguageRegarding the English language: the paper is generally well-written. Just one editorial comment:
line 362: interested sites ---> an interested site
Author Response
- How does the proposed proposal compare with other similar proposals in terms of security goals? I think the provided comparison between your proposal and others should be updated to include the pros and cons of your proposal in terms of security goals.
Response) In terms of security goals, both our work and related work exploit the blockchain mechanisms for maintaining data confidentiality, integrity, and availability. However, we resolve the problems of researcher-subject collusion, early unblinding, fake experimental results, and information monopoly to prevent illegal benefit transfer and insider trading in the medicine market and avoid damaging public health by incorrect trial results. By contrast, the others did not resolve these problems.
- Throughout the article I only saw a performance evaluation of the proposed platform, thus I think that adding a formal analysis of the security goals provided by the proposal to the paper would enhance its value. In particular, evaluate the platform by analyzing some abuse cases that potentially could exploit it, from which you can abstract and give arguments for your claimed security goals. (e.g. To prevent collusion between researchers and participants).
Response) Thanks for the suggestion. As we know, none of the related work in Table 1 provides a formal analysis of the security goals in their publications because they trust and rely on the blockchain mechanism to promise security goals. In addition, the publications [1] and [2] have done a deep survey and analysis of proposed blockchains and consensus algorithms on the issues of security, scalability in networks, cyber-attacks, energy efficacy, etc. Due to considering paper length and our focus, we did not add the analysis of the security goals in the revised paper. However, we have explained how our approach prevents collusion between researchers and participants more clearly as shown on page 9.
[1] Huaqun Guo, Xingjie Yu, A survey on blockchain technology and its security, Blockchain: Research and Applications, Volume 3, Issue 2, 2022, 100067. https://doi.org/10.1016/j.bcra.2022.100067.
[2] David Berdik, Safa Otoum, Nikolas Schmidt, Dylan Porter, Yaser Jararweh, A Survey on Blockchain for Information Systems Management and Security, Information Processing & Management, Volume 58, Issue 1, 2021, 102397, https://doi.org/10.1016/j.ipm.2020.102397.
Reviewer 3 Report
Comments and Suggestions for AuthorsThe paper presents a novel platform called "BlindBox," which addresses key challenges in traditional double-blind trials in pharmaceutical research. These challenges include the risks of data loss, tampering, and biases due to centralized data management by a single institution. The BlindBox platform leverages distributed ledger technology, specifically a combination of Ethereum and IOTA blockchains, along with smart contracts and RFID technology, to enhance the security, integrity, and fairness of clinical trials.
Strengths
1. The platform creatively integrates blockchain technology, smart contracts, and RFID to address major issues in clinical trial management.
2. The use of distributed ledgers and smart contracts ensures data integrity, security, and transparency throughout the trial process.
3. The platform effectively reduces costs and increases efficiency in managing and accessing trial data.
4. BlindBox covers many aspects of a clinical trial, from experiment setup and participant recruitment to data collection and result dissemination.
Weaknesses:
1. While the paper integrates blockchain technology into clinical trials, it doesn't clearly delineate how its approach is significantly different or superior to existing blockchain-based solutions in healthcare. The literature review shows several other blockchain implementations in similar contexts, but the paper does not thoroughly compare its novel contributions against these existing frameworks​​.
2. The methodology, particularly in the design and implementation of the smart contracts and RFID system, lacks detailed technical descriptions. For researchers or practitioners looking to understand or replicate the system, the paper does not provide sufficient depth, especially regarding the technical specifications and security mechanisms of the smart contracts.
3. The evaluation mainly focuses on deployment time and operational efficiency​​. However, it lacks a comprehensive assessment of the platform's performance in real-world scenarios, such as its behavior under varied load conditions, its resilience to network failures, or its response to security threats. These aspects are crucial for a platform handling sensitive clinical trial data.
4. The paper does not address scalability concerns in depth. For instance, how the system performs when handling a large number of participants or dealing with extensive data sets typical in Phase 3 clinical trials is not discussed. Performance metrics under such conditions would be critical for understanding the platform's real-world applicability.
5. While the use of blockchain inherently suggests a focus on security and data integrity, the paper does not provide a detailed analysis of how the platform ensures data privacy and complies with regulatory requirements, which are crucial aspects in clinical trials.
6. The paper proposes the use of RFID for experimental item matching, but it does not delve into the practical challenges of implementing RFID in a clinical trial setting, such as issues with signal interference, hardware reliability, and the logistics of tagging and scanning a large number of items.
7. I can't really understand the figure 16 and match it with the description.
The "BlindBox" paper presents an innovative approach to enhancing the integrity and efficiency of double-blind clinical trials through the integration of blockchain technology, smart contracts, and RFID. However, the paper has several specific areas of weakness that need to be addressed. These include a lack of clear distinction in novelty from existing blockchain-based solutions in healthcare, insufficient methodological depth, limited evaluation in realistic scenarios, and inadequate comparative analysis with related work. Additionally, concerns about scalability, data privacy and security, practical challenges of RFID implementation, and performance under real-world conditions are not thoroughly addressed.
Author Response
- While the paper integrates blockchain technology into clinical trials, it doesn't clearly delineate how its approach is significantly different or superior to existing blockchain-based solutions in healthcare. The literature review shows several other blockchain implementations in similar contexts, but the paper does not thoroughly compare its novel contributions against these existing frameworks​​.
Response) Thanks for the comment. We have described our contributions against these existing frameworks more clearly on page 9.
- 2. The methodology, particularly in the design and implementation of the smart contracts and RFID system, lacks detailed technical descriptions. For researchers or practitioners looking to understand or replicate the system, the paper does not provide sufficient depth, especially regarding the technical specifications and security mechanisms of the smart contracts.
Response) We have added an implementation section in the revised manuscript to provide insight into the critical codes of executing double-blinding trials in the proposed platform, as shown in Section 4. The smart contract used for the implementation of the proposed platform is version 0.8.11. We employ the require() and assert() statements to adhere to the Checks-Effects-Interactions pattern. In addition, the used RFID device is DRD 050, which executes the EPC Class 1 Gen 2 V2 communication protocol, and its source has been published on Git Hub. We access the RFID reader/writer using the node-hid package and conduct experiments with sticker-type labels.
The evaluation mainly focuses on deployment time and operational efficiency​​. However, it lacks a comprehensive assessment of the platform's performance in real-world scenarios, such as its behavior under varied load conditions, its resilience to network failures, or its response to security threats. These aspects are crucial for a platform handling sensitive clinical trial data.
Response) As shown in Table 1, all the related work uses only private blockchains to implement their proposed platforms, and only one shows simulation results. By contrast, we use hybrid blockchains to implement our platform and show the experimental results of our platform’s performance. In the future, we will conduct a comprehensive evaluation in real-world scenarios by collaborating with hospitals as described on page 30.
The paper does not address scalability concerns in depth. For instance, how the system performs when handling a large number of participants or dealing with extensive data sets typical in Phase 3 clinical trials is not discussed. Performance metrics under such conditions would be critical for understanding the platform's real-world applicability.
Response) Thanks for the comment. In this paper, we focus on resolving the problems of researcher-subject collusion, early unblinding, fake experimental results, and information monopoly to prevent illegal benefit transfer and insider trading in the medicine market and avoid damaging public health by incorrect trial results. According to the experimental results, the performance of the proposed platform is basically dominated by the efficiency of blockchain networks. The scalability of blockchain networks has been discussed and addressed in the publications [1][2][3]. Although we did not discuss the scalability issue in this paper, we will enhance the scalability of the proposed platform in the future, as described on page 30.
[1] J. Xie, F. R. Yu, T. Huang, R. Xie, J. Liu and Y. Liu, A Survey on the Scalability of Blockchain Systems, in IEEE Network, vol. 33, no. 5, pp. 166-173, Sept.-Oct. 2019, doi: 10.1109/MNET.001.1800290.
[2] Q. Zhou, H. Huang, Z. Zheng and J. Bian, Solutions to Scalability of Blockchain: A Survey, in IEEE Access, vol. 8, pp. 16440-16455, 2020, doi: 10.1109/ACCESS.2020.2967218.
[3] M. H. Nasir, J. Arshad, M. M. Khan, M. Fatima, K. Salah, R. Jayaraman, Scalable blockchains — A systematic review, in Future Generation Computer Systems, vol. 126, 2022, pp. 136-162, https://doi.org/10.1016/j.future.2021.07.035.
While the use of blockchain inherently suggests a focus on security and data integrity, the paper does not provide a detailed analysis of how the platform ensures data privacy and complies with regulatory requirements, which are crucial aspects in clinical trials.
Response) Thanks for the comment. As described on page 9, anonymity is one of the characteristics of blockchain. In the proposed platform, experimental participants are identified by blockchain addresses rather than their personal information. Moreover, the questionnaire record will only show the questionnaire serial number and will not reveal the subject of the questionnaire. Therefore, no one will know the subject’s reaction to the experimental items unless they themselves reveal it.
The paper proposes the use of RFID for experimental item matching, but it does not delve into the practical challenges of implementing RFID in a clinical trial setting, such as issues with signal interference, hardware reliability, and the logistics of tagging and scanning a large number of items.
Response) Thanks for the reviewer’s comment. Since the hardware fault tolerance is not our focus, we did not discuss these practical problems of using RFIDs. As we know, using filters and shielding is one solution to minimize signal interference from nearby electronic devices. Preparing backup RFID can increase hard reliability. Developing a standardized tagging procedure and automatic scanning are solutions to address the logistics of tagging and scanning a large number of items. We will address these problems to enhance the reliability of the proposed system, as described on page 30.
I can't really understand the figure 16 and match it with the description.
Response) We have modified our description for the figure on page 28.
Round 2
Reviewer 2 Report
Comments and Suggestions for AuthorsI think the changes made to the paper enhanced its value and clarity. However, I still think it can be further enhanced by analyzing its security goals more thoroughly, i.e. a proper response to the second point from my previous review is still missing.
Author Response
Thanks for the suggestion. For security analysis, we have scanned the Solidity codes of the proposed platform by Slither, which is a smart contract static analyzer, and discussed the impact and solution of found program vulnerabilities in Section 5.5 from page 30 to page 32. We will do our best to patch the found smart contract vulnerabilities by modifying the Solidity source codes to prevent malicious users or miners from attacking the proposed platform in the future, as described on page 33.
