Automated Cytogenetic Biodosimetry at Population-Scale
, Eliseos J. Mucaki 1, Ben C. Shirley 2, Yanxin Li 2, Ruth C. Wilkins 3, Farrah Norton 4, Olga Sevriukova 5, Ngoc-Duy Pham 6, Ed Waller 7 and Joan H. M. Knoll 2,8Round 1
Reviewer 1 Report
The authors evaluated the performance of the automated dicentric chromosome identifier and dose estimator in terms of processing time and accuracy. Considering inherent challenges of the conventional approach for dicentric chromosome identification and its relevant dose estimation, the automated approach based on machine learning on a multiprocessor supercomputer can be promising and useful in a mass casualty radiation accident. Although the manuscript was well written regarding the processing performance, it is not sufficient to provide information about validity (i.e., accuracy), particularly for dose estimation. The manuscript can be more developed with incorporation of following comments:
- Are there any data to validate does estimations from the ADCI-HT?
- It would be more informative to add uncertainty about dose estimation in low dose levels in the introduction section or other relevant sections. Is there any detection limit of dose in using the ADCI-HT?
- It is not clear if the ADCI-HT system is fully automated including the process from cell culture to capture of metaphase cell images.
- Biodosimetry (e.g., dose-response courve from the analysis of dicentric chromosome) possibly differs according to age. Can the ADCI-HT cover all age groups?
Author Response
- Are there any data to validate does estimations from the ADCI-HT?
Response: The elements of program code that performs calibration curve derivation and dose estimation are identical to the ADCI Windows version. In Rogan et al. Radiation Dose Estimation by Completely Automated Interpretation of the Dicentric Chromosome Assay. Rad. Prot. Dosimetry 186(1), 42-47 (2019), we determined the accuracy for whole body radiation assessment to be <0.5 Gy. The primary differences between the HT and Windows versions are the scheduling software which manages compute resources to accelerate processing of sample images, which is the bottleneck in dose estimation. We will revise the introduction to expand on the similarities and differences between these versions, and make clear we have verified that dose estimates produced with either HT or Windows versions are the same.
- It would be more informative to add uncertainty about dose estimation in low dose levels in the introduction section or other relevant sections. Is there any detection limit of dose in using the ADCI-HT?
Response: We have shown that the dose estimates are accurate to as little as 0.25 Gy , however this does depend on the quality of the calibration curve. See:
Rogan PK, Shirley B, Li Y, Knoll J, Sevriukova O, Ngoc Duy P, Moquet J, Ainsbury E, Sudprasert W, Wilkins R, Norton F. (2019). Determination of radiation exposure levels by fully automated dicentric chromosome analysis: Results from IAEA MEDBIODOSE (CRP E35010) interlaboratory comparison. International Congress of Radiation Research, Manchester, United Kingdom. Zenodo. http://doi.org/10.5281/zenodo.4012749
ADCI Windows and ADCI-HT optimize calibration curve quality by image selection segmentation algorithms.
- It is not clear if the ADCI-HT system is fully automated including the process from cell culture to capture of metaphase cell images.
Response: ADCI-HT automates the selection of metaphase images, the interpretation dicentric chromosomes in metaphase images, calibration curve generation, and dose estimates from the dicentric chromosome frequencies obtain partial or whole body dose estimates. It does not perform cell culture, fixation, slide preparation, Giemsa staining, or image capture. Existing automation systems already exist for these purposes. If given the opportunity, we will revise the methods of the paper to make this clear.
- Biodosimetry (e.g., dose-response courve from the analysis of dicentric chromosome) possibly differs according to age. Can the ADCI-HT cover all age groups?
Response: This issue is not addressed directly in this paper. However, the user can, if desired, prepare different calibration curves stratified by age with ADCI, then evaluate dose of samples from age-matched individuals exposed to unknown levels of radiation. If a sufficient number of samples were evaluated for each age range, it would be feasible to use ADCI-HT to assess differences in estimated radiation dose as a function of age.
Reviewer 2 Report
The authors have presented the parallel implementation of previously developed The Automated Dicentric Chromosome Identifier and Dose Estimator (ADCI) software called ADCI-HT. According to the authors, this software used on a computational supercluster will analyze thousands of images in parallel. That will decrease the time for triage of the entire population on a moderate-sized city scale from a month to several days. Paper has scientifically sounded, has a logical structure, and an exact representation of results. Several questions may be to add to the discussion.
For example, how to transfer million of image data to supercluster? Will it be done through the web-app? In that case, handling a million images will require the use of a database. Or the plan is to collect all images at one location, and then the trained operator will perform the batch submission.
Another question is about "the minimum criteria for dicentric assay," postulated on page 8, section "Sample set creation." I was not able to find any in the provided references. Please correct.
Please also provide information about CPU and memory amount on super-computer provided for each core of the node. Without this information, the results of differences in the processing time of the ADCI-Windows version, ADC-HT, and ADCI BG/Q are not complete.
Thank you
Author Response
The authors have presented the parallel implementation of previously developed The Automated Dicentric Chromosome Identifier and Dose Estimator (ADCI) software called ADCI-HT. According to the authors, this software used on a computational supercluster will analyze thousands of images in parallel. That will decrease the time for triage of the entire population on a moderate-sized city scale from a month to several days. Paper has scientifically sounded, has a logical structure, and an exact representation of results. Several questions may be to add to the discussion.
For example, how to transfer million of image data to supercluster?
Response: IBM-BG/Q (and other compute clusters) sacrifice I/O performance for compute power, since there is a single node that performs I/O operations. For reasonable performance, it is necessary to combine images from multiple patients into sample sets through archiving (tar) operations before uploading small numbers of large files. In this paper, the transfers were performed using scp, untarred, then a pool of images from each laboratory source was randomly sampled to form a set of samples from the same laboratory source. This method was used to create thousands of synthetic samples with 500 images each. While it did demonstrate the performance of ADCI-HT, it did not replicate transfers of millions of images. This could be accomplished expeditiously if necessary using IBM’s Aspera FASP transfer protocol.
Will it be done through the web-app? In that case, handling a million images will require the use of a database. Or the plan is to collect all images at one location, and then the trained operator will perform the batch submission.
Response: It is unlikely that all of the images would be generated at a single location based on the capacity limitations of individual laboratories. It would be more likely that a network consisting multiple commercial and academic laboratories would distribute and process the samples in parallel (as envisioned by the RENEB project), then the results of each sample would be combined (using tar function) and multiple sample archives from the same laboratory and calibration samples be assembled into sample sets. These would be uploaded from different laboratories to a federated cloud storage repository, where it would be accessed by the compute cluster running ADCI-HQ. ADCI-HQ dearchives the sample sets in onboard RAM and processes samples. There, processing of each sample set would first generate image selection-optimized, laboratory-specific calibration curve and then evaluate all of the samples of unknown exposure with that curve.
Another question is about "the minimum criteria for dicentric assay," postulated on page 8, section "Sample set creation." I was not able to find any in the provided references. Please correct.
Response: We regret that the incorrect reference 21 was provided in the Literature section in the original submission of this paper. The following monograph should have been referenced:
International Atomic Energy Agency. Cytogenetic Dosimetry: Applications in Preparedness for and Response to Radiation Emergencies (Vienna: IAEA) (2011). (https://www-pub.iaea.org/MTCD/publications/PDF/EPR-Biodosimetry%202011_web.pdf)
Specifically: Section 9.7.2 (page 63) of this technical document indicates that 500 cells or 100 dicentric chromosomes are usually sufficient for dose estimation. However, analysis of addition cells may be required when greater accuracy in the dose estimate is required.
Please also provide information about CPU and memory amount on super-computer provided for each core of the node. Without this information, the results of differences in the processing time of the ADCI-Windows version, ADC-HT, and ADCI BG/Q are not complete.
Response: We have corrected and rewritten the section on page 9 titled “ADCI-HT Resource Allocation.” It now reads:
"In general, the CPU resources allocated were limited to 4 node boards by the default job queuing system. A node board contains 32 nodes and a node is a "system-on-a-chip" compute node containing a 16 core 1.6GHz PowerPC A2 CPU and 16Gb of RAM. Priority scheduling was approved by system administrators to determine performance in a machine-optimized environment. Higher priority runs maximized the number of processors that could be simultaneously allocated in the supercomputer (though processes could still be delayed due to the BG/Q queuing system). These included 1 and 4 node board runs of sample sets (100 runs with 400 samples each), with each node board containing 512 cores (4 threads per core)."
Round 2
Reviewer 1 Report
The authors have adequately addressed all the concerns I raised. Thank you.
