Recurrent and Concurrent Prediction of Longitudinal Progression of Stargardt Atrophy and Geographic Atrophy towards Comparative Performance on Optical Coherence Tomography as on Fundus Autofluorescence
Abstract
:1. Introduction
2. Methods
2.1. Imaging Dataset and Ground Truth
2.2. Neural Network Structure
2.3. Atrophy Prediction
2.3.1. Prediction of Future GA and Stargardt Atrophy Regions Using Longitudinal FAF Images (ReConNet)
2.3.2. Prediction of Progression of Stargardt Atrophy Using Longitudinal SD-OCT Images (ReConNet-Ensemble)
2.3.3. Prediction of Interval Growth of GA and Stargardt Atrophy Regions Using Longitudinal FAF Images (ReConNet-Interval)
2.4. Performance Evaluation
3. Results
4. Discussions
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Strauss, R.W.; Ho, A.; Muñoz, B.; Cideciyan, A.V.; Sahel, J.A.; Sunness, J.S.; Birch, D.G.; Bernstein, P.S.; Michaelides, M.; Traboulsi, E.I.; et al. The natural history of the progression of atrophy secondary to Stargardt disease (ProgStar) studies: Design and baseline characteristics: ProgStar report no. 1. Ophthalmology 2016, 123, 817–828. [Google Scholar] [CrossRef]
- Schönbach, E.M.; Wolfson, Y.; Strauss, R.W.; Ibrahim, M.A.; Kong, X.; Muñoz, B.; Birch, D.G.; Cideciyan, A.V.; Hahn, G.A.; Nittala, M.; et al. Macular sensitivity measured with microperimetry in Stargardt disease in the progression of atrophy secondary to Stargardt disease (ProgStar) study: Report no.7. JAMA Ophthalmol. 2017, 135, 696–703. [Google Scholar] [CrossRef] [PubMed]
- Strauss, R.W.; Muñoz, B.; Ho, A.; Jha, A.; Michaelides, M.; Mohand-Said, S.; Cideciyan, A.V.; Birch, D.; Hariri, A.H.; Nittala, M.G.; et al. Incidence of atrophic lesions in Stargardt disease in the progression of atrophy secondary to Stargardt disease (ProgStar) study: Report no. 5. JAMA Ophthalmol. 2017, 135, 687–695. [Google Scholar] [CrossRef]
- Strauss, R.W.; Muñoz, B.; Ho, A.; Jha, A.; Michaelides, M.; Cideciyan, A.V.; Audo, I.; Birch, D.G.; Hariri, A.H.; Nittala, M.G.; et al. Progression of Stargardt disease as determined by fundus autofluorescence in the retrospective progression of Stargardt disease study (ProgStar report no. 9). JAMA Ophthalmol. 2017, 135, 1232–1241. [Google Scholar] [CrossRef] [PubMed]
- Ma, L.; Kaufman, Y.; Zhang, J.; Washington, I. C20-D3-vitamin A slows lipofuscin accumulation and electrophysiological retinal degeneration in a mouse model of Stargardt disease. J. Biol. Chem. 2010, 286, 7966–7974. [Google Scholar] [CrossRef]
- Kong, J.; Kim, S.R.; Binley, K.; Pata, I.; Doi, K.; Mannik, J.; Zernant-Rajang, J.; Kan, O.; Iqball, S.; Naylor, S.; et al. Correction of the disease phenotype in the mouse model of Stargardt disease by lentiviral gene therapy. Gene Ther. 2008, 15, 1311–1320. [Google Scholar] [CrossRef]
- Binley, K.; Widdowson, P.; Loader, J.; Kelleher, M.; Iqball, S.; Ferrige, G.; de Belin, J.; Carlucci, M.; Angell-Manning, D.; Hurst, F.; et al. Transduction of photoreceptors with equine infectious anemia virus lentiviral vectors: Safety and biodistribution of StarGen for Stargardt disease. Investig. Ophtalmol. Vis. Sci. 2013, 54, 4061–4071. [Google Scholar] [CrossRef] [PubMed]
- Mukherjee, N.; Schuman, S. Diagnosis and management of Stargardt disease. EyeNet Magazine 2014. [Google Scholar]
- Kong, X.; Ho, A.; Munoz, B.; West, S.; Strauss, R.W.; Jha, A.; Ervin, A.; Buzas, J.; Singh, M.; Hu, Z.; et al. Reproducibility of measurements of retinal structural parameters using optical coherence tomography in Stargardt disease. Transl. Vis. Sci. Technol. 2019, 8, 46. [Google Scholar] [CrossRef] [PubMed]
- Bressler, N.M.; Bressler, S.B.; Congdon, N.G.; Ferris, F.L., 3rd; Friedman, D.S.; Klein, R.; Lindblad, A.S.; Milton, R.C.; Seddon, J.M.; Age-Related Eye Disease Study Research Group. Potential public health impact of Age-Related Eye Disease Study results: AREDS report no. 11. Arch. Ophthalmol. 2003, 121, 1621–1624. [Google Scholar] [PubMed] [PubMed Central]
- Davis, M.D.; Gangnon, R.E.; Lee, L.-Y.; Hubbard, L.D.; Klein, B.E.; Klein, R.; Ferris, F.L.; Bressler, S.B.; Milton, R.C.; Age-Related Eye Disease Study Group. The age-related eye disease study severity scale for age-related macular degeneration: AREDS report no. 17. Arch. Ophthalmol. 2005, 123, 1484–1498. [Google Scholar] [PubMed]
- Ferris, F.L.; Davis, M.D.; Clemons, T.E.; Lee, L.Y.; Chew, E.Y.; Lindblad, A.S.; Milton, R.C.; Bressler, S.B.; Klein, R.; Age-Related Eye Disease Study (AREDS) Research Group. A simplified severity scale for age-related macular degeneration: AREDS report no. 18. Arch. Ophthalmol. 2005, 123, 1570–1574. [Google Scholar]
- Klein, R.; Klein, B.E.; Jensen, S.C.; Meuer, S.M. The five-year incidence and progression of age-related maculopathy: The Beaver Dam Eye Study. Ophthalmology 1997, 104, 7–21. [Google Scholar] [CrossRef]
- Blair, C.J. Geographic atrophy of the retinal pigment epithelium: A manifestation of senile macular degeneration. Arch. Ophthalmol. 1975, 93, 19–25. [Google Scholar] [CrossRef] [PubMed]
- Schmitz-Valckenberg, S.; Holz, F.; Bird, A.; Spaide, R. Fundus autofluorescence imaging: Review and perspectives. Retina 2008, 28, 385–409. [Google Scholar] [CrossRef]
- Huang, D.; Swanson, E.A.; Lin, C.P.; Schuman, J.S.; Stinson, W.G.; Chang, W.; Hee, M.R.; Flotte, T.; Gregory, K.; Puliafito, C.A.; et al. Optical coherence tomography. Science 1991, 254, 1178–1181. [Google Scholar] [CrossRef]
- Fujimoto, J.G.; Bouma, B.; Tearney, G.J.; Boppart, S.A.; Pitris, C.; Southern, J.F.; Brezinski, M.E. New technology for high-speed and high-resolution optical coherence tomography. Ann. N. Y. Acad. Sci. 1998, 838, 96–107. [Google Scholar] [CrossRef] [PubMed]
- Mishra, Z.; Wang, Z.; Sadda, S.R.; Hu, Z. Using Ensemble OCT-Derived Features beyond Intensity Features for Enhanced Stargardt Atrophy Prediction with Deep Learning. Appl. Sci. 2023, 13, 8555. [Google Scholar] [CrossRef] [PubMed]
- Mishra, Z.; Wang, Z.; Sadda, S.R.; Hu, Z. Automatic Segmentation in Multiple OCT Layers for Stargardt Disease Characterization via Deep Learning. Transl. Vis. Sci. Technol. 2021, 10, 24. [Google Scholar] [CrossRef]
- Kugelman, J.; Alonso-Caneiro, D.; Chen, Y.; Arunachalam, S.; Huang, D.; Vallis, N.; Collins, M.J.; Chen, F.K. Retinal boundary segmentation in Stargardt disease optical coherence tomography images using automated deep learning. Transl. Vis. Sci. Technol. 2020, 9, 12. [Google Scholar] [CrossRef]
- Charng, J.; Xiao, D.; Mehdizadeh, M.; Attia, M.S.; Arunachalam, S.; Lamey, T.M.; Thompson, J.A.; McLaren, T.L.; De Roach, J.N.; Mackey, D.A.; et al. Deep learning segmentation of hyperautofluorescent fleck lesions in Stargardt disease. Sci. Rep. 2020, 10, 16491. [Google Scholar] [CrossRef]
- Schmitz-Valckenberg, S.; Brinkmann, C.K.; Alten, F.; Herrmann, P.; Stratmann, N.K.; Göbel, A.P.; Fleckenstein, M.; Diller, M.; Jaffe, G.J.; Holz, F.G. Semiautomated image processing method for identification and quantification of geographic atrophy in age-related macular degeneration. Investig. Ophthalmol. Vis. Sci. 2011, 52, 7640–7646. [Google Scholar]
- Chen, Q.; de Sisternes, L.; Leng, T.; Zheng, L.; Kutzscher, L.; Rubin, D.L. Semi-automatic geographic atrophy segmentation for SD-OCT images. Biomed. Opt. Express 2013, 4, 2729–2750. [Google Scholar] [PubMed]
- Wang, S.; Wang, Z.; Vejalla, S.; Ganegoda, A.; Nittala, M.; Sadda, S.; Hu, Z. Reverse engineering for reconstructing baseline features of dry age-related macular degeneration in optical coherence tomography. Sci. Rep. 2022, 12, 22620. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
- Wang, Z.; Sadda, S.R.; Lee, A.; Hu, Z. Automated segmentation and feature discovery of age-related macular degeneration and Stargardt disease via self-attended neural networks. Sci. Rep. 2022, 12, 14565. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
- Hu, Z.; Wu, X.; Hariri, A.; Sadda, S. Multiple Layer Segmentation and Analysis in Three-Dimensional Spectral-Domain Optical Coherence Tomography Volume Scans. J. Biomed. Opt. 2013, 18, 076006. [Google Scholar] [CrossRef] [PubMed]
- Hu, Z.; Medioni, G.G.; Hernandez, M.; Hariri, A.; Wu, X.; Sadda, S.R. Segmentation of the Geographic Atrophy in Spectral-Domain Optical Coherence Tomography and Fundus Autofluorescene Images. Investig. Ophthalmol. Vis. Sci. 2013, 54, 8375–8383. [Google Scholar] [PubMed]
- Wang, Z.; Sadda, S.R.; Hu, Z. Deep learning for automated screening and semantic segmentation of age-related and juvenile atrophic macular degeneration. In Proceedings of the Medical Imaging 2019: Computer-Aided Diagnosis, San Diego, CA, USA, 16–21 February 2019; International Society for Optics and Photonics: San Diego, CA, USA, 2019; Volume 10950, p. 109501Q. [Google Scholar] [CrossRef]
- Hu, Z.; Wang, Z.; Sadda, S. Automated segmentation of geographic atrophy using deep convolutional neural networks. In Proceedings of the SPIE Medical Imaging 2018: Computer-Aided Diagnosis, Houston, TX, USA, 10–15 February 2018; Volume 10575, p. 1057511. [Google Scholar] [CrossRef]
- Saha, S.; Wang, Z.; Sadda, S.; Kanagasingam, Y.; Hu, Z. Visualizing and understanding inherent features in SD-OCT for the progression of age-related macular degeneration using deconvolutional neural networks. Appl. AI Lett. 2020, 1, e16. [Google Scholar] [CrossRef] [PubMed]
- Schmidt-Erfurth, U.; Bogunovic, H.; Grechenig, C.; Bui, P.; Fabianska, M.; Waldstein, S.; Reiter, G.S. Role of Deep Learning-Quantified Hyperreflective Foci for the Prediction of Geographic Atrophy Progression. Am. J. Ophthalmol. 2020, 216, 257–270. [Google Scholar] [CrossRef]
- Ramsey, D.; Sunness, J.; Malviya, P.; Applegate, C.; Hager, G.; Handa, J. Automated image alignment and segmentation to follow progression of geographic atrophy in age-related macular degeneration. Retina 2014, 34, 1296–1307. [Google Scholar] [PubMed]
- Liefers, B.; Colijn, J.M.; González-Gonzalo, C.; Verzijden, T.; Wang, J.J.; Joachim, N.; Mitchell, P.; Hoyng, C.B.; van Ginneken, B.; Klaver, C.C.W.; et al. A Deep Learning Model for Segmentation of Geographic Atrophy to Study Its Long-Term Natural History. Ophthalmology 2020, 127, 1086–1096. [Google Scholar] [CrossRef]
- Chu, Z.; Wang, L.; Zhou, X.; Shi, Y.; Cheng, Y.; Laiginhas, R.; Zhou, H.; Shen, M.; Zhang, Q.; de Sisternes, Let al. Automatic geographic atrophy segmentation using optical attenuation in OCT scans with deep learning. Biomed. Opt. Express 2022, 13, 1328–1343. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
- Pramil, V.; de Sisternes, L.; Omlor, L.; Lewis, W.; Sheikh, H.; Chu, Z.; Manivannan, N.; Durbin, M.; Wang, R.K.; Rosenfeld, P.J.; et al. A Deep Learning Model for Automated Segmentation of Geographic Atrophy Imaged Using Swept-Source OCT. Ophthalmol. Retina 2023, 7, 127–141. [Google Scholar] [CrossRef] [PubMed]
- Kalra, G.; Cetin, H.; Whitney, J.; Yordi, S.; Cakir, Y.; McConville, C.; Whitmore, V.; Bonnay, M.; Lunasco, L.; Sassine, A.; et al. Machine Learning-Based Automated Detection and Quantification of Geographic Atrophy and Hypertransmission Defects Using Spectral Domain Optical Coherence Tomography. J. Pers. Med. 2023, 13, 37. [Google Scholar] [CrossRef] [PubMed]
- Ji, Z.; Chen, Q.; Niu, S.; Leng, T.; Rubin, D.L. Beyond Retinal Layers: A Deep Voting Model for Automated Geographic Atrophy Segmentation in SD-OCT Images. Transl. Vis. Sci. Technol. 2018, 7, 1. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
- Manaswi, N.K. RNN and LSTM. In Deep Learning with Applications Using Python; Apress: Berkeley, CA, USA, 2018. [Google Scholar] [CrossRef]
- Calin, O. Deep Learning Architectures; Springer Nature: Cham, Switzerland, 2020; 555p, ISBN 978-3-030-36720-6. [Google Scholar]
- Graves, A.; Schmidhuber, J. Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks. In Advances in Neural Information Processing Systems 21, Proceedings of the Twenty-Second Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 8–11 December 2008; MIT Press: Cambridge, MA, USA, 2009; pp. 545–552. [Google Scholar]
- Kugelman, J.; Alonso-Caneiro, D.; Read, S.A.; Vincent, S.J.; Collins, M.J. Automatic segmentation of OCT retinal boundaries using recurrent neural networks and graph search. Biomed. Opt. Express 2018, 9, 5759–5777. [Google Scholar] [CrossRef]
- Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef]
- Monner, D.; Reggia, J.A. A generalized LSTM-like training algorithm for second-order recurrent neural networks. Neural Netw. 2012, 25, 70–83. [Google Scholar] [CrossRef] [PubMed] [PubMed Central]
- Dixit, A.; Yohannan, J.; Boland, M.V. Assessing Glaucoma Progression Using Machine Learning Trained on Longitudinal Visual Field and Clinical Data. Ophthalmology 2021, 128, 1016–1026. [Google Scholar] [CrossRef]
- Lee, J.; Wanyan, T.; Chen, Q.; Keenan, T.D.L.; Glicksberg, B.S.; Chew, E.Y.; Lu, Z.; Wang, F. Predicting Age-related Macular Degeneration Progression with Longitudinal Fundus Images Using Deep Learning. In Machine Learning in Medical Imaging: 13th International Workshop, MLMI 2022, Held in Conjunction with MICCAI 2022, Singapore, 18 September 2022, Proceedings; Lian, C., Cao, X., Rekik, I., Xu, X., Cui, Z., Eds.; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2022; Volume 13583. [Google Scholar] [CrossRef]
- Santeramo, R.; Withey, S.; Montana, G. Longitudinal detection of radiological abnormalities with time-modulated LSTM. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, 20 September 2018, Proceedings; Springer International Publishing: Cham, Switzerland, 2018; pp. 326–333. [Google Scholar]
- Hong, X.; Lin, R.; Yang, C.; Zeng, N.; Cai, C.; Gou, J.; Yang, J. Predicting Alzheimer’s disease using LSTM. IEEE Access 2019, 7, 80893–80901. [Google Scholar] [CrossRef]
- Banerjee, I.; de Sisternes, L.; Hallak, J.A.; Leng, T.; Osborne, A.; Rosenfeld, P.J.; Gregori, G.; Durbin, M. Prediction of age-related macular degeneration disease using a sequential deep learning approach on longitudinal SD-OCT imaging biomarkers. Sci. Rep. 2020, 10, 15434. [Google Scholar] [CrossRef]
- Zhang, Y.; Zhang, X.; Ji, Z.; Niu, S.; Leng, T.; Rubin, D.L.; Yuan, S.; Chen, Q. An integrated time adaptive geographic atrophy prediction model for SD-OCT images. Med. Image Anal. 2021, 68, 101893. [Google Scholar] [CrossRef] [PubMed]
- Hernandez, M.; Medioni, G.G.; Hu, Z.; Sadda, S.R. Multimodal registration of multiple retinal images based on line structures. In Proceedings of the 2015 IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA, 5–9 January 2015; pp. 907–914. [Google Scholar]
- Sadda, J.; Abdelfattah, N.S.; Sadda, S.R.; Hu, Z. Inter-Grader Repeatability of Geographic Atrophy Measurements from Infrared Reflectance Images. Investig. Ophthalmol. Vis. Sci. 2018, 59, 3245. [Google Scholar]
- Abdelfattah, N.S.; Sadda, J.; Wang, Z.; Hu, Z.; Sadda, S. Near-Infrared Reflectance Imaging for Quantification of Atrophy Associated with Age-Related Macular Degeneration. Am. J. Ophthalmol. 2020, 212, 169–174. [Google Scholar] [CrossRef] [PubMed]
- Stojanov, D. Phylogenicity of B.1.1.7 surface glycoprotein, novel distance function and first report of V90T missense mutation in SARS-CoV-2 surface glycoprotein. Meta Gene 2021, 30, 100967. [Google Scholar] [CrossRef] [PubMed]
Accuracy | Dice Coefficient | Sensitivity | Specificity | ||
---|---|---|---|---|---|
Stargardt Atrophy | ReConNet1-Initial | 0.919|0.904 (0.072) | 0.568|0.577 (0.163) | 0.406|0.432 (0.163) | 0.998|0.996 (0.008) |
ReConNet2-Final | 0.980|0.973 (0.021) | 0.922|0.895 (0.086) | 0.876|0.84 (0.120) | 0.998|0.996 (0.007) | |
p-value | <0.001 | <0.001 | <0.001 | 0.020 | |
GA | ReConNet1-Initial | 0.901|0.896 (0.060) | 0.867|0.827 (0.129) | 0.84|0.839 (0.107) | 0.971|0.938 (0.089) |
ReConNet2-Final | 0.928|0.919 (0.042) | 0.893|0.864 (0.113) | 0.964|0.945 (0.062) | 0.920|0.901 (0.077) | |
p-value | <0.001 | <0.001 | <0.001 | <0.001 |
Accuracy | Dice Coefficient | Sensitivity | Specificity | ||
---|---|---|---|---|---|
Stargardt Atrophy | ReConNet1-Ensemble-Initial | 0.955|0.92 (0.093) | 0.742|0.662 (0.238) | 0.691|0.63 (0.292) | 0.992|0.977 (0.032) |
ReConNet2-Ensemble-Final | 0.98|0.968 (0.033) | 0.906|0.882 (0.101) | 0.912|0.894 (0.086) | 0.991|0.983 (0.029) | |
p-value | <0.001 | <0.001 | <0.001 | 0.140 |
Accuracy | Dice Coefficient | Sensitivity | Specificity | ||
---|---|---|---|---|---|
Stargardt Atrophy | ReConNet1-Interval | 0.988|0.985 (0.009) | 0.559|0.557 (0.094) | 0.676|0.673 (0.145) | 0.993|0.991 (0.006) |
GA | ReConNet1-Interval | 0.968|0.959 (0.025) | 0.601|0.612 (0.089) | 0.718|0.711 (0.155) | 0.981|0.975 (0.020) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Mishra, Z.; Wang, Z.C.; Xu, E.; Xu, S.; Majid, I.; Sadda, S.R.; Hu, Z.J. Recurrent and Concurrent Prediction of Longitudinal Progression of Stargardt Atrophy and Geographic Atrophy towards Comparative Performance on Optical Coherence Tomography as on Fundus Autofluorescence. Appl. Sci. 2024, 14, 7773. https://doi.org/10.3390/app14177773
Mishra Z, Wang ZC, Xu E, Xu S, Majid I, Sadda SR, Hu ZJ. Recurrent and Concurrent Prediction of Longitudinal Progression of Stargardt Atrophy and Geographic Atrophy towards Comparative Performance on Optical Coherence Tomography as on Fundus Autofluorescence. Applied Sciences. 2024; 14(17):7773. https://doi.org/10.3390/app14177773
Chicago/Turabian StyleMishra, Zubin, Ziyuan Chris Wang, Emily Xu, Sophia Xu, Iyad Majid, SriniVas R. Sadda, and Zhihong Jewel Hu. 2024. "Recurrent and Concurrent Prediction of Longitudinal Progression of Stargardt Atrophy and Geographic Atrophy towards Comparative Performance on Optical Coherence Tomography as on Fundus Autofluorescence" Applied Sciences 14, no. 17: 7773. https://doi.org/10.3390/app14177773