Author Contributions
Conceptualization, F.R.-R., R.V.-M. and J.M.-S.; methodology, F.R.-R. and R.V.-M.; software, F.R.-R. and R.V.-M.; validation, F.R.-R., R.B.-V., R.V.-M., J.M.-S. and I.S.-N.; formal analysis, R.B.-V., R.V.-M. and J.M.-S.; investigation, F.R.-R., R.B.-V., R.V.-M. and J.M.-S.; resources, R.B.-V., R.V.-M., J.M.-S. and I.S.-N.; data curation, R.V.-M. and I.S.-N.; writing—original draft preparation, F.R.-R. and R.V.-M.; writing—review and editing, F.R.-R., R.B.-V., R.V.-M., J.M.-S. and I.S.-N.; visualization, R.B.-V., R.V.-M., J.M.-S. and I.S.-N.; supervision, R.B.-V., R.V.-M. and J.M.-S.; project administration, R.V.-M. and I.S.-N.; funding acquisition, R.V.-M. and I.S.-N. All authors have read and agreed to the published version of the manuscript.
Figure 1.
Distribution of the temporal (T), temporal superior (TS), nasal superior (NS), nasal (N), nasal inferior (NI), and temporal inferior (TI) sectors in both eyes of a subject.
Figure 1.
Distribution of the temporal (T), temporal superior (TS), nasal superior (NS), nasal (N), nasal inferior (NI), and temporal inferior (TI) sectors in both eyes of a subject.
Figure 2.
Distribution of the sectors. (a) Retinography of the right eye of a patient with the width of the rim from the manual segmentations of the optic disc (in dotted blue line) and excavation (in dotted green line). (b) Peripapillary B-scan OCT with the measurement of the thickness of the retinal nerve fiber layer delineated between the red line (top) and the light blue line (bottom). In both figures the yellow lines indicate the borders of each sector of the eye.
Figure 2.
Distribution of the sectors. (a) Retinography of the right eye of a patient with the width of the rim from the manual segmentations of the optic disc (in dotted blue line) and excavation (in dotted green line). (b) Peripapillary B-scan OCT with the measurement of the thickness of the retinal nerve fiber layer delineated between the red line (top) and the light blue line (bottom). In both figures the yellow lines indicate the borders of each sector of the eye.
Figure 3.
(a) Representation on retinal fundus images of the division by sectors and measurement of the thicknesses of the right eye of a healthy patient and (b) Cartesian coordinate representation of the RDR location and the division by sectors. The contour of the optic disc delineated in blue, the contour of the excavation in green, and the difference between the two in red. The horizontal lines show the mean values of each of these curves. The × mark indicates the angle of the RDR.
Figure 3.
(a) Representation on retinal fundus images of the division by sectors and measurement of the thicknesses of the right eye of a healthy patient and (b) Cartesian coordinate representation of the RDR location and the division by sectors. The contour of the optic disc delineated in blue, the contour of the excavation in green, and the difference between the two in red. The horizontal lines show the mean values of each of these curves. The × mark indicates the angle of the RDR.
Figure 4.
Thickness of the RNFL in OCT versus rim width in retinography considering right eyes (OD) or left eyes (OS) of healthy (’o’) and glaucoma (’+’) patients in (a) G sector, (b) T sector, (c) N sector, and (d) NI sector. The approximation by a linear regression model is also plotted.
Figure 4.
Thickness of the RNFL in OCT versus rim width in retinography considering right eyes (OD) or left eyes (OS) of healthy (’o’) and glaucoma (’+’) patients in (a) G sector, (b) T sector, (c) N sector, and (d) NI sector. The approximation by a linear regression model is also plotted.
Figure 5.
Relation of G−sector values obtained in RET versus OCT differentiating between healthy and glaucoma patients using the following asymmetry metrics: (a) , (b) , (c) , and (d) . The centroids (or centers of mass, CMs) are also depicted for all patients and each subset.
Figure 5.
Relation of G−sector values obtained in RET versus OCT differentiating between healthy and glaucoma patients using the following asymmetry metrics: (a) , (b) , (c) , and (d) . The centroids (or centers of mass, CMs) are also depicted for all patients and each subset.
Figure 6.
Relevance assigned by the decision tree algorithm to each sector of , , and , extracted from retinographies.
Figure 6.
Relevance assigned by the decision tree algorithm to each sector of , , and , extracted from retinographies.
Figure 7.
Evolution of the classification tree metrics as function of MNS trained with extracted from retinographies.
Figure 7.
Evolution of the classification tree metrics as function of MNS trained with extracted from retinographies.
Figure 8.
(a) Evolution of the sensitivity and specificity of the SVM model generated using all the metrics extracted from retinographies as a function of . (b) Confusion matrix of the SVM model using all the features from retinographies with .
Figure 8.
(a) Evolution of the sensitivity and specificity of the SVM model generated using all the metrics extracted from retinographies as a function of . (b) Confusion matrix of the SVM model using all the features from retinographies with .
Figure 9.
(a) Evolution of the sensitivity and specificity of the SVM model generated using and extracted from OCT as a function of . (b) Confusion matrix particularized to .
Figure 9.
(a) Evolution of the sensitivity and specificity of the SVM model generated using and extracted from OCT as a function of . (b) Confusion matrix particularized to .
Figure 10.
Evolution of the sensitivity and specificity of the decision tree trained with of retinographies and and of OCT as a function of MNS.
Figure 10.
Evolution of the sensitivity and specificity of the decision tree trained with of retinographies and and of OCT as a function of MNS.
Figure 11.
Evolution of decision tree metrics generated from MSPs extracted from OCT with as a function of MNS.
Figure 11.
Evolution of decision tree metrics generated from MSPs extracted from OCT with as a function of MNS.
Figure 12.
Optimal decision tree, generated from the MSPs extracted from OCT with and MNS .
Figure 12.
Optimal decision tree, generated from the MSPs extracted from OCT with and MNS .
Table 1.
Sex and age range of the healthy and glaucoma patients of the dataset.
Table 1.
Sex and age range of the healthy and glaucoma patients of the dataset.
| Healthy | Glaucoma | Total |
---|
Age (years) | | | |
Gender (male/female) | 54/107 | 19/28 | 73/135 |
Total | 161 patients | 47 patients | 208 patients |
Table 2.
Number and percentage of healthy patients (out of 161) and glaucoma patients (out of 47) of the PAPILA dataset complying with ISNT, IST, and IS rules in one eye (left or right) or both eyes simultaneously.
Table 2.
Number and percentage of healthy patients (out of 161) and glaucoma patients (out of 47) of the PAPILA dataset complying with ISNT, IST, and IS rules in one eye (left or right) or both eyes simultaneously.
| Healthy (161 Patients) | Glaucoma (47 Patients) |
---|
| ISNT | IST | IS | ISNT | IST | IS |
---|
Right eye | 88 (54.66%) | 97 (60.25%) | 97 (60.25%) | 16 (34.04%) | 27 (57.44%) | 27 (57.44%) |
Left eye | 71 (44.10%) | 90 (55.90%) | 90 (55.90%) | 13 (27.66%) | 25 (53.19%) | 25 (53.19%) |
Both eyes | 49 (30.43%) | 59 (36.65%) | 59 (36.65%) | 6 (12.76%) | 17 (36.17%) | 17 (36.17%) |
Table 3.
Performance of decision trees trained with asymmetry features extracted from retinographies.
Table 3.
Performance of decision trees trained with asymmetry features extracted from retinographies.
| Input Asymmetry Characteristics |
---|
| All RET + + | | | | | | | + + | + | + |
---|
Sensitivity | 44.7% | 40.4% | 40.4% | 29.8% | 48.9% | 40.4% | 38.3% | 44.7% | 48.9% | 51.1% |
Specificity | 78.3% | 80.1% | 78.9% | 78.9% | 78.9% | 80.1% | 82.0% | 77.0% | 74.5% | 77.0% |
Precision | 37.5% | 37.3% | 35.8% | 29.2% | 40.4% | 37.3% | 38.3% | 36.2% | 35.9% | 39.3% |
Accuracy | 70.7% | 71.1% | 70.2% | 67.8% | 72.1% | 71.1% | 72.1% | 69.7% | 68.75% | 71.1% |
Table 4.
Performance of the models generated with the most significant parameter of , MSP, and its combinations with and .
Table 4.
Performance of the models generated with the most significant parameter of , MSP, and its combinations with and .
| Input Asymmetry Characteristics |
---|
| MSP | MSP + + | MSP + | MSP + |
---|
Sensitivity | 46.8% | 40.4% | 44.7% | 46.8% |
Specificity | 78.3% | 77.6% | 70.0% | 77.0% |
Precision | 38.6% | 34.5% | 36.2% | 37.3% |
Accuracy | 71.1% | 69.2% | 69.7% | 70.2% |
Table 5.
Performance of SVM models considering different sets of input features from retinographies.
Table 5.
Performance of SVM models considering different sets of input features from retinographies.
| Input Asymmetry Characteristics |
---|
| All RET + + | All RET | All RET + | All RET + | | | | | |
---|
Sensitivity | 44.7% | 46.8% | 46.8% | 44.7% | 29.8% | 2.1% | 0.0% | 6.4% | 44.7% |
Specificity | 93.2% | 93.2% | 93.2% | 92.5% | 96.3% | 97.5% | 100% | 100% | 92.5% |
Precision | 65.6% | 66.7% | 66.7% | 63.6% | 70.0% | 20.0% | 0.0% | 100% | 63.6% |
Accuracy | 82.2% | 82.7% | 82.7% | 81.7% | 81.3% | 76.0% | 77.4% | 78.8% | 81.7% |
Table 6.
Performance of decision trees considering the asymmetry metrics extracted from OCTs in all sectors and the most significant sectors of and .
Table 6.
Performance of decision trees considering the asymmetry metrics extracted from OCTs in all sectors and the most significant sectors of and .
| Input Asymmetry Characteristics |
---|
| | | | | | | All OCT | MSP | MSP |
---|
Sensitivity | 48.9% | 59.6% | 68.1% | 68.1% | 48.9% | 61.7% | 70.2% | 72.3% | 72.3% |
Specificity | 93.2% | 87.0% | 91.9% | 91.3% | 93.2% | 89.4% | 88.8% | 92.5% | 90.7% |
Precision | 67.6% | 57.1% | 71.1% | 69.6% | 67.6% | 63.0% | 64.7% | 73.9% | 69.4% |
Accuracy | 83.2% | 80.8% | 86.5% | 86.1% | 83.2% | 83.2% | 84.6% | 88.0% | 86.5% |
Table 7.
Performance of decision trees based on OCT characteristics when modifying the maximum number of splits (MNS) with and .
Table 7.
Performance of decision trees based on OCT characteristics when modifying the maximum number of splits (MNS) with and .
| Input Asymmetry Characteristics |
---|
| , MNS = 20 | , MNS = 3 | , MNS = 20 | , MNS = 3 |
---|
Sensitivity | 68.1% | 55.3% | 68.1% | 66.6% |
Specificity | 91.9% | 94.4% | 91.3% | 95.0% |
Precision | 71.1% | 74.3% | 69.6% | 79.5% |
Accuracy | 86.6% | 85.6% | 86.1% | 88.5% |
Table 8.
Performance of SVM models trained with different asymmetry metrics using characteristics extracted from OCTs.
Table 8.
Performance of SVM models trained with different asymmetry metrics using characteristics extracted from OCTs.
| Input Asymmetry Characteristics |
---|
| | | | | | | All OCT |
---|
Sensitivity | 2.1% | 55.3% | 0.0% | 53.2% | 0.0% | 51.1% | 51.1% |
Specificity | 100% | 96.6% | 100% | 98.8% | 100% | 95.7% | 96.3% |
Precision | 100% | 83.9% | 0.0% | 92.6% | 0.0% | 77.4% | 80.0% |
Accuracy | 77.9% | 87.5% | 77.4% | 88.5% | 77.4% | 85.6% | 86.0% |
Table 9.
Performance of classification trees generated with combinations of retinography and OCT features.
Table 9.
Performance of classification trees generated with combinations of retinography and OCT features.
| Input Asymmetry Characteristics |
---|
| All RET and All OCT | Best Metrics RET and OCT | Best Metrics + + | Best Metrics + | MSP Best Metrics RET and OCT | MSP Best Metrics RET and OCT + + | MSP Best Metrics RET and OCT + |
---|
Sensitivity | 68.1% | 68.1% | 68.1% | 68.1% | 70.2% | 68.1% | 68.1% |
Specificity | 87.6% | 90.1% | 90.1% | 90.1% | 90.7% | 90.1% | 90.1% |
Precision | 61.5% | 66.7% | 66.7% | 66.7% | 68.8% | 66.7% | 66.7% |
Accuracy | 83.2% | 85.1% | 85.1% | 85.1% | 86.0% | 85.1% | 85.1% |
Table 10.
Performance of SVM models using characteristics extracted from retinographies and OCTs.
Table 10.
Performance of SVM models using characteristics extracted from retinographies and OCTs.
| Input Asymmetry Characteristics |
---|
| All RET and All OCT | All RET and , from OCT |
---|
Sensitivity | 61.7% | 61.7% |
Specificity | 88.8% | 95.0% |
Precision | 61.7% | 78.4% |
Accuracy | 82.7% | 87.5% |
Table 11.
Summary table with the results provided by the main machine learning models designed throughout the work.
Table 11.
Summary table with the results provided by the main machine learning models designed throughout the work.
Method | Image Modality | Features Applied | Hyperparameter Modified | Sensitivity (%) | Specificity (%) | Precision (%) | Accuracy (%) |
---|
Decision trees | RET | All + + | − | 44.7 | 78.3 | 37.5 | 70.7 |
| − | 29.8 | 78.9 | 29.2 | 67.8 |
| − | 48.9 | 78.9 | 40.4 | 72.1 |
MSP | − | 46.8 | 78.3 | 38.6 | 71.1 |
MSP + + | − | 40.4 | 77.6 | 34.5 | 69.2 |
| MNS = 3 | 43.5 | 93.0 | − | 81.0 |
= 4.2 | 61.7 | 65.2 | 34.1 | 64.4 |
OCT | All | − | 70.2 | 88.8 | 64.7 | 84.6 |
| − | 68.1 | 91.9 | 71.1 | 86.5 |
| − | 68.1 | 91.3 | 69.6 | 86.1 |
MSP | − | 72.3 | 92.5 | 73.9 | 88.0 |
MSP | − | 72.3 | 90.7 | 69.4 | 86.5 |
MSP | MNS = 3 | 55.3 | 94.4 | 74.3 | 85.6 |
MSP | MNS = 3 | 66.6 | 95.0 | 79.5 | 88.5 |
MSP | = 3.2 | 80.9 | 85.1 | 61.3 | 84.1 |
MSP | = 2.2 | 80.9 | 87.0 | 64.4 | 85.6 |
MSP | MNS = 10, = 2.2 | 80.9 | 88.2 | 66.7 | 86.5 |
RET + OCT | All | − | 68.1 | 87.6 | 61.5 | 83.2 |
MP | − | 68.1 | 90.1 | 66.7 | 85.1 |
MSP MP | − | 70.2 | 90.7 | 68.8 | 86.0 |
MSP MP | = 1.8 | 78.7 | 87.6 | 64.9 | 85.6 |
SVM | RET | All | − | 46.8 | 93.2 | 66.7 | 82.7 |
All + + | − | 44.7 | 93.2 | 65.6 | 82.2 |
All | = 3.5 | 63.8 | 83.2 | 52.6 | 78.8 |
OCT | All | − | 51.1 | 96.3 | 80.0 | 86.0 |
, | − | 59.6 | 97.5 | 87.5 | 88.9 |
, | = 3.5 | 68.1 | 93.2 | 74.4 | 87.5 |
RET + OCT | All | − | 61.7 | 88.8 | 61.7 | 82.7 |
All RET, , OCT | − | 61.7 | 95.0 | 78.4 | 87.5 |
All RET, , OCT | = 3.8 | 83.0 | 81.4 | 56.5 | 81.7 |