Improvement of Retinal Images Affected by Cataracts

Round 1

Reviewer 1 Report

Authors present a method to improve retinal images of eyes with different grades of cataract, based on contrast enhancement by histogram equalization applied in three color spaces (RGB, CIELAB and HSV). The method is applied to two groups of real images obtaining large improvement in the detection of vascularization, optics disk, macula, fibrovascular proliferations or micro-hemorrhages, among others.

Authors show a practical way to improve retinal images, and although they do not present to many details of the methodology used, the manuscript is well written and easy to understand.

There are a few points that should be modified/clarified before the publication.

1. The Methodology section does not indicate whether authors have permission to use the second sets of images (private data set) and if the study were conducted following the tenets of the Declaration of Helsinki. Nevertheless, in the Results section (lines 175-76), it is mentioned (although it is not clear) if authors are referring to just the jpg retinal images corresponding to Figure 5-8, or to all the images obtained the University Hospital of Santiago de Compostela. Authors should clarify this point and include the permission in section 2 (Method). How many images from the second set were used?
2. In lines 134-141 authors said: “After analyzing most of the images in 136 our data base with different stages of cataract we found that the default limit of 137 the function, 256, provides acceptable results for all of them, and all images in 138 this work have been processed with this bin number.” How many images were analyzed to stablish the CLAHE parameters, and in how many new images were there tested? If all images were used to obtain acceptable results, then the best results were obtained for just that group of images, but that doesn’t guarantee that it will work for another set, unless the group of images uses correspond to a normal distribution. More details/explanations should be given in this regard.
3. Related to the previous points: how were the images from Figures 5-8, 10 and 11 selected? Do they correspond to the whole set of figures? If not, author should give details about whether their methodology works for the rest of images and include an example of the average improvement achieved.
4. It will be nice if the authors can include in the Discussion any reason why enhancement seems to be worse in the HSV Color Space. Is the dependency on the wavelength of the scattering by the cataractous lens paying any role of the differences found on each color space?
5. Line 99-100 defined “CLAHE” which is already defined in line 55.
6. Figure 11: Figure caption should indicate what profile (horizontal or vertical) represents the upper and lower figures on the right side.

Author Response

We thank the reviewer's comments. We answer all of them in the following paragraphs. Our reply is in green.

1) The Methodology section does not indicate whether authors have permission to use the second sets of images (private data set) and if the study were conducted following the tenets of the Declaration of Helsinki. Nevertheless, in the Results section (lines 175-76), it is mentioned (although it is not clear) if authors are referring to just the jpg retinal images corresponding to Figure 5-8, or to all the images obtained the University Hospital of Santiago de Compostela. Authors should clarify this point and include the permission in section 2 (Method).

 

We have permission to use the second set of images. In lines 177 to 180 of the manuscript revised by the reviewer, we wrote “Before taking the images, the permission to use them for research purposes was explained to each patient, and written informed consent was obtained. The study was conducted in accordance with the tenets of the Declaration of Helsinki.” We meant, all images taken at the University Hospital of Santiago de Compostela have permission.

This is explicitly written now in lines 82 to 88 of revised manuscript:

“The second set of 17 JPG retinal images was taken from the private database of images of the University Hospital of Santiago de Compostela (obtained with a Midriatic Retinographer NIDEK CO., LTD at the Ophthalmology service). Before taking the images, the permission to use them for research purposes was explained to each patient, and written informed consent was obtained. The study was conducted in accordance with the tenets of the Declaration of Helsinki”

 

How many images from the second set were used?

 

The selection of the 15 images from the public database was made by picking 5 we thought they are representative of each cataract stage. Retinal images from public database don’t show retinal pathologies that’s why we added 4 images of our database. We developed the algorithm for easing diagnostics. Therefore, images shown in figures 5-11 were selected to show the improvement in the detection of the pathological features (see sentence in lines 175 to 177). Our methodology improves retinal images blurred by cataracts independent of the used database.

2) In lines 134-141 authors said: “After analyzing most of the images in our data base with different stages of cataract we found that the default limit of the function, 256, provides acceptable results for all of them, and all images in this work have been processed with this bin number.” How many images were analyzed to stablish the CLAHE parameters, and in how many new images were there tested? If all images were used to obtain acceptable results, then the best results were obtained for just that group of images, but that doesn’t guarantee that it will work for another set, unless the group of images uses correspond to a normal distribution. More details/explanations should be given in this regard.

 

NTiles defines the block size or local region around a pixel from which the histogram is equalized. This size should be larger than the size of features to be preserved or enhanced. Moreover, the number of histogram bins used for histogram equalization within a block should be smaller than the number of pixels in the block. All images from both data sets are 8-bits images and their size is 2592x1728 or 2464x1632 for the public dataset and 2976x2976 for our dataset. In all the processed images presented in the manuscript we used NTiles=16 and NBins=256. Larger values are not meaningful and reducing the number of bins reduces the dynamic range of the output image, by limiting the change in intensity to improve contrast. For the images presented in this work of ROI’s containing the optical disk we used NTiles=8 and NBins= 256. Nevertheless, the developed interface allows the user to choose different values for NBins and NTiles depending on the bit depth and region of interest to be processed

This answer to the referee comment has been included in the manuscript from line 131 to 144. We did also include the values of ClipLimit that we used(paragraph from line 145)

 

3) Related to the previous points: how were the images from Figures 5-8, 10 and 11 selected? Do they correspond to the whole set of figures? If not, author should give details about whether their methodology works for the rest of images and include an example of the average improvement achieved.

The number of images of our database is 17. Images presented in figures 5-8 and 10-11 do not correspond therefore to the complete dataset, only to 4 of them. Our database includes retinal images degraded by cataracts with and without pathological features. We developed the algorithm for easing diagnostics. Therefore, images shown in figures 5-11 were selected to show the improvement in the detection of the pathological features (see sentence in lines 175 to 177). Nevertheless, our methodology works for all the images of our and the public database.

 

4) It will be nice if the authors can include in the Discussion any reason why enhancement seems to be worse in the HSV Color Space. Is the dependency on the wavelength of the scattering by the cataractous lens paying any role of the differences found on each color space?

We did not find yet an explanation why HSV provide worse results at the retinal layer or why the choroid is better seeing. But it is an interesting and unexpected result. We can say from results that if the processing is done to improve the retinal layer it would be recommended the use of CIELAB or RGB color spaces, but if the target is the choroid, then HSV is the color space.

 

5) Line 99-100 defined “CLAHE” which is already defined in line 55.

We removed definition in the corresponding line.

 

6) Figure 11: Figure caption should indicate what profile (horizontal or vertical) represents the upper and lower figures on the right side.

We improved the description of Figure 11 in the caption as well as the sentence right above the figure. Lines 253 to 255.

 

Reviewer 2 Report

The authors explore in this study, the use of a computational method to improve color images of the eye fundus using the algorithm of contrast limited adaptive histogram equalization (CLAHE). They provide results using this technique, showing a good performance to enhance diagnostic features found in the eye fundus, not only for the non-cataract eye but for different levels of blurring due to different stages of cataracts, where the major impact is for easing the clinical assessment of retinal pathologies. The method is thought to be used with retinal fundus images, which make it easily available for most ophthalmology centers. 

The study is relevant, well done, and of interest, especially for the field of diagnosis of retinal pathologies in retinal images affected by cataracts.  A major (and some minor) point needs to be addressed, but that doesn’t detract from the value of the study.

 

Major comment

The study addresses a new methodology to get relevant information from images degraded by the presence of cataract blurring, but the paper doesn’t present a discussion about the possible appearance of artifacts or non-real features in the post-processed images. Questions like: how accurate is the post-processed image? Or, how accurate are the features found after the proposed method is applied? needs to be addressed to give support to the method in a diagnosis stage. One idea to get information to do that discussion, is to generate some simulated cataract images from eye fundus images with well know pathologies and features associated with them, adding a cataract-like blurring or contrast reduction.  Then, those simulated cataract eye fundus images can be processed with the proposed method, comparing the non-cataract images and known features with the simulated post-processed cataract images, to see the accuracy of the method or the appearance of non-real features.

Minor comments:
Line 29: add a reference supporting that statement about the change of color temperature of retina images.  It is important for a deeper understanding of interested readers.
Lines 82, 83: Discuss how do you select the set of 15 images from the public database? What were your selection criteria, since that database has 100 images?  After downloading two images from the database, it was found bigger resolution than that reported by you, then it is better to report which specific images you used and if you did some preprocessing to those images, as an area selection or resizing, etc.
Line 86: The classification done by the two clinicians over the 15 images selected from the public database was done without any discrepancy? Do both clinicians choose exactly the same classification for all 15 images?
Line 91: please report how many images from the hospital database do you process with your software?
Line 130: review the sentence: “…image details can subjectively be better detected”.
Lines 135 to 141: is the ‘Nbin’ parameter linked to the bit depth of the image? Which is the bit depth of the images used in your study?
Lines 142 to 146: What was the ‘NTiles’ parameter chosen for the 1668x1668 pixels of the images taken from the public database?
Line 199: review the sentence: “…non-cataract cataract with bleeding (left) and, severe cataract (right).“
Line 263: Maybe is better a full stop (period) between ‘study’ and ‘future’.

Comments for author File: Comments.pdf

Author Response

We want to thank the reviewer's comments. We answered all the questions in the following paragraphs. Our replies are in green.

The study addresses a new methodology to get relevant information from images degraded by the presence of cataract blurring, but the paper doesn’t present a discussion about the possible appearance of artifacts or non-real features in the post-processed images.

Due to the block histogram equalization, CLAHE minimizes noise-like artifacts in homogeneous regions that appear when equalization is applied to the full image. Nevertheless, we have included a median filter to reduce remained high frequency noise introduced by CLAHE (as we mentioned in the manuscript).

Images were analyzed by the clinicians coauthors of this work. They agreed and recognized the different retinal features and did not find artifacts that could resemble retinal details. Artifacts in the shape of lines or squares appear always in severe cataracts and in mild and moderate cataract images in the blurry regions close to the edges of the images. Therefore, the appearance of these artifacts is closely related to the degree of blurriness of the image. The spatial frequency of the lines or squares depends on block size. The bigger the value of NTiles the higher the frequency of the artifacts. This sentence has been included in the manuscript in lines 241 to 247.

 

Questions like: how accurate is the post-processed image? Or, how accurate are the features found after the proposed method is applied? needs to be addressed to give support to the method in a diagnosis stage.

Quality of images and enhancement of features depends on the choice of NBins and NTiles values.

NTiles defines the block size or local region around a pixel for which the histogram is equalized. This size should be larger than the size of features to be preserved or enhanced. Moreover, the number of histogram bins used for histogram equalization within a block should be smaller than the number of pixels in the block. All images from both data sets are 8-bits images and their size is 2592x1728 or 2464x1632 for the public dataset and 2976x2976 for our dataset. All processed images presented in the manuscript were obtained with NTiles=16 and NBins=256. Larger values are not meaningful and reducing the number of bins reduces the dynamic range of the output image, by limiting the change in intensity to improve contrast. For the images presented in this work of ROI’s containing the optical disk we used NTiles=8 and NBins= 256. Nevertheless, the developed interface allows the user to choose different values for NBins and NTiles depending on the bit depth and region of interest to be processed.

This answer to the referee comment has been included in the manuscript from line 132 to 145. We did also include the values of ClipLimit that we used (paragraph from line 146)

These paragraphs also answer your questions about NBins and NTiles below (*)

 

One idea to get information to do that discussion, is to generate some simulated cataract images from eye fundus images with well know pathologies and features associated with them, adding a cataract-like blurring or contrast reduction.  Then, those simulated cataract eye fundus images can be processed with the proposed method, comparing the non-cataract images and known features with the simulated post-processed cataract images, to see the accuracy of the method or the appearance of non-real features.

We do not think a simulation can add value to this work.

Minor comments:

Line 29: add a reference supporting that statement about the change of color temperature of retina images.  It is important for a deeper understanding of interested readers.

We included the following reference

van den Berg, T. J. Intraocular light scatter, reflections, fluorescence and absorption: what we see in the slit lamp. Ophthalmic Physiol Opt. 2018, 38, 6-25.

Lines 82, 83: Discuss how do you select the set of 15 images from the public database? What were your selection criteria, since that database has 100 images?  After downloading two images from the database, it was found bigger resolution than that reported by you, then it is better to report which specific images you used and if you did some preprocessing to those images, as an area selection or resizing, etc.

The selection of the 15 images from the public database was made by picking 5 we thought they are representative of each cataract stage. Retinal images from public database don’t show retinal pathologies that’s why we added 4 images of our database. We developed the algorithm for easing diagnostics. Therefore, images shown in figures 5-11 were selected to show the improvement in the detection of the pathological features (see sentence in lines 175 to 177). Our methodology improves retinal images blurred by cataracts independent of the database used.

As you pointed out, there was an error in the manuscript. We thank the reviewer for its detection. The images of the public dataset present two different sizes 2592x1728, 2464x1632. This has been corrected in the manuscript.

Line 86: The classification done by the two clinicians over the 15 images selected from the public database was done without any discrepancy? Do both clinicians choose exactly the same classification for all 15 images?

One of the clinicians pick up the 5 images for each grade of cataract. The images were classified and analyzed by all clinicians, and they agreed in cataract classification as well as features detection.

Line 91: please report how many images from the hospital database do you process with your software?

We processed all images but only showed those with retinal pathologies because as explained above we developed the algorithm for easing diagnostics.

Line 130: review the sentence: “…image details can subjectively be better detected”.

We reviewed the sentence. Now it stands as follows: “we believe that image details can be easier detected “

(*) Lines 135 to 141: is the ‘Nbin’ parameter linked to the bit depth of the image? Which is the bit depth of the images used in your study? Lines 142 to 146: What was the ‘NTiles’ parameter chosen for the 1668x1668 pixels of the images taken from the public database?

NTiles defines the block size or local region around a pixel for which the histogram is equalized. This size should be larger than the size of features to be preserved or enhanced. Moreover, the number of histogram bins used for histogram equalization within a block should be smaller than the number of pixels in the block. All images from both data sets are 8-bits images and their size is 2592x1728 or 2464x1632 for the public dataset and 2976x2976 for our dataset. All processed images presented in the manuscript we have used NTiles=16 and NBins=256. Larger values are not meaningful and reducing the number of bins reduces the dynamic range of the output image, by limiting the change in intensity to improve contrast. For the images presented in this work of ROI’s containing the optical disk we used NTiles=8 and NBins= 256. Nevertheless, the developed interface allows the user to choose different values for NBins and NTiles depending on the bit depth and region of interest to be processed.

Line 199: review the sentence: “…non-cataract cataract with bleeding (left) and, severe cataract (right).“

Figure caption of figure 10 has been changed in accordance with the suggestion of the reviewer. Now it stands as follows: “Optical disc region for: non-cataract with bleeding (left) and, severe cataract (right).”

Line 263: Maybe is better a full stop (period) between ‘study’ and ‘future’.

Thank you very much for the correction.

Round 2

Reviewer 2 Report

Thanks to the authors for considering the previous comments.

I know that the major impact of your study is for easing the clinical assessment of retinal pathologies, but I still think that it is not clear enough if your method is 100% free of inducing a false feature detection.  I suggest adding a paragraph where you establish this is a concept probe study and that the method worked for the images used in the study, giving the total number of images you analyzed (15 from the public database + 17 from your own database, is it not?)

At the new text you added at the end of page 3, please review the sentence: “All processed images presented in the manuscript we have used NTiles …”

Author Response

We answer in red all the reviewer’s comments

• I know that the major impact of your study is for easing the clinical assessment of retinal pathologies, but I still think that it is not clear enough if your method is 100% free of inducing a false feature detection.  I suggest adding a paragraph where you establish this is a concept probe study and that the method worked for the images used in the study, giving the total number of images you analyzed (15 from the public database + 17 from your own database, is it not?)

We added the following paragraph to the Conclussion section:

The present work is a concept probe study where we tested the proposed algorithm with 15 images from a public database and 17 images of our own database. In these images the clinicians involved in the study did not identify any false feature that could have been induced by the image processing algorithm.

We included also the words “concept proof study” in the abstract. Now the abstract is written as follows:

Eye fundus images are used in clinical diagnosis for the detection and assessment of retinal disorders. When retinal images are degraded by scattering due to opacities of the eye tissues, the precise detection of abnormalities is complicated depending on the grading of the opacity. This paper presents a concept proof study on the use of contrast limited adaptive histogram equalization (CLAHE) technique for better visualization of eye fundus images for different levels of blurring due to different stages of cataracts. Processing is performed in three different color spaces: RGB, CIELAB and HSV, with the aim of finding which one enhances better the missed diagnostic features due to blur. The experimental results show that some fundus features not observable by naked eye can be detected in some of the space color processed with the proposed method. In this work, we do also develop and provide an online image processing, which allows clinicians to tune the default parameters of the algorithm for a better visualization of the characteristics fundus of the images. It also allows the choice of a region of interest (ROI) within the images that provide better visualization of some features than those enhanced by the processing of the full picture.

 

• At the new text you added at the end of page 3, please review the sentence: “All processed images presented in the manuscript we have used NTiles …”

We changed the sentence into: We used NTiles=16 and NBins=256 in all the processed images presented in the manuscript