Enhancing Autofocus in Non-Mydriatic Fundus Photography: A Fast and Robust Approach with Adaptive Window and Path-Optimized Search

Featured Application

In this study, we report on fast and robust autofocus for non-mydriatic fundus photography under less stringent imaging conditions. Based on image sharpness evaluations, the adaptive focus window and path-optimized search are used to detect the focus position accurately. This method can be applied to fundus cameras or similar ophthalmic instruments, effectively alleviating imaging challenges posed by patients with fixation difficulties.

Abstract

Non-mydriatic fundus photography (NMFP) plays a vital role in diagnosing eye diseases, with its performance primarily dependent on the autofocus process. However, even minor maloperations or eye micro-movements can compromise fundus imaging quality, leading to autofocus inaccuracy and a heightened risk of misdiagnosis. To enhance the autofocus performance in NMFP, a fast and robust fundus autofocus method with adaptive window and path-optimized search is proposed. In this method, the adaptive focus window is used to suppress irrelevant image contents and correct the sharpness curve, and the path-optimized search is constructed to overcome the curve’s local extrema, in order to achieve rapid focus position convergence. This method was simulated and clinically studied with the self-developed autofocus system for NMFP. The results of 80 cases of human eye imaging show that, compared with similar autofocus methods, this method achieves a focus success rate of 90% with the least axial scanning, and can adapt to non-ideal imaging conditions such as pupil misalignment, eyelash occlusion, and nystagmus.

1. Introduction

Imaging the fundus and its underlying lesions provides indispensable evidence for diagnosing various ophthalmic diseases, such as diabetes, glaucoma, and age-related macular degeneration [1,2,3,4,5]. Non-mydriatic fundus photography (NMFP) is a non-invasive inspection technique for high-resolution imaging of the fundus [6]. Its success is attributed to NMFP eliminating the need for pupil dilation using pharmaceutical agents, which improves patient comfort and ease of examination. Combined with recently developed wide-field imaging techniques [7], NMFP applied in telemedicine is capable of imaging retina tissues, and even the entire fundus.

The NMFP workflow involves adjusting the device to the ideal spatial position near the eye via near-infrared fundus preview, followed by the immediate activation of the argon arc flash to capture color fundus images [6]. Considering the differences in eye diopter, this convenient and unique NMFP mechanism also imposes a strict requirement: the objective focal plane needs to overlap with the retinal surface. Any slight defocus drastically degrades the sharpness of fundus images, potentially hiding lesions and leading to misdiagnosis [8]. However, due to the eye’s structural complexity and its independence from the imaging system [9,10], satisfying this requirement presents a formidable challenge. Throughout the focusing process, maloperations and eye micro-movements can easily cause the device to shift from its ideal position [11], leading to (i) misalignment of the pupil center (impairing imaging quality) and (ii) tiny displacements between the retina and focal plane. Consequently, NMFP devices often require swift movements of the objective lens position to compensate for eye defocus while adapting to underlying non-ideal imaging conditions.

Determining the best focus position by eye is the gold standard for delivering superior image quality. However, this is cumbersome and time-consuming, particularly in community-based medical screenings that image numerous subjects. To address this issue, image-based autofocus (IBAF) methods are widely implemented in NMFP, which analyze the image to distinguish its focus degree without additional optical modifications [12,13]. IBAF methods are mainly divided into learning-based detection (LBD) and maximum sharpness detection (MSD) [14]. LBD entails constructing and training a convolutional neural network based on extensive data, then rapidly predicting the focus position according to one or more defocused images [15,16]. MSD has favorable applicability and is free of learning, and it drives axial scanning to determine the focal plane by calculating the sharpness value of different defocused images. Given the difficulty in collecting high-quality data from diverse patients and data labeling, this paper only concentrates on the latter, which includes two key issues: image sharpness evaluation and search strategy [17].

Sharpness evaluation describes the focus level by quantifying the sharpness characteristics of images, and the obtained evaluation curve should have good unimodality, unbiasedness, and sensitivity [18]. In NMFP scenarios, numerous studies have achieved positive progress on sharpness evaluations [11,19,20,21,22,23]. Computing the directional variance of the normalized discrete cosine transform provides superior results [21]. To improve evaluation curve performance, the focus window of interest (usually fundus vessels and discs) is generally used to replace the entire image [24], such as manual windows [21], fixed windows [25], and multi-grid windows [18]. According to the evaluation curve of an image stack, search strategy finds the highest score to determine the focus position. To expediently find this maximum, the preference leans towards employing greedy local searches (such as hill climbing [20] and golden section search [18]) instead of stable global searches (such as exhaustive search [26] and function fitting [27]). Recent research introduced mechanisms, including coarse-to-fine [28] and adaptive step-size [29], to constrain the scanning range, thereby speeding up the convergence and enhancing robustness. However, the above methods are only designed for the ideal NMFP imaging condition. Low-quality fundus images caused by non-ideal imaging conditions are common (around 21%), such as stray light, motion blur, and nonhomogeneous brightness [30,31]. These cases degrade the performance of traditional sharpness evaluations and require additional customization for focus windows. In addition, poor imaging environments exacerbate curve noise, which makes search strategies susceptible to local extrema and decelerates the scanning process.

To solve the above problems, a fast and robust IBAF method is introduced for enhanced autofocus in NMFP. First, an adaptive window was designed using imaging prior knowledge, grayscale features, grid screening, and morphological operations to improve the performance of sharpness evaluation when irrelevant image contents interfere. Then, the exponentially weighted moving average (EWMA) was integrated into the traditional hill-climbing method to optimize the search path in real-time, which improves the ability to suppress local extrema and reduces axial sampling. In simulation analyses and clinical studies involving 40 volunteers, the proposed method performed well under non-ideal imaging conditions such as pupil misalignment, eyelash occlusion, and nystagmus, which are difficult to resolve using existing IBAF methods.

2. Proposed Methods

The framework of the autofocus method in this paper consists of three parts: adaptive focus window, sharpness evaluation, and path-optimized search. This method provides image regions of interest containing accurate fundus information through an adaptive focusing window, and then quantifies image sharpness using existing evaluation criteria [11,21,22,23]. Based on the sharpness curve of an image stack, the path-optimized search is designed to modify local extrema, so that the accurate fundus autofocus can be achieved.

2.1. Adaptive Focus Window

To improve autofocus speed and accuracy, most IBAF methods select partial regions as the focus window for detection. An ideal focus window should meet the following criteria [18]: Firstly, the evaluation curve of selected regions has satisfactory performance; secondly, the focus position of selected regions can represent the actual focus position; thirdly, the window is automatically calculated according to image contents.

Considering the window selection in a fundus image collected by the device shown in Figure 1, if the window covers contents such as out-of-field redundant information (ORI) and stray light (SL), irrelevant regions are involved in sharpness calculation, which leads to multimodal and flat evaluation curves. Rich and accurate target fundus contents with steeper curves contribute to reliable autofocus, for which the following adaptive focus window is proposed (Figure 2).

2.1.1. Selection of Fundus FOV

Since NMFP primarily adopts annular coaxial illumination that avoids corneal reflection [32], image contents of interest are mainly concentrated in the central circular FOV. The FOV region is considered extracted to avoid the influence of ORI. Compared with other eye anatomies, the green illumination component is more easily absorbed by hemoglobin-rich retinal tissues, such as vessels [5]. Therefore, the green channel image with higher contrast is first extracted, and an appropriate circular Mask_FOV is selected (Figure 2a). Then, grayscale normalization and down-sampling are performed to overcome potential nonhomogeneous illumination.

2.1.2. Selection of SL Region

OTSU threshold segmentation [33] is performed on the processed image to further select and eliminate the SL region. The algorithm divides the entire image into target and background regions, then finds the optimal solution that maximizes their interclass variance as the threshold. Although the grayscale difference between the SL and target FOV is significant (Figure 2b), the pixel ratio imbalance can easily lead to OTSU calculation failure. Hence, a grid screening mechanism was designed to weaken this effect: (i) the image is divided into adjacent grids with the same size; (ii) the SL’s central highlight pixels are located using a preset saturation threshold, then used as a benchmark to screen all grids; (iii) the grayscale of the grids without highlight pixels is set to 0 and called “turning off”; otherwise, it is called “light up” without operation (Figure 2b); (iv) the Mask_SL is obtained by performing the OTSU algorithm only on “light up” grids.

Since the SL region may be extremely large or small due to subtle variations in imaging conditions, Mask_SL is then modified based on statistical knowledge to improve generalization ability. The probability of a pixel X in the FOV existing in area α is defined as follows:

$$p\left(X\in \alpha \right)=N_{\alpha }/N_{Mas{k}_{FOV}}$$

(1)

where N_α represents the number of pixels contained in α. If $p(X\in Mas{k}_{FOV}\cap \overline{Mas{k}_S})$$<p(X\in Mas{k}_{FOV}\cap Mas{k}_S)$, the SL regions become dominant in the image contents, and this case is excluded from the sharpness evaluation. If $p(X\ge th)<p(X\in Mas{k}_{FOV}\cap {\alpha }_{grid})/2$, the pixel ratio imbalance still potentially exists, and the highlighted area of the SL region is selected as Mask_SL.

2.1.3. Mask Fusion and Optimization

The adaptive window is chosen by fusing the obtained Mask_FOV and Mask_SL, mathematically expressed as their intersection, $Mas{k}_{FOV}\cap \overline{Mas{k}_{SL}}$. To further optimize the window, appropriate structural elements are selected for morphological opening–closing operations to fill small holes and cracks while overcoming the edge diffusion of the SL region. Finally, the largest connected domain in the selected region is positioned as the final window (Figure 2c).

2.2. Path-Optimized Search with EWMA Control

Unlike static focus scenarios [27,28,29], NMFP requires rapid execution within relatively ideal imaging conditions to mitigate the impact of low-quality fundus images. In this study, a high-performance autofocus search should have the following: (i) fast convergence; (ii) a strong ability to suppress local extrema.

The hill-climbing method [20] converges immediately upon meeting termination conditions, yet the search path is susceptible to becoming trapped in a local optimum. Exponentially weighted moving average (EWMA) [34,35], a data processing method for process control, provides a heuristic solution. Its core idea is to fuse the current measurement value and historical statistical values to overcome data disturbances. Based on this mechanism, a path-optimized search with EWMA control (POSE) was designed, which skillfully directs the search path of hill-climbing towards the focal plane by dynamically rectifying the evaluation curve noise.

Figure 3 illustrates the initial consideration of equal-step sampling, where the sharpness values of the image stack are stored in a vector F = [F₁, F₂, …, F_t], with t representing the current sampling point. Then, the EWMA vector E = [E₁, E₂, …, E_t] is calculated with the following recursive formula:

$$E_t=\beta E_{t-1}+\left(1-\beta \right)F_t$$

(2)

where β is the smoothing coefficient (0 < β < 1), and E_t−1 represents historical statistical values of sharpness information before t (E₀ = 0). Expanding E_t−1 in (2) step-by-step, E_t can be written as follows:

$$E_t={\displaystyle \sum _{n=1}^t\left(1-\beta \right){\beta }^{t-n}{F}_n}$$

(3)

It is regarded as the inner product of vector F and an index weight vector (1 − β) [β^t−1, β^t−2, …, β¹, β⁰], and the weight away from t decreases exponentially. In theory, the curve described by vector E can effectively smooth the local errors within the search path. Simultaneously, it also enables real-time tracking of the sharpness curve described by vector F due to its capacity to disregard outdated data gradually. With the performance advantages of EWMA, the first stage of POSE is realized through the following steps:

• STEP 1. Perform continuous axial scanning with a large step D (D < Δ, Δ is the depth of field (DOF) of the imaging system) while utilizing the sampling interval to update vectors F and E;
• STEP 2. If vector E decreases continuously for three steps, E_t−2 > E_t−1 > E_t (meaning the scanning starts to leave the curve’s peak), then proceed to STEP 3; otherwise, go back to STEP 1.
• STEP 3. Pause the scanning and calculate the coarse focus position, P_coarse = $[\arg \max (\mathit{F})]·D$.

β is an essential parameter to determine the performance of POSE. Assuming that the contribution of low-weight points (less than 1/e times the weight of t position) is ignored, EWMA can effectively average about 1/(1 − β) sampling points. Increasing β means improving the smoothness of vector E but weakening its sensitivity (leading to slow convergence). To balance the trade-off between search efficiency and noise immunity, smoothing sampling points within Δ are considered sufficient to overcome most local extrema in the search path, and β that satisfies this constraint is given by the following:

$$\beta =1-D/\Delta$$

(4)

The second stage of POSE provides high focus accuracy by compressing the search interval and step size. To ensure that the search path covers the curve’s peak, the minimal fine-search interval [l_A, l_B] around P_coarse is first determined, where l_A = P_Coarse + (Δ + D)/2, l_A = P_Coarse − (Δ + D)/2. Then, scanning is driven to sample with a small step d (target accuracy) in this interval and obtain an image stack S. The focus position is calculated as follows: P_fine = $l_B-\{\arg \max [\mathit{F}(\mathit{S})]\}·d.$

3. Experimental Results and Discussion

As shown in Figure 4, the self-developed autofocus system for NMFP mainly consists of a non-mydriatic fundus camera (860B; MEDA, Tianjin, China), an industrial CCD sensor (MU3S640C; CatchBest, Beijing, China), a 3D translational stage, a stepper motor module (7TRSM424X with RS-485 communication; Shidaichaoqun, Beijing, China), a driving power (HY3005B; Huayi Electronics, Hangzhou, China), and a PC. In this system, the fundus camera previews the retina through near-infrared illumination (680–830 nm) and switches to white light illumination (580 nm) to obtain fundus images (1280 × 1024 pixels). The stepper motor module (driven by the power) changes the object distance by shifting the camera’s objective lens under the PC control, thereby realizing fundus autofocus within the range of [−16 D, +16 D]. The PC’s specifications include an Intel Core i5-9500 CPU with a main frequency of 3.0 GHz and 4 GB of memory. The entire workflow, including image acquisition, processing, and motor control, was implemented in Visual Studio 2013 using C++.

3.1. Experiment on Adaptive Focus Window

To verify the effectiveness of the adaptive window, this window was applied to three types of representative images summarized in 80 sets of human eye focus data. As shown in Figure 5, the proposed window can avoid irrelevant image contents, and the selected region basically covers all visible fundus details, which meets the selection criteria of ideal windows.

To compare with traditional methods, full window (entire image), inverted T window [25] (fixed region), manual window [21] (expert annotation), and proposed window were applied to image stacks of representative examples in Figure 5 (sampling interval is half DOF). The sharpness evaluation curves were calculated using the criteria of [11,21,22,23], as shown in Figure 6. For a fair comparison, all calculation results are divided by the corresponding window area (the number of pixels included), and then normalized as the sharpness evaluation values (SEV).

In the ideal imaging condition, evaluation curves of three automatic windows accurately indicate the actual focus position. However, because the adaptive window can actively exclude the contribution of image invalid features, the curve shows better unimodality, facilitating fast focus search convergence. In non-ideal imaging conditions, the proposed method maintains a curve distribution that is similar to the ideal case, without obvious local extrema. In contrast, the other two automatic windows regard the grayscale mutation of SL and ORI as effective sharpness characteristics, leading to evaluation curves that fail to reflect the ‘defocus-focus-defocus’ trend. These results demonstrate that adaptive window enhances robustness and focus accuracy for sharpness evaluation criteria.

It can also be concluded from Figure 6 that the evaluation curves of the adaptive window are comparable to those of the manual window, although the peak sensitivity [17] of the latter is slightly higher, which does not affect the focus accuracy. In this case, the proposed window only needs 2 ms to complete the calculation of a single image, which is 1000 times faster than the manual window (taking about 2 s), thereby significantly shortening the fundus examination time.

3.2. Simulation Analysis of POSE

To measure the ability of the proposed strategy to suppress local extrema, POSE was compared with the improved hill-climbing search (IHCS) [29] in simulation experiments. The Gaussian equation F_i^(j) (i = 1, 2, …, 25, p = 13) in (5) generates sets of simulated sharpness evaluation curves [36], where j, p, σ_f, and α denote the image stack index, focus position, Gaussian standard deviation, and offset, respectively. A normally distributed random number noise(ε_f) generates the noise bias with mean 0 and standard deviation ε_f to simulate local extrema. The above hyperparameters are obtained by Gaussian fitting of the clinically collected curves.

$$F_i^{\left(j\right)}=A\exp \left(-\frac{{\left(i-p\right)}^2}{2{\sigma }_f^2}\right)+\alpha +noise\left({\epsilon }_f\right)$$

(5)

The experiment generates 10,000 sets of simulation curves, and their peak positions are estimated through the above strategies (with i = 1 as the start point). As shown in

Table 1. Simulation results on search strategies.

Quantitative Criteria	IHCS	POSE
Average number of local extrema	3.38	0.65
Average search success rate	71.56%	93.40%

, compared with IHCS, POSE has 2.73 fewer local extrema in the search path, and the search success rate is increased by 21.84%. Hence, the proposed strategy holds stronger robustness to the noise.

Some search results were further visualized to explain this performance improvement of POSE. Even if the curve oscillates violently near the start point or peak (Figure 7a,b), POSE can still obtain an accurate search path utilizing the error smoothness and data awareness of EWMA, while IHCS only judges the convergence based on the original data, resulting in spurious peaks. Furthermore, POSE fails when the curve exhibits a significant dip (Figure 7c) near the start point. Since the historical statistical value of EWMA in the early search stage appears unreliable, the convergence condition is met once the search path enters the ‘valley’. However, such cases are rarely observed in practical fundus focus. This limitation can be overcome by optimizing imaging conditions or conducting autonomous searches within partitions.

Due to individual differences in eye diopter, the curve peak often occurs in off-center positions (p ≠ 13). To demonstrate the applicability of POSE, the peak position (let p = 8, 9, …, 18, corresponds to [+10 D, −10 D] diopter, respectively) in (5) is changed, and the search process is repeatedly performed to obtain the curves shown in Figure 8. It can be seen that the search success rate of POSE is higher than that of IHCS in all cases. As the diopter changes from positive to negative, the defocused range of the curve widens, and the disturbance introduced by noise bias becomes more pronounced. Hence, the overall success rate tends to decline. However, POSE exhibits an 85% search success rate, even with a −10 D diopter, which proves the proposed strategy applies to different individuals.

3.3. Clinical Study

To evaluate the proposed method’s overall performance and actual focus effect, 40 volunteers (17 women, 23 men; mean age = 35 ± 12 years, range 18–75; mean refractive error = +0.89 ± 0.32 D, range −0.55–+4.5 D, no history of other eye diseases) were recruited to participate in a clinical study. The Ethics Committee of Tianjin University approved the study, and all participants provided informed consent to participate in the retinal examination (both eyes), in compliance with the Declaration of Helsinki. Manual focus was performed in each case (Experts A, B), as well as three search strategies for autofocus: Gaussian function fitting (GFF) [37], IHCS, and POSE. Autofocus was considered to be successful if an expert believed that that automatic and manual focus results were consistent [16]. This study adopted adaptive window and sharpness evaluation criteria in [21] for all autofocus cases with uniform search accuracy (1/4Δ) and scanning start point. The comparison results and some autofocus results are shown in

Table 2. Performance comparison of three search strategies with adaptive focus window.

Strategy	Expert	Success	Failure	Success Rate	Motor Movement Count	Motor Steps *
IHCS	A	56	24	70.0%	22	7120
	B	58	22	72.5%
GFF	A	65	15	81.3%	32	12,010
	B	64	16	80.0%
POSE	A	72	8	90.0%	10	5626
	B	71	9	88.8%

* Motor movement count and motor steps are used to measure the search efficiency [17].

and Figure 9.

The study results show that the combination of adaptive window and POSE achieves the highest autofocus success rate of up to 90%, confirming the simulation results. Moreover, the proposed method is the least expensive regarding motor movement count and steps (each autofocus takes about 2.1 s), thus striking a better balance between search efficiency and accuracy. From Figure 9a,b, three strategies achieved acceptable focus results under both ideal and pupil misalignment (generating SL) conditions, mainly due to the excellent evaluation curve performance provided by the adaptive window. However, both local IHCS and global GFF strategies suffer from local extrema when nystagmus and eyelash occlusion occur, resulting in defocused images, as seen in rows 1–2 of Figure 9c,d. Conversely, POSE still provides focused images containing distinct and recognizable retinal anatomies, including vessels and leopard patterns, as seen in row 3 of Figure 9c,d. These results illustrate that the proposed method can rapidly and accurately provide autofocused images of superior quality under complex fundus imaging conditions. In addition, our method exhibits 10% failure in cases that are primarily attributed to (i) involuntary eye movements or blinking during autofocus initiation; (ii) motor speed limitations that prevent faster autofocus. Using faster system configurations (such as piezoelectric ceramic stages and GPU parallel computing) and good individual guidance before examination can help further improve the autofocus success rate.

4. Conclusions

This research proposes a fast and robust NMFP autofocus method to enhance the quality of fundus imaging, particularly under poor imaging conditions. The method first improves the overall performance of the sharpness evaluation curve by calculating the adaptive window, then constructs a path-optimized search based on EWMA to overcome local extrema and rapidly converge to the focus position. Experiments and simulations supported insights into issues of the focus window and search strategy. In a clinical study of 80 cases, the proposed method provided the highest autofocus success rate of 90%, with the least axial scanning compared to similar IBAF methods. Furthermore, the method exhibits robustness in non-ideal imaging conditions such as pupil misalignment, eyelash occlusion, and nystagmus. These benefits are expected to improve the efficiency and accuracy of community-based fundus screenings and alleviate challenges faced by individuals with difficulty in vision fixation, including children and the elderly. Future research will involve more extensive clinical studies, including patients with eye diseases that make it difficult to focus normally, as well as autofocusing peripheral retinal regions, providing possibilities for achieving precise fundus image stitching and ultra-wide-angle fundus imaging.