High-Resolution Retinal Imaging: Technology Overview and Applications

Abstract

Adaptive optics (AO) has been used in many applications, including astronomy, microscopy, and medical imaging. In retinal imaging, AO provides real-time correction of the aberrations introduced by the cornea and the lens to facilitate diffraction-limited imaging of retinal microstructures. Most importantly, AO-based retinal imagers provide cellular-level resolution and quantification of changes induced by retinal diseases and systemic diseases that manifest in the eye enabling disease diagnosis and monitoring of disease progression or the efficacy of treatments. In this paper, we present an overview of our team efforts over almost two decades to develop high-resolution retinal imagers suitable for clinical use. Several different types of imagers for human and small animal eye imaging are reviewed, and representative results from multiple studies using these instruments are shown. These examples demonstrate the extraordinary power of AO-based retinal imaging to reveal intricate details of morphological and functional characteristics of the retina and to help elucidate important aspects of vision and of the disruptions that affect delicate retinal tissue.

1. Introduction

As an extension of the brain, the retina is the only part of the central nervous system (CNS) that can be imaged in vivo non-invasively by optical methods at cellular-level resolution [1,2]. Retinal photoreceptors convert light into electrical signals and send them to the brain along a circuitry consisting of several types of retinal neurons followed by the long axons of the ganglion cells. Any dysfunction along this path can negatively affect an individual’s vision and quality of life. All retinal diseases disrupt some aspects of signal transfer in one or multiple chain links, leading to vision problems. The leading causes of blindness worldwide are age-related macular degeneration (AMD), diabetic retinopathy (DR), and glaucoma. The number of cases for these three eye diseases in the U.S. (age 50 and older) grew from 8.03 million in 2000 to 12.48 million in 2010 and is expected to rise above 25 million by 2050 [3], with a devastating impact on the quality of life and a staggering economic burden on society [4]. For example, DR impairs retinal blood flow and weakens the retinal circuit’s ability to transmit visual signals to the brain. Following a diagnosis of type 1 diabetes (T1D) in pediatric patients, the incidence of clinically detectable DR is ~8% at 3 years, 25% at 5 years, 60% at 10 years, and 80% at 15 years [5]. Testing the proper functionality of this circuitry generally relies on the subject’s response to specific light stimuli, which can be subjective, coarse, and might depend on other external factors. Recently, there have been significant efforts to develop methods to measure photoreceptor activity and the electrical signal transfer following controlled light stimuli without requiring the subject’s feedback. Functional testing of retinal circuitry can provide an unbiased evaluation of vision, enable early detection of potential signs of trouble, and monitor treatment response.

Blood vessels facilitate the delivery of oxygen and nutrients and the evacuation of waste from the tissue. Some blood vessels alter their caliber and their permeability in response to the changing needs of the neural tissue, processes that can be affected by diseases. Vascular complications are usually the first to be detected in DR. The major retinal arteries and veins are visible in standard fundus photography; however, this technique does not have the resolution to visualize retinal capillaries. Fluorescein dyes are used to improve the visualization of the retinal vasculature. However, this technique still lacks the resolution for the finest capillaries and is an invasive method with potential adverse effects due to the contrast agent.

As part of the CNS and of the cardiovascular system, the retina is an open window into the body and systemic diseases that have manifestations in the eye can be diagnosed through retinal investigations, a field known as Oculomics [1,2]. Alzheimer’s disease (AD) is the most common age-related neurodegenerative disorder characterized by memory loss and cognitive deficits in elderly adults, usually over 65 years of age, affecting over 50 million people worldwide [6]. With no cure and limited options for early noninvasive diagnosis, this devastating and always fatal disease is a major medical, sociological, and economical challenge worldwide [6,7]. Post-mortem analysis of the brain for amyloid β-protein (Aβ) plaques and neurofibrillary tangles comprising the microtubule protein tau remains the gold standard for AD diagnosis [8,9]. Therefore, biomarkers for early detection of AD pathology have been sought over the last few decades. High-resolution optical imaging of the retina is being tested as a noninvasive approach to visualize Aβ plaques in the retina in AD patients, knowing that retinal Aβ plaques share properties with those in the brain [10,11].

Clearly, there is a significant need for technology that allows early detection of retinal pathology, which would enable improved diagnosis and treatment, and therefore, a better outcome for patient vision and health. Cellular-level retinal imaging based on adaptive optics (AO) is a powerful tool to capture the most detailed picture of retinal health and to visualize cells in vivo and has been pursued in the living eye over the last two decades. Many cells and cellular structures including retinal pigment epithelium (RPE) [12,13,14,15,16], cones and rods [17,18,19,20,21], retinal capillaries [22,23], vessel walls [24,25], retinal ganglion cells (RGCs) [26,27], nerve fiber bundles [28,29,30], and microglia [27,31] have been resolved using different imaging modalities. AO enables cellular-level imaging in the living eye and opens the door to understanding retinal function and malfunction, and to potential early diagnosis. Developed initially for astronomy [32,33,34,35] to overcome the image degradation induced by the turbulent atmosphere, AO has advanced in vivo imaging at the cellular level in a wide range of applications in ophthalmology [36,37,38,39]. It has been integrated into flood illumination retinal cameras, confocal scanning laser ophthalmoscopes (SLOs) for reflectance and fluorescence imaging, and optical coherence tomography (OCT) instruments for retinal imaging in humans and animals [36,37,38,39,40,41,42,43,44,45,46]. AO is being used to help elucidate the structural and functional aspects of vision, from complex retinal circuitry to neurovascular physiology, and the signatures of cellular pathologies and processes during disease progression. AO overcomes fundamental limitations imposed by ocular geometry and optics. The eye optics can focus a parallel beam to a nearly diffraction-limited spot on the retina for a small pupil diameter of less than approximately 2 mm. However, to increase lateral resolution in retinal imaging, the eye pupil needs to be dilated to increase the numerical aperture (NA) of the focused beam, given the fixed length of the eye. The main issue with a larger pupil is that additional ocular aberrations due to the cornea and lens distort the wider wavefront and blur the image. AO is used to correct ocular aberrations to achieve the actual potential resolution of a higher NA from a dilated pupil. Examples of the extraordinary power of AO-based instruments to reveal the intricacies of the retinal structure are shown in Figure 1.

In the Section 2 we present an overview of various platforms developed for high-resolution retinal imaging in human and small animal (mice and rats) eyes. We illustrate representative results obtained in both healthy and diseased eyes in the Section 3 and then we highlight important observations about these images in the Section 4.

2. Materials and Methods

AO systems sense ocular aberrations that arise primarily from the cornea, lens, and tear film with a wavefront sensor (WS) and correct them in a closed-loop manner with a wavefront compensator (deformable mirror—DM) [47,48]. Typically, a large diameter beam (7–8 mm diameter covering a dilated pupil) is incident on the eye pupil, and the eye optics focuses it to a spot on the retina. A WS camera records the images of this spot through a lenslet array that creates an array of spots. The local displacements of the spots (called slopes) with respect to their calibration position indicate the local wavefront distortion as compared to a reference (plane wave) wavefront. Following a calibration procedure, these slopes are used to produce a shape on the DM which corrects the optical aberrations that distorted the wavefront. During the calibration procedure, the actuators of the DM are poked one at a time, and the slopes are recorded as the step response of the system to each actuator poke. All the slopes are included in a large matrix called the influence function, which is then inverted into the so-called pseudo-inverse matrix. Any set of slopes for a new wavefront distortion multiplied with the pseudo-inverse matrix will generate a set of voltages that are sent to the DM to correct the distortion. The voltage calculation is performed in real-time in a closed-loop control algorithm to provide diffraction-limited imaging. This calibration procedure is repeated as needed if the instrument becomes misaligned. The dynamic range of the WS determines the magnitude of the aberrations that can be sensed and depends on the lenslet array pitch (spacing). The stroke and actuator count of the DM determine the magnitude and number of higher-order aberrations that can be corrected in the wavefront.

Over the last 25 years, efforts have been geared toward improving AO retinal imaging performance, especially with respect to resolution and imaging speed. Although AO enables high-resolution imaging of the eye fundus, specific microstructures such as blood vessel wall cells, fine capillaries, and other types of cells are mostly transparent and produce no contrast in confocal SLO images. About 12 years ago, a new method was developed to reveal the details of the retinal vasculature network with unprecedented clarity [25] by using a larger, offset pinhole in the AO detector to capture multiply scattered light (instead of light directly backscattered). In this way, contrast is obtained from forward scattering, light-diffracting structures such as erythrocytes and vessel walls. Shortly thereafter, in 2014, split-detector imaging [21], introduced initially in differential phase contrast microscopy [49], was used to visualize the outer segment of cone photoreceptors by using two offset apertures optically conjugated to the opposite sides of the illumination spot. The subtraction of the two offset images divided by their sum produces the split-detector image which highlights edges (such as blood vessel walls) perpendicular to the split direction; however, the method is less sensitive to refractive index discontinuities along the direction of the two offsets producing directionality artifacts. A “multi-offset” approach demonstrated in 2017 [26] by repositioning the offset aperture sequentially over a large combination of distances and angles with respect to the illumination spot can remove directionality artifacts and significantly increase the imaging contrast; however, sequential imaging in many detection configurations is very tedious, time-consuming, and difficult to use in the clinic.

A new method in which four offset apertures and the confocal channel are simultaneously acquired was first presented at ARVO 2018 [50] and then at CLEO 2018 [51]. Light-collecting fibers acting as apertures are arranged in a compact bundle placed in the detection plane. Alf Dubra’s group introduced two orthogonal split configurations [52], while Austin Roorda’s group presented the use of a bundle with seven fibers for enhancing the signal-to-noise ratio (SNR) by over-sampling the Airy disk followed by pixel reassignment [53]. Roorda’s group showed [54] that by magnifying the Airy disk in the focal plane, they can collect both the confocal and multiple offsets. However, a special mask is needed to create the confocal aperture since the diameter of the central fiber is too large.

The simple and elegant 5-fiber configuration developed by Physical Sciences Inc (PSI) has been integrated over the last few years into multiple imaging platforms, which are described below, together with relevant results from human and animal imaging studies.

As described in detail in [55], split-detection analysis, sometimes called differential phase contrast, is interpreted as phase derivative. One can reconstruct the phase from these derivatives or directly calculate the phase gradient. In our imaging configuration, split-detection analysis is performed using multiple combinations of the four offset images. Two orthogonal split (SPL) images are generated for orthogonal fiber pairs 1–3 and 2–4 (SPL 1 and 2). Two additional SPL images (SPL 3 and 4) are obtained by adding first adjacent fibers and then performing subtraction divided by the sum of the two added images. Therefore, we obtain four SPL images, two for horizontal and vertical directions and two diagonal (+/−45 deg). Each of the four SPL images highlights structural edges such as blood vessel walls along different directions given the directionality associated with the split (offset) direction. The phase gradient (PG) images are obtained from either one of these two pairs of SPL images. Standard deviation (SD) among the four simultaneous offsets [26] highlights the differences among these images, which are predominant at the blood vessel walls, due to the detection directionality diversity. The sum of all four offset images (S) combines all detected scattered light in a ring-detection configuration while rejecting the direct backscattered light detected in the confocal image through the central fiber (SLO or C). In addition, a mean of the four split stds (STD) is calculated, isotropically highlighting the blood flow vasculature as motion contrast.

Over the last two decades, PSI has developed a broad portfolio of retinal imaging instruments customized to the needs of the early adopters of AO technology, with support from multiple NIH institutes, including NEI, NIDDK, NIBIB, and NIA. All instruments are multimodal and have simultaneous acquisition of OCT and SLO images, both assisted by AO for diffraction-limited retinal imaging. The imagers have been designed for a compact footprint, are significantly smaller than typical research AO instruments, and are placed on mobile carts suitable for clinical environments. Another common feature of all our instruments is that the OCT beam is also used as the WS beacon, eliminating the need for an additional light source. A pupil camera helps position the eye at the instrument pupil, while a point-spread-function (PSF) camera enables improved imager calibration. Additional imaging modalities such as PSI proprietary line-scanning ophthalmoscopy (LSO) [56] for large area mapping and other features such as retinal tracking, dual DM, Badal focusing or trial lenses to accommodate a large range of focus correction, controlled light stimulation from cone-level to large areas, fluorescence/autofluorescence, multispectral analysis, or high-speed OCT have been explored and implemented in different versions of the retinal imaging platform. A gallery of the high-resolution retinal imaging instruments is shown in the Supplemental Figure S1 illustrating the very compact form factor as compared to other research instruments.

The early adopters of AO technology that have used various versions of the PSI instruments include Joel Schuman and Gadi Wollstein at the University of Pittsburgh Medical Center (UPMC)/New York University (NYU)/Wills Eye Hospital, Philadelphia; Anne Fulton and Jim Akula at Boston Children’s Hospital (BCH); Carol Westall and Tom Wright at Sick Kids Hospital Toronto; Melanie Campbell at University of Waterloo; Michel Paques, Kate Grieve, and Elena Gofas at the Paris Eye Imaging Unit, Quinze Vingts Ophthalmology Hospital; Jennifer Sun and Konstantina Sampani at Joslin Diabetes Center (JDC), Beetham Eye Institute, Boston; Richard Rosen at New York Eye and Ear Infirmary (NYEEI); and Marco Lombardo at Fondazione G.B. Bietti, Rome.

Several academic groups worldwide have developed their own research AO-SLO or AO-OCT retinal imaging platforms in a rather small but vibrant research community that includes scientists and clinicians. Extraordinary results are being published extensively as novel contrast mechanisms are developed based on AO-SLO/OCT images. However, what appears to be lacking is a comprehensive clinical validation of the technology’s diagnostic power before large-scale adoption into clinical practice can occur.

2.1. Human Subjects and Imaging Procedure

All observational studies have been conducted under IRB approval. Depending on the purpose of the study, specific imaging protocols have been designed at each research institution. The exclusion criteria included pupillary miosis or an inability to dilate, prior panretinal photocoagulation, media opacities limiting the ability to acquire high-quality AO-SLO/OCT images, and hypertension. Each study eye underwent mydriasis, ultrawide fundus photography, and AO-SLO/OCT imaging in a single-visit study.

2.2. MAORI

The multimodal AO retinal imager (MAORI) [57] is a modular, compact, clinical prototype that enables researchers to explore in the clinic or the lab, with near-isotropic micron-level resolution, the fine cellular and lamellar retinal structure of subjects’ retinas. Previous publications have extensively described the platform [58,59], including design characteristics and performance. Several versions of MAORI have been built with different combinations of wavelengths for multiple imaging modalities, including spectrometer-based OCT at 840 nm or swept-source OCT at 1060 nm, depending on the functional modules incorporated and the specific requirements of the application as required by the user. A block diagram of the typical system is shown in Figure 2 [58], and a simplified optical setup is shown in Figure 3. In addition to the simultaneous AO-SLO and AO-OCT channels mentioned above, there is a retinal tracker (RT) for image stabilization and an LSO imager for navigation. The LSO generates 2D images by scanning a focused line on the retina and detecting the de-scanned backscattered light with a linear array CCD. The lateral resolution in the scanned direction is ~20 µm, and the non-AO corrected LSO offers a large angle (~30°) view of the retina. The width of the LSO focus at the CCD (image plane) matches the height of the CCD pixels, making the LSO essentially a quasi-confocal setup that significantly suppresses multiply scattered light. Therefore, compared to conventional fundus imaging modalities based on flood illumination, the LSO module provides improved resolution and contrast.

To enable autofluorescence imaging of the RPE mosaic, one of the MAORI systems (Paris) has been modified as shown in the simplified optical schematic in Figure 4 [60]. More recent improvements of the Paris system include the implementation of the fiber bundle to simultaneously acquire one confocal and four offset channels, and the implementation of cone-level light stimulation for optoretinography studies.

A slightly different design of the MAORI platform, as tested at JDC, is illustrated in Figure 5. One of the main differences between the designs shown in Figure 3 and Figure 5 is the order in which the DM and the beam-steering module are placed in pupil conjugates. Both versions, with the DM or the scanning module placed in the last pupil conjugate in front of the eye, have been described in the literature [55,58,59]. Another notable difference is the diameter of the last two spherical mirrors in front of the eye. Larger mirrors enable more flexible combinations of fixation steering and scanning offsets to cover wider spread imaging locations on the retina, while smaller mirrors allow for a more compact design, limited, however, to mostly fixation steering. The main design specifications of this MAORI version are image size: 1–5° user selectable (640 × 640 pixels); SDOCT/WS beacon: 830 nm, >60 nm FWHM, and 750 µW at pupil; SLO: 750 nm SLD, 20 nm FWHM, and 400 µW at pupil; resonant scanner frequency: 12 kHz; pupil diameter: 6.7 mm; lateral resolution: ~2.5 µm; frame rate: >30 Hz; field of view: 20° × 30° (fixation).

2.3. CAORI

Several generations of the Compact adaptive optics retinal imager (CAORI) have been developed, with generous support from NEI, which evolved into a versatile tool for researchers and clinicians to investigate and monitor changes in retinal microstructures due to disease progression or response to treatment. CAORI has a very small clinical footprint and can be easily moved around the clinic, as illustrated in the Supplemental Figure S1.

Earlier versions took advantage of the PSI line scanning technology and were designed as adaptive optics line-scanning ophthalmoscopes (AO-LSO) [61,62]. Both the line illumination and the OCT A-line were scanned together with a single galvanometer, generating synchronized AO-LSO and AO-OCT images, the same as in MAORI. The simplified hardware requirements resulting from removing the resonant scanner and more in-depth engineering design enabled a more compact design of CAORI than for MAORI, as seen in the Supplemental Figure S1. Of course, these miniaturizations came at a cost: reduced flexibility in imaging configurations. The image size was set laterally by the length of the linear CCD. However, the image size of 3° × 5° was quite large compared to standard AO-SLO instruments and enabled the rapid scanning of large retinal areas, which is quite important in clinical applications.

CAORI has been designed to utilize a custom-made optical breadboard and several independent sub-assemblies that can be rapidly assembled, aligned, and tested separately and then placed on the breadboard. From a manufacturing perspective, it is much easier to put together individual sub-assemblies and then assemble them on the main structure. The sub-assemblies are Delay Line, Wavefront Sensing, Fiber Optic, Deformable Mirror,

Line-scan Camera, Fixation Target, Patient Interface, and Main Structure (primary optical bench). The CAORI electronics box included the power supply and drivers for the light sources (SLDs) and the DM.

CAORI had a frame rate of up to 40 frames/second with stable AO control. A set of 100 frames was acquired in 2.5 s, which was comfortable for the patient. The acquisition of multiple patches and strip scans took less than one minute. A total patient imaging session for one eye, including patient alignment and repeat scans, required approximately 5–10 min.

Dark-field imaging capabilities have been implemented into the second version of CAORI. As recognized at the time, dark-field imaging approaches in standard SLOs provide complementary imaging capabilities that are not available in confocal imaging. PSI developed a new method based on a line-scan time-domain integration (TDI) camera to implement split-detection simultaneously with bright-field confocal imaging. However, split-detection in a line-scanning configuration is only possible in one orientation, with the split direction perpendicular to the line. As mentioned before, directionality artifacts limit the ability to image blood vessels oriented along the line. Therefore, a third version of CAORI has been converted to flying-spot AO-SLO instead of AO-LSO, which brings all the advantages of simultaneous multi-offset imaging into the very compact form factor of CAORI. Additional improvements include independent AO-SLO and AO-OCT focal plane control, allowing the two modalities to image simultaneously at their maximum axial and lateral resolution in two different layers. The instrument uses off-the-shelf APDs from Thorlabs. An 8-channel electronics board was designed and fabricated by PSI to condition the detector signals to meet the digitizer input requirements. The 4-channel Matrox analog framegrabber card (that became obsolete) was replaced with a 16-channel digitizer (ATS9416, Alazar). The main design specifications of CAORI are currently as follows: standard image dimensions: 640 × 640 pixels; pupil diameter: 7 mm; estimated resolution: ~2.5 µm for a normal eye; nominal field size: 1 ÷ 5° (user selectable); SDOCT/WS beacon: 830 nm, >60 nm FWHM, and 750 µW at pupil; SLO: 750 nm SLD, 20 nm FWHM, and 400 µW at pupil; field of view: 20° × 30° (fixation); frame rates up to 40 Hz.

2.4. MAOSI

The general MAORI platform described above has been adapted into the multimodal adaptive optics small animal imager (MAOSI) suitable for small animal (mice and rats) high-resolution retinal imaging. The availability of rodent models of human eye disease has increased the demand for in vivo animal fundus imaging. Transgenic models have been developed for many forms of retinal degeneration including glaucoma, diabetic retinopathy, inherited vision disorders, and neurodegenerative diseases such as Alzheimer’s. The small eye of the mouse, with its relatively large pupil, provides among the largest NA of any mammal eye, several times that of the human eye. Thus, AO imaging has the potential to provide micron-level resolution, even enabling in vivo multi-photon biomicroscopy. Therefore, a multi-modal AO imaging platform dedicated to the living rodent eye is intended to fill an important laboratory research niche.

MAOSI is a flexible, high-resolution (<1 µm) benchtop system with confocal reflectance and fluorescence AO-SLO (r/fAO-SLO) channels with simultaneous AO-OCT for routine laboratory imaging applications in animal research. Novel platform design elements include independent software-controlled focal depths for rAO-SLO and fAO-SLO channels with near-diffraction limited axial resolution (<10 µm); a dual-axis galvanometer scan engine with software-adjustable scan patterns; a 6-degree-of-freedom goniostage for easy and precise animal positioning to select the desired image location on the retina; a low-flow anesthesia system with integrated digital micro vaporizer; and a dual motorized translation stage for adjusting the pupil size and the image magnification/focus. Simultaneous, co-scanning AO-OCT/SLO raster scans provide significant benefits for image guidance, cross-modal registration, and averaging for enhanced detection sensitivity and ease of use in anesthetized animals. The system is configurable either for multiple discrete laser and SLD sources, or for a single commercial supercontinuum source for maximum spectral flexibility. The ability to use low light levels, along with the use of contact lenses and animal body temperature maintenance at ~37 °C in a custom heated animal holder prevents damage to the retina and minimizes cornea and lens opacifications.

Initially developed with only the AO-SLO confocal channel, MAOSI has been upgraded recently with the fiber bundle that enables simultaneous confocal and multi-aperture offset imaging. Figure 6 shows a simplified block diagram of MAOSI, and Figure 7 illustrates a simplified optical configuration. A photograph of the complete MAOSI system on a mobile cart is shown in the Supplemental Figure S2 on the left, while the right-side image shows a rat in the animal holder with the nose cone attached to the isoflurane anesthesia machine.

Dual wavelength (blue 422 nm and 440 nm) simultaneous AO-SLO imaging has been implemented in MAOSI to evaluate the possibility of obtaining oxygen saturation (SpO) maps of the retinal vasculature with cellular-level resolution in vivo in the rodent eye, which is needed to study the early onset of diseases such as diabetes. Oxygenation assessment requires the measurement of differential absorptions of oxygenated and deoxygenated blood at a minimum of two different wavelengths [63]. 422 nm is close to an isobestic point while 440 nm exhibits a large differential absorption for oxygenated and deoxygenated blood. In addition to the existing OCT and red SLO imaging channels, two more detection paths have been implemented for 422 nm and 440 nm in a different version of MAOSI, replacing the fluorescence channel. Appropriate dichroic beamsplitters have been used to combine the illumination and separate the detection in these paths. Detection in each blue path is performed using a multichannel fiber bundle developed by PSI, as described above. The sum image S collects photons transmitted through the structure of interest (blood vessel) and back-reflected by the more reflecting layers, such as the RPE, underneath the retinal capillary beds. Therefore, S can be used to evaluate light absorption through blood relative to the surrounding tissue. Using the extinction coefficients for oxygenated and deoxygenated blood and the measured absorption at the two wavelengths, one can calculate SpO at the capillary level.

Multi-spectral imaging capabilities have also been explored recently and implemented in MAOSI for spectral discrimination of retinal tissue to identify Aβ deposits in the retina as a biomarker for AD. In yet another version of MAOSI, the illumination/detection modules were modified to include five wavelengths: 405 nm, 450 nm, 488 nm, 520 nm, and 640 nm. Nanosecond pulsed lasers (Thorlabs) were controlled with a custom trigger delay circuit that delayed the laser pulsing to generate a train of laser pulses of different colors, each individually detected by the digitizer. The software then separated them into five different images for each channel. The multi-spectral detection approach was applied in our system to multiple detection channels simultaneously, one confocal channel and four offset-aperture channels, and provided imaging in both reflectance (confocal channel) and transmission (offset aperture) configurations for the specific microstructures of interest. A spectral similarity score is calculated for each image pixel with respect to a reference spectrum and enables differentiation between regions with and without Aβ. The proof-of-principle has been demonstrated on model eyes and a limited set of transgenic mice (AD model) [64]. This optical imaging technique can identify specific chemical compounds in vivo in the retina with cellular-level resolution, non-invasively, and without the use of contrast agents.

To summarize, the main classes of instruments discussed here are listed in

Table 1. Highlights of instruments particularities.

Instrument	Capabilities	Human/Animal Use
MAORI	large field of view: 20° × 30° (fixation); retinal tracking and large area LSO capabilities for image stabilization and navigation; autofluorescence and light stimulus capabilities	human
CAORI—line scan	large area scan (3° × 5°); difficult and limited offset capability; large field of view: 20° × 30° (fixation)	human
CAORI—raster scan	easy multi-offset imaging; small area scan (1° to 5°); large field of view: 20° × 30° (fixation)	human
MAOSI	easy multi-offset imaging; adjustable pupil and image size with Badal; fluorescence and multispectral capabilities; adjustable goniostage for animal positioning	mice and rats

highlighting the particularities of each version.

3. Results

3.1. MAORI

The multimodal systems described here have been specifically designed for a clinical environment where space is at a premium. Our collaborators have tested all versions of MAORI and CAORI in clinical applications on a wide range of diseases, including glaucoma, DR, AMD, Stargardt disease, Retinopathy of Prematurity, and Cone/Rod Dystrophy. Representative results are highlighted below.

Recently, a non-rigid registration method for aligning a stack of confocal images has been presented [65]. The images used to demonstrate that method were obtained with the BCH MAORI. At the time, the imager had only the confocal AO-SLO channel. An example of the cone mosaic from a healthy volunteer is shown in Figure 8, with several magnified regions shown in Figure 9 for better visualization. The fovea is in the center (white *), and the image demonstrates that the cones are well resolved to within 0.3°–0.4° of the fovea.

An upgraded version of the BCH MAORI included two offsets simultaneously recorded with the confocal channel. Figure 10 shows an example of the confocal, SPL, SD, and STD images obtained from a healthy volunteer. The BCH MAORI was later upgraded again to include all four offsets and the confocal in a configuration that was included in all PSI retinal imaging platforms.

Figure 11 and Figure 12 show examples of the images obtained with the JDC version of MAORI, which illustrate the complementarity of the confocal and offset images.

Figure 13 shows an extremely detailed panorama of the vasculature and nerve fiber bundles obtained by our collaborators Joel Schuman and Gadi Wollstein at UPMC with an earlier MAORI platform [66]. The SLO confocal focus was pulled up into the retinal nerve fiber layer (RNFL).

3.2. CAORI

Examples of a cone photoreceptor image (AO-LSO) and of an AO-OCT image are shown in Figure 14 for a healthy control volunteer and in Figure 15 for a Cone/Rod Dystrophy (Fundus albipunctatus) patient. These images were obtained with the line-scanning version of CAORI.

Figure 16 shows examples of the split-detection images (A and D), bright-field confocal images (B and E), and STD image (C). The top line and the center images (A, B, and C) are at the same location, while the bottom line is at the edge of the optic nerve head (ONH).

Figure 17 shows an example of the four offset images (O1–4) and the confocal (C) image simultaneously acquired with the OCT B-scan from a healthy subject. Figure 18 shows all the images derived from these four offsets: SPL 1–4, S, STD, SD, P, and PG. Figure 19 and Figure 20 show the images from glaucoma patients obtained at NYU with the upgraded CAORI (with flying-spot SLO and fiber bundle multi-offset detection).

3.3. MAOSI

Figure 21 shows the simultaneously acquired AO-SLO reflectance (top of A, B, and C) and AO-OCT images (bottom of A, B, and C) focused at different depths axially, as indicated by the dashed line in the OCT images. Panels D and E in Figure 21 show the simultaneously acquired AO-SLO reflectance and fluorescence (in false color) images of activated green fluorescence protein (GFP)-expressing microglia. Figure 22 shows a collection of images with different contrast mechanisms, including confocal, SPL, STD, and fluorescence. Figure 23 shows simultaneously acquired confocal (top row) and SPL (bottom row) images, highlighting the complementarity of the two imaging modalities.

Figure 24 shows the images obtained in the visible range at 422 nm and 440 nm from rats in a study performed at JDC using the dual-wavelength imaging version of MAOSI. The top row shows S images at the two wavelengths and the resulting SpO map, while the bottom row shows the ability to zoom in and image the same retinal structures at different magnifications. The image on the right shows a larger area SpO map illustrating the blood transition from arteries to veins.

4. Discussion

The retinal images shown above illustrate the extraordinary power of AO to reveal the structural and functional characteristics of the living eye with cellular-level resolution. In diseased eyes, in particular, certain disturbances can be visualized, measured, and quantified for diagnostic purposes and for monitoring disease progression or the response to treatment. However, additional studies are needed for clinical validation, which is outside the scope of this paper.

Figure 1 shows the capillary maps (A and B) obtained from motion contrast without additional contrast agents. Panel A shows the fovea avascular zone, which has diagnostic value in many eye diseases, including DR and glaucoma [67,68], and its shape and size are generally assessed with OCT angiography (OCTA). Panel B shows a beautiful capillary arborization branching out of a larger vessel. One problem with OCTA is that it is a mostly binary, vessel/no vessel mapping procedure. Significant effort is underway to provide a quantitative assessment of the flow velocity with OCTA. Motion contrast is non-binary, and as seen in Figure 1, the shades of gray may be converted to velocity through a potential calibration method. Panels C and D show in great detail the vessel wall structure visualized as SPL images. Panel E shows a cone mosaic in a confocal image, while panel F shows the typical bubble-wrap appearance of the cone photoreceptors as obtained in our earlier implementation of the offset aperture AO-SLO.

Figure 8 shows the confocal images of the cone mosaic near the fovea, and Figure 9 shows the four magnified patches (from the yellow squares locations in Figure 8). The density of the cones varies across the eye. The largest density is needed at the center of the fovea for the best visual acuity. However, the eye geometry limits the resolution of the AO-SLO/OCT imaging system, and foveal cones are very difficult to resolve.

Figure 10 shows the four images obtained with different contrast mechanisms derived from the simultaneously acquired confocal and offset channels: confocal, SPL, SD, and STD. They illustrate the complementarity of these images. Nerve fiber bundles can be clearly seen in the SLO image but not in the others. The capillary map is nicely visible in the STD image, hard to see in SPL and SD, and completely hidden in the SLO. SPL and SD reveal details of the vessel walls that are not seen in the other images. Similarly, Figure 11 also illustrates this complementarity. Both figures were obtained from healthy volunteers. However, Figure 11 shows several microstructures, as highlighted by the circles in the SPL and PG images, which have no correspondence in the STD and, therefore, no flow associated with them. They could be harmless, or they may raise flags as potential locations for disease initiation, especially the ones near the vessel wall.

Figure 12 shows a couple of blurred blobs in the SPL images that are hard to interpret from those images alone. The focal plane is located in the upper retinal layers, and the capillaries are in sharp focus, as illustrated in the SPL and STD images. The simultaneously acquired OCT image shows the added benefit of the multimodal imaging approach [69]. A drusen-type disturbance at the RPE layer (indicated by the arrow) corresponds to a fainter blur in the SPL image. It is assumed that the other two blurred blobs in SPL are of a similar cause, which prompts to additional monitoring of an otherwise healthy retina.

Figure 13 shows a remarkably detailed panorama of the main retinal vasculature and of the spatial distribution of the nerve fiber bundles, as produced with an earlier version of the confocal MAORI at UPMC. The typical arcuate arrangement of the fiber bundles from around the fovea (darker spot on the center left) and collected into the optic disc (black spot on the right) can be seen nicely illustrated in Figure 13.

The first version of CAORI, based on line-scanning (AO-LSO), enabled the rapid acquisition of large-area scans. Figure 14 and Figure 15 show typical examples of the cone mosaic obtained with CAORI for a healthy eye and a Cone/Rod Dystrophy patient as imaged at BCH. In the normal control case, cone photoreceptors can be identified and counted very close to the fovea (to within 0.5°) in the AO-LSO image (right), and the OCT image (left) shows a nice, layered structure (smooth, continuous layers). However, the diseased eye in Figure 15 has cones only on the lower right side of the LSO mosaic, while the central part shows virtually no cones. The complementary OCT image (inset) shows a drusen-like deposit and layer distortion in the RPE complex corresponding to the white spot in the LSO image.

The second version of CAORI used a TDI camera for offset aperture imaging in addition to the confocal camera. The TDI camera enabled integration over a thick band shifted from the line illumination that was imaged with the confocal line camera. Two bands above and below the line illumination were projected side by side on the TDI camera and were recorded simultaneously, providing two offset aperture images and, therefore, the split-detector image. Figure 16 shows examples of the SPL and confocal AO-LSO images. Panel A shows the detailed structure of a blood vessel wall and the location of several capillaries in the SPL image, while panel B shows the nerve bundles in the confocal image. The STD image in panel C significantly increases the contrast, illustrating the capillaries not visible in the split-detection or the confocal image. As expected, however, vertical edges are not well resolved since the split direction is vertical, perpendicular to the horizontal line illumination. The bottom row of Figure 16 shows in the split-detection image (D) detailed structures at the edge of the ONH not visible in the confocal image (E). The confocal image shows nerve bundles converging into the ONH. The ONH is on the top-right side of these images.

A third version of CAORI has been converted to the flying-spot AO-SLO configuration, and the detection module consists of the fiber bundle simultaneously acquiring the confocal and the four offset channels, as in MAORI. Figure 17 shows an example of the acquired images (C and O 1–4), and Figure 18 shows the derived images (SPL 1–4, SD, PG, STD, and S). It should be noted here that SD, STD, S, and PG are all isotropic, direction-independent, as they were derived from all four offset images and are, therefore, directionally diverse. STD extraordinarily reveals the maps of capillaries with much better contrast than in any other imaging modality, in a mode similar to OCTA. STD can also be used to segment the flow location in large vessels and to quantify the inside diameter of the vessels. The complementarity of the derived images should also be noted in Figure 18. Some edges, like the vessel wall and some capillaries, are visible in some SPL images but not in others in the top row, while the bottom row shows all edges and capillaries in the isotropic images.

It is also important to note here that we can measure and quantify structures in these images. For example, one can measure in Figure 19 the inside (L) and the outside (D) diameter of a blood vessel [55]. The wall-to-lumen ratio((D-L)/2L) has diagnostic value [70,71,72], and with offset/split techniques, one can measure it in vivo with cellular-level resolution. We can see the detailed structure of the vessel wall, which is not possible with standard confocal imaging. The arrow in Figure 19 shows the local thickening of the vessel wall, which might indicate a potential future disruption. The circled area could be a cyst, a lipid deposit, or activated microglia approaching the injured location. Additional investigations are needed to confirm such a hypothesis, and multiple imaging sessions at the same location (not available in this case) would show if that round structure moves or changes in time. The right side of Figure 19 shows significant tortuosity in a glaucoma patient imaged with CAORI at NYU. Increased tortuosity of retinal vessels has been suggested as an indicator of arterial hypertension, retinopathy, cerebral vessel disease, stroke, and ischemic heart disease [73].

The images shown in Figure 20 are from a glaucoma subject imaged with CAORI at NYU. The patient has primary open-angle glaucoma of both eyes, mild stage, and ocular hypertension, bilateral. What is intriguing in this scan is the significant difference between the left and right sides of each image. The right side shows a very clear dotted pattern that resembles the cone mosaic (orange arrow) and no capillaries. The left side shows in the STD image a very distinct and sharply focused capillary vasculature that is usually located in the upper layers of the retina. The depth of focus in AO-SLO is generally very narrow (<50 µm); therefore, capillary layers and cones are not seen in the same image plane. One potential explanation is that the retina has a ridge laterally, as suggested in the sketch on the right side of Figure 20. This scenario can bring into focus different depths of the retina in different parts of the image. Additional investigations at the same location, such as an OCT volume scan (not available), are needed for clarification. Nevertheless, the encircled microstructures are also interesting, and potential correlation with the clinical diagnosis is of interest. The only available OCT image for this scan is shown at the top right of Figure 20, and its location is indicated by the dotted line in the PG image. The darker green arrow shows a large blood vessel in the RNFL running along a portion of the OCT scan.

Figure 21, Figure 22 and Figure 23 show a gallery of retinal images obtained with MAOSI at BCH. These images demonstrate the extended capabilities of MAOSI to acquire confocal, offset, and fluorescence AO-SLO images simultaneously with the AO-OCT images. The focal plane can be adjusted axially from the top of the retina down to the RPE layer, focusing on nerve fiber bundles in the RNLF, on capillary layers, or on the RPE layer, depending on the desired target. Examples of the activated GFP-expressing microglia are shown in panel E in Figure 21 and in panels H–J in Figure 22 as imaged with the fluorescence channel in MAOSI on transgenic mice. Microglia are the first and main form of active immune defense in the CNS. Physically surveying their environment regularly, if microglia detect any foreign material, damaged or apoptotic cells, neurofibrillary tangles, DNA fragments, or plaques, they will activate and phagocytose the material or cell. In this manner, they act as “housekeepers”, cleaning up cellular debris. Their activation results in the release of various pro-inflammatory mediators and increased proliferation and migration, severely affecting retinal neurons and causing increased apoptosis and subsequent nerve fiber layer thinning [74].

The confocal images (panels A and B in Figure 22 and the top row in Figure 23) show stunning details of the fiber bundles in the RNFL and of the vasculature, while SPL images (panels C–E in Figure 22 and the bottom row in Figure 23) enable the visualization of additional microstructures not visible in the confocal images. However, there seems to be a good correlation between the white spots seen in the confocal images in the top row of Figure 23 and the SPL images in the bottom row. Additional investigations are needed to identify those white spots properly.

Figure 24 shows the sum images at 422 nm and 440 nm in the top row. The resulting SpO map shows the transition from oxygenated to deoxygenated blood (red to blue) as the blood is collected from small capillaries into a larger vein. Similarly, the right-side image in Figure 24 shows a large area map of the vasculature where the right side is mostly red, indicating oxygenated blood in capillaries drawn from an artery, while the left side is mostly blue, indicating deoxygenated blood collected into a vein. As the blood transitions from an artery to a vein, the oxygen saturation decreases, and the SpO map shows a gradient from red to blue, as expected. At this point, these SpO maps are primarily qualitative. An additional calibration procedure is needed to quantify the oxygen saturation precisely.

It should be noted here that currently, there are no commercial retinal imaging platforms including both AO-SLO and AO-OCT channels as described here and offered by PSI. There are several research platforms developed by academic research groups, mostly focusing on either AO-SLO or AO-OCT. These systems are generally spread over large holographic tables, take up a lot of space, and cannot be moved to the clinic. They are hard to operate, and sometimes require multiple people and two computers to control the instrument. The instruments presented here have been designed for the clinical environment. They are mobile, on portable carts and height-adjustable tables, and very compact, as can be seen in the Supplemental Figure S1. The largest breadboard is 2′ × 2′ (64 cm × 64 cm). The patient interface and the controlling software have been designed for easy patient alignment, patient comfort, and easy operability. The instruments can be operated by clinical personnel without advanced education in science and technology. They were developed as research platforms and can be customized for specific applications; however, additional engineering is needed to make them truly clinical instruments at affordable cost for clinical practice.

Another barrier currently to the large-scale adoption of AO technology in clinics is their need for clinical diagnostics. The increasing body of literature highlighting the extraordinary results in revealing cellular-level details in healthy and diseased retinas (see the excellent review [75]) needs to be followed by large-scale clinical studies that will demonstrate the clinical utility of such technology. We expect that the main drive towards the clinics will not come from the extraordinary power to image in vivo the retinal structure at the cellular level, but from its ability to quantify the changes in this structure due to disease, and in particular, due to treatment. Having the ability to monitor changes at the micron-level will enable early interventions to provide a better health outcome and to help elucidate disease pathways at the early stages of the disease. At PSI, we will continue to develop these technologies and add new features for improved clinical performance to facilitate the transition of this amazing technology towards clinical practice.

5. Conclusions

In conclusion, the multimodal AO retinal imaging platforms with multiple implementations, as described here, provide advanced capabilities for researchers and clinicians to investigate and monitor changes in a wide range of retinal microstructures due to disease progression or response to treatment. MAORI/CAORI/MAOSI are ideally suited to fundamental research essential for the early detection, diagnosis, and quantification of retinal complications due to retinal or systemic diseases, and we expect that their continuous improvement will enable significant advances in vision research.

6. Patents

The patents that resulted from this work are listed under references [56,57,62].