Three-Dimension Resolution Enhanced Microscopy Based on Parallel Detection

Abstract

Pixel reassignment image scanning microscopy (PRISM) is a useful tool to improve the resolution of confocal laser scanning microscopy (CLSM) only equipped with a detector array. However, while it can improve the lateral resolution, it has little effect on the axial resolution. Here, new microscopy has been proposed which combines three-dimension fluorescence emission difference microscopy (3D FED) with PRISM to further improve three-dimension resolution. We call this method three-dimension pixel reassignment fluorescence emission difference microscopy (3D-PRFED). Detailed theoretical analysis and simulation are presented in this paper. Additionally, the performance of lateral and axial resolution improvement of this method has been demonstrated by imaging 100 nm fluorescent beads and nuclear pore complexes samples. Experiment results show that this method in our system can improve lateral resolution by a factor of 1.85 and axial resolution by a factor of 1.48 compared with CLSM.

1. Introduction

Over the past several decades, confocal laser scanning microscopy (CLSM) has become a basic tool for most fluorescence microscopy applications, such as observing morphologies and dynamics in living cells, which has contributed to its ability to produce high-contrast, optical sectioned images while providing enough versatility to satisfy plentiful samples and application demands [1]. It is significantly noted that the resolution of CLSM, including lateral resolution and axial resolution, can even be improved by a factor of $\sqrt{2}$ with infinitely small pinhole compared to wide-field fluorescence microscopy, which cannot be achieved in practice for the awful signal-to-noise (SNR) and the inexistence of infinitely small pinhole [2,3]. Actually, compared with wide-field fluorescence microscopy, CLSM can supply little lateral resolution improvement but superior sectioned imaging with a finite-size pinhole [4]. It is well known that the resolution of classical fluorescence microscopy is limited by the wavelength of light according to Abbe’s diffraction limit theory, restricting lateral resolution to ca. 200 nm and axial resolution to ca. 500 nm at a typical excitation and emission wavelength of around 500 nm [5]. To further dig out the potential of CLSM, for the past thirty years, lots of super-resolution microscopy methods based on CLSM have been proposed, such as stimulated emission depletion microscopy (STED) [6], fluorescence emission difference microscopy (FED) [7], and image scanning microscopy (ISM) [8].

Fluorescence emission difference microscopy (FED), breaking the resolution limit by nonlinearly decreasing the peripheral fluorescence signal of the focus through digital subtraction between positive confocal image and negative confocal image, is versatile to all kinds of fluorophores as same as CLSM [7]. Over the past several years, several three-dimension fluorescence emission difference microscopy (3D FED) methods have been proposed which utilize three-dimension negative focus modulated by spatial light modulator (SLM) to realize 3D resolution improvement [9,10]. There are numerous advantages using an SLM to generate a doughnut focus, such as flexibly modulating excitation light and applying adaptive optics and wavefront optimization to correct system aberrations [11,12]. Using SLM, a compact system can be designed to generate a three-dimension doughnut focus [10].

ISM, which has succeeded all advantages of CLSM, is a useful and much more approachable technique to achieve super-resolution in CLSM by replacing a single point detector with a detector array or fast wide-field CCD (charge-coupled device) detection [1,8,13]. This method can double the lateral resolution of classical CLSM, which is similar to structure illumination microscopy (SIM) in principle [8,14]. In 2013, Colin J.R. Sheppard further clarified the principle of ISM using a detector array by pixel reassignment and discuss much more details about the effect of detector array size on resolution and signal collection efficiency, which can be called pixel reassignment image scanning microscopy (PRISM) [13]. Recently, PRISM has been combined with many other methods to further improve the resolution, such as FED, which just improves lateral resolution for it only makes use of the transverse vortex modulated light [15].

In recent years, as research for activities of the subcellular structure continues, an increasing number of researches require insights into intrinsic three-dimension organization of subcellular structures, all-directional resolution of microscopy including lateral and axial resolution should be much more improved than ever before. To follow this trend, here, a novel three-dimension resolution-enhanced microscopy is proposed by merging the 3D FED technique and PRISM technique, named with three-dimension pixel reassignment fluorescent emission different microscopy (3D-PRFED), which can further improve both lateral and axial resolution. Here, we will supply theoretical analysis and experiments to demonstrate the feasibility of 3D-PRFED.

2. Methods

2.1. Theoretical Analysis

Here will show the theoretical analysis of 3D-PRFED. At first, CLSM with a single detector and pinhole can be theoretically described by [16]

$${\mathrm{I}}_{\mathrm{c}}=\mathrm{S}\otimes {(\mathrm{PSF}}_{\mathrm{ex}}\times \left({\mathrm{PSF}}_{\mathrm{em}}\otimes \mathrm{Pinhole}\right))$$

(1)

where I_c is the intensity distribution of the confocal image, S is the intensity distribution of a sample, PSF_ex is the point spread function of excitation light at the sample plane, PSF_em is the point spread function of emission light for the system and Pinhole is the pinhole function, $\otimes$ is the convolution operator. Because of the existence of a pinhole, CLSM can achieve much more excellent lateral and axial resolution performance than conventional wide-field microscopy. However, with the pinhole size decreasing, a more effective signal would be blocked by the pinhole, as in Figure 1A.

PRISM can further improve the resolution of CLSM without reducing signal, which can be achieved by replacing a single pinhole with a detector array. The imaging process for different detectors in the array can be mathematically described by [8,17,18]

$${\mathrm{I}}_{\mathrm{n}}(\mathrm{x},\mathrm{y},\mathrm{z})=\mathrm{S}\otimes {(\mathrm{PSF}}_{\mathrm{ex}}(\mathrm{x},\mathrm{y},\mathrm{z})\times \left({\mathrm{PSF}}_{\mathrm{em}}(\mathrm{x},\mathrm{y},\mathrm{z})\otimes {\mathrm{D}(\mathrm{x}+\mathrm{d}}_{\mathrm{n}}\right))$$

(2)

$${\mathrm{I}}_{\mathrm{PRISM}}={{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{I}}_{\mathrm{n}}(\mathrm{r}-\raisebox{1ex}{${\mathrm{d}}_{\mathrm{n}}$}\!\left/ \!\raisebox{-1ex}{$2$}\right.,\mathsf{\phi })$$

(3)

where I_n(x,y,z) is the volume image recorded at the nth pinhole, S is the sample distribution function, PSF_ex(x,y,z) is the volume point spread function of excitation light, PSF_em(x,y,z) is the volume point spread function of emission light, D(x + d_n) is the pinhole function of the nth pinhole in the array, d_n is the relative distance between the nth detector and the center of the array, I_PRISM is the reconstruction image based on PRISM algorithm, and I_n(r−d_n/2,φ) is the image shifted by a distance d_n/2. According to Equations (1)–(3) and without considering Stokes shift, the equivalent PSF (point spread function) analysis of confocal and PRISM was shown in Figure 1. The detector array used for simulation was depicted in Figure 1D. Figure 1A shows us the relative intensity and resolution change for different pinhole sizes, which indicates the smaller the pinhole size is, the worse the signal-to-noise ratio will be gotten. Figure 1B demonstrates the image translation for different detectors at the detector array because of the existence of pinhole offset. By measuring the full width at half maximum (FWHM) of the normalization profile crossing the PSF Gaussian center of CLSM and PRISM, the resolution of PRISM is 1.25 times enhanced than conventional CLSM with one AU (Airy Unit) pinhole size, as in Figure 1C.

To further enhance three-dimension resolution of PRISM, here three-dimension fluorescence emission difference microscopy is introduced. Then a novel method called three-dimension pixel reassignment fluorescence emission difference microscopy (3D-PRFED) is proposed. Referred to 3D FED algorithm [9,10,15], 3D-PRFED can be described by

$${\mathrm{I}}_{3\mathrm{D}-\mathrm{PRFED}}{=\mathrm{I}}_{\mathrm{PRISM}}{-\mathsf{\gamma }\mathrm{I}}_{3\mathrm{D}-\mathrm{negative}}$$

(4)

$${\mathrm{I}}_{3\mathrm{D}-\mathrm{negative}}={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}\mathrm{S}\otimes {\left(\right(\mathrm{PSF}}_{\mathrm{vortex}}{+\mathrm{PSF}}_{\mathrm{hollow}})\times \left({\mathrm{PSF}}_{\mathrm{em}}\otimes {\mathrm{D}(\mathrm{x}+\mathrm{d}}_{\mathrm{n}}\right)))$$

(5)

where I_3D-PRFED is the image of 3D-PRFED, I_PRISM is the volume image of PRISM, I_3D-negative is the volume image modulated by 3D negative PSF which is incoherently combined by lateral negative PSF and axial negative PSF, as in Figure 2. γ is the subtraction factor of two images, which would affect final resolution and effective information retention, S is the sample distribution function, PSF_vortex is the lateral negative PSF of excitation light modulated by vortex phase mask, PSF_hollow is the axial negative PSF of excitation light modulated by hollow phase mask, and PSF_em is the PSF of emission light. D(x + d_n) is the same definition as equation 2. Normally this value would be set as 0.7, which is an empirical value and can be set based on the SNR of the result images to keep a balance between the best resolution and the minimum ghost images [7,9,10,19].

Vortex phase mask can be used in generating lateral negative PSF which can enhance the transverse resolution but not improve axial resolution. Meanwhile, 0–π phase mask can generate axial negative PSF which can enhance the axial resolution but not improve transverse resolution. These two phase retardation functions can be described by [9,10,19,20]

$$\Delta {\theta }_{xy}\left(r,\phi \right)=\phi , \phi \in \left[0,2\pi \right]$$

(6)

$$\Delta {\theta }_z\left(r,\phi \right)=\left\{\begin{array}{c}\pi , r\le r_{max}/\sqrt{2} \\ 0, r>r_{max}/\sqrt{2} \end{array}\right.$$

(7)

where $\Delta {\theta }_{xy}$ is the vortex 0–2π mask and $\Delta {\theta }_z$ is the 0–π mask, and r is the radius of the mask. According to Equations (6) and (7), vortex PSF and hollow PSF can be generated by phase modulation based on the vector diffraction model, as shown in Figure 2.

3D negative PSF as in Figure 2E can be incoherently combined by a lateral negative focus as in Figure 2B and an axial negative focus as in Figure 2D, which can be used to enhance three-dimension resolution in 3D-PRFED. For a single excitation light beam, the p-polarization component of the excitation light can be modulated by 0–2π vortex phase mask as in Figure 2A to generate a lateral negative PSF as in Figure 2B and the s-polarization component of excitation light can be modulated by 0–π hollow phase mask as in Figure 2C to generated an axial negative PSF as in Figure 2D. For the sake of making sure that these two components can be incoherently combined at the focus plane, the required light path difference should be introduced into these two components to destroy their coherence.

According to Debye integral [9,21,22,23] and Equations (1)–(4), the simulation results in Figure 3 show us the PSF comparison between CLSM, PRISM, and 3D-PRFED. By measuring the lateral and axial full width half maximum (FWHM) of the equivalent PSF profile along the x axis and the z axis, as shown in Figure 3D,E, the transverse resolution in 3D-PRFED is 1.89 times better than CLSM and 1.5 times better than PRISM. Meanwhile, the axial resolution in 3D-PRFED is 1.68 times better than CLSM and 1.58 times better than PRISM, which indicates that 3D-PRFED can well improve transverse and axial resolution compared with CLSM and PRISM.

2.2. Experiment Setup

The 3D-PRFED system set-up is depicted in Figure 4. Excitation is provided by a 640 nm laser (PDL828 Sepia II, PicoQuant, Berlin, Germany) with a pulse frequency of 80 MHz. For the sake of making sure that vortex modulated light can incoherently combine with hollow modulated light, a light path delay (~100 ps) was introduced between the p-polarization component and s-polarization component of excitation light at first.

The excitation beam then passed a half wave plate before projecting into the SLM (X13139 LCOS-SLM, Hamamatsu Photonics K.K., Hamamatsu, Japan), which could adjust the angle of the polarization direction of excitation light and made sure that the direction of p-polarization component was parallel to the module direction of SLM. The p-polarization component of excitation light was modulated by the R part of SLM. A double pass through a quarter wave plate (QWP) rotated two polarization states by 90 degrees, which changed the original s-polarization into p-polarization that could be modulated by the L part of SLM, while the original p-polarization became s-polarization and would not be affected by the pattern in the L part. The lens L1 and mirror M4 and lens L2 and lens L3 were made up as 4f lens groups, which ensured that the R part and L part of SLM were conjugated with the x galvanometer. After the TriScanner, the excitation beam would continue to pass through a scan lens (f = 50 mm) and a tube lens (f = 200 mm) which were also constituted as a 4f lens group to ensure the x galvanometer was conjugated with the entrance pupil of OL. The conjugation relation can make sure that SLM is operating at the Fourier plane of OL. Finally, a 100x objective lens (OL, Apo TIRF, 100×/1.49 NA, Nikon, Tokyo, Japan) was used to focus the excitation light on the sample plane.

The TriScanner, used for deflecting excitation light to scan the sample and de-scanning the fluorescence signal from the sample into the fixed detector array, consists of three galvanometers, two deflecting mirrors (X and Y1) and one correcting mirror (Y2) as shown in Figure 4 inset (b). It should be noted that the mirror X close to the scan lens is conjugated with the entrance pupil of OL, which makes sure the SLM can be conjugated with the entrance pupil of OL. The correct mirror Y2 should make sure the defecting light from Y1 mirror is projected into the conjugate point of X mirror, which means that theoretically the correct mirror should be rotated double angle than Y1 deflection mirror.

The excited fluorescence (~670 nm) was collected by OL, relayed by TL and SL, de-scanned by the TriScanner, separated with the illumination light by a dichroic mirror (DM, ZT405/488/561/647rpc, Chroma, Bellows Falls, VT, USA), and then passed through a fluorescence filter (FF660-Em02-B, Semrock, Rochester, NY, USA) and focused by a lens (L4, f = 400 mm) onto a fiber bundle consisting of 19 fibers (125 μm cladding diameter and 105 μm core diameter) that transfer the fluorescence signal to 19 avalanche photodiodes (APD, SPCM-AQRH-16-FC, Excelitas Technologies, Waltham, MA, USA) respectively. The fiber bundle and 19 APDs are made up of our detector array.

The whole system was controlled by a DAQ box (NI Card 6366, National Instruments, Austin, TX, USA) connected with a PC and operated by a LabView program.

It is noted that the center of the detector array should be aligned with Gaussian intensity center of the fluorescent signal, which means the energy detected by the 1st detector should be the highest, and the energy detected by the 2nd to 7th detectors should be medium, and the energy detected by the 8th to 19th detectors should be the lowest. In order to avoid the stray light from environment and the zeroth order excitation light into the system, an orthometric grating would be applied in the unmodulated and modulated place of the pattern, as shown in the inset (b) in Figure 4.

3. Experimental Results

It is known from the principle of 3D-PRFED that the detectors array should be well collimated with the Gaussian center of emission light focus and the 3D negative focus should also be close to the distribution of the theoretical model which will affect the ultimate resolution of 3D-PRFED. Here, 200 nm nanoparticles (ATTO 647N) were imaged to calibrate the system for its’ preferable quantum efficiency and appropriate size. In order to guarantee the detector array is aligned with the Gaussian center of emission light focus in the imaging plane, the energy distribution from the center to outer detectors should be diminishing which means that the image intensity captured by the center detector is the highest and the image intensity captured by the second ring detectors is medium and the image intensity captured by the peripheral detectors is the lowest, as shown in Figure 5A. Figure 5B shows the modulation pattern for solid focus, in which the gratings inside and outside the modulation places are different because the different grating patterns can avoid the stray light from environment and the zeroth order unmodulated light into the system. By summing up the photons of each image, the energy distribution for each detector was shown in Figure 5C. By means of Zernike correction [10,11,12], the Gaussian energy distribution for the detector array can be promised. By adding vortex pattern and hollow pattern into the solid focus pattern as Figure 5D, the negative confocal image scanned by 3D negative focus can be easily generated, as in Figure 5E. It is needed to illustrate that the solid focus pattern and 3D negative focus pattern are using the same Zernike correction because they are the same excitation light at the common optical path. Figure 5F,G shows the preferable 3D negative focus after Zernike correction while Figure 5H,I are the images before correction.

100 nm nanoparticles (Abberior nanoparticles 4C flour. 100 nm) and nuclear pore complexes (Abberior Cells 3C NPC-START RED) were imaged to further test 3D-PRFED performance. Figure 6 shows the nanoparticles imaging results which were performed with 6μs acquisition time per scan position and transverse scan step size of 30 nm and axial scan step size of 50 nm. The ultimate images were up to 5 nm pixel size through an interpolation algorithm. Therefore, before processing these images, we would get 19 group images and each group with 50-layer images. The confocal volume image resulted from 19 group images summing up without translation processing. Because the magnification of our system is 800 times, the equivalent pinhole size of the confocal volume image is 1.4 AU. The PRISM volume images were performed with pixel reassignment algorithm and the equivalent pinhole size is 0.25 AU. The 3D-PRFED volume image was performed with the subtraction coefficient of 0.7 and the negative value was set to zero. As shown in Figure 6, 3D-PRFED can obviously reduce the noise of CLSM and PRISM and improve transverse and axial resolution. The transverse and axial resolution of 3D-PRFED can be up to 116 nm and 406 nm. In order to reduce measurement error, 18 nanoparticles were selected to measure their transverse profile data and axial resolution. The results indicate that the mean transverse resolution of CLSM, PRISM, and 3D-PRFED are 264 nm, 191 nm, and 143 nm and meanwhile the mean axial resolution of CLSM, PRISM, and 3D-PRFED are 769 nm, 645 nm, and 518 nm respectively. The nanoparticles imaging experiment indicates that the transverse resolution of 3D-PRFED can be improved 1.85 times better than CLSM and 1.33 times better than PRISM, and the axial resolution of 3D-PRFED can be improved 1.48 times better than CLSM and 1.24 times better than PRISM.

In order to test the performance of 3D-PRFED applied in biological imaging, we imaged a slice of commercial nuclear pore complexes (NPCs) samples (Abberior Cells 3C NPC-START RED), as shown in Figure 7. The primary function of NPCs is as the key regulator of molecular traffic between the cytoplasm and the nucleus, and meanwhile this structure participates in a considerably broader range of cellular activities on both sides of the nuclear envelope. NPCs, among the largest proteinaceous assemblies in the cell, are made up of multiple copies of ~30 different proteins called nucleoporins (Nups). The NPC in vertebrate consists of a ~125 nm diameter core structure, which includes eight spokes in a radially symmetrical arrangement [24].

In the axial direction, 47 layers of NPC with 50 nm separation between consecutive layers have been acquired. Figure 7A–C are the results of CLSM, PRISM, and 3D-PRFED at 350 nm, which suggest that 3D-PRFED for the NPC imaging can obviously improve the resolution comparing with CLSM and PRISM. The normalized line profile in Figure 7G from Figure 7D–F, which are the local images come from the dash boxes of Figure 7A–C, suggest that two NPC structure can be obviously observed in 3D-PRFED but not in CLSM and PRISM. Figure 7H–J represent the cross-section view extracted along the white dashed line depicted in Figure 6A–C, which shows that the axial resolution can also be obviously improved in 3D-PRFED comparing with two other ways. As shown in Figure 7K which is the normalized line profile along the green line in Figure 7H–J, two NPC structures in different layers can be distinctly observed in 3D-PRFED but not in CLSM and PRISM. The NPC imaging experiments show that the excellent performance of 3D-PRFED can be well applied to biological imaging and other research projects.

4. Discussion

The 3D-PRFED method has proven excellent to improve the transverse and axial resolution and reduce the background noise compared with conventional CLSM, which is an optional means to observe multiple kinds of subcellular organelles and other fine structures. Based on SLM, some advantages of 3D-PRISM should be mentioned. At first, 3D-PRFED succeeds all benefits of CLSM such as flexibility with almost all fluorescent dyes. Secondly, different from the traditional 3D vortex focus generation method based on phase plate [20], through Zernike correction, 3D-PRFED based on SLM can correct the aberration of the system and greatly optimize the excitation PSF, which is beneficial to make sure the quality of 3D negative focus. Additionally, the detector array should be well collimated with the Gaussian center of emission light focus which is related to the excitation light focus. Utilizing the modulation advantage of SLM, the alignment problem would be easier to solve.

However, it is noted that the 3D negative focus should be close to the theoretical model because the quality of the 3D negative focus would greatly affect the ultimate resolution and the shape of the structure presented on the image, which is limited by the theory of 3D-PRFED. Moreover, it should be pointed out that this method would cost double time than CLSM because two different images should be gathered, positive confocal images and negative confocal images. Additionally, this method is not suitable for living cell researches because the movement of cells would change the place of second images scanned by a 3D negative focus and increase the possibility of a ghost image.

In the future, besides CLSM, other imaging technologies, e.g., two-photons microscopy [25], can be also upgraded with 3D-PRFED to further enhance their resolution for the use of SLM.

5. Conclusions

In conclusion, compared with CLSM, 3D-PRFED can greatly enhance the transverse and axial resolution and reduce the background noise. The nanoparticle imaging experiment results suggest that the transverse and axial resolution in 3D-PRFED would be improved by a factor of 1.85 and 1.48 compared with CLSM, which would provide a great improvement to the biological research based on CLSM. Simultaneously, the NPC volume imaging experiments show that 3D-PRFED can also exhibit excellent capability in biological research. We can envision that this method could be employed to a wide field of applications including biotechnology, material research, among others.