XL-SIM: Extending Superresolution into Deeper Layers

Abstract

Of all 3D-super resolution techniques, structured illumination microscopy (SIM) provides the best compromise with respect to resolution, signal-to-noise ratio (S/N), speed and cell viability. Its ability to achieve double resolution in all three dimensions enables resolving 3D-volumes almost 10× smaller than with a normal light microscope. Its major drawback is noise contained in the out-of-focus-signal, which—unlike the out-of-focus signal itself—cannot be removed mathematically. The resulting “noise-pollution” grows bigger the more light is removed, thus rendering thicker biological samples unsuitable for SIM. By using a slit confocal pattern, we employ optical means to suppress out-of-focus light before its noise can spoil SIM mathematics. This not only increases tissue penetration considerably, but also provides a better S/N performance and an improved confocality. The SIM pattern we employ is no line grid, but a two-dimensional hexagonal structure, which makes pattern rotation between image acquisitions obsolete and thus simplifies image acquisition and yields more robust fit parameters for SIM.

1. Introduction

In widefield epifluorescence microscopy, image contrast and resolution is impaired by fluorescence emission originating from layers above and below the focus plane [1] (out-of-focus). The gold-standard for curing this problem is the confocal microscope, which employs optical means i.e., a pinhole for rejecting out-of-focus light. A totally different cure for the same problem is provided by structured illumination microscopy (SIM). Here, a structured illumination pattern is used for excitation and the emission is detected without a pinhole. By recording several raw images at different pattern positions on the sample, the out-of-focus emission can be removed computationally from the recorded raw images [2,3]. The phase shifted SIM raw images are called phase images in the following.

While the resolution of an optical microscope depends on the minimum distance of two distinguishable radiating points, confocality describes its sectioning power, i.e., its ability to suppress out-of-focus information [4,5]. With respect to confocality, SIM outperforms every confocal microscope [6]. On top of that, it has the potential to increase the resolution beyond the Rayleigh limit [7,8,9,10]. While this resolution enhancement is not as great as with other super resolution techniques like e.g., PALM [11], STORM [12] or STED [13], SIM provides several distinct advantages:

• It requires orders of magnitude less laser power than other super resolution approaches, hence it is much more gentle to live cells [14].
• It is compatible with all available fluorescent dyes [15].
• It is much faster [14,16].

Thus, SIM appears to provide the most viable path to (3D)-super resolution when resolution gain is weighed against speed, signal-to-noise ratio (S/N) and the capability to examine live cells in two [17] and in three dimensions [15]. However, the S/N of SIM is compromised by the detected out-of-focus signal [18], an effect which increases with sample thickness and with increased extent of staining. This has limited the usage of 3D-SIM to samples no thicker then a few micrometers. Here, we present a novel concept which combines SIM with line-confocal detection (L-SIM), thus optically reducing the amount of out-of-focus information before mathematics remove the rest. This way we allow SIM to go significantly deeper without compromising the signal-to-noise ratio.

Commercial SIM systems [19] and most other SIM approaches [9,10,20,21,22] employ line grid patterns. For super resolution, these patterns need to be rotated to at least three different orientations in order to obtain an increased resolution in all directions of the $(x,y)$ image plane. In order to extend super resolution SIM to three dimensions, sets of phase images at different grid positions and orientations have to be recorded for several z-positions. This way, a resolution improvement by a factor of two can also be achieved in a z-direction [21].

Given that the precision of all three parameters (grid position, grid orientation and z-position) critically enters the most crucial steps in the evaluation of SIM images/3D-SIM volumes, namely, determination of the frequency vectors of the pattern changes calculation of the images. This needs to be performed for every SIM image processing, and any error here will lead to artifacts as well as to a reduction of resolution in the final processed image.

In the following, we shall present a combination of two-dimensional hexagonal patterns for SIM (hexagonal SIM = X-SIM) [23] with line-confocal detection (line-confocal SIM = L-SIM), called hexagonal line-confocal SIM (XL-SIM) as a powerful super resolution tool for live cell imaging. While two-dimensional SIM patterns have been described before [24,25], the novelty of our approach lies in the fact that we need to shift the grid pattern in one dimension only between phase image acquisitions [23]. The other novelty lies in the confocal spatial filtering, which improves confocality and—by discarding out-of-focus emission in the detected phase images—makes SIM more applicable to thicker samples.

2. Results

In order to achieve a line-confocal excitation, we scan a laser-line over the SIM-pattern substrate and image the latter onto the sample. The laser line is generated by a Powell lens (for details, see methods Section 4.1 and the drawing in Figure 1). The result is a two-dimensionally modulated intensity profile, which—for a given region of the sample—lights up and subsequently fades again as the laser-line sweeps over the corresponding area of the pattern mask. An sCMOS camera chip with a rolling shutter is used for detection, and the movement of the structured excitation line is synchronized with the movement of the rolling shutter active pixel area. Thus, unlike in previously described combinations of SIM with line-confocal microscopy [22], phase images are obtained in a single sweep, which, in the experiments shown, takes no more than 10 $\mathrm{ms}$ for a field of $2048\times 1024$ pixels.

2.1. Theory

To generate an intensity profile with hexagonal modulation, a binary phase pattern with hexagonal symmetry and a phase step of $\lambda /2$ is placed into the intermediate image-plane of the microscope’s excitation beam path. Here, it was designed to produce hexagonal diffraction orders for the excitation wavelength $\lambda =488$ nm, with zero order contain about 70–80% of the power of the first diffraction orders, as described in [21] (for details, see Supplementary Note 5 and Supplementary Figure S5). Given that the pattern contrast is caused by interference, the laser line focus must encompass a minimum width. A narrower focused line would reduce interference in the direction orthogonal to the line and would thus reduce pattern contrast in this direction.

A similar effect is illustrated in Figure 2a, where the hexagonal diffraction orders are broadened in one direction by the line shaped excitation. Reducing the line width relative to the pattern period, would increase confocality, but at the expense of modulation depth of the hexagonal pattern frequencies of the integrated excitation intensity ${\tilde{I}}^{\mathrm{int}}(k_x,k_y)$, shown on the right in Figure 2a. Reducing the pattern frequency to overcome this would—in turn—increase modulation depth again, but at the expense of a reduced resolution. This means that there is a trade-off between pattern frequency (resolution) and excitation line width of the laser focus (confocality). In the present experiments, our main emphasis was on resolution, hence the pattern base frequency was chosen slightly below the microscope objective’s cut-off frequency and the full-width-at-half-maximum of the excitation line focus width was approximately 3 pattern periods, as described in Supplementary Note 2 and Supplementary Figure S4. The sectioning ability of a microscope, also called confocality, is defined as the axial response to a thin fluorescent layer to defocussing action [6]. Several methods to quantify confocality have been introduced, e.g., in [4,5], they all encompass measuring z-stacks of a thin fluorescent sheet. In SIM microscopy, the axial response depends on the pattern contrast as a function of defocus. In cases where the whole area of interest is illuminated simultaneously—as in conventional (3D)SIM—several planes of excitation contrast exist due to the Talbot-effect, they are called Talbot planes. The axial extent of the excitation contrast (or equivalently the number of Talbot planes) is limited by the spatial coherence of the excitation laser light [21]. In contrast to conventional line grid SIM, the location of local intensity maxima in all Talbot planes corresponds to their location in the main focus plane with hexagonal patterns.

With line-confocal excitation, this is different as illustrated in Figure 2b: there is a symmetry break in the hexagonal pattern intensity, which causes the local maxima of the Talbot planes to shift towards the center of the excitation line. Thus, moving the line across the pattern substrate leads to moving local intensity maxima in the Talbot planes, but not in the focus plane. This is illustrated in Supplementary Video 1 and further explained in Supplementary Figure S2. As a consequence, the integrated excitation pattern contrast is reduced in the Talbot planes above and below the focus plane, and this, in turn, leads to improved confocality and axial z-resolution of XL-SIM. This is shown in Figure 2c, which depicts mathematical simulations of the integrated excitation contrast as a function of defocus for conventional SIM and XL-SIM illumination at $\lambda =488$ nm. The objective entered into the simulation was a $63\times$ NA 1.4 oil immersion. A plot of the simulated axial response (confocality) for XL-SIM and SIM is shown in Figure 2d.

The height of the rolling shutter window determines line-confocality. It can be chosen narrower than the excitation line; in the simulation, it was assumed to be two hexagon periods, which corresponds to approximately 10 camera pixels or an exposure time of 100 $\mathsf{\mu }\mathrm{s}$ in the fast mode of the camera. The reduced excitation contrast in the Talbot layers leads to a reduced axial response in these layers and thus better confocality when using XL-SIM.

Hexagonal SIM

In order to separate all spatial frequency bands contained in the detected signal, several phase images have to be recorded and the pattern has to be phase shifted between phase images. As described in [23], it is sufficient to shift the hexagonal pattern along one spatial direction, provided one chooses the right angle between the pattern symmetry axes and the shift direction so as to allow separation of all frequency bands contained in the detected phase images with a unitary matrix. The angle depends on the number of frequency orders contained in the excitation intensity spectrum.

The period of the hexagonal pattern is chosen such that only the seven first diffraction orders of the hexagonal pattern can pass the microscope objective pupil. As illustrated in Figure 2a and Supplementary Figure S3, these seven first diffraction orders will interfere to totally 19 frequency orders in the excitation intensity spectrum, oriented in six different directions. Further details about the hexagonal frequencies and the shift direction can be found in Supplementary Note 1.

Assuming linear fluorescence excitation, the m-th recorded XL-SIM phase image can thus be written as follows:

$$\begin{array}{cc}\hfill I_m(x,y)& =I^{\left(0\right)}(x,y)+\sum _{n=1}^9{I}^{\left(n\right)}(x,y)cos({\overrightarrow{k}}^{\left(n\right)}\overrightarrow{x}-{\varphi }_{n0}-{\alpha }_m^{\left(n\right)})\hfill \\ \hfill & =I^{\left(0\right)}(x,y)+\frac{1}{2}\sum _{\begin{array}{c}n=-9\\ n\ne 0\end{array}}^9{I}^{\left(n\right)}(x,y)e^{i{\overrightarrow{k}}^{\left(n\right)}\overrightarrow{x}}{e}^{-i{\varphi }_{n0}}{e}^{-i{\alpha }_m^{\left(n\right)}}.\hfill \end{array}$$

(1)

$I^{\left(n\right)}(x,y)$ denotes the modulation of the n-th frequency order ${\overrightarrow{k}}^{\left(n\right)}$ on a pixel $(x,y)$, and $I^{\left(0\right)}$ is the unmodulated background signal. The relative phase shift (i.e., the phase shift in the m-th recorded phase image relative to the first one) is given by ${\alpha }_m^{\left(n\right)}$ and ${\varphi }_{n0}$ is the absolute phase of the n-th frequency in the first recorded image, which only depends on the origin of the chosen coordinate system. The Fourier transform of Equation (1) is consequently given by the superposition of 19 so called frequency bands ${\tilde{I}}^{\left(n\right)}(k_x,k_y)$, which correspond to 19 unknowns. Thus, a minimum of 19 phase images have to be recorded for solving all frequency bands contained in the phase images ${\tilde{I}}_m(k_x,k_y)$.

In the above scheme, the rolling shutter width not only determines the camera exposure time, but also the degree of slit-confocality for a given phase image. Thus choosing a narrow rolling shutter width to obtain better confocality—to account for increased sample thickness, for example—also reduces the time the sample is exposed to excitation light in a given phase-image scan. To increase signal strength, again in case of weak fluorescence signals, one can resort to a slower scan-mode of the camera or increase the number of recorded phase images beyond the minimum number. The minimum acquisition time of the PCO Edge camera (pco.edge 4.2) (Kelheim, Germany) for one XL-SIM phase image amounts to 10 $\mathrm{ms}$, for a full frame image ($2000\times 1000$) pixels. If fewer camera lines are read out, the total acquisition time is reduced accordingly. Thus, a field of $2000\times 70$ pixels corresponding to an area of 1000 $\mathsf{\mu }$m² can be imaged within 30 ms. With an image stack consisting of 100 layers, spaced 100 nm apart, one can record a 10,000 $\mathsf{\mu }$m³ XL-SIM image volume in 2 s.

By mounting the hexagonal phase substrate on a rotatable disc, it is possible to adjust the pattern-orientation relative to the shift-direction. Unless the disk-rotation is used to insert alternative patterns into the beam path, the correct pattern orientation does not need to be changed anymore. The pattern frequency vectors can be calibrated e.g., using a thin layer of fluorescent dye, as also used in [4]. Calibration is performed by pairwise comparison of the phase shifts of the hexagonal frequency orders in the phase images.

In most SIM approaches and commercial solutions, line grid patterns are used for SIM, which must be rotated to at least three orientations and a minimum number of at least five phase images have to be recorded for every orientation [21]. After solving the system of linear equations for these unknown frequency bands, the bands have to be shifted to their original positions in Fourier space. These shifts require quite precise knowledge of the pattern frequency vectors. When the frequency bands are at their original positions, their values ${\tilde{I}}^{\left(n\right)}(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n\right)})$ can be compared (respecting the influence of the microscope objective emission OTF) with the 0th frequency band ${\tilde{I}}^{\left(0\right)}\left(\overrightarrow{k}\right)$ in order to determine a complex factor (which is called $c_n{e}^{i{\varphi }_{n0}}$ in [21]) for each frequency band. These factors reflect the contrast of the n-th pattern frequency in the sample ($c_n$) as well as the absolute position (phase) of the n-th pattern frequency in the first recorded phase image (${\varphi }_{n0}$). They are essential for the correct superposition of the frequency bands.

As a consequence, errors in the frequency vectors will lead to errors in the determined phase factors and to errors in the superposition itself because overlapped frequency values of the object don’t match. Errors of the frequency vector will lead to inhomogeneities across the SIM image. Errors of the phase factors (especially errors of ${\varphi }_{n0}$) will lead to a loss of information and will deteriorate the resolution that can be reached in the final processed SIM image/volume (see Supplementary Note 3 and Supplementary Figure S7).

Because the orientation of the pattern changes in every measurement, the frequency vectors have to be determined in every measurement directly from the measured data [21].

In contrast to the conventional SIM solutions with line grids, the frequency vectors can be kept invariant when using a hexagonal pattern for SIM. Thus, the determination of the pattern frequency vector becomes a calibration step. The only parameter that has to be fitted in the SIM processing is the complex factor ($c_n{e}^{i{\varphi }_{n0}}$) for each frequency band. Furthermore, only one z-stack of phase images has to be recorded instead of three when using line-grid SIM patterns. This avoids any possible errors in the z-positions of recorded layers, which can occur for thick samples/large z-stacks.

A consequence of an interferometric pattern contrast generation is its dependence on the polarization of the interfering waves [26]. Using a line grid pattern and linearly polarized light, the polarization can be adapted to the different pattern orientations. Using hexagonal patterns with linearly polarized light leads to different levels of contrasts for the different pattern intensity frequencies. This can be accounted for in the SIM processing [27], but the frequency bands exhibiting less contrasts will contribute more noise to the final processed image, especially in the higher order frequency bands which are essential for the resolution improvement.

However, this S/N degrading effect is more than compensated for by the following difference between 1D and 2D patterns: when line grids are used for SIM, only phase images for one pattern orientation are used for the calculation of frequency bands, which correspond to this pattern orientation. For example, if a total of $3\times 5=15$ phase images is recorded, there are five phase images recorded for each of the three pattern orientations and only five (of 15) images are used to calculate any non-zero frequency band. In contrast to the 0th frequency band, which is obtained three times, all other frequency bands are calculated by image data that corresponds to $1/3$ of the total exposure time. Hexagonal SIM allows for using all phase images for the calculation of all frequency bands and the 0th frequency band is obtained only once. This gives an improvement of S/N by a factor of $\sqrt{3}$ when using X-SIM.

In essence, the polarization dependent effect on S/N is compensated by a three times lower variance in the non-zero frequency bands, thus the S/N for X-SIM is theoretically at least as good as for conventional SIM with line grids. The S/N situation can probably be further improved by using azimuthal instead of linear polarization.

2.2. Resolution Doubling by XL-SIM

In order to test the resolution of XL-SIM qualitatively, we used a tubulin sample. Further information about this sample can be found in the Material and Methods section.

A z-stack of phase images was recorded with a total number of 19 phase images in each layer. Thus, the total acquisition time was 190 $\mathrm{ms}$ for every layer for all required phase images. Because the sample thickness amounts to only a few micrometers, the rolling shutter width has been set to a value of 500 pixels. Thus, the phase images have been detected non-confocally. The resulting images permit illustrating the effect of out-of-focus removal by XL-SIM as compared to the calculated (quasi-) wide field (WF) images . Figure 3 shows a comparison of a XL-SIM image (a) with a WF image (b) of the tubulin sample.

2.3. SIM Image Reconstruction

The XL-SIM image has been calculated by separation of all 19 frequency bands that are contained in the phase images (compare Equation (1)). The complex factor $c_n{e}^{i{\varphi }_{n0}}$ is then determined for every frequency band and the frequency bands are shifted along their corresponding pattern frequency vectors in Fourier space. In order to provide enough space for shifting the frequency orders, the Fourier space images/volumes have to be padded with zeros before the shifts are applied. This is equivalent to a pixel interpolation. Usually, the evaluated pixels sizes are half of the camera pixel sizes [10,21].

Finally the frequency values of all bands are superimposed, according to Equation (2), which is a generalized Wiener filter [20,21,28,29]:

$${\tilde{I}}_{\mathrm{XL}-\mathrm{SIM}}\left(\overrightarrow{k}\right)=\frac{{\displaystyle \sum _{n=-9}^9}{\tilde{I}}^{\left(n\right)}(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n\right)})O_n^{*}(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n\right)})}{{\displaystyle \sum _{n^{\prime }=-9}^9}{\left|O_{n^{\prime }}(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n^{\prime }\right)})\right|}^2+{ϵ}^2}\tilde{A}\left(\overrightarrow{k}\right).$$

(2)

$O_n(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n\right)})$ denotes the optical transfer function (OTF) of the n-th frequency band, which is shifted along the frequency vector ${\overrightarrow{k}}^{\left(n\right)}$. The parameter $ϵ$ is the Wiener parameter, which has been chosen empirically and $\tilde{A}\left(\overrightarrow{k}\right)$ is an apodization function [21,29].

Apart from the shifts, the OTFs only differ concerning their functional dependency in the $k_z$-direction of frequency space. In contrast to conventional SIM, the OTFs in Equation (2) depend on the confocality of the detection i.e., the rolling shutter width. Furthermore, their functional dependency on the $k_z$ direction depends on the excitation contrasts of the hexagonal frequencies as functions of defocus.

All XL-SIM images have been calculated according to Equation (2) without any further deconvolution. Figure 3c,d shows the region of interest (ROI) that is marked by `1’ for XL-SIM and WF. As one can see, this ROI hardly contains any out-of-focus emission and the XL-SIM image shows several features of the sample that are unresolved in the WF image. The contrast enhancement along the white horizontal lines in Figure 3c,d is illustrated as line profile plot in Figure 3g. Note that the intensity profile of the WF image has been scaled and an offset has been subtracted in order to overlap the intensity profiles for comparison. Figure 3e,f shows the enlarged ROI `2’. In contrast to ROI `1’, ROI `2’ contains detected out-of-focus emission, which leads to a loss of resolution in the WF image. The S/N in the evaluated XL-SIM image will be smaller for areas of the sample where more out-of-focus emission is detected, which finally also means a loss of resolution in these sample areas.

2.4. Quantitative Resolution Measurement

The resolution of XL-SIM has also been tested quantitatively in two different ways: with nanorulers from the company Gattaquant and with fluorescent microspheres from the company ThermoFisher.

Nanorulers are based on DNA origamis and consist of two distinct fluorescent marks separated by a fixed distance, which is 120 nm for the nanorulers we used in our experiments. They offer a convenient way to test respectively optimize the resolution of a microscope system. Figure 4a,b shows evaluated nanoruler images for WF and XL-SIM. The latter clearly resolves the two parts of the nanorulers. To provide a smoother image reconstruction, we employed a mathematical pixel interpolation factor of 4, bringing the pixel size in the reconstructed images down to 21 $\mathrm{nm}$. For better comparison of WF with XL-SIM images, the same pixel-interpolation was also applied to the WF image in Figure 4a. Because the nanoruler sample is very thin (only one layer of nanorulers), we used non-line-confocal detection i.e., a rolling shutter width of ≈500 pixels. As each nanoruler carries only of a few molecules, laser intensity was reduced to avoid bleaching and the total exposure time was set to 760 $\mathrm{ms}$ by recording a set of 76 phase images for a given z-layer.

A criterion for the quantification of resolution, which is often used in the field of scanning electron microscopy, is Fourier-ring-correlation (FRC) [30]. FRC has also recently been used in optical microscopy [31,32]. A set of 76 phase images was split into two statistically independent sets of 38 phase images and two independent XL-SIM images were processed, which are shown in Supplementary Figure S12. Subsequently, the FRC was calculated from the two XL-SIM images of the nanoruler sample as described, e.g., in [31]. We used the value of $1/7\approx 0.143$ as the threshold criterion, which has also been used, e.g., in [32]. A graphical illustration of the FRC is shown in Figure 4c. The FRC drops below the threshold for a distance of 106.1 $\mathrm{nm}$. This determined resolution value approximately corresponds to the theoretically expected value for a doubled resolution, which is approximately 100 $\mathrm{nm}$.

Another method for the quantification of resolution is the analysis of the microscope PSF as the image of point light sources. We used fluorescent microspheres with a diameter of approximately 50 nm, which is far below the expected resolution. A z-stack of phase images was recorded, and 19 phase images were recorded in every layer. The rolling shutter width was set to the minimum value of 10 pixels in order to get line-confocal detection (LC). Figure 4d,e shows corresponding $(x/y)$ views of the LC and XL-SIM ROIs of a volume of fluorescent microspheres. Several intensity profiles that are overlapped in Figure 4d can be recognized separately in Figure 4e; they are marked by arrows. Figure 4g,h shows the corresponding $(x/z)$ views of the LC and XL-SIM volume. The remaining out-of-focus signal in the LC volume is removed mathematically in the XL-SIM volume. The whole field of view for the focus layer is shown in Supplementary Figure S8. In order to obtain a quantitative evaluation from the microsphere measurement, we fitted a Gaussian model to 537 bead intensity distributions and determined the full-width-at-half-maximum (FWHM) values for each bead in $(x/y)$ and z. The determined lateral FWHM values are plotted against the axial FWHM values in Figure 4f. The lateral resolution value, which is given by the mean value of the FWHM amounts $(106.1\pm 10.0)$ nm, and the axial resolution is $(296.8\pm 38.2)$ nm. Finally, ten separated intensity profiles have been overlapped with subpixel accuracy and radially averaged. The corresponding averaged PSFs for XL-SIM and WF are shown in Supplementary Figures S9 and S10.

Iterative constrained nonlinear deconvolution algorithms as described, e.g., in [33], are a routinely used tool in 3D microscopy [34]. They allow to enhance resolution beyond conventional limits [35] and are also widely used in SIM microscopy [36]. However, these deconvolution methods make the reconstruction of a given structure depend on the object itself [21]. Thus, we only used Equation (2) without further deconvolution for image reconstruction in order to characterize the resolution enhancement by XL-SIM only.

The values for the lateral resolution, which were ascertained by FRC and the FWHM of the microsphere images, correspond very well to each other as well to the theoretical expectation, which is also the case for the axial resolution. Both values correspond to the resolution of other SIM microscopes with line grids [10,21,34].

2.5. XL-SIM with Thick Samples

To demonstrate XL-SIM on thicker samples we used a FluoCells^® Prepared Slide #3 (for details see methods section), which is a mouse kidney section stained with Alexa Fluor^® 488 WGA, Alexa Fluor^® 568 Phalloidin, and $4^{\prime },6$-Diamidin-2-phenylindol (DAPI). The section thickness (15–20 $\mathsf{\mu }\mathrm{m}$) makes this sample not suitable for SIM with WF excitation and detection.

We set the rolling shutter to a width of 10 pixels corresponding to 840 $\mathrm{nm}$. The laser power for the excitation line was set to 30 mW at 488 $\mathrm{nm}$. We recorded a z-stack consisting of 170 layers, spaced 100 $\mathrm{nm}$ apart. The total exposure time was set to 1.1 $\mathrm{ms}$ by recording 38 phase images for every z-position. This corresponds to a total acquisition time of 380 $\mathrm{ms}$ for each layer.

Figure 5a shows the XL-SIM image for the central layer of the z-stack, and Figure 5d shows the corresponding line-confocal image. See Supplementary Video 2 for a focus series through the volume. The marked ROIs `1’ and `2’ are shown enlarged in Figure 5b,c for the XL-SIM and in Figure 5e,f for the LC image. For both ROIs, the resolution of the LC image is not diffraction-limited because of the sample thickness, which causes notable magnitudes of out-of-focus detection in the line-confocal images.

This residual out-of-focus detection is removed mathematically and resolution is improved beyond the Rayleigh limit by XL-SIM: one can clearly see structures far below 1 $\mathsf{\mu }\mathrm{m}$ in both XL-SIM ROIs.

The XL-SIM image in Figure 5a is shown in Figure 5g as an $(x/y)$ section of a three-dimensional volume. The corresponding volume sections in $(x/z)$ and $(y/z)$ directions are also shown. The whole volume is shown in Figure 5h as a 3D object. This embodiment has been created with the software Amira 3D Software for Life Sciences from the company FEI. The whole volume has a size of $(86\times 86\times 17)$ ${\mathsf{\mu }\mathrm{m}}^3$. An animated movie of the 3D object can be seen in Supplementary Video 3.

In order to allow a direct comparison of XL-SIM and ordinary line-grid 3D-SIM, we recorded stacks of the same mouse kidney sample as used in Figure 5, using a DeltaVision OMX Imaging System from GE Healthcare Life Sciences at the Center of Advanced Light Microscopy (CALM) of our university. A sample location of comparable thickness was chosen, showing features similar to our corresponding measurements. The left column in Figure 6 shows OMX images taken at selected planes (in a), and an $(x/z)$ section in b, whereas the right columns show corresponding images/sections acquired with our instrument. The chosen ROIs are 15 $\mathsf{\mu }$m × 15 $\mathsf{\mu }$m, the thickness of the SIM stack is approximately 15 $\mathsf{\mu }$m and the XL-SIM stack thickness amounts to approximately 17 $\mathsf{\mu }$m. The ROI shown in (c) and (d) is marked with `3’ in Figure 5a. The whole field of view for the ROI displayed in (a) and (b) is shown in Supplementary Figure S13.

One can clearly see how out-of-focus signal and noise degrade the image quality in the six focus layers of the SIM stack shown in Figure 6a, whereas the XL-SIM images in Figure 6c retain a similar image quality independant of sample-depth. One can hardly detect a difference to the performance obtained from much thinner sample (e.g., the tubulin sample shown in Figure 3. The $(x/z)$ slice in Figure 6b shows that an out-of-focus signal and noise in the SIM stack constitutes a kind of an echo from the in-focus signal, corresponding to the side lobes of the SIM axial response as shown in Figure 2d. In contrast, there’s virtually no out-of-focus echo in the $(x/z)$ slice shown in Figure 6d, consistent with the theoretically much smaller side lobes of the XL-SIM axial response shown in Figure 2d.

3. Discussion

We have shown that SIM using hexagonal patterns achieves the same resolution improvement as the conventional SIM approaches. However, XL-SIM offers the advantages of much simpler and more stable acquisition and processing of the 3D-SIM raw images, especially in the case of a thick biological specimen. Any errors in the determination of the pattern frequency vectors and z-positions, which can deteriorate the resolution and the homogeneity in the final processed SIM image, are minimized by using hexagonal patterns. The resolution abilities of XL-SIM were demonstrated qualitatively as well as quantitatively and a lateral resolution of approximately 106 $\mathrm{nm}$ has been verified. The axial resolution of the XL-SIM microscope has been shown to be approximately 297 $\mathrm{nm}$. Both the lateral and axial resolution values double the resolution of the conventional optical microscope, which often makes a substantial difference in biological applications.

Furthermore, we have shown that XL-SIM also achieves better images from thicker and densely stained samples, e.g., from a 17 $\mathsf{\mu }\mathrm{m}$ thick mouse kidney slice. In spite of contrast inhomogeneities, introduced by the interference of linear polarized diffraction orders, hexagonal SIM achieves the same resolution and S/N as standard SIM, but the usage of a hexagonal pattern essentially simplifies the raw image acquisition and offers an improved confocality. In principle, any (quasi-)confocal SIM microscope can be modified to become a super resolution SIM microscope, simply by changing the SIM substrate to a hexagonal phase pattern. In the case of 3D-SIM, only one (instead of three) stack(s) of raw images has to be recorded, which avoids any possible z-positioning errors, especially for large image stacks.

Furthermore, the SIM processing is simplified and accelerated by using hexagonal patterns because the frequency vectors of the SIM pattern can be calibrated before the measurement and don’t need to be changed and remeasured. The SIM pattern substrate has been mounted on a rotatable disc, whose rotation was only used to adjust the optimum orientation angle of the pattern relative to the shift direction. For future implementations of XL-SIM, we plan to introduce a linear stage for changing several SIM-patterns, e.g., for several excitation wavelengths.

In order to extend the working range of SIM-based super resolution to thicker samples(3D-)SIM, we combined (3D-)SIM with line-confocal excitation and detection. This was achieved by creating a line-shaped, hexagonally modulated excitation laser line, whose movement was synchronized with the movement of the rolling shutter of the sCMOS camera. The resulting line-confocality reduces the amount of out-of-focus light in the raw SIM phase-images, thus increasing their image contrast and hence the S/N of the resulting SIM image. The benefit of this approach increases with increasing sample thicknesses.

While confocality could be improved by using a narrower excitation line focus, this would diminish the excitation pattern contrasts (see also Figure 2a). To compensate for this, one could choose a smaller pattern base frequency, but that, in turn, would reduce the achievable SIM resolution. Thus, for thicker samples, a trade-off between resolution and confocality has to be made.

4. Materials and Methods

4.1. Fluorescence Excitation

An iBeam-Smart-488-S laser from the company TOPTICA (Gräfelfing, Germany) with a maximum output power of 200 mW has been used for excitation. A model/scheme of the excitation beam path is shown in Figure 1. The laser is shaped into a line by a custom Powell mirror (LT-Ultra, Herdwangen-Schönach, Germany). The line is scanned by a galvanometric scan mirror 6210H (Cambridge Technology, Bedford, MA, USA) across the field of view. A custom scan lens with $f=40.2$ mm (Sill Optics GmbH, S4LFT4345, Wendelstein, Germany) was used to focus the line onto the phase mask in the intermediate image. The galvanometers are driven by digital DC900 controller (SmartMove, Cambridge Technology). The back trace is faster than $0.5$ ms. A micro controller employing an embedded Linux system was used for the synchronization between the movement of the laser line and the rolling shutter window of the camera. The illumination pattern image is shifted by the rotation of a 5 mm glass window (Thorlabs WG11050-A, Newton, NJ, USA) mounted on a 6220H galvanometer (Cambridge Technology) behind the phase mask. A glass rotation of $1^{\circ }$ shifts the pattern by ≈30 $\mathsf{\mu }$m within 1 ms. There is no significant optical error up to a rotation of $\pm {10}^{\circ }$. The excitation tube lens was a $f=164.5$ mm lens from Zeiss (425308) (Oberkochen, Germany).The phase shift unit is a shaped Thorlabs WG11050-A on a CTI 6220H galvanometer.

4.2. Microscope Platform

The microscope platform is a novel microscope prototype from the company TILL I.D. (Planegg/Martinsried, Germany) A 3D model of the platform is shown in Supplementary Figure S2.

A Zeiss plan apochromat $63\times$ NA1.4 oil DIC was used as objective, which is designed for infinity space and a coverslip thickness of 170 $\mathsf{\mu }\mathrm{m}$. The objective was used in combination with Immersol^TM 518F/ISO8036 immersion oil ($n_e=1.518$ (${23}^{\circ }$), ${\nu }_e=45$).

We used a filter set from the company Alluxa (Santa Rosa, CA, USA) consisting of a dichroic, an excitation and an emission filter suitable for a fluorescence excitation at 488 $\mathrm{nm}$. The binary phase pattern has been designed as described in Supplementary Note 5 and was manufactured by the company Collischon Optik-Design in Erlangen, Germany. We chose the pattern period in the sample to be approximately twice the objective’s minimum resolvable distance. For the given objective, this corresponds to a period of $p\approx$ 400 $\mathrm{nm}$.

A user-defined tube lens (denoted as tube lense 2 in Figure 1 with a focal length of $f_{\mathrm{tubus}}=(203.2\pm 1)$ mm has been designed using the software ZEMAX. In combination with the objective, we have a total magnification factor of approximately 77×−78×.

A pco.edge 4.2 rolling shutter camera with a minimum acquisition time of 10 $\mathrm{ms}$ was used for the detection of the emission signal. By synchronization of the excitation scanning line movement with the movement of the rolling shutter active pixel area, we realized a slit confocal excitation and detection. The pixel pitch is 6.5 $\mathsf{\mu }\mathrm{m}$. We used the stage micrometer R1L3S2P (Thorlabs) for determination of pixel sizes in the sample and obtained a value of 84 $\mathrm{nm}$.

The camera has two modes with different line times, which correspond to two different exposure times. Choosing a chip window of $1000\times 1000$ pixels and a rolling shutter height of 10 pixels corresponds to an exposure time of 100 $\mathsf{\mu }\mathrm{s}$ with the fast mode. In some measurements, we used the slow mode, for which a rolling shutter height of 10 pixels corresponds to an exposure time of 275 $\mathsf{\mu }\mathrm{s}$. Exposure times below 550 $\mathsf{\mu }\mathrm{s}$ can be realized by shifting the pattern $N\ge 19$ times. An exposure time of 550 $\mathsf{\mu }\mathrm{s}$ can, e.g., be realized by shifting the pattern 19 times and recording two phase images for every pattern position. This strategy avoids the shifts to become too small and the relative shift errors to become too large. If the 19 pattern shifts are carried out two times successively, two versions of each frequency band can be calculated. This allows for correcting thermal drift between the successive acquisitions analogously to [21].

4.3. Data Processing

The recorded raw image volumes have been processed with the software MATLAB (MATLAB R2013a, The MathWorks Inc., Natick, MA, USA).

4.4. Microscopic Samples

4.4.1. Nanorulers

Gatta-SIM nanorulers [37] (Gattaquant, Braunschweig, Germany) have been used in order to test and quantify the resolution of XL-SIM. Those nanorulers have two distinct fluorescent marks consisting of Alexa488 with a fixed distance of 120 $\mathrm{nm}$.

4.4.2. Fluorescent Microspheres

We used green fluorescent aqueous microspheres with a diameter of 50 nm in order to test and characterize the resolution of XL-SIM (Thermo Fisher Scientific G50, Waltham, MA, USA). A cover slip (Zeiss $\#1.5$) was coated with polylysin, washed and dried. A drop of a diluted microsphere suspension was applied onto the coated coverslip. After some minutes, the surface was washed with water and dried. Then, the sample was sealed to a slide with UV-adhesive NOA63. The contact time of uncured adhesive and the microspheres has to be minimized.

4.4.3. Tubulin Sample

As previously described [38], mouse c2c12 cells were fixed for 10 min with 2% formaldehyde and subsequently washed three times with phosphate buffered saline (PBS). Cells were premeabilized for 10 min with 0.5% Triton X-100 diluted in PBS. After two washing steps with PBS plus 0.02% Tween (PBST) unspecific binding was blocked by incubation of the sample for 1 h with 2% BSA diluted in PBST. Tubulin was immunolabeled with a mouse anti tubulin monoclonal antibody (1:200) and after four washing steps (PBST) stained with a Alexa Fluor 488 conjugated secondary antibody (goat anti mouse 1:500). After four washing steps (PBST), cells were postfixed (4% formaldehyde), washed four times (PBST), and embedded in Vectashild fluid and surrounded by nail polish.

4.4.4. Mouse Kidney Sample

The mouse kidney sample was the FluoCells^® prepared slide #3 from Thermo Fisher Scientific (Waltham, MA, USA), which is stained with Alexa Fluor^® 488 WGA, Alexa Fluor^® 568 Phalloidin, and DAPI. The average sample thickness is approximately 16 $\mathsf{\mu }\mathrm{m}$, and the maximum thickness is approximately 20 $\mathsf{\mu }\mathrm{m}$.

4.4.5. Thin Fluorescent Layers

The thin fluorescence layers that have been used for the orientation adjustment of the hexagonal pattern have been produced by spin coating: 2 ml $1\%$ PMMA in ${\mathrm{CHCl}}_3$ were mixed with 40 $\mathsf{\mu }$L ${10}^{-6}$ mol/L Perylen-diimid-solution in ${\mathrm{CHCl}}_3$. A single drop of approximately 8 $\mathsf{\mu }\mathrm{L}$ were deposited on an with 2000 rpm rotating coverslip (Zeiss $\#1.5$). The sample has been sealed to the slide with the UV-adhesive NOA63.

4.5. Adjustment of Shift Angles and Calibration

The SIM pattern substrate has been mounted on a rotatable disc. The shift angles of the hexagonal pattern intensity in the sample have been adjusted by varying the orientation of the disc relative to the pattern shift direction. The orientation alignment procedure as well as the calibration of the shift widths are described in detail in Supplementary Note 1.

4.6. Parameter Fitting in SIM Processing

The parameters $c_n$ and ${\mathrm{\Phi }}_{n0}$ are determined by complex linear regression of ${\tilde{I}}^{\left(n\right)}(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n\right)})O_0\left(\overrightarrow{k}\right)$ against ${\tilde{I}}^{\left(0\right)}\left(\overrightarrow{k}\right)O_n(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n\right)})$ using the before calibrated values of ${\overrightarrow{k}}^{\left(n\right)}$ [21]. Explicitly, the parameter $c_n{e}^{i{\varphi }_{n0}}$ is evaluated by

$$c_n{e}^{i{\varphi }_{n0}}=\frac{{\displaystyle \sum _{\overrightarrow{k}\in {\mathbb{O}}_n}}{\left|{\tilde{I}}^{\left(0\right)}\left(\overrightarrow{k}\right)O_n(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n\right)})\right|}^2}{{\displaystyle \sum _{\overrightarrow{k}\in {\mathbb{O}}_n}}{\left[{\tilde{I}}^{\left(0\right)}\left(\overrightarrow{k}\right)O_n(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n\right)})\right]}^{*}·\left[{\tilde{I}}^{\left(n\right)}(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n\right)})O_0\left(\overrightarrow{k}\right)\right]},$$

where ${\mathbb{O}}_n$ is the set of frequencies within the overlap of the nth frequency band ${\tilde{I}}^{\left(n\right)}(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n\right)})$ with the 0th frequency band ${\tilde{I}}^{\left(0\right)}\left(\overrightarrow{k}\right)$. Each information component ${\tilde{I}}^{\left(n\right)}\left(\overrightarrow{k}\right)$ is multiplied with its phase factor $c_n{e}^{i{\varphi }_{n0}}$ before evaluation of Equation (2). Analogously to the image processing described in [21], each information component ${\tilde{I}}^{\left(n\right)}\left(\overrightarrow{k}\right)$ has been multiplied by a filter function $F\left(\overrightarrow{k}\right)$:

$$F\left(\overrightarrow{k}\right)=\frac{O_n^{*}\left(\overrightarrow{k}\right)\tilde{A}(\overrightarrow{k}-{\overrightarrow{k}}^{\left(n\right)})}{{\displaystyle \sum _{n^{\prime }}}{\left|O_{n^{\prime }}(\overrightarrow{k}+{\overrightarrow{k}}^{\left(n^{\prime }\right)}-{\overrightarrow{k}}^{\left(n\right)})\right|}^2+{ϵ}^2}$$

before shifting them in frequency space.

5. Conclusions

We have demonstrated that SIM, using a hexagonal pattern (X-SIM), achieves the same resolution enhancement as line-grid-SIM, but image acquisition and processing is simplified and become more stable – particularly for thicker samples – which require acquisition of extended z-stacks. These need to be recorded once only, instead of three times, as in conventional line-grid SIM. Another advantage is that pattern frequency vectors can be calibrated prior to the experiment.

By combining X-SIM with a line-confocal approach (XL-SIM), we were able to extend the working range of 3D-SIM into deeper layers, which line-grid SIM cannot reach because of the rapid drop in contrast when the whole field of view is excited and measured at once. Slit-confocality is achieved by scanning a hexagonally structured slit over the sample and synchronizing the rolling shutter window of a sCMOS cameras such that it acts as (slit-)confocal pinhole.

XL-SIM works equally well as a fast quasiconfocal sectioning tool and—simply by changing the pattern substrate—as super resolution instrument.