PSF Estimation of Space-Variant Ultra-Wide Field of View Imaging Systems

Abstract

Ultra-wide-field of view (UWFOV) imaging systems are affected by various aberrations, most of which are highly angle-dependent. A description of UWFOV imaging systems, such as microscopy optics, security camera systems and other special space-variant imaging systems, is a difficult task that can be achieved by estimating the Point Spread Function (PSF) of the system. This paper proposes a novel method for modeling the space-variant PSF of an imaging system using the Zernike polynomials wavefront description. The PSF estimation algorithm is based on obtaining field-dependent expansion coefficients of the Zernike polynomials by fitting real image data of the analyzed imaging system using an iterative approach in an initial estimate of the fitting parameters to ensure convergence robustness. The method is promising as an alternative to the standard approach based on Shack–Hartmann interferometry, since the estimate of the aberration coefficients is processed directly in the image plane. This approach is tested on simulated and laboratory-acquired image data that generally show good agreement. The resulting data are compared with the results of other modeling methods. The proposed PSF estimation method provides around 5% accuracy of the optical system model.

1. Introduction

Modern photonic imaging systems, used in astronomy and in other areas of science, are required to perform with high imaging quality, a wide aperture (i.e., low F-numbers), and high spatial resolution. These systems are known as UWFOV systems. The wide-angle optical elements used in UWFOV systems introduce a significant shift variance [1], which complicates the modeling task. Many recent research works have focused on finding a model of UWFOV systems that is suitable for reducing the uncertainties in reconstructing image data from wide-field microscopic systems [2,3,4], security cameras [5,6,7], and all-sky cameras [8,9,10,11,12,13,14,15]. Any imaging system can be described by its spatial impulse response function, commonly referred to as the Point Spread Function (PSF). PSF is used to represent the aberrations of UWFOV devices, and can be applied either to the entire image or within regions uniquely defined by the isoplanatic angle. However, it can sometimes be complicated to obtain a PSF model of the system. It can be difficult to achieve the desired accuracy using conventional methods—not from the measurement point of view, but because it is difficult to describe the space variance (SV) of the PSF over the Field of View (FOV). This leads to an obvious issue. From the measurement point of view, it is necessary to obtain the PSF for all discrete points of the entire FOV. Then the PSF of the system is described by such a field of individual PSFs. Born [16] calls this type of PSF partially space-invariant because there are just discrete points (PSFs) in parts of the FOV. However, a space-variant PSF is described by a model in all points of the acquired image. Of course, this brings high demands on the precision of the model. Issues associated with sampling, pixel size and the pixel sensitivity profile must be taken into account in the design. It is very difficult to obtain the parameters of the model, especially in image areas with an aberrated impulse response and for objects with a small Full Width at Half Maximum (FWHM) parameter in relation to the pixel size. These objects, captured, e.g., in microscopy using all-sky cameras and/or special optical elements, can hardly be processed using current techniques, which are typically oriented to multimedia content.

A widely-used approach for obtaining the PSF is to model wavefront aberrations using Zernike polynomials. These polynomials were introduced by Zernike in [17]. This work was later used by Noll [18], who introduced his approach to indexing, which is widely used in the design of optics. Zernike polynomials are usually applied to rotationally symmetric optical surfaces. Zernike polynomials form a basis in optics with Hopkins [19] field dependent wave aberration. Recently, Sasián [20] reformulated the work of Hopkins. This new formulation is more practical for optical design. A more recent contribution by Ye et al. [21] uses Bi-Zernike polynomials, which is an alternative to the method mentioned in Gray’s article [22].

Zernike polynomials are used in modern interferometers for describing wave aberrations. However, researchers have begun to think about using these polynomials for an overall description of lens optics. When the first all-sky camera projects appeared, the question of how to model aberrations of fisheye lenses arose. Weddell [23] proposed the approach of a partially space-invariant model of PSF based on Zernike polynomials. Another all sky project, Pi of the Sky [1], focusing on gamma ray bursts, attempted to make a model of PSF. Their approach was quite different from Weddell’s [23]. First, they calculate the dependency of the coefficients of the polynomials on the angle to the optical axis for several points. Then they interpolate these coefficients over the entire FOV. The WILLIAM project [24] (WIde-field aLL-sky Image Analyzing Monitoring system) faces the issue of aberrations at an extremely wide-field FOV (see also [25,26]). Gray [22] proposed a method for describing the spatial variance of Zernike polynomials. This approach is derived from the description provided by Hopkins [19], and laid the foundations for truly space-variant Zernike polynomials. Of course, there is also the problem of space-variant PSF. In fact, we cannot be limited to rotationally symmetric imaging systems only. Hasan [27] and Thompson [28] came up with a more general description of the pupil function for elliptical or rotationally non-symmetrical Zernike polynomials. This description can be used for calculating the wave aberration in optical materials such as calomel for acousto-optical elements [29]. The first optical approach to wave aberration estimation was described by Páta et al. [30].

The search for the best description of UWFOV systems is not limited to astronomical imaging. Microscopy is another area where the UWFOV type of lens is used. Measurements of the special spherical aberration using a shearing interferometer were described in [31]. A promising new approach was formulated in [32]. It is based on aberration measurements of photolithographic lenses with the use of hybrid diffractive photomasks. The aperture exit wavefront deformation is modeled in [33] for wide field-of-view fluorescence image deconvolution with aberration-estimation from Fourier ptychography.

UWFOV cameras are also used in surveillance systems. However, an image affected by aberrations can have a negative effect in criminal investigations [6]. Ito et al. [5] face this issue, and they propose an approach that estimates a matrix of coefficients of the wavefront aberrations. Many authors have reported on investigations of space-variant imaging systems. An estimate of the PSF field dependency is critical for restoring the degraded image. Qi and Cheng proposed their linear space-variant model [34] and focused on the restoration algorithm for imaging systems. Heide et al. [35] proposed techniques for removing aberration artifacts using blind deconvolution for the imaging system with a simple lens.

In our work, we propose a modeling method for PSF estimation for space-variant systems. Since we would like to use this model for general optical systems, the method is based on modeling the PSF of the system without knowledge of the wavefront. Thus, the method can be an alternative to the Shack–Hartmann interferometer [4,36], or to other direct wavefront measurement methods, since we can estimate wavefront aberrations from fitting PSF in the image plane. The following section begins with a description of wavefront aberrations.

2. Wavefront Aberration Functions

An ideal diffraction limited imaging system transforms a spherical input wave with an inclination equal to the plane wave direction. As described by Hopkins [19], real imaging systems convert the input plane wave into a deformed wavefront. The difference between an ideal wavefront and an aberrated wavefront in the exit aperture can be expressed as

$$W(x,y)=W_{ab}(x,y)-W_{sp}(x,y)$$

(1)

where W(x, y) are wavefronts—the surface of points characterized by the same phase. Figure 1 illustrates the shape of the ideal spherical wavefront $W_{sp}(x,y)$ and aberrated wavefront $W_{ab}(x,y)$. It can be useful to introduce the coordinate system illustrated in Figure 2. The object plane is described by the ξ, η coordinates; the exit pupil uses x, y notation, and the image plane uses u, ν notation. Then we introduce the $\widehat{x},\widehat{y}$ normalized coordinates (see Figure 2b,c) at the exit pupil and ρ, θ polar coordinates. The normalized image plane coordinates are $\widehat{u},\widehat{v}$ and H, ϕ, respectively.

The aberrated wavefront can be represented by a set of orthonormal base functions, known as Zernike polynomials. Zernike polynomials, which are described in [17] and in [37,38,39,40], are a set of functions orthogonal over the unit circle, usually described in polar coordinates as $Z_n^m(\rho ,\theta )$, where ρ is the radius and θ is the angle with respect to the $\widehat{x}$-axis in the exit aperture (see Figure 2b). They represent functions of optical distortions that classify each aberration using a set of polynomials. The set of Zernike polynomials is defined in [17]; other adoptions are in [37,38,39,40], and it can be written as

$$\begin{array}{cc}{Z}_n^m(\mathsf{\rho },\mathsf{\theta })=N_n^m{R}_n^{\left|m\right|}(\mathsf{\rho })\cos (m\mathsf{\theta })& m\ge 0\\ Z_n^m(\mathsf{\rho },\mathsf{\theta })=-N_n^m{R}_n^{\left|m\right|}(\mathsf{\rho })\sin (m\mathsf{\theta })& m<0\end{array},$$

(2)

with n describing the power of the radial polynomial and m describing the angular frequency.

$$N_n^m=\sqrt{\frac{2(n+1)}{1+{\mathsf{\delta }}_{m0}}},$$

(3)

is the normalization factor with the Kronecker delta function ${\mathsf{\delta }}_{m0}$ = 1 for m = 0, and ${\mathsf{\delta }}_{m0}$ = 0 for m ≠ 0, and

$$R_n^{\left|m\right|}(\mathsf{\rho })={\displaystyle \sum _{s=0}^{(n-\left|m\right|)/2}\frac{{(-1)}^s(n-s)!}{s!\left[\frac{(n+\left|m\right|)}{2}-s\right]!\left[\frac{(n-\left|m\right|)}{2}-s\right]!}}{\mathsf{\rho }}^{(n-2s)},$$

(4)

is the radial part of the Zernike polynomial.

Any wavefront phase distortion over a circular aperture of unit radius can be expanded as a sum of the Zernike polynomials as

$$W(\mathsf{\rho },\mathsf{\theta })={\displaystyle \sum _n^k{\displaystyle \sum _{m=-n}^n{A}_n^m}}{Z}_n^m(\mathsf{\rho },\mathsf{\theta }),$$

(5)

which can be rewritten using Equation (2) as

$$W(\mathsf{\rho },\mathsf{\theta })={\displaystyle \sum _n^k{\displaystyle \sum _{m=0}^n{A}_n^m}{N}_n^m{R}_n^{\left|m\right|}(\mathsf{\rho })\cos (m\mathsf{\theta })-{\displaystyle \sum _n^k{\displaystyle \sum _{m=-n}^{-1}{A}_n^m}{N}_n^m{R}_n^{\left|m\right|}(\mathsf{\rho })\sin (m\mathsf{\theta })}},$$

(6)

where m has values of $-n,-n+2,\dots n,$ k is the order of the polynomial expansion, and $A_n^m=A_n^m(H,\mathsf{\phi })$ is the coefficient of the $Z_n^m$ mode in the expansion, i.e., it is equal to the root mean square (RMS) phase difference for that mode. The wavefront aberration function across the field of view of the optical system can then be described as

$$W(\mathsf{\rho },\mathsf{\theta },H,\mathsf{\phi })={\displaystyle \sum _{n=0}^k{\displaystyle \sum _{m=-n}^n{A}_n^m\left(H,\mathsf{\phi }\right)}}{Z}_n^m(\mathsf{\rho },\mathsf{\phi }-\mathsf{\theta })$$

(7)

where the wavefront is described with normalized polar coordinates. A better understanding can be provided by

Table A1. The relation between indices and coefficients of the Zernike polynomials.

${\mathit{Z}}_{\mathit{n}}^{\mathit{m}}\mathbf{(}\mathsf{\rho }\mathbf{,}\mathsf{\theta }\mathbf{)}$	${\mathit{A}}_{\mathit{n}}^{\mathit{m}}$	Expansion Coefficient Function	Name
$Z_0^0(\mathsf{\rho },\mathsf{\theta })$	$A_0^0$	$\begin{array}{l}\frac{1}{2}[\frac{5}{9}{W}_{0,10,0}+\frac{2}{5}{W}_{080}+\frac{1}{2}{W}_{060}+\frac{2}{3}{W}_{040}+W_{020}+H^6\left(W_{620}+\frac{1}{2}{W}_{622}\right)\dots \hfill \\ +H^4\left(W_{420}+\frac{1}{2}{W}_{422}+\frac{2}{3}{W}_{440}+\frac{1}{3}{W}_{442}+\frac{1}{4}{W}_{444}\right)\dots \hfill \\ +H^2\left(W_{220}+\frac{1}{2}{W}_{222}+\frac{2}{3}{W}_{240}+\frac{1}{3}{W}_{242}+\frac{1}{2}{W}_{260}+\frac{1}{4}{W}_{262}\right)]\dots \hfill \\ +W_{800}{H}^8+W_{600}{H}^6+W_{400}{H}^4+W_{200}{H}^2+W_{000}\hfill \end{array}$	Piston
$Z_1^1(\mathsf{\rho },\mathsf{\theta })$	$A_1^{+1}$	$\begin{array}{l}[-\frac{16}{35}{W}_{171}+\frac{1}{2}{W}_{151}+\frac{2}{3}{W}_{131}+W_{111}+H^6{W}_{171}+H^4\left(W_{511}+\frac{2}{3}{W}_{531}+\frac{1}{2}{W}_{533}+\right)\dots \hfill \\ +H^2\left(W_{311}+\frac{1}{2}{W}_{351}-\frac{9}{40}{W}_{533}+\frac{2}{3}{W}_{331}+\frac{1}{2}{W}_{333}+\frac{3}{5}{W}_{353}\right)]H\cos (\mathsf{\phi })\hfill \end{array}$	Tilt
$Z_1^{-1}(\mathsf{\rho },\mathsf{\theta })$	$A_1^{-1}$	$\begin{array}{l}[-\frac{16}{35}{W}_{171}+\frac{1}{2}{W}_{151}+\frac{2}{3}{W}_{131}+W_{111}+H^6{W}_{171}+H^4\left(W_{511}+\frac{2}{3}{W}_{531}+\frac{1}{2}{W}_{533}+\right)\dots \hfill \\ +H^2\left(W_{311}+\frac{1}{2}{W}_{351}-\frac{9}{40}{W}_{533}+\frac{2}{3}{W}_{331}+\frac{1}{2}{W}_{333}+\frac{3}{5}{W}_{353}\right)]H\sin (\mathsf{\phi })\hfill \end{array}$
$Z_2^0(\mathsf{\rho },\mathsf{\theta })$	$A_2^0$	$\begin{array}{l}\frac{1}{2}[\frac{20}{21}{W}_{0,10,0}+\frac{4}{5}{W}_{080}+\frac{9}{10}{W}_{060}+W_{040}+W_{020}+H^6\left(W_{620}+\frac{1}{2}{W}_{622}\right)\dots \hfill \\ +H^4\left(W_{420}+\frac{1}{2}{W}_{422}+W_{440}+\frac{1}{2}{W}_{442}+\frac{3}{8}{W}_{444}\right)\dots \hfill \\ +H^2\left(W_{220}+\frac{1}{2}{W}_{222}+W_{240}+\frac{1}{2}{W}_{242}+\frac{9}{10}{W}_{260}+\frac{9}{20}{W}_{262}\right)]\hfill \end{array}$	Focus
$Z_2^2(\mathsf{\rho },\mathsf{\theta })$	$A_2^{+2}$	$\begin{array}{l}\frac{1}{2}\left[\frac{3}{5}{W}_{262}+\frac{3}{4}{W}_{242}+W_{222}+H^4{W}_{622}+H^2\left(W_{422}+\frac{3}{4}{W}_{444}+\frac{3}{4}{W}_{442}\right)\right]H^2\cos (2\mathsf{\phi })\hfill \end{array}$	Astigmatism
$Z_2^{-2}(\mathsf{\rho },\mathsf{\theta })$	$A_2^{-2}$	$\begin{array}{l}\frac{1}{2}\left[\frac{3}{5}{W}_{262}+\frac{3}{4}{W}_{242}+W_{222}+H^4{W}_{622}+H^2\left(W_{422}+\frac{3}{4}{W}_{444}+\frac{3}{4}{W}_{442}\right)\right]H^2\sin (2\mathsf{\phi })\hfill \end{array}$
$Z_3^1(\mathsf{\rho },\mathsf{\theta })$	$A_3^{+1}$	$\begin{array}{l}\frac{1}{3}\left[\frac{6}{5}{W}_{171}+\frac{6}{5}{W}_{151}+W_{131}+H^4\left(W_{531}+\frac{3}{4}{W}_{533}\right)+H^2\left(W_{531}+\frac{3}{4}{W}_{333}+\frac{6}{5}{W}_{351}+\frac{9}{10}{W}_{353}\right)\right]H\cos (\mathsf{\phi })\hfill \end{array}$	Coma
$Z_3^{-1}(\mathsf{\rho },\mathsf{\theta })$	$A_3^{-1}$	$\begin{array}{l}\frac{1}{3}\left[\frac{6}{5}{W}_{171}+\frac{6}{5}{W}_{151}+W_{131}+H^4\left(W_{531}+\frac{3}{4}{W}_{533}\right)+H^2\left(W_{531}+\frac{3}{4}{W}_{333}+\frac{6}{5}{W}_{351}+\frac{9}{10}{W}_{353}\right)\right]H\sin (\mathsf{\phi })\hfill \end{array}$
$Z_4^0(\mathsf{\rho },\mathsf{\theta })$	$A_4^0$	$\begin{array}{l}\frac{1}{6}\left[\begin{array}{l}\frac{25}{14}{W}_{0,10,0}+\frac{12}{7}{W}_{080}+\frac{3}{2}{W}_{060}+W_{040}+H^4\left(W_{440}+\frac{1}{2}{W}_{442}+\frac{3}{8}{W}_{444}\right)\dots \\ +H^2\left(W_{240}+\frac{1}{2}{W}_{242}+\frac{3}{2}{W}_{260}+\frac{3}{4}{W}_{262}\right)\end{array}\right]\hfill \end{array}$	Spherical
$Z_3^3(\mathsf{\rho },\mathsf{\theta })$	$A_3^{+3}$	$\frac{1}{4}\left(W_{333}+\frac{1}{20}{W}_{353}+H^2{W}_{533}\right)H^3\cos (3\mathsf{\phi })$	Elliptical Coma
$Z_3^{-3}(\mathsf{\rho },\mathsf{\theta })$	$A_3^{-3}$	$\frac{1}{4}\left(W_{333}+\frac{1}{20}{W}_{353}+H^2{W}_{533}\right)H^3\sin (3\mathsf{\phi })$
$Z_4^2(\mathsf{\rho },\mathsf{\theta })$	$A_4^{+2}$	$\frac{1}{4}\left[\frac{1}{2}{W}_{242}+\frac{2}{3}{W}_{262}+\frac{1}{2}{H}^2\left(W_{444}+W_{442}\right)\right]H^2\cos (2\mathsf{\phi })$	Oblique Spherical
$Z_4^{-2}(\mathsf{\rho },\mathsf{\theta })$	$A_4^{-2}$	$\frac{1}{4}\left[\frac{1}{2}{W}_{242}+\frac{2}{3}{W}_{262}+\frac{1}{2}{H}^2\left(W_{444}+W_{442}\right)\right]H^2\sin (2\mathsf{\phi })$
$Z_5^1(\mathsf{\rho },\mathsf{\theta })$	$A_5^{+1}$	$\frac{1}{10}\left[\frac{12}{7}{W}_{171}+W_{151}+H^2\left(W_{351}+\frac{3}{4}{W}_{353}\right)\right]H\cos (\mathsf{\phi })$	5th Coma
$Z_5^{-1}(\mathsf{\rho },\mathsf{\theta })$	$A_5^{-1}$	$\frac{1}{10}\left[\frac{12}{7}{W}_{171}+W_{151}+H^2\left(W_{351}+\frac{3}{4}{W}_{353}\right)\right]H\sin (\mathsf{\phi })$
$Z_6^0(\mathsf{\rho },\mathsf{\theta })$	$A_6^0$	$\frac{1}{20}\left[\frac{25}{9}{W}_{0,10,0}+2W_{080}+W_{060}+H^2\left(W_{260}+\frac{1}{2}{W}_{262}\right)\right]$	5th Spherical
$Z_5^3(\mathsf{\rho },\mathsf{\theta })$	$A_5^{+3}$	$\frac{1}{20}{W}_{353}{H}^3\cos (3\mathsf{\phi })$
$Z_5^{-3}(\mathsf{\rho },\mathsf{\theta })$	$A_5^{-3}$	$\frac{1}{20}{W}_{353}{H}^3\sin (3\mathsf{\phi })$
$Z_4^4(\mathsf{\rho },\mathsf{\theta })$	$A_4^{+4}$	$\frac{1}{8}{W}_{444}{H}^4\cos (4\mathsf{\phi })$
$Z_4^{-4}(\mathsf{\rho },\mathsf{\theta })$	$A_4^{-4}$	$\frac{1}{8}{W}_{444}{H}^4\sin (4\mathsf{\phi })$
$Z_7^1(\mathsf{\rho },\mathsf{\theta })$	$A_7^{+1}$	$\frac{1}{35}{W}_{171}H\cos (\mathsf{\phi })$
$Z_7^{-1}(\mathsf{\rho },\mathsf{\theta })$	$A_7^{-1}$	$\frac{1}{35}{W}_{171}H\sin (\mathsf{\phi })$
$Z_6^2(\mathsf{\rho },\mathsf{\theta })$	$A_6^{+2}$	$\frac{1}{30}{W}_{262}{H}^2\cos (2\mathsf{\phi })$
$Z_6^{-2}(\mathsf{\rho },\mathsf{\theta })$	$A_6^{-2}$	$\frac{1}{30}{W}_{262}{H}^2\sin (2\mathsf{\phi })$
$Z_8^0(\mathsf{\rho },\mathsf{\theta })$	$A_8^0$	$\frac{1}{28}{W}_{0,10,0}+\frac{1}{70}{W}_{080}$	7th Spherical
$Z_{10}^0(\mathsf{\rho },\mathsf{\theta })$	$A_{10}^0$	$\frac{1}{252}{W}_{0,10,0}$	9th Spherical

in Appendix A and by Figure 3, which describes the indexing of Zernike polynomials and the kind of aberration associated with an index of Zernike polynomials.

3. Field Dependency of the Wavefront

A description of the wavefront aberration of a circular rotationally symmetric optical system can be adopted from [22], originally from [19]

$$W(\mathsf{\rho },\mathsf{\theta };H,\mathsf{\phi })={\displaystyle \sum _p^{\infty }{\displaystyle \sum _n^{\infty }{\displaystyle \sum _m^{\infty }{W}_{klm}}}}{H}^k{\mathsf{\rho }}^l{\cos }^m(\mathsf{\phi }-\mathsf{\theta }),$$

(8)

where k = 2p + m and l = 2n + m. Symbol $W_{klm}$ is used for the expansion coefficients; the coordinates are defined in Figure 2c. The field dependency of $A_n^m(H,\mathsf{\phi })$ can be solved by comparing wavefront aberrations using Equations (7) and (8) as

$${\displaystyle \sum _{n=0}^k{\displaystyle \sum _{m=-n}^n{A}_n^m\left(H,\mathsf{\phi }\right)}}{Z}_n^m(\mathsf{\rho },\mathsf{\phi }-\mathsf{\theta })={\displaystyle \sum _p^{\infty }{\displaystyle \sum _n^{\infty }{\displaystyle \sum _m^{\infty }{W}_{klm}}}}{H}^k{\mathsf{\rho }}^l{\cos }^m(\mathsf{\phi }-\mathsf{\theta }).$$

(9)

Equation (10) can be obtained by expanding Equation (7) and rewriting terms ${\cos }^m(\mathsf{\theta },\mathsf{\phi })$ into terms containing a set of goniometric functions $\cos (m\mathsf{\theta }),\sin (m\mathsf{\theta }),r\in N$ and comparing the coefficients in this form with expanded Zernike polynomials (as in Table A1). The resulting coefficients $A_n^m(H,\mathsf{\phi })$ describing the space variation of Zernike polynomials are presented in [22]. Coefficients $W_{klm}$ are then used for describing the aberration of the space-variant optical system. The order of the aberration terms is defined by Hopkins in [19] as

$$order=(sumofthepowersof\mathsf{\rho }and\mathsf{\theta })-1.$$

(10)

4. The Space-Variant Point Spread Function

Wide-field optics is usually affected by various aberrations, such as distortion which makes the PSF of the system space-variant, i.e., the PSF changes across the field of view. It is, therefore, necessary to use a complicated model of the system’s PSF. Let us begin with a linear space-invariant optical imaging system, which can be expressed as

$$h(u,v)=|FT\{P(x,y)\}{|}^2={\left|FT\left\{p(x,y)\exp \left(-i\frac{2\mathsf{\pi }}{\mathsf{\lambda }}W(x,y)\right)\right\}\right|}^2,$$

(11)

where $p\left(x,y\right)$ defines the shape, the size, and the transmission of the exit pupil, and $W(x,y)$ is the phase deviation of the wavefront from a reference sphere. The generalized exit pupil function is described in [16] as

$$P(x,y)=p(x,y)\exp \left(-i\frac{2\mathsf{\pi }}{\mathsf{\lambda }}W(x,y)\right).$$

(12)

An imaging system is space-invariant if the image of a point source object changes only in position, not in the functional form [41]. However, wide-field optical systems give images where the point source object changes both in position and in functional form. Thus, wide-field systems and their impulse responses lead to space-variant optical systems. If we consider a space-variant system, we cannot use the convolution for expressing the relation between object and image. When computing SVPSF, we have to use the diffraction integral [6,42]. SVPSF can then be expressed as

$$\begin{array}{cc}h(u,v,\xi ,\eta )=& \frac{1}{{\lambda }^2{z}_1{z}_2}{\displaystyle \underset{-\infty }{\overset{\infty }{\int }}{\displaystyle \underset{-\infty }{\overset{\infty }{\int }}p}}(x,y)\exp \left(-i\frac{2\pi }{\lambda }W(x,y)\right)\cdots \\ & \cdots \exp \left\{-j\frac{2\pi }{\lambda z_2}[(u-M\xi )x+(v-M\eta )y]\right\}dxdy\end{array},$$

(13)

with a defined magnification of the system

$$M=-\frac{z_2}{z_1},$$

(14)

where z₁ is the distance from the object plane to the principal plane and z₂ is the distance from the principal plane to the image plane.

5. Proposed Method

The method proposed in this paper is based on modeling the PSF of the system, and comparing it with real image data without requiring measurements of the wavefront aberrations. The fitting of the model takes place in the image plane.

There are three conditions for acquiring the set of calibration images. The first criterion is that FWHM of the image of the light source should be sufficient for PSF estimation. The experimental results show that FWHM size greater than 5 px is enough for the algorithm. FWHM of the diffractive image of the source should be smaller than 1 px. Otherwise the diffractive function has to be added to the model of the system. The image of the 200 µm pinhole has FWHM size 0.6 µm which is less than the size of the pixel of our camera. The influence of the source shape has to be taken into account. The second criterion is the Signal-to-Noise Ratio (SNR), which should be greater than 20 dB. Section 6 contains a comparison of results obtained with different SNR. The third criterion is that the number of test images depends on the space variance of the system, i.e., a heavily distorted optical system will require more test images to satisfy this criterion. We have to choose the distance of two different PSFs in such a way that

$${RMSE}_{IN}(f_i,f_{i+1})=\sqrt{\frac{1}{M\times N}{\displaystyle \sum _{u=0}^{N-1}{\displaystyle \sum _{v=0}^{M-1}(}}{f}_i-f_{i+1}{)}^2}<5\%.$$

(15)

where $f_i$ and $f_{i+1}$ are the images of the point light source in the image plane. M and N are the sizes of the $f_i$ and $f_{i+1}$ images. In our example, a grid of 24 positions of the PSFs in one quadrant of the optical system is sufficient to estimate the model. The wavefront is modeled using Zernike polynomials and known optical parameters. In addition to the input image data, we need to know camera sensor parameters such as resolution, the size of the sensor and optical parameters such as the focal length (crop factor, if included), the F-number and the diameter of the exit pupil. The obtained model of the PSF of the optical system is based on the assessment of differential metrics.

We can describe the modeling of real UWFOV systems as a procedure with three main parts: the optical part, the image sensor, and the influence of the sensor noise. The space-variant impulse response $h(u,v,\mathsf{\xi },\mathsf{\eta })$ can include the influence of the image sensor (e.g., pixel shape and sensitivity profile, noise or quantization error). The sensor has square pixels with uniform sensitivity. Then, the PSF of the imaging system can be described as

$$h(u,v,\mathsf{\xi },\mathsf{\eta })=h_{opt}(u,v,\mathsf{\xi },\mathsf{\eta })\ast h_{sen}(u,v),$$

(16)

where $h_{sen}(u,v)$ is the PSF of the sensor and $h_{opt}(u,v,\mathsf{\xi },\mathsf{\eta })$ is the PSF of the optical part. Symbol * describes the convolution. $h_{opt}(u,v,\mathsf{\xi },\mathsf{\eta })$ can be calculated from the system parameters and wavefront deformations using the Fourier transform as described in Equation (13). The wavefront deformation is modeled using Zernike polynomials for the target position in the image plane (see Equation (7)). Ultra-wide field images typically have angular dependent PSF. High orders of Zernike polynomials are therefore used in the approximation. The wavefront approximation is used up to the 8th order plus the 9th spherical aberration of the expansion function. The field dependence of the coefficients is formulated in [22]. In our work, the set of field-dependent coefficients was expanded up to the 8th order plus the 9th spherical aberration. The actual table of the used $W_{klm}$ coefficient is attached in Appendix A as Table A1.

Let us assume an imaging system with an unknown aberration model. We then obtain with this system a grid of K test images of a point light sources covering the entire FOV as

$${\left\{f_i(u,v)\right\}}_{i=1}^K.$$

(17)

Then, let

$${\left\{f_d(\widehat{u},\widehat{v})\right\}}_{d=1}^L,$$

(18)

be the d-th realization of the model in the corresponding position in the image as the original object. Sub-matrix $f_i$ is the image of the point light source in the image plane, while matrix $f_d$ is the sub-image model computed by our method. The size of the sub-arrays must be sufficient to cover the whole neighborhood of the point light object on the positions ${\left\{u_{0i},v_{0i})\right\}}_{i=1}^K$.

As was mentioned above, symbols $\widehat{u},\widehat{v}$ are used for image plane coordinates in the normalized optimization space, and $u,v$ are image plane coordinates. Note that $f_i$ and $f_d$ can be located at any point over the entire field of view. The step between the positions of the point light source has to be chosen to cover the observable difference of the acquired point. The positions of the point light source in the field of view play an important role in the convergence efficiency. In our example, we divided the field of view each 10 degrees uniformly horizontally and vertically. We therefore obtain a matrix of PSFs sufficient for a description of the model. It turned out that finer division of the FOV is not necessary and does not improve the accuracy of the model and it satisfies the condition in Equation (15).

The algorithm uses two evaluation methods. The first method is based on calculating the root mean square error (RMSE) over the difference matrix in Equation (21) and optimizing the $W_{klm}$ parameters and the $\Delta \widehat{u},\Delta \widehat{v}$ positions for decreasing residuals. The second method is based on deducting the original and model matrix to obtain the maximum difference (MAXDIFF). Then, the residuals of this method indicate the deviation against the original matrix. The first method with RMSE calculation provides a better shape description. However, this method can result in local extremes. The reduction can be resolved by using the MAXDIFF calculation method. This method minimizes the local extremes, because it is focused on minimizing the maximum difference between the $f_i$ and $f_d$ object matrices, but the output can be a more general shape of PSF.

Let us now introduce operators $RMSE(f_i,f_d)$ and $MAXDIFF(f_i,f_d)$, which are used as descriptors of the differences between the original PSF and the model of PSF ($f_i$ and $f_d$).

$$RMSE(f_i,f_d)=\sqrt{\frac{1}{M\times N}{\displaystyle \sum _{\tilde{u}=0}^{N-1}{\displaystyle \sum _{\tilde{v}=0}^{M-1}(}}{f}_i(u,v)-f_d(\widehat{u},\widehat{v}){)}^2}.$$

(19)

$$MAXDIFF(f_i,f_d)={\max }_{(u,v)}\left(\left|f_i(u,v)-f_d(\widehat{u},\widehat{v})\right|\right).$$

(20)

M and N are the sizes of the $f_i$ and $f_d$ sub-arrays. The optimization parameter of the $W_{klm}$ coefficients uses the Nelder–Mead optimizing algorithm, which is described in detail in [43]. Let $R_{f_i,f_d}$ be the optimizing operator, then

$$R_{f_i,f_d}(\widehat{u},\widehat{v})=\underset{W_{klm},\Delta u,\Delta v}{\min }\left[RMSE(f_i,f_d)\vee MAXDIFF(f_i,f_d)\right],$$

(21)

where $W_{klm},\Delta \widehat{u},\Delta \widehat{v}$ are variables for minimizing the cost function. For multiparameter optimization tasks, the challenge is to find the global minimum of the function. Considering this issue, we can find an appropriate set of starting parameters of the fit algorithm by smart selection of on-axis points (PSFs) on which we will obtain appropriate $W_{klm},\Delta \widehat{u},\Delta \widehat{v}$ variables, and we will then increase the precision of our model by selecting off-axis points and improving $W_{klm},\Delta \widehat{u},\Delta \widehat{v}$ variables. The process of point selection is illustrated in Figure 4, and the steps in the algorithm are as follows:

●

Select a point placed on the optical axis (this point is considered as SV aberration free).

○

The $W_{klm}$ optimization coefficient is based on minimizing RMSE or the MAXDIFF metrics. Then we obtain

$$R_{f_i,f_d}(0,0)\Rightarrow W_{klm}(1),\Delta \widehat{u},\Delta \widehat{v},$$

(22)

○

where $W_{klm}\left(1\right)$ is the first realization of the fit, and $\Delta \widehat{u},\Delta \widehat{v}$ represent the displacement of the object point in the image plane.

●

The next calibration point will be placed on the $\widehat{u}$-axis and next to the first point.

○

All $W_{klm}$ coefficients from the first point fit will be used as start conditions in the next step of the fit.

●

In the next step, we will fit all the points along the $\widehat{u}$-axis by increasing distance H.

○

The previous result is used as the start condition for the next point.

$$R_{f_i,f_d}(\widehat{u},0)\Rightarrow W_{klm}(d),\Delta \widehat{u},\Delta \widehat{v}.$$

(23)

●

Then, we can continue along the $\widehat{v}$-axis by increasing distance H. This procedure gives the first view of the model.

$$R_{f_i,f_d}(0,\widehat{v})\Rightarrow W_{klm}(d),\Delta \widehat{u},\Delta \widehat{v}.$$

(24)

○

where $W_{klm}\left(d\right)$ is the d-th realization of the fit.

●

After fitting all the on-axis points, we will start to fit all the off-axis points.

○

The example in this paper uses 24 points.

$$R_{f_i,f_d}(\widehat{u},\widehat{v})\Rightarrow W_{klm},\Delta \widehat{u},\Delta \widehat{v}.$$

(25)

●

After fitting all the points, we need to evaluate the output $W_{klm}$ coefficients which can describe the field dependency of our model.

●

We verified experimentally that the median applied to the set of estimated $W_{klm}$ coefficients provides better results of the output model than other statistical methods. Thus, we need to find the median of every $W_{klm}$ coefficient over all fit realizations (the number of realizations is L) of the used points. This step will eliminate extreme values of $W_{klm}$ coefficients which can occur at some positions of the PSF due to convergence issues caused by sampling of the image or overfitting effects caused by high orders polynomials. Extreme values indicate that the algorithm found some local minimum of the cost function and not the global minimum. The values of the $W_{klm}$ coefficients are then significantly different from the coefficients obtained in the previous position. These variations are given by the goodness of fit.

●

The output set of $W_{klm}$ coefficients then consists of values verified over the field.

As is illustrated in Figure 4, the described procedure repeats in every order (as defined in Equation (10)) of $W_{klm}$ coefficients. The estimates of higher order coefficients (6th and 8th) come from lower order coefficients that have already been estimated from lower orders. If we want to describe our optical system with coefficients up to the 8th order, we will first need to obtain coefficients of the 4th order. Then, it is necessary to repeat the procedure from Equation (22) to Equation (25) of the previously described procedure assuming 4th order $W_{klm}$ coefficients as a starting condition. Following the whole procedure, we will estimate the 6th order coefficients. Then, repeating the procedure again with 6th order coefficients as a starting condition, we can finally calculate 35 coefficients of the 8th order. The result is a set of coefficients (the number of coefficients depends on the order that is used) related to the optical system with the field dependency described in Section 3.

6. Results

This section is divided into two subsections. Our first goal was to verify the method for estimating the $W_{klm}$ coefficients. We, therefore, simulated an artificial imaging system and tested the convergence of the introduced algorithm to find proper $W_{klm}$ parameters. The second subsection of results involves modeling real imaging systems with the results of different orders of the Zernike polynomials. The input conditions, such as SNR > 20 dB (peak to noise) of all considered PSFs, the number of calibration images and FWHM, mentioned in Section 5 have to be taken into account before acquiring the calibration images and applying the algorithm of the PSF estimate.

6.1. Numerical Stability Verification

This section deals with verifying the stability of the optimization convergence. The test pattern, used for verifying the functionality of the algorithm was an image obtained by generating random $W_{klm}$ coefficients (i.e., random optical aberrations) and placing PSFs in locations covering the entire field of view (see Figure 5), assuming a rotational symmetric imaging system. To verify the algorithm, we test PSFs in the locations marked with red circles.

Table 1. The sensor size, the resolution and the optical parameters of the simulated artificial imaging system.

Resolution	3358 × 2536 px
Sensor size	18.1 × 13.7 mm
Pixel size	5.39 µm
Lens focus distance	10 mm
FOV	110°

summarizes the parameters of the simulated system.

Table 2. Selected on-axis points results of verification of the algorithm, where normalized distance H is from 0 to 0.83 and ϕ is equal to zero. Image distance H is normalized according to the sensor, and it is related to an FOV of 110°; all optical parameters are summarized in Table 1. Differences between the original PSF and the model are within the order of thousandths. Thus, the proposed algorithm can find the exact $W_{klm}$ coefficients used for generating the test pattern illustrated in Figure 5.

Metrics	Normalized Image Distance H (-)
	0	0.17	0.33	0.50	0.67	0.83
RMSE (10−5)	8.2	7.3	8.8	16	57	270
MAXDIFF (‰)	0.14	0.13	0.16	0.36	2.1	6.4

, Figure 6 and Figure 7 show successive verification of the proposed algorithm by values of MAXDIFF and the RMSE operator, a fitted curve illustrating the trend of the results, and standard deviation error bars. We can see that the difference between the original and the model is within the order of thousandths. The difference is given only by quantization noise and the sampling of the test pattern. The sampling of the original PSF appears as a serious issue, and low resolution causes problems within the convergence in the optimization of the $W_{klm}$ parameters. The resolution of the image may affect the accurate positioning of the PSF. For an explanation, we can find the maximum of the PSF by calculating the center of the mass precisely, if the sampling of the pattern is finer. Therefore, if we can find the maximum precisely, we can use precise positioning $\widehat{u},\widehat{v}$ for calculating the PSF of the system.

The graphs mentioned above provide information about the accuracy of the model against the positioning of the PSF in the image. However, it can also be useful to mention the accuracy of the model when the original image includes noise. For this reason, we performed a test where one PSF, in the center of the image, was affected by additional Gaussian noise. A test was performed in which the SNR in the image under test was from 28 dB to 9 dB (see Figure 8). We can see that the algorithm works quite well from 28 dB to 20 dB, and MAXDIFF is less than 3%. However, when SNR is below 20 dB, the error decreases rapidly. This is due to the fact that our model focuses on optical aberrations, not on modeling the noise.

Figure 9 illustrates a direct comparison between the original PSF, the obtained model and the difference between them in relation to RMSE and the MAXDIFF operator. This verification process was performed to verify whether the algorithm is able to find the model $W_{klm}$ parameters that are closest to the $W_{klm}$ coefficients used in generating the pattern illustrated in Figure 5.

6.2. Experimental Laboratory Results

The second part of the results describes the performance of our method when dealing with real data. The experimental images were obtained with the setup illustrated in Figure 10. A small white LED diode was used as a light source, together with a pinhole 200 μm in diameter. The image data were acquired with a G2-8300 astronomical camera [44], with a 3358 × 2536 pixel CCD sensor, corresponding to a size of 18.1 × 13.7 mm, which gives a pixel size of 5.39 µm. The camera was equipped with a Sigma 10 mm EX DC HSM fisheye lens [45], which is the diagonal type of fisheye lens. A camera was mounted on a 2D rotational stage. This configuration, together with an observation distance of 5 m, gives FWHM size of a PSF of around 6 pixels. Different positions of the light source, to cover the entire FOV, were achieved by rotating the camera in both axes. The advantage of this approach is that the same distance is kept between the light source and the camera, assuming that nodal mounting has been used.

Table 3. The sensor size, the resolution and the optical parameters of the experimental imaging system used for acquiring the image dataset.

Resolution	3358 × 2536 px
Sensor size	18.1 × 13.7 mm
Pixel size	5.39 µm
Lens focus distance	10 mm
FOV	110°

summarizes parameters of the simulated system, which are the same as for the simulated system in Section 6.1. However, it is necessary to take into account that the optical aberrations are different. The graphs in Figure 11, Figure 12 and Figure 13 and

Table 4. Selected on-axis point results for the MAXDIFF difference between the original and the estimated model, where H is from 0 to 0.83 and ϕ is equal to zero. The MAXDIFF differences of the polynomial orders are given as a percentage. Highlighted columns contain results used later in the comparison.

Metrics	Normalized Image Distance H (-)
	0	0.17	0.33	0.50	0.67	0.83
4th order	4.6	5.1	5.3	7.8	7.5	9.4
6th order	5.4	7.7	6.5	9.4	7.7	12.5
8th order	5	6.4	6.1	7.7	7.6	10

show the results of the fit with a different order of $W_{klm}$ coefficients, a fitted curve illustrating the trend of the results, and standard deviation error bars. Figure 14 shows a comparison of the results with other PSF modeling methods. It can be seen that the best result is obtained with coefficients up to the 4th order. Then, with a higher order of the coefficients, we obtain slightly worse results, even for the PSFs placed on the optical axis. With increasing image distance, the situation seems to deteriorate, and the difference increases. One reason for this is the optimization convergence, which is worse for the more involved $W_{klm}$ coefficients in 6th and 8th orders. Another explanation is provided by Figure 15 and

Table 5. A direct comparison of results estimated by 4th, 6th and 8th order polynomials. This table is related to Figure 15. The results are calculated for one position of the light source at H = 0.33. The total flux difference was calculated for enumerating the overall intensity difference between the original PSF and the model.

Metrics	4th Order	6th Order	8th Order
RMSE (-)	0.032	0.039	0.036
MAXDIFF (%)	5.3	6.5	6.1
Total flux difference (‰)	0.31	0.35	0.37

. It can be seen that the absolute difference is slightly worse for higher orders of $W_{klm}$ coefficients, but that the shape of the model or the precision of the obtained shape of the PSF is better. The image of the differences in Figure 15 provides information proving that the maximum of the difference is located in a smaller number of pixels in the central part of the PSF. Thus the model seems to be well described in terms of the shape description. We can conclude that the higher order (6th and 8th) $W_{klm}$ coefficients provide better parameters of the model in terms of the shape description and better localization of aberrations, but sometimes at the cost of a slightly worse absolute difference. Table 4 shows another interesting phenomenon. The results obtained by 8th order fitting are slightly better than the results obtained by 6th order fitting. We observed that the 6th order model aberrations are overfitted; the model adds aberrations which are actually not present. However, use of the 8th order model compensates these overfitting effects. Finally, we have to admit that it is not always necessary to use high order Zernike polynomials, and our results illustrate that a simple solution including only basic aberrations (up to the 4th order) provides results of several percent.

When we compare our method for estimating field-dependent PSF with other approaches mentioned in the introduction, we obtain similar results. Results from all implemented approaches are illustrated in Figure 14. An approach based on work by Piotrowski [1], labeled as the Interpolated model, provide results of the MAXDIFF operator from 7.8% to 9.9% over the FOV. Another approach based on Weddel’s method [23], labeled as the Sectorized model, which divides the image into smaller invariant parts, provides similar results to the Interpolated model. However, it can be seen that the Sectorized model may fail when the PSF model in one sector does not precisely fit to all PSFs inside the sector. This can be seen in Figure 14, where the results for the Sectorized model (marked with a black+) at H = 0.83, 0.88, and 0.92 vary from 8% to 17%. The last method included here is fitting the PSFs with a Gaussian function [46] which was choosen because it can be use as a model of the diffraction limited optical system. This approach provides results from 14% to 22%; however, we had not expected very good results from this method. The Gaussian model was used as a complement to the space-variant methods. We also implemented an approach that uses the Moffat function [46]. However, the results provided by fitting the Moffat function start from 20% and go up to 40%. We therefore concluded that the Moffat model is inappropriate, and we did not include these results in Figure 14.

Our method includes detailed information about the shape of the PSF of the imaging system. Information about the shape of the PSF can be important for PSF fitting photometry in astronomy [47], for deconvolution algorithms in microscopy [48,49] and for other applications. Our model of the UWFOV system was successfully used by Fliegel [50] for a comparison of the deconvolution methods. In this context, a direct estimate of field-dependent Zernike polynomials brings a new approach to the description of space-variant imaging systems. The method was also used for modeling the WILLIAM all-sky camera [51], where the modeling method faced the issue of the presence of the Bayer mask in the system.

7. Conclusions

The proposed method for estimating PSF from an optical system is novel in a direct estimate of optical aberrations of the optical system. In our work, we have used findings reported in previous papers devoted to a complex description of optical systems and their aberrations. It is a difficult task to describe a system of this type, and UWFOV systems further greatly complicate this situation, since they are heavily aberrated. This paper has summarized a complex mathematical approach, and provides an algorithm for modeling the PSF of space-variant optical systems. The proposed algorithm has been verified with simulated data, and has been applied to real image data, showing error of the model around 5%. A comparison of images with different SNR provide MAXDIFF results around 3% for images with SNR greater than 20 dB. The algorithm has been compared with other space-variant modeling methods. It has been shown to be competitive since our results are better than or equal to the results provided by other modeling methods. Other space-variant models provide accuracy around 8.5%. Our results demonstrate that the approach described here is also suitable for UWFOV systems. The results compare models of imaging systems of 4th, 6th, and 8th orders of Zernike polynomials and show some benefits of using different orders. The contribution of this paper to the description of aberration is via a method for obtaining aberration coefficients in an unknown optical system and for using them in the model of space-variant PSF. The algorithm was used to model the WILLIAM system [51], and the model provided by the approach described here was used in a comparison [50] of deconvolution methods. Further research should address pixel profile sensitivity and sampling methods aimed at improving the way in which the model describes sensor PSF.