Modeling Signal-to-Noise Ratio of CMOS Image Sensors with a Stochastic Approach under Non-Stationary Conditions

Abstract

A stochastic model for characterizing the conversion gain of Active Pixel Complementary metal–oxide–semiconductor (CMOS) image sensors (APS), assuming stationary conditions was recently presented in this journal. In this study, we extend the stochastic approach to non-stationary conditions. Non-stationary conditions occur in gated imaging applications. This new stochastic model, which is based on fundamental physical considerations, enlightens us with new insights into gated CMOS imaging, regardless of the sensor. The Signal-to-Noise Ratio (SNR) is simulated, allowing optimized performance. The conversion gain should be determined under stationary conditions.

1. Introduction

Gated imaging is an active imaging technique using a laser light source for seeing objects at desired distances. Gated imaging is suitable for various applications such as defense, security, automotive, and robots for consumer electronics [1,2,3,4,5].

Time-gated measurement is a class of time-of-flight ranging or imaging technologies where a sensor with tightly controlled opening and closing times of the shutter is used in coincidence with a high-power pulsed light source. Signal-to-noise ratio (SNR) is enhanced by time-gated technique since it limits the exposure time of the sensor to the return time of an emitted light pulse from an object at a defined distance r [6,7,8,9,10].

The time-gated technologies may be divided into two distinct classes: single-shot and multi-shot. For example, single-shot light detection and ranging (Lidar) captures the returned light from one single light pulse while multi-shot LIDAR integrates the returned light pulses during several laser periods. Furthermore, the implementation of the time-gated sensing may be based on two different gating modes: software gating and hardware gating.

In this work, we address active gating where the sensor is a CMOS Image sensor, as described in our previous paper [11]. Gating is used for depth measurements and to achieve 3D imaging. Non-stationary conditions occur in such applications. Therefore, the stochastic approach presented in ref [11] needs to be re-defined.

Section 2 presents the basic operation model of the gating approach. Section 3 presents the non-stationary stochastic modeling. The simulation of the signal-to-noise ratio (SNR) evaluated with the non-stationary stochastic approach is presented in Section 4. Section 5 summarizes the results.

2. The Gated Imaging Basic Operation Model

The basic principle of gated imaging was described earlier for various applications [12,13]. A camera and a pulsed laser are placed close to each other. The idea is to send a pulse of light and hold the camera in the off position until an optical echo from a specific range (r) returns to the camera, as shown in Figure 1.

The advantages of gated imaging are obvious. Photons, traveling at the speed of light, that are reflected at different ranges, arrive at different points in time. The imaging sensor or camera only “opens the gate” (i.e., starts the exposure) after a certain time delay for a very short period. Therefore, the sensor is not affected by scattered photons or parasitic light sources. Only the photons that arrive within the right period contribute to the resulting image. The time delay (t) determines the depth measurement range according to r = ct/2, where c is the speed of light. Therefore, the resulting image consists of information only from reflected photons at the distance of interest.

Active gated-imaging technology incorporates range-gating technology, based on multiple “Time-of-Flight” events per single read-out image frames in the sensor. Active gating technology combines two key components: a pulsed light source (i.e., Vertical Cavity Surface Emitting Laser (VCSEL)) and a specially designed gated sensor that exposes and blocks light at high speeds of the order of nanoseconds up to microseconds. Each reflected light source pulse is accumulated in the gated sensor based on a specific “Time-of-Flight” event. Once a desired signal (corresponding to a desired depth-of-field) is accumulated in the sensor, the image is read out. This method provides high SNR with relatively low peak power illumination.

A system employing such a gated timing is shown in Figure 1.

Figure 2 demonstrates pulse propagation from and towards the sensor. For the sake of clarity, we assume that the laser pulse as well as the gating time are rectangular. Real laser pulses are Gaussian, and the edges could be described by the error function. This would just make the expressions in Section 3 more complicated, without getting new physical insight.

3. Mean and Variance of the Output Voltage in Non-Stationary Regimes

3.1. Mathematical Setup

The relations between the mean and variance of the random process representing the output voltage obtained for an arbitrary time-varying illumination are discussed here.

The equivalent electrical circuit of the photodiode assumed here is described in Figure 3 and composes a random current source $I\left(t\right)$ that injects electrons into the output impedance of the photodiode, namely, a capacitor $\mathrm{C}$ in parallel with a resistor $\mathrm{R}$, both assumed to be constant.

For the non-stationary case, the random current injected by the photodiode is described as:

$$I(t)=q_0\sum _{k=-\infty }^{\infty }{X}_k\cdot \delta (t-t_k)$$

(1)

where $q_0$ is the electronic charge, $\delta (\cdot )$ the Dirac function, $\left\{t_k\right\}$ the center points of narrow and non-overlapping time intervals denoted by $\left\{\mathsf{\Delta }{t}_k\right\}$, and that overall covers the entire time under inspection. The random reality is here introduced by a series of mutually independent random variables $\left\{X_k\right\}$, where each random variable $X_k$ is equal to the number of injected electrons within the corresponding time interval $\mathsf{\Delta }{t}_k$. When $\mathsf{\Delta }{t}_k\to 0$ $X_k$ one may assume only the values 1 or 0, where the probability for the value 1 is determined by an underlying time-function (commonly known as the Illumination function) $\varphi \left(t\right)$, such that for $\mathsf{\Delta }{t}_k\to 0$, $P\left\{X_k=1\right\}=\varphi \left(t_k\right)\cdot \mathsf{\Delta }{t}_k$ while $P\left\{X_k=0\right\}=1-\varphi \left(t_k\right)\cdot \mathsf{\Delta }{t}_k$.

Hence $E\left\{X_k\right\}=E\left\{X_k^2\right\}=\varphi \left(t_k\right)\cdot \mathsf{\Delta }{t}_k$, so that $\varphi \left(t\right)$ represents the expected number of electrons injected by the photodiode per unit time, owing to some external light source/sources. Since the number of injected electrons within non-overlapping time intervals are independent, it is apparent that when $\mathsf{\Delta }{t}_k\to 0$ and $\mathsf{\Delta }{t}_j\to 0$,

$$E\left\{\left(X_k-\varphi \left(t_k\right)\Delta t_k\right)\cdot \left(X_j-\varphi \left(t_j\right)\mathsf{\Delta }{t}_j\right)\right\}\to \left\{\begin{array}{c}\begin{array}{cc}E\left\{X_k^2\right\}=\varphi \left(t_k\right)\mathsf{\Delta }{t}_k& if j=k\end{array}\\ \begin{array}{cc}0& if j\ne k\end{array}\end{array}\right\}$$

(2)

The temporal voltage across the photodiode can be represented as the response of the current charging the junction capacitance, using a typical capacitor charging response function, being charged at a random time $T_k$ with $\alpha ={\left(RC\right)}^{-1}\left[{\mathrm{s}}^{-1}\right]$ and $u\left(t\right)$ the unit step function:

$$h\left(t\right)=\frac{q_0}{C}\cdot u\left(t\right)\cdot \exp (-\alpha \cdot t)$$

(3)

It follows from Equations (1) and (3) that the temporal voltage across the photodiode output can be represented as a random process:

$$V(t)=\frac{1}{C}{\int }_{-\infty }^{t}I\left(\vartheta \right)\ast h\left(\vartheta \right)\cdot d\vartheta =\frac{q_0}{C}\sum _{k=-\infty }^{\infty }{X}_k\cdot u(t-t_k)\cdot \mathit{exp}\{-\alpha \cdot (t-t_k)\}$$

(4)

The accumulating time-integral of this voltage $S\left(t\right)$ is the random process:

$$S(t)={\int }_{-\infty }^{t}V\left(\vartheta \right)\cdot d\vartheta =q_{0}R\cdot \sum _{k=-\infty }^{\infty }{X}_k\cdot u(t-t_k)\cdot \left[1-\mathit{exp}\{-\alpha (t-t_k)\}\right]$$

(5)

Equations (4) and (5) describe two different cases of voltage measurement. Equation (4) describes the temporal voltage measurement (case 1) while Equation (5) describes the integrated voltage measurement (case 2).

3.2. The Mean and Variance of $V\left(t\right)$ and $S\left(t\right)$

The mean and variance of both $V\left(t\right)$ and $S\left(t\right)$ are important for analyzing the performance of the measurement. Since $E\left\{X_k\right\}=\varphi \left(t_k\right)\cdot \mathsf{\Delta }{t}_k$, using Equation (4), the mean of $V\left(t\right)$ is:

$$\begin{array}{c}{m}_V(t)=E\{V(t)\}=\frac{q_0}{C}{\displaystyle \sum _{k=-\infty }^{\infty }}\varphi \left(t_k\right)\cdot \mathsf{\Delta }{t}_k\cdot u(t-t_k)\cdot \mathit{exp}\{-\alpha (t-t_k)\}\stackrel{\hspace{1em}\mathsf{\Delta }{t}_k\to 0\hspace{1em}}{\to }\\ \frac{q_0}{C}{\int }_{-\infty }^{\infty }\varphi \left(\vartheta \right)\cdot u(t-\vartheta )\cdot \mathit{exp}\{-\alpha (t-\vartheta )\}\cdot d\vartheta =\\ =\frac{q_0}{C}\cdot \mathit{exp}\{-\alpha t\}\cdot {\int }_{-\infty }^t\varphi \left(\vartheta \right)\cdot \mathit{exp}\{\alpha \vartheta )\}\cdot d\vartheta \end{array}$$

(6)

Likewise, using Equation (5), the mean of $S\left(t\right)$ is:

$$m_S(t)=E\left\{S\left(t\right)\right\}=q_{0}R\cdot \sum _{k=-\infty }^{\infty }\varphi \left(t_k\right)\cdot \mathsf{\Delta }{t}_k\cdot u(t-t_k)\cdot \left[1-\mathit{exp}\{-\alpha (t-t_k)\}\right]$$

(7)

Which, upon letting $\mathsf{\Delta }{t}_k\to 0$, becomes:

$$\begin{array}{c}{m}_S\left(t\right)=E\left\{S\left(t\right)\right\}=q_{0}R\cdot {\int }_{-\infty }^{\infty }\varphi \left(\vartheta \right)\cdot u(t-\vartheta )\cdot \left[1-\mathit{exp}\{-\alpha (t-\vartheta )\}\right]\cdot d\vartheta =\\ =q_{0}R\cdot {\int }_{-\infty }^t\varphi \left(\vartheta \right)\cdot \left[1-\mathit{exp}\{-\alpha (t-\vartheta )\}\right]\cdot d\vartheta =\\ =q_{0}R\cdot \mathit{exp}\{-\alpha t\}\cdot {\int }_{-\infty }^t\varphi \left(\vartheta \right)\cdot \left[\mathit{exp}\{\alpha t\}-\mathit{exp}\{\alpha \vartheta \}\right]\cdot d\vartheta \end{array}$$

(8)

Next, using Equation (2), the variance of $V\left(t\right)$ can be expressed as:

$$\begin{gathered} {\sigma }_V^2\left(t\right)=E\left\{{\left[V\left(t\right)-m_V\left(t\right)\right]}^2\right\}= \\ =E\left\{{\left[\frac{q_0}{C}\sum _{k=-\infty }^{\infty }\left(X_k-\varphi \left(t_k\right)\cdot \Delta t_k\right)\cdot u(t-t_k)\cdot \mathit{exp}\{-\alpha (t-t_k)\}\right]}^2\right\}= \\ ={\left(\frac{q_0}{C}\right)}^2\cdot \sum _{j,k=-\infty }^{\infty }E\left\{\left(X_j-\varphi \left(t_j\right)\cdot \Delta t_j\right)\cdot \left(X_k-\varphi \left(t_k\right)\cdot \Delta t_k\right)\right\}\cdot u(t-t_j)\cdot u(t-t_k)\cdot \mathit{exp}\{-\alpha (2t-t_j-t_k)\}= \\ ={\left(\frac{q_0}{C}\right)}^2\cdot \sum _{k=-\infty }^{\infty }\varphi \left(t_k\right)\cdot \Delta t_k\cdot u(t-t_k)\cdot \mathit{exp}\{-2\alpha (t-t_k)\}\stackrel{\hspace{1em}\Delta t_k\to o\hspace{1em}}{\to } \\ {\left(\frac{q_0}{C}\right)}^2\cdot {\int }_{-\infty }^{\infty }\varphi \left(\vartheta \right)\cdot u(t-\vartheta )\cdot \mathit{exp}\{-2\alpha (t-\vartheta )\}\cdot d\vartheta = \\ ={\left(\frac{q_0}{C}\right)}^2\cdot \mathit{exp}\{-2\alpha t\}{\int }_{-\infty }^t\varphi \left(\vartheta \right)\cdot \mathit{exp}\{2\alpha \vartheta \}\cdot d\vartheta \end{gathered}$$

(9)

Likewise, from Equations (2), (5) and (7) the variance of $S\left(t\right)$ is:

$$\begin{array}{c}{\sigma }_S^2\left(t\right)=E\left\{{\left[S\left(t\right)-m_S\left(t\right)\right]}^2\right\}=\\ ={\left(\frac{q_0}{C}\right)}^2\cdot {\displaystyle \sum _{j,k=-\infty }^{\infty }}E\left\{\left(X_j-\varphi \left(t_j\right)\right)\cdot \left(X_k-\varphi \left(t_k\right)\right)\right\}\cdot u\left(t-t_j\right)\cdot u\left(t-t_k\right)\cdot \left[1-e^{-\alpha \left(t-t_j\right)}\right]\cdot \left[1-e^{-\alpha \left(t-t_k\right)}\right]=\\ ={\left(\frac{q_0}{C}\right)}^2\cdot {\displaystyle \sum _{k=-\infty }^{\infty }}\varphi \left(t_k\right)\cdot \Delta t_k\cdot u\left(t-t_k\right)\cdot [1-e^{-\alpha \left(t-t_k\right)}{]}^2\stackrel{\hspace{1em}\Delta t_k\to 0\hspace{1em}}{\to }\\ {\left(\frac{q_0}{C}\right)}^2\cdot {\int }_{-\infty }^{\infty }\varphi \left(\vartheta \right)\cdot u(t-\vartheta )\cdot [1-e^{-\alpha \left(t-\vartheta \right)}{]}^2\cdot d\vartheta \\ ={\left(\frac{q_0}{C}\right)}^2\cdot {\int }_{-\infty }^t\varphi \left(\vartheta \right)\cdot [1-e^{-\alpha \left(t-\vartheta \right)}{]}^2\cdot d\vartheta \end{array}$$

(10)

3.3. Signal-to-Noise Ratio of V(t) and S(t)

The ratio ${\left(\frac{S}{N}\right)}_V=\frac{m_V\left(t\right)}{{\sigma }_V\left(t\right)}$ and ${\left(\frac{S}{N}\right)}_S=\frac{m_S\left(t\right)}{{\sigma }_S\left(t\right)}$ are dimensionless quantities that have special significance to the qualities of the measurement. From the expressions presented above it follows that for case 1:

$${\left(\frac{S}{N}\right)}_V=\frac{m_V\left(t\right)}{{\sigma }_V\left(t\right)}=\frac{{\int }_{-\infty }^t\varphi \left(\vartheta \right)\cdot \mathit{exp}\left\{\alpha \vartheta \right\}\cdot d\vartheta }{{\left[{\int }_{-\infty }^t\varphi \left(\vartheta \right)\cdot \mathit{exp}\left\{2\alpha \vartheta \right\}\cdot d\vartheta \right]}^{1/2} }$$

(11)

Hence, in the special case where the illumination function is a step function $\varphi \left(t\right)={\varphi }_0\cdot u\left(t\right)$,

$$\begin{gathered} {\left(\frac{S}{N}\right)}_V=\frac{m_V\left(t\right)}{{\sigma }_V\left(t\right)}=\frac{{\int }_0^t{\varphi }_0\cdot \mathit{exp}\left\{\alpha \vartheta \right\}\cdot d\vartheta }{{\left[{\int }_0^t{\varphi }_0\cdot \mathit{exp}\left\{2\alpha \vartheta \right\}\cdot d\vartheta \right]}^{1/2}}={\left[\frac{2{\varphi }_0}{\alpha }\cdot \frac{\mathit{exp}(\alpha t)-1}{\mathit{exp}(\alpha t)+1}\right]}^{1/2} \\ ={\left[\frac{2{\varphi }_0}{\alpha }\cdot \tanh \left(\frac{\alpha t}{2}\right)\right]}^{1/2} \end{gathered}$$

(12)

In principle, the signal-to-noise ratio increases with the duration of the measurement but approaches the limiting value ${\left(2{\varphi }_0\cdot {\alpha }^{-1}\right)}^{1/2}$ when $1\ll \alpha t$.

When the measured quantity is $S\left(t\right)$ (case 2) the signal-to-noise ratio is:

$${\left(\frac{S}{N}\right)}_S=\frac{m_S\left(t\right)}{{\sigma }_S\left(t\right)}=\frac{{\int }_{-\infty }^t\varphi \left(\vartheta \right)\cdot \left[1-e^{-\alpha (t-\vartheta )}\right]\cdot d\vartheta }{{\left[{\int }_{-\infty }^t\varphi \left(\vartheta \right)\cdot {\left[1-e^{-\alpha \left(t-\vartheta \right)}\right]}^2\cdot d\vartheta \right]}^{1/2}}$$

(13)

Which, for the above step function illumination is:

$$\begin{gathered} {\left(\frac{S}{N}\right)}_S=\frac{m_S\left(t\right)}{{\sigma }_S\left(t\right)}=\frac{{\int }_0^t{\varphi }_0\cdot \left[1-e^{-\alpha (t-\vartheta )}\right]\cdot d\vartheta }{{\left[{\int }_0^t{\varphi }_0\cdot {\left[1-e^{-\alpha \left(t-\vartheta \right)}\right]}^2\cdot d\vartheta \right]}^{1/2}}= \\ =\sqrt{{\varphi }_0}\cdot \frac{{\int }_0^t\left[1-e^{-\alpha (t-\vartheta )}\right]\cdot d\vartheta }{{\left[{\int }_0^t{\left[1-e^{-\alpha \left(t-\vartheta \right)}\right]}^2\cdot d\vartheta \right]}^{1/2}}=\sqrt{{\varphi }_0}\cdot \sqrt{\frac{2\cdot {\left[t-{\alpha }^{-1}(1-e^{-\alpha t})\right]}^2}{\left[2t-4{\alpha }^{-1}(1-e^{-\alpha t})+{\alpha }^{-1}(1-e^{-2\alpha t})\right]}} \end{gathered}$$

(14)

In this case the signal-to-noise ratio increases with the measurement duration, however, in this case without limit and for $1\ll \alpha t$ it becomes asymptotically equal to ${\left({\varphi }_0\cdot t\right)}^{1/2}$.

4. Numerical Modeling and Simulation

The equivalent electrical network in Figure 3 represents the capacitor C in which photo-carriers are being stored, while its parallel resistor R represents the equivalent resistance. An illumination pulse is transmitted towards an object, reflected from it, and then captured by the camera’s photodiode, where it is transformed into photo-carriers and stored in capacitor C. The illumination pulse shape presented in our model is the pulse reflected from the object. Measuring the voltage can be completed in two main approaches, described earlier as case 1, where temporal voltage is being measured, and case 2, where an integrated voltage is being measured.

The model divides the pulse into small time intervals $\mathsf{\Delta }{t}_k$, such that in every time interval there is a probability for creation of a single photo-carrier, or none. The probability for the creation of more than a single photo-carrier in time interval $\mathsf{\Delta }{t}_k\to 0$ is negligible. The model is time-dependent and represents the SNR value at the time within the integration time. Time t = 0 represents the shutter opening. It is assumed that the time t is smaller than the exposure time of the frame.

The first method (case 1) considers the temporal voltage on the capacitor during the measuring time. This method applies when the illumination pulse is very narrow, so that in a short period of time, many photo-carriers are collected in the photodiode. This method can be used in SPADs (Single Photon Avalanche Diode) and Silicon multipliers, where the firing is detected “instantly” [6].

The second method (case 2) is accumulative, which integrates the voltage on the capacitor during the integration time. This method applies for wider pulses in time. In this case, the photodiode acts as a converter of photons to photo-carriers as well as a capacitor storing the photo-carriers, until they are transferred and read by the Pixel’s Source Follower. Therefore, in this case, the accumulative voltage should be considered in the SNR calculations.

4.1. Numerical Modeling of the Results

The uniqueness of the suggested model is that it allows SNR analysis for any system with the same electrical network as shown in Figure 3, for any given illumination function $\varphi \left(t\right)$. In Section 3, general expressions for SNR for both methods are calculated in Equations (11) and (13). The special case where the illumination function is a step function is also calculated in Equations (12) and (14).

For typical values of $R=100\left[\mathrm{M}\mathsf{\Omega }\right]$ [14] and $C=1\left[\mathrm{f}\mathrm{F}\right]$ [11], the resulting $\alpha =1/RC$ value is ${10}^7 \left[{\mathrm{s}}^{-1}\right]$. For example, to measure depth in a range of up to 10 m, the pulse travelling time is $t=2\cdot 10\left[\mathrm{m}\right]/3\cdot {10}^8\left[\frac{\mathrm{m}}{\mathrm{s}}\right]=66\left[\mathrm{n}\mathrm{s}\right]$, resulting in $\alpha t<1$. From Figure 4a, showing the SNR values for measuring times of up to 70 [ns], it is noticeable that the SNR values are higher in the temporal voltage method than the accumulative method, as expected due to the short-time pulse characterization of this method. For longer measurement times, meeting the condition $\alpha t\gg 1$, as seen in Figure 4b, the temporal method SNR values converge into a constant value whereas in the accumulative method the SNR values keep its square root trend, resulting in a higher SNR value, meaning that the accumulative method is more suitable for longer measurement times—the higher the measuring time is, the more photo-carriers are created in the photodiode, resulting in a higher SNR.

4.2. Monte Carlo Simulations

To validate the model results, a Monte Carlo simulation [15] was performed on the model described in Section 3. The simulation was composed of 1000 realizations, each randomizing the total amount of photo-carriers injected into the model according to Poisson distribution and the time in which each of the photo-carriers were injected into the model. The model response was then calculated for both temporal and accumulation methods according to Equations (3)–(5). Mean and variance values over all the realizations were calculated, from which SNR values were extracted. The Monte Carlo simulation results are shown in Figure 5. These results match the numerical modeling detailed in Section 4.1 and shown in Figure 4b.

5. Summary

This study presents a new stochastic model for the case of gated imaging, where non-stationary conditions prevail. The model is based on fundamental physical assumptions. In an earlier paper we analyzed conversion gain of CMOS image sensors based on a stochastic approach. The conversion gain of an imager is best analyzed under stationary conditions. However, in the case of gated imaging, due to the non-stationary nature of the measuring conditions, the signal-to-noise ratio requires extending this approach. The new model is validated using a Monte Carlo simulation.

Improvements in efficiency and size shrinkage of laser sources, as well as technological advances in CMOS sensor technology, in particular CMOS SPAD, have brought the laser-gated imaging to be an established and mature technology. A laser pulse illuminates the scene, functioning as a source of light. The laser’s photons travel towards the object and then some of them are reflected towards the CMOS image sensor. The basic principle of gated imaging is to start the sensor’s exposure only when the reflected photons should return to the sensor, minimizing the sensing time of the photons as much as possible, preventing sensing of parasitic light sources and scattered photons. The gating time sets the depth of view of the scene.

Since in gated imaging the exposure is taking place after a delay from the pulse sending and for a short period of time, the stationary assumption of the earlier paper was revisited, and the SNR value was calculated under non-stationary conditions. The SNR can be measured in two different methods. The first method is based on temporal voltage, which is applied for very short illumination pulses. The second method is based on the accumulated, namely integrated voltage. The second method produces higher SNR values for longer measurement periods, while the first method produces higher SNR values for short measuring.

It should be noted that gating reduces only the background noise. The noise contributed by the sensor is integrated and in fact limits the useful gating time. This contribution is not considered in this study, in order to let the reader get used to the concepts of the modeling.