**A Highly Linear CMOS Image Sensor Design Based on an Adaptive Nonlinear Ramp Generator and Fully Di**ff**erential Pipeline Sampling Quantization with a Double Auto-Zeroing Technique**

#### **Chuangze Li 1, Benguang Han 2, Jie He 1, Zhongjie Guo <sup>3</sup> and Longsheng Wu 1,\***


Received: 28 January 2020; Accepted: 12 February 2020; Published: 14 February 2020

**Abstract:** For a complementary metal-oxide-semiconductor image sensor with highly linear, low noise and high frame rate, the nonlinear correction and frame rate improvement techniques are becoming very important. The in-pixel source follower transistor and the integration capacitor on the floating diffusion node cause linearity degradation. In order to address this problem, this paper proposes an adaptive nonlinear ramp generator circuit based on dummy pixels used in single-slope analog-to-digital converter topology for a complementary metal-oxide-semiconductor (CMOS) image sensor. In the proposed approach, the traditional linear ramp generator circuit is replaced with the new proposed adaptive nonlinear ramp generator circuit that can mitigate the nonlinearity of the pixel unit circuit, especially the gain nonlinearity of the source follower transistor and the integration capacitor nonlinearity of the floating diffusion node. Moreover, in order to enhance the frame rate and address the issue of high column fixed pattern noise, a new readout scheme of fully differential pipeline sampling quantization with a double auto-zeroing technique is proposed. Compared with the conventional readout structure without a fully differential pipeline sampling quantization technique and double auto-zeroing technique, the proposed readout scheme cannot only enhance the frame rate but can also improve the consistency of the offset and delay information of different column comparators and significantly reduce the column fixed pattern noise. The proposed techniques are simulated and verified with a prototype chip fabricated using typical 180 nm CMOS process technology. The obtained measurement results demonstrate that the overall nonlinearity of the CMOS image sensor is reduced from 1.03% to 0.047%, the efficiency of the comparator is improved from 85.3% to 100%, and the column fixed pattern noise is reduced from 0.43% to 0.019%.

**Keywords:** CMOS image sensor; linearity; adaptive nonlinear ramp; fully differential pipeline; double auto-zeroing; high framerate; fixed pattern noise; floating diffusion; readout scheme; ramp generator circuit

#### **1. Introduction**

Highly linear complementary metal-oxide-semiconductor (CMOS) image sensors (CISs) have a wide range of applications in time-of-flight (ToF) ranging, medical imaging, space remote sensing imaging and scientific imaging [1–6]. The most critical element of a CMOS image sensor (CIS) is the column-parallel analog-to-digital converter (ADC). Various architectural designs of column-parallel ADCs have been proposed, including Cyclic-ADC, SAR-ADC, pipeline-ADC, single-slope ADC (SS\_ADC) and multi-slope ADC [7–12]. The single-slope ADC structure is the most popular for column-parallel ADCs in CISs due to its simple circuit topologies and small layout area. The ramp generator circuit and the high-speed comparator in the SS-ADC structure are the most critical modules in determining the performance of the CIS.

Many research studies have been conducted to improve the linearity of CIS. According to the causes of nonlinearity of CISs, the solutions adopted in the literature are categorized into four types. The first type is employing the generated linear ramp signal buffered by the source follower (SF) transistor [8] to eliminate the gain nonlinearity of the SF transistor in pixel. However, the disadvantage of this method is that the nonlinearity of the integration capacitor (*CFD*) caused by the floating diffusion (FD) node still exists. The second type is to utilize the regulating output voltage follower method to calibrate the nonlinearity outside the pixel structure [2,3,13,14]. The third type is utilizing the unit-gain analog buffer instead of the SF in the pixel circuit [1–4] to reduce the gain nonlinearity of the SF transistor. However, this method increases the pixel area and reduces the fill factor. The fourth type is utilizing the off-chip high-precision ADC and digital-to-analog converter (DAC) to calibrate the nonlinearity of the CMOS image sensor system [1–3,15]. The advantages of this correction method are high linearity and high precision. However, the method is complicated and has a high cost.

The SS-ADC in the conventional CIS has different crossing point voltages and different offset voltages of the column-level comparator. In order to address this issue, a fully differential comparator with pipeline sampling quantization based on the double auto-zeroing (AZ) technique was proposed to improve the frame rate and reduce the column fixed pattern noise (FPN) in the spatial domain. In the literature [6], a classic CIS composed of a programmable gain amplifier (PGA) and an SS-ADC has been proposed for the column-level readout circuit. The advantage of this readout structure is that the dynamic range (DR) of the CIS can be extended, and the gain of the PGA can suppress the noises of the next stages (for example, the readout noise of the SS-ADC). However, the PGA itself has high noise, high power consumption, and the readout circuit sampling and quantization work in sequence without concurrent execution. Therefore, the comparator in the SS-ADC will have a fixed idle time, which reduces the efficiency of the comparator. The general readout scheme leads to the problem of large column FPN caused by different comparator crossing voltage levels. In order to address the above-mentioned problems, this paper proposes an adaptive nonlinear ramp generator circuit design and a readout circuit with fully differential pipeline sampling quantization based on the double AZ technique.

The remainder of this paper is organized as follows. Section 2 introduces the architecture of highly linear CIS. Section 3 describes the proposed adaptive nonlinear ramp generator technique and the fully differential pipeline sampling quantization based on the double AZ technique. The principle analysis of highly linear CIS with linear and nonlinear ramp generators is presented in Section 4. The simulation and experimental results of the fabricated CIS are discussed in Section 5, followed by conclusions in Section 6.

#### **2. Image Sensor Architecture**

The CIS proposed in this paper consists of a typical 5T pixel [6] array, an adaptive nonlinear ramp generator, a readout circuit with fully differential pipeline sampling quantization based on the double AZ technique, row/column decoder and driver, timing sequence controller, phase-locked loop (PLL), charge pump, temperature sensor, and high-speed, high-precision, low-power LVDS serial data transfer circuit. The overall architecture of the proposed highly linear CIS is shown in Figure 1.

**Figure 1.** System architecture of the proposed highly linear complementary metal-oxide-semiconductor (CMOS) image sensor.

The central part of Figure 1 is a 2560 × 3072 active pixel array. The active pixel array is distributed around the dummy pixel. The dummy pixel can be used to generate an integration capacitor for an adaptive nonlinear ramp that is highly consistent with the characteristics of floating diffusion node capacitance. The left part of Figure 1 includes bandgap reference voltage generator circuit and current bias circuit, charge pump module, PLL module, timing sequence driving circuit, serial peripheral interface (SPI) and other modules. The right part of Figure 1 includes the temperature sensor, design for testability (DFT) circuit and the right row decoder driver circuit.

#### **3. Proposed Techniques**

The essential blocks of column-parallel SS-ADC in CIS is the ramp generator circuit and the high-speed readout circuit. This paper proposes an adaptive [16,17] nonlinear ramp generator and a fully differential pipeline sampling quantization scheme based on the double AZ technique. The two proposed techniques are dedicated for enhancing the performance of system linearity, improving the frame rate and reducing the column FPN of the CIS.

#### *3.1. Nonlinear Ramp Generation Technique Based on Dummy Pixel Array*

Figure 2 shows the system signal process flow diagram of CIS using the nonlinear ramp generation

technique based on the dummy pixel and fully differential pipeline sampling quantization technique. Figure 3a,b shows the system digital number (DN) outputs using the linear and the nonlinear

ramp generators, respectively. A typical SS-ADC utilizes the linear ramp generation technique, which cannot eliminate the nonlinearity of the pixel. The proposed nonlinear ramp generation technique based on dummy pixels can not only eliminate the gain nonlinearity of the SF, which changes with the input voltage but can also reduce the nonlinearity of *CFD* in the FD node.

**Figure 2.** System signal process flow diagram of a highly linear CMOS image sensor based on nonlinear ramp generator.

**Figure 3.** System digital number (DN) output values with linear and nonlinear ramp generators: (**a**) system DN output with linear ramp generator; (**b**) system DN output with a nonlinear ramp generator.

The proposed technique utilizes the multiple dummy pixel units surrounded by active pixel array, which are connected in parallel to form the *CFD* of the ramp generator circuit. That is, the equivalent *CFD* is generated by the parallel connection of multiple FD nodes of pixel units, and then a discrete sampling negative feedback technique is utilized to adaptively adjust the level of the current source. This technique produces an adaptively current source that creates a discharge path between the FD node's capacitance and the ground, thereby generating a nonlinear ramp signal. Then the nonlinear signal is buffered by the SF transistor that has the same characteristics as the active pixel. The buffered ramp signal is then compared with the pixel output voltage by the fully differential pipeline sampling

quantization comparator based on the double auto-zeroing technique. When the sampled active pixel output voltage signal *VPIX* is equal to the nonlinear ramp signal *Vramp*, the comparator toggles while the counter stops counting and stores the counter value into the static random-access memory (SRAM). Figure 4 shows the schematic diagram of a nonlinear ramp using the dummy pixel based on a typical voltage mode 5T pixel structure.

**Figure 4.** Schematic diagram illustrating the implementation of the integration capacitor by a floating diffusion node based on a dummy pixel array.

The schematic diagram of the adaptive nonlinear ramp generator based on the dummy pixel array is shown in Figure 5. The left part of Figure 5 shows the discrete negative feedback control circuit that generates the corresponding tail current source of nonlinear ramp generator. The right part of Figure 5 shows the circuit of *CFD* based on a dummy pixel. The *CFD* produced by the dummy pixel unit and the operational amplifier circuit constitutes the entire nonlinear ramp generator circuit, which is sampled and corrected by the adaptive discrete negative feedback control technique. The nonlinearity characteristic of the generated ramp signal is the same as the output of the pixel unit. Therefore, this method can reduce the nonlinearity caused by the capacitance of the FD node and SF's variable gain in the pixel. Hence, the system linearity of the CIS is improved.

In the 5T pixel, the row selection transistor and the transfer gate transistor (including EC and TG) are connected to the power supply and analog ground, respectively, while the reset transistor NM2 is controlled by the timing signal (Ramp\_Adj\_P).

The timing of the adaptive nonlinear ramp generator is divided into three phases. During the initial phase, both the pulse Ramp\_Init (driving the switch K9) and the Ramp\_RST (driving the switch K6–K9) are set low in the initial state of the adaptive nonlinear ramp generator circuit, while both are held high for other phases. The non-overlapped clocks Ramp\_Adj\_P and Ramp\_Adj\_N are driving the switches K1–K5 and NM2 to correct and generate the ramp, respectively. The second phase adaptively corrects the slope of the ramp signal. The third phase is the ramp generation phase. The timing diagram of the adaptive nonlinear ramp generator circuit is shown in Figure 6.

**Figure 5.** Schematic diagram of adaptive nonlinear ramp generator based on a dummy 5T pixel.

**Figure 6.** Timing diagram of adaptive nonlinear ramp generator.

#### *3.2. Fully Di*ff*erential Pipeline Sampling Quantization Based on Double Auto-Zeroing Technique*

Generally, the readout circuit for the CMOS image sensor consists of a programmable gain amplifier (PGA) and an SS-ADC. The sampling and the quantization phases are implemented by the PGA module and the SS-ADC, respectively. The sampling and the quantization phases usually work in sequential order. One disadvantage of this design used in the classic CIS is the inefficient use of the SS-ADC. Another disadvantage is that the comparator of SS-ADC has different crossing common-mode voltages that cause the offset and the delay information to vary in different columns, resulting in a large column FPN. In order to overcome the shortcomings of sampling and quantization phases, which cannot operate simultaneously for conventional SS-ADC, a readout circuit of fully differential pipeline sampling quantization based on a double auto-zeroing technique is proposed in this section. The proposed circuit realizes sampling and quantization executions concurrently to improve the frame rate and reduce the column FPN.

In the proposed design, a fully differential analog comparator with pipeline sampling quantization based on the double AZ technique is employed to improve the frame rate and reduce the influence of column FPN on the performance of CIS. Figure 7 shows the double AZ technique comparator circuit for fully differential pipeline sampling.

**Figure 7.** Fully differential comparator with pipeline sampling quantization based on the double auto-zeroing technique.

In Figure 7, *CR* and *CS* are the reset and the signal sampling capacitors, respectively. S1 and S2 represent the sampling switch control signals in the signal and the reset phases, respectively. S1B, S1BN, and S2B, S2BN are the control switches related to the sampling reset and signal, respectively. Clock signal PHI1, PHI2 and PHI2B are utilized to realize double reset operation in the quantization phase. Where PHI2 and PHI2B are driven by a pair of non-overlapping clocks, and PHI2B is a control signal of the comparator in the reset phase ahead of PHI2 in phase by a constant time. Figure 8 shows a detailed working timing diagram for the fully differential comparator with pipeline sampling quantization based on the double AZ technique.

**Figure 8.** Working timing diagram of fully differential comparator with pipeline sampling quantization based on the double auto-zeroing technique.

The column-parallel readout timing of the CIS is illustrated in Figure 9, and the whole frame time is defined as:

$$T\_{frame} = T\_{FOT} + T\_{nvw} \times N\_{nvw} \tag{1}$$

where *TFOT* is the frame overhead time, *Trow* is the row readout time, and *Nrow* is the number of rows of CIS. For the readout circuit structure of sequential sampling quantization, the time of row readout can be defined as:

$$T\_{\text{row\\_scq}} = T\_{\text{sample}} + \max(T\_{\text{ADC\\_\prime}} T\_{\text{LVDS}}) \tag{2}$$

where *Trow\_seq* is the row sequential sampling time, *Tsample* is the row sampling time, *TADC* is the ADC conversion time and *TLVDS* is the LVDS output time. According to the column-parallel pipeline signal processing, the row readout time can be expressed as:

$$T\_{vww} = \max(T\_{sample}, T\_{ADC\_\prime}, T\_{LVDS})\tag{3}$$

**Figure 9.** Readout timing of the column-parallel signal processing.

It can be observed from the above equations that the pipeline sampling structure will reduce the row readout time. Thus, the frame rate will be improved.

The proposed readout circuit combines the advantages of both pipeline sampling quantization and double AZ techniques. The proposed method saves two sampling times (the time of the correlated double sampling), avoids the waiting time as in the conventional readout scheme, increasing the efficiency of the comparator up to 100%, and improves the frame rate from 75 to 86 fps under the clock frequency of 400 MHz compared with the classic sequential sampling and quantization method. Since a double reset operation is performed in the quantization phase (double AZ technique), the values of signal and reset phases are all at the same constant crossing detector voltage level. Thus, the column FPN is significantly reduced and the inter-columns consistency of the CIS is effectively improved.

#### **4. Analysis of Proposed Techniques**

This section presents the analysis of the eliminating principle of nonlinearity when using the linear and the nonlinear ramp generators. Usually, the principle of a typical SS-ADC structure consisting of a high-speed comparator, a ramp generator circuit, and a digital counter is implemented by converting the counter time into the digital code, assuming that the photocurrent of the pinned photodiode is highly linear to the illumination intensity [1–3]. Equivalently, the analysis in this section uses time *t* to represent the digital output code of SS-ADC while conducting the relationship between the digital output code of an SS-ADC and the photocurrent when exploring both the linear and the nonlinear ramp generators.

#### *4.1. Linearity Analysis of CMOS Image Sensor with Linear Ramp*

Using a linear ramp, the ramp output voltage *Vramp* is defined as:

$$V\_{ramp} = \frac{I\_{ramp}}{C} \times t\tag{4}$$

where *Iramp* is the discharging current of the ramp circuit, the integral capacitance *C* is a constant value, and *t* is the integration time of the ramp circuit. Since the illumination intensity changes linearly based on the above assumption, the output photocurrent of the pinned photodiode *Ipd* is also changing

linearly. When the voltage of the FD node voltage discharges for an integration time *T*int with the photocurrent *Ipd*, the resulting voltage *VFD* is defined as:

$$V\_{FD} = \frac{I\_{pd}}{C\_{FD}(V\_{FD})} \times T\_{\text{int}} \tag{5}$$

where, *CFD*(*VFD*) is the total parasitic capacitance of the FD node, which is related to the voltage of the FD node, and is also one of the nonlinear causes of the pixel. The complete capacitance of the FD node is defined as [2,3]:

$$\mathcal{C}\_{\text{FD}} = \mathcal{C}\_{\text{RST\\_}V} + \mathcal{C}\_{\text{TX\\_}V} + \mathcal{C}\_{\text{SF\\_}V} + \mathcal{C}\_{\text{FD\\_VERTClassL}} + \mathcal{C}\_{\text{METAL}} \tag{6}$$

where, *CMETAL*, *CTX\_OV*, and *CRST\_OV* are the parasitic capacitances related to the size of the metal wire, the TX transistor and the reset transistor, respectively. Once the sizes of the TX and the reset transistors in the pixel are determined, the corresponding parasitic capacitances will remain unchanged, while *CFD\_VERTICAL*, and *CSF\_OV* are related to the overlap parasitic capacitances of the floating diffusion node that changes with the voltage of the FD node. The output voltage of pixel *VPIX* is equal to the voltage of the floating diffusion node buffered by the SF, as shown below [2,3]:

$$V\_{PIX} = \frac{I\_{pd}}{C\_{FD}(V\_{FD})} \times T\_{\text{int}} \times G\_{SF}(V\_{FD}) \tag{7}$$

When the comparator of a single-slope ADC utilizes a linear ramp, the comparator toggles if the voltage *Vramp* is equal to *VPIX*, so depending on time *t*, Equation (5) can be obtained as:

$$t = \frac{I\_{pd}}{I\_{ramp}} \times \frac{\mathbb{C}}{\mathbb{C}\_{FD}(V\_{FD})} \times G\_{\text{SF}}(V\_{FD}) \times T\_{\text{int}} \tag{8}$$

where, *GSF*(*VFD*) is the gain of SF in the pixel. It can also be seen from Equation (8) that, due to the nonlinearity of integral capacitance of the FD node and the gain nonlinearity of the SF, nonlinearity exists between the integration time *t* and the photocurrent *Ipd* when *T*int is constant. That is, the integration time of the linear ramp generator has a nonlinear relationship with the integration time of the FD node in the pixel is constant (which can be equivalent to a linear change in the integration time when the illumination intensity is constant). It can be further observed that there is a nonlinear relationship between the digital output code of the SS-ADC and the linear ramp generator. Thus, the overall illumination intensity is not linear between the digital code (by representing the time *t*) of the SS-ADC and the illumination intensity *Ipd*. The nonlinear relationship reduces the system linearity of the CMOS image sensor.

#### *4.2. Linearity Analysis of CMOS Image Sensor with Adaptive Nonlinear Ramp*

When using the adaptive nonlinear ramp proposed in this paper, the nonlinear ramp generator voltage is defined as:

$$V\_{ramp} = \frac{I\_{ramp}}{K \times \mathbb{C}\_{FD}(V\_{FD})} \times G\_{SF}(V\_{FD}) \times t \tag{9}$$

where *K* is the scale factor, *Iramp* is the discharging current of the nonlinear ramp, and *CFD*(*VFD*) is the nonlinear capacitance produced by the dummy pixel, which is related to the voltage of the FD node. When the output voltage of pixel *VPIX* is equal to the ramp generator voltage *Vramp*, the output of the comparator is toggled, and the counter stops counting. At this moment, the output value of the digital

counter represents the pixel signal. There is a linear relationship between the digital number (DN) and the ramp integration time *t* when using a nonlinear ramp. The integration time *t* is:

$$t = K \times \frac{I\_{pd}}{I\_{ramp}} \times \frac{\mathbb{C}\_{FD}(V\_{ramp\\_FD})}{\mathbb{C}\_{FD}(V\_{FD})} \times \frac{\mathbb{G}\_{\text{SF}}(V\_{num\\_FD})}{\mathbb{G}\_{\text{SF}}(V\_{FD})} \times T\_{\text{int}} \tag{10}$$

where *Vramp\_FD* is the voltage of the FD node of the adaptive nonlinear ramp generator. According to Equation (10), when the voltage values *Vramp* and *VPIX* are equal, *CFD*(*Vramp\_FD*), *CFD*(*VFD*), *GSF*(*Vramp\_FD*) and *GSF*(*VFD*) are all also equal. Thus, the two variables in Equation (10) cancel each other out, thereby eliminating the nonlinear error caused by the FD node integral capacitance and the gain nonlinearity of the SF transistor. The linear relationship between the DN and the input derived from the above analysis is given as:

$$t = K \times \frac{I\_{pd}}{I\_{ramp}} \times T\_{\text{int}} \tag{11}$$

where *K* is the number of dummy pixel units used for producing the nonlinear ramp integration capacitance.

#### **5. Results**

#### *5.1. Simulation Results of Nonlinear Ramp Generation Circuit Based on Dummy Pixel*

Figure 10 compares the simulation results of nonlinear and linear ramp generators based on the dummy pixel. The red and gray curves represent linear and nonlinear ramp signals, respectively. It can be seen from the simulation results that the nonlinear ramp signal already contains the nonlinear components of the pixel structure, which is the nonlinearity of the *CFD* of the FD node and the gain nonlinearity of the SF.

**Figure 10.** Comparison of nonlinear and linear ramp signals.

Figure 11 shows the nonlinearity of CIS along with the linear change of photocurrent (*Ipd*) when using the linear and the adaptive nonlinear ramp techniques. Figure 11 includes the nonlinearity characteristics of typical CIS using the linear and the adaptive nonlinear ramp techniques. It can be seen from the simulation results that the nonlinearity is significantly reduced between the output digital code and the photocurrent (light intensity). The maximum differential nonlinearity (DNL) is 0.044 LSB, and the maximum integral nonlinearity (INL) is 0.054 LSB.

**Figure 11.** Simulation results of the linearity of CMOS image sensor (CIS) using the linear ramp and the adaptive nonlinear single-slope analog-to-digital converter (ADC) techniques.

#### *5.2. Simulation Results of Fully Di*ff*erential Pipeline Sampling Quantization with Double Auto-Zeroing Technique*

Figure 12 shows the simulation results of the operation timing of the fully differential comparator with pipeline sampling quantization based on the double AZ technique. It is easy to observe the correct function of the proposed fully differential comparator from the operation timing diagram. The crossing detector voltage level of the input positive level VCP is always the same as the negative level VCN for the comparator.

**Figure 12.** Operation timing of the fully differential pipeline sampling quantization with double auto-zeroing technique.

#### *5.3. Experimental Result*

The size of the pixel array in the proposed design is 2560 × 3072. Figure 13 shows the sample image taken by a typical global shutter pixel 5T structure with an adaptive nonlinear ramp generator and the fully differential comparator based on pipeline sampling quantization with the double AZ technique.

**Figure 13.** Sample image taken from the proposed highly linear CMOS image sensor.

Table 1 compares the performance of the highly linear CMOS image sensor proposed in this paper with those reported in the literature.



#### **6. Conclusions**

In this paper, a novel adaptive nonlinear ramp generator design technique based on dummy pixels and a readout scheme of fully differential pipeline sampling quantization with a double auto-zeroing technique are proposed. The proposed ramp generator design can significantly improve the linearity of the pixel, which eliminates the nonlinearity caused by the capacitance of the floating diffusion node of the pixel and the gain nonlinearity of the SF transistor.

The proposed readout circuit improves the frame rate by 14% in the fully differential comparator with the pipeline sampling quantization technique, while increasing the efficiency of the comparator up to 100%. Moreover, the readout chain with the fully differential comparator based on the double auto-zeroing technique reduces the fixed pattern noise to 0.019%. A wide dynamic range of 81.3dB is also achieved by the proposed design without using the column amplifier gain stages. The resulting readout circuit noise is lowered to 7.9e-, and the system linearity is reduced to 0.047% using the adaptive nonlinear ramp technique.

**Author Contributions:** C.L. and B.H. conceived the ideas and innovations, performed the simulation and analyzed the data; J.H. and Z.G. contributed a lot to the principle verification and results and discussion of this paper; L.W. provided supervision and guidance in this work; C.L. wrote the paper. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by National Major Science and Technology Industry Project grant number 2017ZX01006101-001 and Scientific Research Project of the Shaanxi Education Department grant number 19JC029. The APC was funded by National Major Science and Technology Industry Project.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
