Research on Performance Evaluation and Optimization Theory for Thermal Microscope Imaging Systems

Abstract

Infrared imaging theory is an important theoretical basis for the design of infrared imaging systems, but there is no research on infrared imaging theory for designing thermal microscope imaging systems. Therefore, we studied the performance evaluation and optimization theory of thermal microscope imaging systems. In this paper, we analyzed the difference in spectral radiant flux between thermal microscope imaging and telephoto thermal imaging. The expression of signal-to-noise ratio of the output image of the thermal microscope imaging systems was derived, based on the analysis of the characteristics of thermal microscope imaging. We studied the performance evaluation model of thermal microscope imaging systems based on the minimum resolvable temperature difference and the minimum detectable temperature difference. Simulation and analysis of different detectors (ideal photon detector and ideal thermal detector) were also carried out. Finally, based on the conclusion of theoretical research, we carried out a system design and image acquisition experiment. The results show that the theoretical study of thermal microscope imaging systems in this paper can provide reference for the performance evaluation and optimization of thermal microscope imaging systems.

1. Introduction

Infrared imaging technology shows great advantages in target detection and is widely used in agriculture [1], medicine [2], military [3,4], industrial [5,6,7] and other fields [8]. Due to the limitation of the manufacturing process of infrared imaging detectors, the performance of the infrared imaging system needs to be optimized. Therefore, it is necessary to optimize the design of the infrared imaging system according to different application fields [9,10]. The theoretical research of infrared imaging systems is the theoretical basis for the development of infrared imaging technology [11,12]. Among them, the performance evaluation theory can provide theoretical guidance for the design and optimization of the infrared imaging system [13,14]. MRTD (minimum resolvable temperature difference) and MDTD (minimum detectable temperature difference) have been important parameters for evaluating the performance of infrared imaging systems, mainly because they can quantify the temperature resolution capability of infrared imaging systems [15,16,17,18,19]. MRTD is defined as a standard four-bar blackbody target pattern with a certain spatial frequency and a height ratio of 7:1 in a uniform blackbody background (as shown in Figure 1) [20,21,22]. When observed for a long time, the temperature difference between the target and the background gradually increases from zero until the observer confirms that the target pattern with four-bars can be distinguished (50% probability). The temperature difference between the target and the background is the minimum resolvable temperature difference at that frequency. MDTD is defined as when the observer can just distinguish the shape and position of a square or round black-body target with a certain size through the display screen without time limitation (as shown in Figure 2). The temperature difference between the black-body target and the black-body background is called the MDTD [23,24,25,26]. MRTD and MDTD calculations help in the design of detectors and actual systems. The first mature infrared imaging system performance prediction model (the Ratches Model) was named after the proposer of the theory, and was developed by the American Night Vision Laboratory in 1975 [27]. The FLIR92 model is widely used in the performance evaluation of infrared imaging systems because it adds a three-dimensional noise model, which can predict the performance of scanning and staring imaging systems. By introducing a modulation transfer function and a new human eye model, it adds an analysis of the effect on sampling to better evaluate gaze imaging systems [28,29]. However, the evaluation models of infrared imaging systems are mostly for infrared imaging systems in telephoto mode. Due to the lack of comprehensively considering 1) the characteristics of the TMISs (thermal microscope imaging systems) and 2) the difference in spectral radiant flux, which are factors that cannot be ignored, none of the above-mentioned methods can accurately and comprehensively evaluate the performance of the TMISs, and so there is currently no standard method for evaluating the performance of the TMISs. The imaging device can only be selected based on experience when designing the TMISs, which leads to the failure of the best performance of the imaging device in the process of designing the system, and the inability to select appropriate design schemes for different detection targets.

In this paper, we studied the performance evaluation and optimization theory of TMISs. The thermal microscope imaging process was analyzed, and the performance evaluation model suitable for TMISs was further deduced and established. Based on the performance evaluation theory, we conducted simulation analysis and obtained theoretical results suitable for performance evaluation and optimization of TMISs. Finally, guided by the theoretical analysis results, we built a high-performance thermal microscope imaging system and carried out the system’s image acquisition experiment. The experimental results show that the designed system has high performance, which proves that the theory studied in this paper could provide a theoretical basis for the TMISs design. The block diagram of the research in this paper is shown in Figure 3.

2. Materials and Methods

We analyzed the difference in spectral radiant flux between thermal microscope imaging and telephoto thermal imaging. The expression for the SNR (signal-to-noise ratio) of the output image of the TMISs was derived, based on the analysis of the characteristics of thermal microscope imaging, and we propose novel MRTD and MDTD models that are suitable for evaluating the performance of the TMISs. The block diagram of the research method in this paper is shown in Figure 4.

2.1. Spectral Radiant Flux of the TMISs

In the TMISs, the light radiated by the microfacet of the object to be measured is received by the optical system in a divergent form, and then passes through the optical system focus on the imaging detector. The imaging optical path diagram of the TMISs is shown in Figure 5. Among them, $d{s}_0$ and $d{s}_1$ represent the microfacets of the object to be measured and the detector respectively, $n_0$ and $n_1$ represent the refractive index of the object space and image space respectively, $2u_0$ and $2u_1$ respectively represent the opening angle of the micro-unit of the object to be measured and the micro-unit of the detector of the optical system, $l$ and $l^{\prime }$ represent object distance and image distance, respectively, ${\tau }_0$ represents the spectral transmittance of the optical system, $d_0$ and ${d^{\prime }}_0$ represent the diameter of the optical system in object space and image space, respectively. The radiant flux emitted by the micro-space element $d{s}_0$ in object space to a lens with an aperture $d_0$ can be expressed as [30]

$$d\Phi (\lambda ,T)=\epsilon (\lambda ,T)M_0(\lambda ,T){\tau }_a(\lambda )d{s}_{0}si{n}^2{u}_0$$

(1)

where $\epsilon (\lambda ,T)$ is the spectral emissivity; $2u_0$ is the objective aperture angle of the microscope objective; ${\tau }_a(\lambda )$ is the spectral transmittance of the atmosphere (assuming that ${\tau }_a=1$); $M_0(\lambda ,T)$ is the blackbody spectral emittance at temperature T (Planck’s radiation law) [31]:

$$M_0(\lambda ,T)=\frac{c_1}{{\lambda }^5}\frac{1}{exp(c_2/\lambda T)-1}$$

(2)

After $d\Phi (\lambda ,T)$ passes through a microscope objective with a spectral transmittance ${\tau }_0(\lambda )$, the irradiance on the detector microunit $d{s}_1$ is

$$E(\lambda ,T)=\epsilon (\lambda ,T)M_0(\lambda ,T){\tau }_0(\lambda )\frac{d{s}_0}{d{s}_1}si{n}^2{u}_0$$

(3)

according to the theorem of Helmholtz–Lagrange: $n_0\cdot r_0\cdot sin{u}_0=n_1\cdot r_1\cdot sin{u}_1$ [32], and Equation (3) can be rewritten as

$$E(\lambda ,T)=\epsilon (\lambda ,T)M_0(\lambda ,T){\tau }_0(\lambda )\frac{n_1{}^2}{n_0{}^2}si{n}^2{u}_1$$

(4)

where ${\tau }_0(\lambda )$ is spectral transmittance of the optical system, $n_0$ and $n_1$ are the refractive index of the microscope objective in the object space, and image space, and $2u_1$ is the opening angle of the detector microunit to the system; it can be shown that

$$sin{u}_1=(\frac{d_0}{2})/\sqrt{[{(\frac{d_0}{2})}^2+{l^{\prime }}^2]}$$

(5)

and so the spectral radiant flux of the object received by the detector unit of area $a\times b$ can be calculated as

$$\Phi (\lambda ,T)=E(\lambda ,T)ab=ab\cdot \epsilon (\lambda ,T)M_0(\lambda ,T){\tau }_0(\lambda )(\frac{d_0}{2}{)}^2/[{(\frac{d_0}{2})}^2+{l^{\prime }}^2]$$

(6)

where $l^{\prime }$ is the image distance.

2.2. SNR of the TMISs Output Image

The SNR of the target image received by the imaging system can be expressed as

$$SNR=\frac{S}{N}=\frac{\Delta \Phi (\lambda ,T)R(\lambda )}{\sqrt{{\displaystyle {\int }_0^{\infty }{s}^{\prime }(f)MT{F}_e^2(f)df}}}$$

(7)

where $R(\lambda )$ is the responsivity of the detector; $s^{\prime }(f)$ is the noise power spectrum of the system; $MT{F}_e(f)$ is the transfer function of the electronic filter; $\Delta \Phi (\lambda )$ is the radiant flux difference of the target and the background:

$$\Delta \Phi (\lambda )=ab[{\epsilon }_T(\lambda ,T)M_0(\lambda ,T_T)-{\epsilon }_B{M}_0(\lambda ,T_B)]{\tau }_0(\lambda ){(\frac{d_0}{2})}^2/[{(\frac{d_0}{2})}^2+{l^{\prime }}^2]$$

(8)

where $T_T$ and $T_B$ are the target and background temperatures. When $T_T$ and $T_B$ are not significantly different, calculus can be used instead of the difference. Then, the radiant flux difference of the target and the background is

$$\Delta \Phi (\lambda )\approx ab[\Delta \epsilon M_0(\lambda ,T_B)+{\epsilon }_B\frac{\partial M_0(\lambda ,T_B)}{\partial T}\Delta T]{\tau }_0(\lambda ){(\frac{d_0}{2})}^2/[{(\frac{d_0}{2})}^2+{l^{\prime }}^2]$$

(9)

where $\Delta \epsilon ={\epsilon }_T-{\epsilon }_B$ is the spectral radiation difference between the target and background; ${\epsilon }_T$ and ${\epsilon }_B$ represents the spectral emissivity of the target and background, respectively; $\Delta T=T_T-T_B$ is the target and background temperature difference; $\partial M_0(\lambda ,T_B)/\partial T$ is the rate of change of the blackbody spectral emittance at the background temperature $T_B$ with respect to the temperature difference.

Assuming that the measurement normalized detection rate is $D^{*}$; $V_n$ is the noise voltage corresponding to the unit bandwidth at the measurement point $f_0$; $\Delta f$ is the measurement bandwidth; $A_d=a\times b$ is the area of the detector. Then, the expression relating $D^{*}$ and R is

$$R(\lambda )=\frac{D^{*}(\lambda )V_n}{\sqrt{A_d\Delta f}}=D^{*}(\lambda )\sqrt{\frac{s^{\prime }(f_0)}{A_d}}$$

(10)

substituting Equation (10) into Equation (7) yields:

$$SNR=\frac{\Delta \Phi (\lambda ,T)D^{*}(\lambda )}{\sqrt{A_d}\sqrt{{\displaystyle {\int }_0^{\infty }s(f)MT{F}_e^2(f)df}}}=\frac{\Delta \Phi (\lambda ,T)D^{*}(\lambda )}{\sqrt{A_d\Delta f_n}}$$

(11)

where $s(f)=s^{\prime }(f)/s^{\prime }(f_0)$ is the normalized noise power spectrum; $\Delta f_n$ is the noise equivalent bandwidth:

$$\Delta f_n={\displaystyle {\int }_0^{\infty }s(f)}MT{F}_e^2(f)df$$

(12)

Combining Equations (9) and (11), we obtain the image SNR of the TMISs with working wavelength in the range of $[{\lambda }_1,{\lambda }_2]$:

$$\begin{array}{l}SNR=\frac{\Delta \Phi (\lambda ,T)D^{*}(\lambda )}{\sqrt{A_d\Delta f_n}}\\ =\frac{{\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}ab[\Delta \epsilon M_0(\lambda ,T_B)+{\epsilon }_B\frac{\partial M_0(\lambda ,T_B)}{\partial T}\Delta T]{\tau }_0(\lambda ){(\frac{d_0}{2})}^2/[{(\frac{d_0}{2})}^2+{l^{\prime }}^2]D^{*}(\lambda )d\lambda }}{\sqrt{A_d\Delta f_n}}\\ ={\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}[\Delta \epsilon M_0(\lambda ,T_B)+{\epsilon }_B\frac{\partial M_0(\lambda ,T_B)}{\partial T}\Delta T]}{D}^{*}(\lambda )d\lambda \times \frac{ab{\tau }_0(\lambda ){(d_0/2)}^2}{\sqrt{A_d\Delta f_n}[{(d_0/2)}^2+{l^{\prime }}^2]}\end{array}$$

(13)

assuming that ${\epsilon }_T={\epsilon }_B=1$ and ${\tau }_0(\lambda )= {\tau }_0$. Then, the signal-noise ratio of the target image received by the system is

$$SNR={\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}\frac{\partial M_0(\lambda ,T_B)}{\partial T}\Delta T}{D}^{*}(\lambda )d\lambda \times \frac{\sqrt{A_d}{\tau }_0{D}_{NA}{}^2}{\sqrt{\Delta f_n}}$$

(14)

where $D_{NA}=sin{u}_1=(\frac{d_0}{2})/\sqrt{[{(\frac{d_0}{2})}^2+{l^{\prime }}^2]}$.

3. Results

3.1. MRTD Model

Due to the different considerations and processing methods, MRTD has some different expressions, but its basic ideas are the same. As the derivation process of MRTD is quite complicated, we derive the MRTD model of the TMISs based on the points and results of the derivation by J.M. Lloyd [25]. According to the basic idea of the MRTD derivation, when the image signal-to-noise ratio perceived by the human eye is greater than or equal to the visual signal-to-noise ratio, the temperature difference between the blackbody target and the background is the MRTD.

The signal-to-noise ratio $SN{R}_d$ of the stripe pattern at the output end of the display is

$$SN{R}_d=R(f)SNR{\left[\frac{\Delta f_n}{{\displaystyle {\int }_0^{\infty }s(f)\cdot MT{F}_i{}^2(f)\cdot MT{F}_m^2(f)df}}\right]}^2$$

(15)

where $R(f)$ is the square wave response of the system; $MT{F}_i(f)$ is a modulation transfer function for electronic amplifiers and video processing circuits; $MT{F}_{\mathrm{m}}(f)$ is the modulation transfer function of the display; $\Delta f_n$ is the equivalent power spectrum of noise.

When the observer observes the target, the $SN{R}_d$ will be modified in the following four aspects to obtain the visual $SN{R}_v$:

(1) The eye extracts the stripe pattern. In the case of distinguishable signals, the higher harmonics are filtered and the first harmonics are maintained. At this time, the signal peak attenuation is $\frac{2}{\pi }R(f)=\frac{8}{{\pi }^2}MTF(f)$, $MTF(f)$ is the system modulation transfer function

(2) As a result of time integration, the signal will be sampled and integrated independently once according to the human eye integration time $t_e(=0.2s)$, while the noise is superimposed in the form of the square root, and so the signal-to-noise ratio will be improved by ${(t_e{f}_p)}^{1/2}$, $f_p$ is the frame frequency.

(3) In the vertical direction, the eye will carry out signal space integration, and take the root mean square value of noise along the line, using the vertical instantaneous field of view $\beta$ as the noise correlation length, to obtain the improvement of visual signal-to-noise ratio

$${\left(\frac{L}{\beta }\right)}^{1/2}={\left(\frac{\epsilon W}{\beta }\right)}^{1/2}={\left(\frac{\epsilon }{2f_T\beta }\right)}^{1/2}$$

(16)

where, $L$ is the length of the strip; $W$ is the stripe bandwidth; $\epsilon$ is a stripe length to width ratio, $\epsilon =7$; $f_T$ is the stripe space-frequency $(c/mrad)$.

(4) For the periodic rectangular line target with frequency $f_T$, the filtering effect of the human eye on the narrowband space can be equivalent to a matched filter, filtered by the frequency domain of the human eye, in the horizontal direction, and the integral effect of the human eye can be captured by replacing the actual system bandwidth with the noise bandwidth $\Delta f_{eye}$, which takes into account the effect of eye matching filter

$$\Delta f_{eye}(f_T)={\displaystyle {\int }_0^{\infty }s(f)\times MT{F}_i{}^2(f)\times MT{F}_m^2(f)sin{c}^2(\pi f/2f_T)df}$$

(17)

that is, the SNR is improved to ${\left[\frac{{\displaystyle {\int }_0^{\infty }s(f)\cdot MT{F}_i{}^2(f)\cdot MT{F}_m^2(f)df}}{\Delta f_{eye}(f_T)}\right]}^2$.

Thus, combining the above four effects with $SN{R}_d$, the visual $SN{R}_v$ is obtained

$$SN{R}_v=\frac{8}{{\pi }^2}MTF(f)\times SNR\times {(t_e{f}_p)}^{1/2}\times {(\frac{\epsilon }{2f_T\beta })}^{1/2}\times {(\frac{\Delta f_n}{\Delta f_{eye}})}^{1/2}$$

(18)

by substituting Equation (14) into Equation (18), let $s(f)=1$, and $MT{F}_i(f)=MT{F}_m(f)=1$. Then, the visual signal-to-noise ratio of TMISs can be expressed

$$SN{R}_v=\frac{4\sqrt{14}}{{\pi }^2}MTF(f)\times {\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}\frac{\partial M_0(\lambda ,T_B)}{\partial T}\Delta T}{D}^{*}(\lambda )d\lambda \times \frac{\sqrt{A_d}{\tau }_0{D}_{NA}{}^2}{\sqrt{\Delta f_n}}\times {(\frac{t_e{f}_p{\tau }_d\Delta f_n}{\alpha \beta })}^{1/2}$$

(19)

let the threshold signal-to-noise ratio when the observer can exactly distinguish the band be $SN{R}_{TH}$, then $\Delta T$ solved by Equation (18) is the expression of MRTD. Then, the MRTD of TMISs can be expressed as

$$MRTD=\frac{{\pi }^2}{4\sqrt{14}}\frac{SN{R}_{TH}\sqrt{\Delta f_n}}{MTF(f)\sqrt{A_d}{\tau }_0{\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}\frac{\partial M_0(\lambda ,T_B)}{\partial T}}{D}^{*}(\lambda )d\lambda }\times {(\frac{\alpha \beta }{t_e{f}_p{\tau }_d\Delta f_n})}^{1/2}\times \frac{1}{D_{NA}{}^2}$$

(20)

where ${(\frac{\alpha \beta }{t_e{f}_p{\tau }_d\Delta f_n})}^{1/2}$ is the human eye filter function, with a value ranging from 0.52 to 0.65; $SN{R}_{TH}$ is the visual threshold signal-to-noise ratio, which is generally 2.25.

3.2. MDTD Model

Since MDTD also has many models, we derive the MDTD model of the TMISs based on the points and results of the FLIR 92 model [28]. Assuming that the target is a square with bandwidth W and the average value of the target pattern displayed on the display is $\overline{I(x,y)}\Delta T$, where $I(x,y)$ is the amplitude-normalized square target image, the display signal-to-noise ratio $SN{R}_d$ for each image frame is

$$SN{R}_d={\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}\frac{\partial M_0(\lambda ,T_B)}{\partial T}\overline{I(x,y)}\Delta T}\times D^{*}(\lambda )d\lambda \times \frac{\sqrt{A_d}{\tau }_0{D}_{NA}{}^2}{\sqrt{\Delta f_n}}{\left[\frac{\Delta f_n}{{\displaystyle {\int }_0^{\infty }s(f)MT{F}_m{}^2(f)df}}\right]}^{1/2}$$

(21)

When the observer looks at the target image, the visual signal-to-noise ratio $SN{R}_v$ improvement manifests itself as:

(1)

Time integration enables signal-to-noise ratio improvement ${(t_e{f}_p)}^{1/2}$;

(2)

Set the size of the line expansion function of the system in the vertical direction is $\left|r_y\right|$, then the improvement of the spatial integration on the signal-to-noise ratio is ${\left[(W+\left|r_y\right|/\beta )\right]}^{1/2}$;

(3)

Human-eye frequency domain filtering effect, using f instead of the system bandwidth $\Delta f_{eye}$.

Taking all these factors into account, the visual signal-to-noise ratio can be rewritten as

$$SN{R}_v={\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}\frac{\partial M_0(\lambda ,T_B)}{\partial T}\overline{I(x,y)}\Delta T}\times D^{*}(\lambda )d\lambda \times \frac{\sqrt{A_d}{\tau }_0{D}_{NA}{}^2}{\sqrt{\Delta f_n}}{\left(\frac{t_e{f}_p\Delta f_n}{\Delta f_{eye}}\right)}^{1/2}{\left(\frac{W+\left|r_y\right|}{\beta }\right)}^{1/2}$$

(22)

set the visual threshold $SN{R}_{TH}$ when the observer becomes able to detect, then $\Delta T$ at this point is MDTD, If the r term is ignored, then the expression for MDTD is

$$MDTD=\frac{SN{R}_{TH}\sqrt{2\Delta f_n}}{\overline{I(x,y)}\sqrt{A_d}{\tau }_0{\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}\frac{\partial M_0(\lambda ,T_B)}{\partial T}}{D}^{*}(\lambda )d\lambda }{(\frac{f\beta \Delta f_{eye}}{t_e{f}_p\Delta f_n})}^{1/2}\times \frac{1}{D_{NA}{}^2}$$

(23)

where $\Delta f_{eye}$ indicates the system bandwidth considering human eye filtering; $t_e$ is the human eye integration time, which is generally taken between $0.1-0.25$.

Because the detectors of the TMISs are divided into IPD (ideal photon detector) and ITD (ideal thermal detector), we will derive and analyze these two types.

3.3. Mathematical Model of the IPD

3.3.1. MRTD of the IPD

The spectral detection rate of an IPD can be assumed as follows:

$$D^{*}(\lambda )=\left\{\begin{array}{c}{D}^{*}({\lambda }_p)\lambda /{\lambda }_p, \lambda \le {\lambda }_p \\ 0, \lambda > {\lambda }_p\hfill \end{array}\right.$$

(24)

when $T_T$ and $T_B$ are not significantly different, the rate of change of the spectral emittance at the background temperature $T_B$ with respect to the temperature difference can be approximated as [33]

$$\frac{\partial M(\lambda ,T_B)}{\partial T}\approx \frac{c_2{M}_0(\lambda ,T_B)}{\lambda T_B^2}$$

(25)

according to the hypothesized model of the photon detector’s spectral detection rate, combining Equations (20), (24) and (25), when the detector of the TMISs is an IPD, the MRTD of the system is

$$MRTD(f)=\frac{{\pi }^2}{4\sqrt{14}}\frac{SN{R}_{TH}\sqrt{\Delta f_n}{\lambda }_p{T}_B{}^2}{MTF(f)\sqrt{A_d}{\tau }_0{D}^{*}({\lambda }_P)c_2{\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_m}{M}_0(\lambda ,T)}d\lambda }\times {(\frac{\alpha \beta }{t_e{f}_p{\tau }_d\Delta f_n})}^{1/2}\times \frac{1}{D_{NA}{}^2}$$

(26)

where $c_2=1.4388\times 10^4\mathsf{\mu }\mathrm{m}\times K$ is the second radiation constant; ${\lambda }_m=min\left\{{\lambda }_2\right.,\left.{\lambda }_p\right\}$, ${\lambda }_p$ is the peak response wavelength; $D^{*}({\lambda }_P)$ is the normalized detection rate of the peak response of the detector.

3.3.2. MDTD of the IPD

According to the above derivation, the expression of the MDTD of an IPD is:

$$MDTD=\frac{SN{R}_{TH}\sqrt{2\Delta f_n}{\lambda }_p{T}_B{}^2}{\overline{I(x,y)}\sqrt{A_d}{\tau }_0{c}_2{D}^{*}({\lambda }_P){\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_m}{M}_0(\lambda ,T_B)}d\lambda }{(\frac{f\beta \Delta f_{eye}}{t_e{f}_p\Delta f_n})}^{1/2}\times \frac{1}{D_{NA}{}^2}$$

(27)

3.4. Mathematical Model of the ITD

3.4.1. MRTD of the ITD

For the ITD, $D^{*}(\lambda )=D_0$ [33], the expression of the MRTD is

$$\begin{array}{l}MRTD(f)=\frac{{\pi }^2}{4\sqrt{14}}{(\frac{\alpha \beta }{t_e{f}_p{\tau }_d\Delta f_n})}^{1/2}\frac{1}{D_{NA}^2}\\ \times \frac{SN{R}_{TH}\sqrt{\Delta f_n}{T}_B}{MTF(f)\sqrt{A_d}{\tau }_0{D}_0\left[4{\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_m}{M}_0(\lambda ,T_B)d\lambda +{\lambda }_2{M}_0({\lambda }_2,T_B)-{\lambda }_1{M}_0({\lambda }_1,T_B)}\right]}\end{array}$$

(28)

3.4.2. MDTD of the ITD

The expression of the MDTD is

$$MDTD=\frac{SN{R}_{TH}\sqrt{2\Delta f_n}}{\overline{I(x,y)}\sqrt{A_d}{\tau }_0{D}_0{\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}\frac{\partial M_0(\lambda ,T_B)}{\partial T}}d\lambda }{(\frac{f\beta \Delta f_{eye}}{t_e{f}_p\Delta f_n})}^{1/2}\times \frac{1}{D_{NA}{}^2}$$

(29)

4. Analysis

4.1. Difference Analysis

After completing the research on the evaluation model, we analyzed the difference between the model in the microscopic mode and the telephoto mode. Below, we divide this part into two parts of MRTD and MDTD for analysis.

4.1.1. Relationship with Telephoto Mode Thermal Imaging System MRTD’

According to the derivation by J.M. Lloyd, the MRTD of the telephoto mode thermal imaging system can be represented as

$$MRT{D}^{\prime }=\frac{{\pi }^2}{4\sqrt{14}}\frac{SN{R}_{TH}NET{D}^{\prime }}{MTF(f)}\times {(\frac{\alpha \beta }{t_e{f}_p{\tau }_d\Delta f_n})}^{1/2}$$

(30)

according to the derivation result of the $NET{D}^{\prime }$ in the telephoto mode

$$NET{D}^{\prime }=\frac{4F^2\sqrt{\Delta f_n}}{\sqrt{A_d}{\tau }_0{}^{\prime }{\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}\frac{\partial M_0(\lambda ,T_B)}{\partial T}}{D}^{*}(\lambda )d\lambda }$$

(31)

so the MRTD’ can be expressed as

$$MRT{D}^{\prime }(f)=\frac{{\pi }^2}{4\sqrt{14}}\frac{4F^{2}SN{R}_{TH}\sqrt{\Delta f_n}}{MTF(f)\sqrt{A_d}{\tau }_0{}^{\prime }{\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}\frac{\partial M_0(\lambda ,T_B)}{\partial T}}{D}^{*}(\lambda )d\lambda }\times {(\frac{\alpha \beta }{t_e{f}_p{\tau }_d\Delta f_n})}^{1/2}$$

(32)

where $F=d_0/f^{\prime }$, f’ is the focal length of the lens; ${\tau }_0{}^{\prime }$ is the spectral transmittance of the telephoto mode optical system. Generally, specific values of $F$ and ${\tau }_0{}^{\prime }$ are given in the performance index of the thermal imaging component. Therefore, when using this thermal imaging component to construct the TMISs, the MRTD of the TMISs can be expressed as

$$MRTD=\frac{1}{D_{NA}{}^2} \times \frac{{\tau }_0{}^{\prime }}{4F^2{\tau }_0}\times MRT{D}^{\prime }$$

(33)

4.1.2. Relationship with Telephoto Mode Thermal Imaging System MDTD’

The MDTD of the telephoto mode thermal imaging system can be represented as

$$MDT{D}^{\prime }=\sqrt{2}\frac{SN{R}_{TH}NET{D}^{\prime }}{\overline{I(x,y)}}{(\frac{f\beta \Delta f_{eye}}{t_e{f}_p\Delta f_n})}^{1/2}$$

(34)

similarly, the relationship between MDTD of the thermal imaging system in microscopic mode and the telephoto mode MDTD’ can be expressed as

$$MDTD=\frac{1}{D_{NA}{}^2} \times \frac{{\tau }_0{}^{\prime }}{4F^2{\tau }_0}\times MDT{D}^{\prime }$$

(35)

according to Equations (33) and (35), the changes of MRTD and MDTD of the TMISs concerning D_NA can be analyzed. For example, using an uncooled focal plane detector imaging device with MRTD’ = 100 mK (F = 1) and MDTD’ = 150 mK (F = 1) in telephoto mode, if the difference in objective transmittance is ignored, the MRTD-D_NA and MDTD-D_NA curves of the TMISs can be obtained, as shown in Figure 6.

It can be seen from Figure 6 that the MRTD and MDTD of the microscopic mode are monotonically decreasing functions of D_NA. When a 2x infrared microscope objective is used, its D_NA = 0.24, then we get MRTD = 4.34 MRTD’ = 434 mK and MDTD = 4.34 MDTD’ = 651 mK. Because D_NA will decrease with the increase of the magnification of the microscope objective lens, combining Equations (33) and (35) and Figure 6 shows that the MRTD and MDTD of the TMISs increase with the increase of the magnification of the microscope objective lens; that is, the temperature resolution of the TMISs decreases with the increase of the magnification.

Therefore, when designing the TMISs, it is necessary to consider the magnification and temperature resolution of the system comprehensively. Simply increasing the magnification of the objective lens to increase the spatial resolution of the system will cause the temperature resolution of the system to decrease. Therefore, when the system is applied to the field of microthermal analysis, it is necessary to consider improving the spatial resolution of the system by other means (such as adding micro-scanning equipment) on the premise of ensuring that the system also has a high-temperature resolution.

4.2. Evaluation Model Simulation Analysis

If the hardware parameters of the system are known, the temperature resolution performance of the system can be simulated and evaluated according to the MRTD and MDTD models established above. According to the results of the above derivation, the results of MRTD and MDTD in microscopic mode are consistent, so only the MRTD is used as an example for simulation analysis. Assume that a 2x microscope infrared objective lens with a 7.5 mm radius of the incident aperture on the object side is used, where D_NA = 0.24. To discuss the sensitivity of the ideal photon detector and the ideal thermal detector to temperature, assume that the IPD and the ITD have the same number of pixels $320\times 240$ and detection area $9\times 9$ mm².

4.2.1. Simulation Analysis of MRTD in the IPD

According to the physical limitations of the mechanism of an IPD, IPD is suitable for working at relatively low-temperature conditions. Assuming a background operating temperature of 200 K and a target emissivity difference of 0.01, the simulated MRTD (f) curves of IPD in different bands are shown in Figure 7. To further analyze the difference in temperature resolution of MRTD in different bands, a simulation experiment of MRTD with background temperature $T_B$ was completed. To further analyze the difference in temperature resolution of IPD in different bands, a simulation experiment of MRTD with background temperature $T_B$ was completed (as shown in Figure 8).

As shown in Figure 7, MRTD increases with increasing spatial frequency, which means that when the target to be measured has a certain temperature difference, the higher the spatial frequency, the lower the temperature resolution of the system, which means that the target to be measured is more difficult to be resolved.

As shown in Figure 8, the MRTD increases with increasing background temperature, indicating that even under the condition of constant spatial frequency the increase of the background temperature will also increase the MRTD, resulting in a decrease in the temperature resolution of the system, meaning that for targets with a certain temperature difference, the higher the background temperature, the harder it is to be discriminated. From the above analysis of Figure 8, we find that to make the system have better performance, besides optimizing the system from the perspective of detector type, band selection, and optical design, the operating environment temperature of the system is also an important factor, and based on the newly established MRTD model, we can choose appropriate background temperature for different types of systems.

Combined Figure 7 and Figure 8 show that the expressions of MRTD about f and $T_B$ are monotonically increasing functions. As the spatial frequency f and background temperature $T_B$ increase, MRTD also tends to increase monotonically. The temperature resolution ability of the IPD under the medium-wave is better than the long-wave. For example, when the spatial frequency $f=0.2cyc\times mra{d}^{-1}$, the MRTD of the IPD in the long-wave and medium-wave is 711.6 mK and 320.4 mK. When $T_B$ = 200 K, the MRTD of the ideal photon detector in the long-wave and medium-wave is 740.8 mK and 333.6 mK.

4.2.2. Simulation Analysis of MRTD in the ITD

Assuming the target emissivity difference is 0.01, the simulated MRTD (f) and MRTD ($T_B$) curves of ITD in different bands are shown in Figure 9 and Figure 10.

From Figure 9, it can be seen that the MRTD of the ITD increases with increasing spatial frequency, indicating that under the same conditions, the higher the spatial frequency, the more difficult it is to distinguish the object to be measured.

By comparing Figure 10 and Figure 8, it can be seen that the two trends are consistent, so the analysis and conclusions regarding Figure 8 are equally applicable to ITD, and therefore will not be repeated here.

Both Figure 9 and Figure 10 show that the expressions of MRTD about f and $T_B$ are monotonically increasing functions. As the spatial frequency f and background temperature $T_B$ increase, MRTD also tends to increase monotonically. The temperature resolution ability of ITD under the medium-wave is better than the long-wave. For example, when the spatial frequency $f=0.2cyc\times mra{d}^{-1}$, the MRTD of the ITD in the long-wave and medium-wave are1520.7 mK and 682.9 mK. When $T_B$ = 300 K, the MRTD of the ITD in the long-wave and medium-wave are 1617.8 mK and 721.3 mK.

The imaging principle of IPD (refrigerated detector) is based on the fact that the detector absorbs the photons of the target or background radiation to produce a photoconductive or photovoltaic effect. It is suitable for working at a low background temperature (70–200 K). The simulation results show that the IPD maintains a high temperature resolving ability in both the medium-wave infrared thermal imaging and the long-wave infrared thermal imaging in a low background temperature environment. The imaging principle of ITD (non-refrigerated detector) is based on changes in the resistivity or polarity of the detector caused by the absorption of energy by thermal radiation. Simulation results show that the temperature resolution of ITD in the medium-wave infrared band is higher than that of the long-wave infrared band. Figure 9 and Figure 10 show that as f and $T_B$ increase, the gap between the MRTD in the medium-wave and the long-wave becomes gradually larger. This shows that ITD is more sensitive to the non-uniformity of emissivity in the long-wave infrared band and has higher sensitivity for observing natural objects that do not generate heat with a small difference in emissivity, and it is easier to obtain the difference in target emissivity.

4.3. Frame Rate Superposition of the TMISs

The noise power spectrum ($s(f)=1$) of white noise, and $\Delta f_n$ at this time is called the noise standard bandwidth $\Delta f_0$. Because the system’s electronic filter transfer function is generally a low-pass filter, it can be seen that the relationship between the system’s equivalent noise bandwidth and the 3dB frequency $f_{t0}$ is $\Delta f_n=f_{t0}\pi /2$. Generally, to maintain the maximum value of the light pulse signal waveform, let $f_{t0}=1/(2{\tau }_d)$. Therefore, the relationship between the noise equivalent bandwidth $\Delta f_n$ and ${\tau }_d$ of the system is

$$\Delta f_0=\frac{\pi }{4{\tau }_d}$$

(36)

where ${\tau }_d$ is the dwell time of the detector. For a staring focal plane detector component, the dwell time of the detector is ${\tau }_d=1/f_p$, where $f_p$ is the frame rate, so the noise equivalent bandwidth $\Delta f_n$ of the system can be expressed as

$$\Delta f_n=\frac{\pi f_p}{4}$$

(37)

Combining Equations (18) and (20) shows that reducing the imaging frame rate can reduce the MRTD and the MDTD of the system. Therefore, when designing TMISs with a certain magnification, the temperature resolution of the system can be improved by frame rate superposition.

5. Application and Experiment

5.1. System Design

After completing the above theoretical research, we designed and built the thermal microscope imaging system, which has good performance and potential applications in the field of nondestructive testing of microelectronic devices.

Firstly, according to the conclusion of the analysis in Section 4.1, to ensure the system has better temperature resolution, we choose the magnification factor of the system as 2 times.

Secondly, according to the analysis in Section 4.2, the refrigerated detector has high-temperature resolution in both a medium-wave infrared band and long-wave infrared band. However, the large size and high price of the refrigerated detector greatly limits its application and promotion. Simultaneously, the non-refrigerated detector can also obtain higher temperature resolution through reasonable collocation and scientific operation methods. For example, uncooled detectors have higher sensitivity to the non-uniformity of emissivity in the long-wave infrared band, and it is easier to distinguish the small differences in emissivity of the target to be measured. When the infrared imaging system is used for nondestructive testing, the fault detection is carried out by using the slight difference of the emissivity between the faulty electronic component and the trouble-free electronic components. Therefore, we use a combination of non-refrigerated detector and long-wave infrared band to build a thermal microscope imaging system.

Finally, the sub-pixel imaging technology based on optical micro-scanning can improve the temperature resolution of the imaging system by reducing the imaging frame rate [34]. Therefore, we have designed a 2 × 2 micro-scanning system, and built a new type of micro-scanning thermal microscope imaging system, as shown in Figure 11. This system consists of an infrared magnifying objective (Magnification: 2), a micro-scanner (Model: GIP-101, micro scan step size: 4.5 pixel size), an infrared image acquisition card (Model: Tau2-324), an image processing system (Computer name: LAPTOP-7DJSLF4O), and a power supply (220 V), as shown in the block diagram of a high-resolution thermal microscope imaging system in Figure 12 [35,36,37].

5.2. Image Acquisition Experiment

After completing the construction of the system, we carried out the system acquisition image experiment; Figure 13 shows the experimental results. Figure 13a is the low-resolution thermal microscope image collected by the system. This is resized using bilinear interpolation, as shown in Figure 13b. Figure 13c is the micro-scan reconstructed image collected by the system, which is obtained by interpolating the four low-resolution images collected by the system according to the 2 × 2 micro-scan [37]. Since the ideal high-resolution image cannot be obtained in this experiment, instead we provide the image information entropy (SNT) for assessment [38,39].

According to the experimental results, the information entropy of the original image is 5.4627, the information entropy of the image after bilinear interpolation is 5.6503, and the information entropy of the micro-scan reconstructed image is 5.7148. The micro-scan reconstructed image can distinguish more detailed information, and, at the same time, the spatial resolution of the original low-resolution image is significantly improved. Compared with the two-line interpolation image, the image after system reconstruction has clearer details and has the advantage of zooming in and not distorting the details, which proves the effectiveness of the system design. It also proves that the theory provided in this article can provide a theoretical basis for the design of TMISs.

6. Conclusions

This paper studied the imaging characteristics of the TMISs and analyzed the differences between microscopic and telescopic thermal imaging. Theoretical models of minimum resolvable temperature difference (MRTD) and minimum detectable temperature difference (MDTD) of the TMISs were established. If the performance parameters of the hardware part of the system are known, the temperature resolution of TMISs can be evaluated by simple calculations based on the model. Through the simulation analysis of the model, we have drawn new conclusions about the performance optimization theory of the thermal microscopy imaging system:

(1)

In the case of the same infrared detector components, the temperature resolution of the TMISs decreases as the magnification increases.

(2)

IPD maintains high temperature resolving ability in both medium-wave infrared thermal imaging and long-wave infrared thermal imaging in a low background temperature environment.

(3)

The use of medium-wave infrared can make the system have a higher temperature resolution ability than using long-wave infrared.

(4)

The use of long-wave infrared can make the system more sensitive to the non-uniformity of the emissivity. It has a higher sensitivity for observing natural objects that do not generate heat with a small difference in emissivity, and it is easier to obtain the difference in target emissivity.

(5)

Lowering the imaging frame rate can reduce the MRTD and MDTD of the system, so it can improve the temperature resolution of the TMISs.

(6)

In addition to the detector type, wavelength band, optical design, and other factors, it is also necessary to set the appropriate operating environment temperature for different systems to achieve a higher temperature resolution.

In addition, we built a high-performance thermal microscope imaging system based on the theoretical research results. Through comparative experiments, it was proved that the system we built has high performance and demonstrates the effectiveness of the theoretical research in this paper.

In this paper, through the research on the performance evaluation theory of TMISs, the theoretical results about the performance evaluation and optimization of TMISs are obtained, and the theoretical results are applied to practical applications to build a high-performance thermal microscope imaging system. The results of this research can be applied to the design of TMISs in the future, and can guide designers to build high-performance TMISs based on different hardware conditions and application environments. In addition, the conclusions about optimizing system performance are also applicable to other thermal imaging systems.