1. Introduction
Any object in nature that is above absolute temperature (−273 °C) radiates heat (electromagnetic waves) outward [
1]. Electromagnetic waves with a wavelength range of 760 nm to 1 mm are called infrared and cannot be seen by the naked eye. The higher the temperature of an object, the greater the energy radiated. Infrared thermographic (IRT) technology involves sensing infrared waves through special materials, converting them into electrical signals, and then converting the electrical signals into digital images. Using thermal imaging technology, the detection device (an infrared thermal imager) receives varying degrees of infrared radiation from the surface of a sample, generating a temperature field map. This temperature field map characterizes the infrared radiation distribution and can be used to evaluate the differences in the external and internal structures of the sample. This is because the differences in the external and internal structures of the evaluated object will generate different heat conduction in the material, thereby affecting the heat flow [
2]. This means that samples with defects, due to differences in internal structure, will cool or heat up at different ratios, resulting in different thermal contrasts in infrared thermal radiation imaging.
Therefore, IRT technology can be used in the field of defect detection [
3], especially in the electronic [
4,
5] and renewable industries [
6]. According to the structural characteristics and defect properties of different materials, different types of thermal excitation sources need to be designed to actively heat the surface or interior of the tested object. The thermal excitation source can be modulated or not. Common excitation sources include flash/halogens lamps, hot air, lasers, ultrasound, electromagnetics, etc. Due to the presence of defects on the surface or inside of the tested object, there will be certain differences in the ways in which the thermal waves generated propagate towards the surface of the object. The main advantages of IRT over other technologies are: (1) non-contact and non-invasive; (2) high-speed; (3) large-area; (4) simple operation; (5) intuitive and easy-to-understand results; and (6) a wide range of inspection objects such as metallic, non-metallic, and composite materials. For example, with IRT technology, it is possible to measure the temperature of extremely hot objects or dangerous products (e.g., strong acid, hot steel) at high speed in a non-contact, non-invasive, and large-area way so that their temperature distribution can be safely measured and users can be kept away from danger [
5,
7]. In addition, it is possible to perform high-speed scanning not only of stationary targets but also of fast-moving targets. In contrast to the harmful radiation effects of techniques such as X-ray imaging, IRT is radiation-free and suitable for long-term and repeated use.
Figure 1 illustrates the search results for the citation frequency and publication count of IRT keywords in the Web of Science database. The chart clearly reflects a gradual increase in both citation frequency and publication count, underscoring the continuous growth of research interest and study in the field of IRT. This upward trend suggests the increasing significance of IRT across various academic disciplines, motivating researchers to delve deeper into the applications and advancements of IRT technology. This is distinctly demonstrated in the chart, providing robust support and impetus for current and future IRT research.
In the past decade, the global photovoltaic (PV) market has grown almost exponentially in size. PV solar energy has strong competitiveness in the global energy market and has become a mainstream renewable energy technology [
6]. The IRT imaging method is an efficient and potent tool for qualitative examination of PV modules when compared to conventional I–V characteristics. It can reliably pinpoint the specific position of defects in PV power plants in addition to detecting their presence in the system. For example, for a normal PV module, the incident irradiance causes a uniform temperature distribution on its surface. On the contrary, for most faulty PV modules, the thermal behavior of the PV module affects its surface temperature distribution, resulting in various inhomogeneities in the temperature distribution. This means that with minimal instrumentation, no direct contact, and no interruption of the functioning of the PV system in real-world conditions [
8,
9], details regarding the thermal characteristics and the precise physical location of the fault can be quickly obtained to quantitatively diagnose the presence of a faulty cell, cell bank, or module.
The electronics industry, stemming from the advancement and application of electronic science and technology, is not only one of the pillar industries of the national economy, but also an emerging science and technology development industry. In integrated circuits, for example, the electronic circuits of printed circuit boards (PCBs) are widely made [
10,
11], and these contain a high density of electronic components in the board power supply and many electronic connections, which are potential manufacturing defects. And the identification and localization of these defects are critical to the error-free performance of PCBs. Typically, defects produce abnormal temperature patterns that can be detected by the IRT. For example, transparent components are a key core component of smart terminals (one of the pillar industries of the electronics industry). Common transparent components mainly include the cover of the display, the light guide plate, etc., in this field [
3]. 3D glass cover components are prone to defects (scratches, microcracks, microbubbles, water ripples, etc.) during the manufacturing process. According to statistics, the yield rate of 3D glass cover components is less than 75% [
12,
13,
14], so high-performance detection of defects to improve the yield rate of the final smart terminal products is a technical challenge to be overcome. However, the defect detection process of transparent components has special characteristics with high light transmission and reflectivity, and the existing process is mainly manual. Relative to the traditional optical machine vision detection method, thermal spray infrared imaging will be controlled by a high-temperature gas through a moving nozzle, heating intelligent terminal transparent components due to the thermal resistance effect, component defects of the geometry, spatial location, etc. In the process of heat transfer, therefore, the defects in the vicinity of the spatial temperature evolution have a certain degree of variability compared to the normal, so the difference can be captured by thermal infrared imaging.
Due to the rapid expansion of the renewable energy sector, a dedicated section has been included in this paper to delve into the intricacies and developments within this industry. The fusion of IRT and advanced deep learning techniques represents a substantial leap forward in improving the accuracy and efficacy of detecting and diagnosing defects in PV panels [
15]. For example, commonly used algorithms include convolutional neural networks (CNN), chaos synchronization detection method (CSDM), and genetic algorithm (GA) [
16,
17,
18,
19]. This integration harnesses the power of IRT’s thermal imaging to capture nuanced temperature variations across PV panel surfaces, and when combined with deep learning algorithms [
20], the system can not only identify defects but also offer enhanced predictive capabilities [
21]. Through this interdisciplinary approach, the ability to precisely pinpoint and diagnose issues in PV panels is substantially elevated [
22,
23].
The detection of electronic components presents a unique set of challenges owing to their complex and intricate structures [
24]. In this context, the process of detecting defects and anomalies typically necessitates external excitation to induce heating within these electronic components. This, in turn, enables the capture of thermal radiation emitted by the object under inspection, facilitating the creation of a thermal image. In the realm of infrared thermal imaging for the detection of electronic components, lasers have emerged as a common and preferred excitation source [
25,
26]. The utilization of lasers at the 808 nm wavelength has demonstrated several advantages in electronic component inspection [
27]. Firstly, it ensures the accurate targeting of specific areas of interest on the component, facilitating a controlled heating process. Additionally, this wavelength is well-matched to the spectral response of many infrared cameras, enhancing the efficiency of data acquisition [
28]. As a result, the thermal images captured exhibit clarity and detail, enabling the detection of defects or anomalies with high precision.
3. Renewable Industry
PV solar power generation has become an indispensable component of the global energy landscape [
71,
72]. The long-term performance and overall reliability of PV modules are significantly influenced by faults occurring both in real-world operational conditions and during transportation and installation [
73,
74]. These faults lead to specific abnormal operations, primarily characterized by reduced power output, abnormal module surface temperature distribution, excessive thermal/mechanical stress, and even safety risks [
75,
76]. Traditional electrical performance testing of PV modules is a mature testing method, but it has limited fault-detection capabilities [
77]. With the advent of digital cameras, charge-coupled devices (CCDs), and uncooled focal plane array (UFPA) detectors, optical-based infrared thermal imaging detection has gained popularity [
78,
79]. Specifically, electroluminescence (EL) and IR imaging prove to be potent tools for the qualitative assessment of PV modules, enabling the detection of faults in PV installations and precise identification of their exact locations [
80]. In conducting this research, a total of 94 literature reviews published between 2000 and 2023 were identified on the Web of Science. These reviews covered various domains, including energy fuels, engineering, and computer science, and were obtained by limiting the search to reviews related to IRT detection in PV.
Table 2 presents the top five most-cited reviews in this domain.
Regarding the number of literature searches on the infrared detection of photovoltaic panels in Web of Science,
Table 3 provides an overview of key annual performance indicators. The table demonstrates a noticeable upward trajectory in the volume of the literature, reflecting the growing interest in the research field. However, it is worth noting that both the average citation count and H-Index for each publication exhibit a declining trend, as indicated in
Table 3. This trend can be attributed to the tendency for older literature to accumulate more citations. Of particular interest is the anomaly in 2019, where, despite a substantial increase in publications, there was a sharp decrease in the average citation count per publication and the average yearly citation rate per publication.
Numerous investigations have been carried out, and there has been a recent surge in publications focusing on assessing the suitability of IRT for the detection of PV anomalies [
84]. Kandeal et al. [
21] accomplished this by meticulously analyzing the available data from the Scopus database and using the VOSviewer tool [
85] to create a bibliometric network to illustrate the literature. These networks were presented in the schematic representation of keyword relationships (
Figure 8). In this illustration, the size of each circle signifies the occurrence frequency of keywords, the thickness of the connecting lines represents how frequently these keywords co-occur, and the color-coding denotes the year of publication. As depicted, the IRT method has been widely employed across various imaging applications and has found substantial utility in the monitoring of PV conditions, particularly from 2016 onward.
3.1. Optical Degradation
One of the pivotal attributes of high-quality PV front encapsulation materials is achieving optimal optical transmission efficiency [
86,
87]. However, when deployed in real-world conditions, PV modules encounter an array of environmental challenges, including elevated temperatures, humidity, exposure to ultraviolet (UV) radiation, wind, and snow pressure [
88,
89]. Among these environmental stressors, moisture can infiltrate the interior of the solar panel through various pathways, including its edges, rear section, or any voids like cracks in the panel structure [
90,
91]. The pathway leading to optical deterioration in PV modules as a consequence of moisture infiltration is depicted in
Figure 9a [
92]. As time progresses, the concern regarding optical degradation intensifies and, in the most severe scenarios, can result in a reduction of over 50% in the rated power output of the PV module [
93]. Therefore, it is crucial to understand the attributes of imperfections and the fault mechanisms responsible for the optical deterioration of PV devices. This understanding is essential for preventing further degradation and the development of additional failure mechanisms [
94].
IR imaging offers insights into the temperature distribution across the surface of the PV module and the location of defects or fault modes [
95]. Faulty cells result in mismatch losses, thereby leading to an uneven distribution of cell temperature (
Tc) across the PV module. The malfunctioning cells operate at elevated
Tc levels, creating hotspots that subsequently affect the module temperature (
Tm) [
96].
Figure 9b displays the IR image of PV Module X, while
Figure 9c–e present magnified EL images of the highlighted regions in
Figure 9b [
97]. These highlighted areas in
Figure 9b are in proximity to the module’s frame and represent the most critical hotspots, indicating the presence of significant leakage current during operation. It’s worth noting that hotspots are distributed throughout the module. The positioning of hotspot cells near the PV module’s frame aligns with the findings from the electroluminescence (EL) images. The abundance of hotspot cells implies that a substantial portion of the cells in field-aged PV Module X are experiencing various stages of degradation. In
Figure 9c, no evident cracks are detected, but the highlighted region in
Figure 9b shows hotspots. These hotspots in
Figure 9b may result from metal grid corrosion and/or solar cell degradation. Moving to
Figure 9d, it reveals the existence of microcracks, with the warmest cells identified in this area on the IR image (as seen in
Figure 9b). In contrast,
Figure 9e displays some cracks, but the hotspots in its corresponding area on the IRT image are not as pronounced as those in
Figure 9d. The significance of cracks in facilitating current flow underscores the occurrence and severity of the hotspots observed in
Figure 9d. The Δ
T of PV Module X was approximately ∼8.2 ± 2 °C [
98].
Figure 9.
PV module in the field: (
a) under environmental stressors e.g., high humidity, temperature, and UV radiation, moisture can enter the PV module [
92]; (
b) IRT characteristics of PV Module X acquired under clear sky outdoor conditions; (
c–
e) EL characteristics acquired under
Isc bias conditions of the corresponding marked areas in (
b) [
97].
Figure 9.
PV module in the field: (
a) under environmental stressors e.g., high humidity, temperature, and UV radiation, moisture can enter the PV module [
92]; (
b) IRT characteristics of PV Module X acquired under clear sky outdoor conditions; (
c–
e) EL characteristics acquired under
Isc bias conditions of the corresponding marked areas in (
b) [
97].
Addressing the current drawbacks in industrial production lines, such as low defect detection efficiency, limited data, and high error rates, is crucial due to the significant impact of defects in the silicon photovoltaic (Si-PV) cell manufacturing process on the normal power generation of PV systems. Hence, defect detection is of utmost importance. Du et al. [
99] introduced a defect detection and classification method for Si-PV cells based on IRT and CNN. The method involved fine-tuning the LeNet-5, VGG-16, and GoogleNet models after generating the dataset. After 71 training iterations, the GoogleNet model consistently achieved 100% defect classification accuracy with a verification accuracy of 100% and a loss of 0.002. However, training was halted at this point since no significant improvements were observed, and the model reached its peak stability at the highest accuracy. The VGG-16 model attained its highest defect classification accuracy after 121 training iterations, achieving a verification accuracy of 97.67% and a loss of 0.15. While the LeNet-5 model could also achieve a 100% precision value, it exhibited instability and significant fluctuations during the training process. Balasubramani et al. [
100] proposed a method for detecting ethylene vinyl acetate (EVA) discoloration and delamination defects based on the thermal pixel counting (TPC) algorithm. Temperature indicators, namely T
15 and T
20, were introduced to highlight the temperature pixel distribution at ΔT °C = 15 °C and 20 °C, respectively. These indicators were compared with healthy panels to validate the algorithm’s effectiveness. The classification was automated using a fuzzy classifier, adjusting classification boundaries by modifying fuzzy IF-THEN rule certainty levels while keeping membership function parameter values constant. This approach, particularly the use of the certainty factor (CF) in the fuzzy classifier, significantly improved classification accuracy, surpassing other methods by an average of 10%.
3.2. Electrical Mismatches and Degradation
The term “electrical mismatches” encompasses a range of fault types, including cell cracks, snail trails, broken interconnecting ribbons and busbars, shunts, and poor soldering [
100]. These faults are not always discernible through straightforward visual inspection, especially when it comes to optical degradation. Typically, power loss and thermal degradation in faulty modules can lead to an increased risk of safety issues in the entire PV system [
6]. Mismatched voltage characteristics can lead to uneven current distribution, thereby affecting the overall performance of the system. Current imbalances between different components may result in electrical mismatch issues among the modules [
101].
The commonly employed method for diagnosing faults in solar PV panels is the measurement of Current–Voltage (
I–
V) characteristics. However, this approach is time-consuming and lacks the ability to categorize defects like delamination, discoloration of EVA, and isolation of cell parts resulting from cell cracks [
102]. Pei and Hao [
103] presented fault indicators based on current and voltage to detect faults in PV systems. According to the experimental results by Tsanakas et al. [
6], cracks in PV modules were actually diagnosed through the
I–
V characteristics. These interconnection material issues in a single cell or within a cell string occur due to physical strains during transport or installation, thermal cycling leading to thermomechanical stresses, subpar soldering, and potential hotspots arising from extended PV system operation in real-world conditions [
104]. Detecting broken interconnections is straightforward using optical techniques such as EL, IRT, ultraviolet (UV) imaging, or through basic
I–
V characterization (see
Figure 10).
Figure 10a illustrates the typical
I–
V characteristic output.
Figure 10b,c display the thermal images of PV modules with electrical mismatches, attributed to interconnection ribbon fractures (
Figure 10b) and soldering/busbar defects (
Figure 10c) as observed through IRT.
Figure 10a shows the typical
I–
V characteristic output, while
Figure 10b,c display thermal images of PV modules. These thermal images reveal electrical mismatches due to interconnection ribbon fractures (
Figure 10b) and soldering/busbar defects (
Figure 10c), as observed through IRT. Belhaouas et al. [
105] employed thermal imaging to investigate the performance of solar PV modules after outdoor exposure. The thermographic inspection revealed that the temperature of PV cells inside the PV modules ranges from 32 °C to 68.2 °C, as given in
Figure 10d. This temperature variation occurs while the average ambient temperature during the thermal inspection is 23 °C. The thermal inspection found that the deployed PV modules, regardless of their glass types, primarily experience minor temperature mismatch (Δ
T) at 90.27%, followed by major Δ
T mismatch at 9.58%, and a critical Δ
T mismatch case at 0.13%. Nonetheless, PV modules with textured glass exhibit slightly lower thermal stress levels compared to those with float glass. Tsanakas et al. [
106] assessed the suitability of thermal image processing and edge detection for defect detection in PV modules. The approach combined image segmentation with Canny edge detection and has yielded favorable results through on-site thermal imaging measurements of two PV arrays: PV-1 and PV-2. It successfully identified 13 out of 14 faulty cells in PV-1 and 27 out of 29 faulty cells in PV-2 by detecting hotspots within the edge maps. These identified hotspots were validated against the standard electrical tests conducted on each module before the experiments, revealing a performance decline of 9.5% for PV-1 and 9.7% for PV-2, respectively. Aziz et al. [
107] exploited continuous wavelet transform to generate two-dimensional (2D) images from PV system data and utilized CNN for PV system fault classification, achieving a circuit fault detection accuracy of 73.53%.
3.3. Non-Classified Faults
In addition to optical degradation and electrical mismatches and degradation, faults such as potential-induced degradation (PID) and defective bypass diodes (short circuits) are informally referred to as “non-classified” faults. PID is a relatively newly identified fault mechanism in operational PV modules and remains an area with limited comprehensive research and understanding. It involves a crucial externally induced factor, typically accelerated in hot and humid conditions, resulting in significant degradation and power loss within the affected PV modules [
108,
109].
Researchers have conducted various algorithm and laboratory tests to detect “non-classified” faults. For instance, Bouaichi et al. [
110] assessed the PID recovery process in affected PV modules using IR evaluation. PID can be considered a factor affecting the durability and power output of crystalline silicon modules. Lu et al. [
16] employed a hybrid algorithm combining chaos synchronization detection method (CSDM) and CNN for the investigation of fault detection in PV modules. The discussion encompassed four prevalent states observed in PV modules: the normal state, module damage state, module contact defect state, and module bypass diode failure state. The research findings showcased the proposed method’s remarkable recognition accuracy of 99.5% when 400 sets of randomly generated fault data (with 100 data points for each fault) were inputted, surpassing the traditional edited nearest neighbor (ENN) algorithm’s recognition rate of 86.75%. Tao et al. [
17] introduced a genetic algorithm-optimized deep belief network (GA-DBN) for diagnosing PV faults, covering normal operation, grounded short circuit, open-circuit in series, partial shadow, and abnormal aging. Although achieving an impressive overall diagnostic accuracy of 95.73%, it’s important to note that the average training time was relatively long at 316.34 s, primarily due to the intricate optimization process involving the initial weight and bias of the DBN through GA. Manno et al. [
18] achieved optimal performance with CNN using thresholding as a preprocessing method, achieving a 99% accuracy on mid-range CPUs in less than 30 min. Additionally, simplification of thermal imaging images, representing various operational states of PV modules, can achieve high precision. Considering a dataset consisting of 200 sliced images, the same configuration resulted in 90% accuracy for the MLP network and 100% accuracy for CNN.
Figure 11 displays various thermographic images utilized for CNN training, the thermographic image in
Figure 11a was taken by an operator using a standard lens.
Figure 11b shows a non-perpendicular thermographic image angle, and
Figure 11c, captured with a standard lens, includes multiple PV modules. In
Figure 11d, the thermographic image was acquired using a wide-angle lens and encompasses several PV modules. Mellit [
111] adopted an embedded system for fault detection and diagnosis in PV modules, utilizing IRT and deep convolutional neural networks (DCNNs). Two DCNN-based models were developed, one for fault detection and the other for fault diagnosis. Despite the limited dataset size, simulation results indicate a remarkable accuracy of 99% for fault detection and a quite impressive 95.55% accuracy for fault diagnosis. As shown in
Figure 11e, the classifier accurately identifies instances of dust deposition on the PV surface, with a recognition accuracy of only 95.5%. In fact, this is due to the similarity in contours between partial shading effects and dust accumulation, as well as PV modules with short circuits and damaged bypass diodes. Dhimish et al. [
112] imported a novel PV hotspot fault detection algorithm based on cumulative density function (CDF) modeling technique, achieving an accuracy of 80%.
3.4. Summary
The advantages of the machine-learning-based method over traditional methods are manifold. Machine learning algorithms can adapt and learn from data, allowing them to improve their performance over time without the need for manual adjustments. This adaptability is a significant advantage when dealing with complex and dynamic systems [
113]. Machine-learning-based methods undoubtedly offer numerous advantages for IRT applications. However, like any approach, they do come with certain disadvantages that need to be considered in the context of IRT. Machine learning models, especially deep learning models, require large amounts of data for effective training. In the case of IRT, acquiring a substantial dataset, particularly for rare or specific defects, can be challenging [
114]. Furthermore, efforts must be made to make machine learning models more interpretable and transparent in the context of IRT to establish trust and confidence in their results.
Machine learning is a widely-used technology that relies on algorithms and models to enable computers to learn from data and make decisions. Deep learning, on the other hand, is a branch of machine learning that involves artificial neural networks, which can simulate the workings of the human brain to process vast amounts of complex data. The integration of IRT with deep learning plays a pivotal role in detecting and diagnosing defects in PV panels [
115,
116]. Initially, the technique of IRT is employed to capture thermal images of the PV panels. These thermal images depict the temperature distribution across the surface of the PV panels, where defects typically manifest as anomalous temperature patterns. Preprocessing of the thermal images may be necessary to eliminate noise, enhance contrast, or adjust image dimensions to ensure compatibility with deep learning models [
117]. Deep learning models [
118], such as CNN [
99] or GA-DBN [
17], are then utilized to learn and extract features pertaining to defects from the thermal images [
119]. These models possess the capability to autonomously acquire knowledge and recognize patterns within the thermal images, including potential defects. The deep learning models excel in automatically discerning complex patterns and temperature distributions within the IRT, thereby enhancing the accuracy of fault detection and diagnosis [
120]. This amalgamation enables the automation of the detection and diagnosis processes, reducing the reliance on manual intervention and significantly enhancing overall efficiency.
For instance, despite the relatively limited scale of the dataset employed in Mellit’s study [
111], simulation results demonstrated a fault detection accuracy of 99% and a fault diagnosis accuracy of 95.55%, as shown in the
Figure 12. In most cases, this method can identify various types of defects in PV panels, including but not limited to hotspots, cracks, dirt, and cell damage. Performance metrics for detection may encompass accuracy, recall, and precision, among others, and these metrics are generally contingent on the specific problem and model configurations. In summary, the fusion of IRT and deep learning offers an efficient and highly accurate solution for detecting defects in PV panels. It holds the potential to play a crucial role in the monitoring and maintenance of PV energy systems.
Table 4 summarizes the application of the combination of IRT and deep learning techniques for defect detection and diagnosis of PV panels.
Table 5 presents a comprehensive comparative analysis between research conducted by scholars in the past and the current state of research. Historically, the majority of studies were primarily focused on the conventional methods for PV panel inspection. In contrast, contemporary scholars are placing significant emphasis on the integration of deep learning with IRT techniques. This shift in focus reflects the evolving landscape of research in this field and the recognition of the potential of advanced methods for more precise and efficient PV panel defect detection. The utilization of deep learning in conjunction with IRT is emerging as a promising avenue for achieving higher accuracy and reliability in the inspection of PV panels.
Automatic photovoltaic inspection has garnered significant interest from researchers in recent years. Numerous studies have explored automatic photovoltaic inspection using various imaging methods. Demant et al. [
130] employed a support vector machine algorithm for the automatic classification of cracks in photoluminescence (PL) images. Stromer et al. [
131] proposed an enhanced EL image crack segmentation framework. Li et al. [
132] adopted image processing algorithms for the automatic detection of snail trails and dust in visible light images. Su et al. [
133] utilized newly proposed feature descriptors to classify manufacturing defects in solar cell EL images. However, there has been limited research on the application of deep learning for defect detection in photovoltaic component images. These studies, including those by Chen et al. [
134], Ding et al. [
135], and Li et al. [
136], have leveraged deep learning techniques to detect defects in visible light (red, green, blue) RGB images. Demant et al. [
137] used CNN for automatic quality assessment and control during the production of solar cells in PL images. Deitsch et al. [
138] and Akram et al. [
139] employed deep learning methods for the automatic detection of faults in solar cell EL images. This represents a notable shift toward utilizing deep learning approaches for photovoltaic inspection.
4. Electronic Industry
With the progression of information electronic devices towards high reliability, miniaturization, light weight, and multifunctionality, high-density integrated circuits with numerous functional components have found extensive applications [
140,
141]. PCBs, serving as critical structures for electrical and pneumatic interconnection, signal transmission, mechanical linkage, and electronic system support, are also the primary failure-prone areas of components. The long-term reliability of PCBs has become a focal research topic, resulting in challenges associated with effectively and reliably detecting PCBs’ defects. Traditional PCB defect detection methods have limitations, but active IRT, including techniques like pulsed thermography and lock-in thermography, has found extensive use in non-destructive testing for PCBs. The development of very large-scale integration (VLSI) technology, increasing silicon wafer diameters, and decreasing integrated circuit linewidths have imposed higher demands on silicon wafer manufacturing processes and surface quality [
142]. During semiconductor silicon wafer production, the formation of microcrack defects is common, ultimately affecting the quality of silicon-based microelectronic products. Ensuring the quality and performance of products necessitates non-destructive testing of silicon wafers. Surface mount components achieve interconnection between chips/packages and substrates or PCBs using solder bumps. However, common manufacturing defects, including opens, cracks, or missing solder bumps, persist. As solder bumps are concealed within packages after assembly, the increasing trend towards high-density and ultra-fine pitch has made defect detection progressively more challenging, severely impeding the advancement of surface mount technology. Detecting defects in solder bump protrusions has become a critical issue in integrated circuit manufacturing technology. Concealed solder bump protrusions impede the entry of light beams, and infrared imaging proves to be an effective detection technique capable of identifying nearly all solder bump defects.
Refer to the number of literature searches on Web of Science on the application of infrared thermal imaging technology in electronic industry defect detection, and the results are shown in
Table 6.
Table 6 provides an overview of the key annual performance indicators. As can be seen from the table, the number of literatures is in a slightly fluctuating state each year, indicating that people’s interest in this field has not changed much. It is worth noting that the average number of citations and the H-Index of each publication are almost horizontal, but suddenly decline in 2022. This trend can be attributed to the lack of in-depth research in the field. Of particular interest is the anomaly of 2017, in which the average number of citations per publication rose sharply, even though the number of publications was not as high as before.
Table 7 makes a comprehensive comparative analysis of the research of past scholars and the current research status. Historically, the feasibility of infrared non-destructive testing technology has brought a lot of convenience to the electronics industry, and accumulated experience for the subsequent research. With the improvement of technology and the deepening of research, it can be seen that contemporary scholars have added the cost factor to the concern of non-destructive testing in the electronics industry. Future cost reductions will also make infrared non-destructive testing technology have a better market.
4.1. Chip
Since the 1960s, advancements in semiconductor technology have profoundly transformed our lives and facilitated the development of high-performance electronic devices. The emergence of smartphones, for instance, would not have been possible without the progress in miniaturized and high-performance semiconductors. The demand for lighter, more compact smartphones necessitates the production of smaller, thinner, and higher-performing semiconductor chips. With the growing trend of using thinner wafers for semiconductor chips, various issues have emerged, including a significant concern related to microcracks that can be found on the surface and sub-surface, varying in size from a few micrometers to several tens of micrometers. Semiconductor chip materials are inherently brittle, making them susceptible to stress-induced cracks during chip manufacturing and assembly. These cracks manifest primarily as scratches, fractures, orange peel effects, and pits [
150]. Surface cracks can adversely affect the performance and reliability of the final electronic device, thus escalating the demand for inspecting surface cracks in semiconductor chips during the manufacturing process. Efficient and high-precision non-destructive testing is crucial for semiconductor chip inspection. Optical visual methods, while offering non-contact and non-destructive three-dimensional chip characterization, have limitations in detecting concealed defects. Active IRT bestows the following advantages for semiconductor chip inspection: complete non-contact, non-destructive, and non-invasive testing, along with the capability to examine large areas in a single test. IRT has emerged as one of the most promising techniques in non-destructive testing and evaluation [
145].
Introducing non-contact active IRT technology into chip defect detection involves the use of an external heat source, such as a flash lamp or laser, for active thermal imaging. When subjected to external heating, the presence of defects within the chip leads to abnormal thermal resistance, enabling the capture of thermal distributions using an infrared imaging device. Analyzing thermal images aids in defect identification, with laser excitation being the most frequently used method for semiconductor chip defect detection among various external excitation techniques. Bu et al. [
151] investigated a method utilizing Barker code-modulated pulse compression waveforms for detecting microcrack defects in semiconductor silicon wafers. This technique employed an optical infrared thermal imaging device for transmission, where an infrared camera captured the thermal wave signal response to the laser-modulated Barker code waveform. The acquired images were stored as sequences and analyzed for detectability using a full-harmonic distortion algorithm, resulting in improved defect detectability. An et al. [
26] introduced the line laser lock-in thermal imaging technique for semiconductor chip inspection. This technique integrated a line-scanning laser source, an infrared camera with a dedicated lens, and a control computer, assembling a novel line laser lock-in thermal imaging system as shown in
Figure 13a. The continuous wave laser beam was modulated into a pulsed laser beam by the excitation unit, and the cylindrical lens transformed the pulsed laser beam shape from point-like to linear. The control unit then issued control signals to the galvo scanner, directing the line laser beam onto the target surface. Subsequently, the line laser beam generated a thermal wave along the desired excitation line, performing horizontal and vertical scans on the target surface, effectively detecting randomly oriented cracks, as shown in
Figure 13b. Yang et al. [
152] proposed a multi-point laser lock-in thermal imaging system for real-time imaging of semiconductor chip cracks, as shown in
Figure 13c. This system employed multi-point pulsed laser beams to simultaneously generate thermal waves at multiple points on the target semiconductor chip surface. The corresponding thermal response was measured using a high-speed infrared camera, enabling real-time detection during the semiconductor chip manufacturing process.
Figure 13b,d illustrates a comparative diagram of semiconductor chip defect detection using the same excitation source—laser—in different modes. The integration of infrared sensing technology with the lock-in method significantly improved the sensitivity and resolution of thermal imaging. The sensitivity of thermal imaging was increased by two orders of magnitude, reaching approximately 100 μK, while the resolution for surface defects was lowered to 5 μm [
153].
4.2. PCBs
PCBs serve as crucial structures for achieving electrical and pneumatic interconnection, signal transmission, mechanical linkage, and support for electronic systems. They also represent the primary failure-prone areas for components, especially in high-frequency and high-voltage circuits. Hence, the detection and maintenance of faults in PCBs are critical due to their complex multi-layered structures, leading to various defects such as layer separation, delamination, breakdown damage, and micro-holes during processing and usage. Conventional defect detection techniques for PCBs encompass visual inspection by human operators and automated optical inspection, X-ray, CT imaging, ultrasound, laser ultrasonics, and terahertz imaging. While manual visual inspection and automated optical inspection are the most common methods, they are limited to detecting visible surface-level defects and cannot guarantee the absence of internal flaws. IRT inspection, as a non-contact measurement method, has gradually found application in the field of PCB fault detection. PCB fault detection methods based on IRT mainly involve three steps: thermal source identification, feature extraction, and thermal pattern recognition [
154].
Figure 14a shows 2D and 3D views of the PCBs transient amplitude images. Wang et al. [
155] employed laser-induced lock-in thermography to detect various real defects in rigid or flexible PCBs. Phase characteristic images enabled effective detection of delamination defects with a depth of 1.2 mm and micro-hole defects with a depth of 400 µm. The reference regions for both defective and non-defective areas are illustrated in
Figure 14c. Experimental results demonstrated that laser-induced thermography is suitable for detecting multiple types of PCB defects. Avdelidis et al. [
156] utilized two different integrated pulse thermography systems: thermoscope and echotherm. In both cases, mid-wave infrared cameras were used; a merlin 3–5 µm thermoscope system and a phoenix 3–5 µm echotherm system. Both systems were state-of-the-art portable non-destructive testing and electronic inspection systems with integrated flash heating capability. The results showed that pulse thermography can be used for defect detection in circuit boards (i.e., delamination and/or soldering defects). Cong et al. [
157] proposed and utilized optical/thermal fusion imaging technology to inspect PCBs. A semiconductor laser diode with a wavelength of 808 nm was employed as the radiation source. Sample data and images were acquired using a mid-infrared camera. Phase-locked thermal imaging was employed for the study of layered defects in PCBs, as illustrated in
Figure 14b. Six different fusion algorithms were applied in the experimental study of image fusion, and four metrics were introduced to evaluate the fusion performance. The experimental results indicate that this fusion technology maintains a high level of accuracy and precision under diverse imaging conditions.
4.3. Weld
Solder joints constitute crucial components on PCBs. Apart from serving as electrical conduits, they also provide mechanical connections between electronic components and the substrate. Solder joints are more susceptible to defects such as cracks, voids, and missing balls, as depicted in
Figure 15a [
159]. These flaws can adversely affect the performance and lifespan of flip-chip packages, leading to erratic circuit behavior and intermittent instability. This poses significant risks for debugging, operation, and maintenance of circuits. Therefore, the assessment of solder joint integrity holds paramount importance. Presently, conventional non-destructive testing methods such as X-ray, optical inspection, and flying probe testing struggle to effectively detect such welding defects. In contrast, infrared non-destructive testing offers a wide applicability, non-contact measurement, rapid detection, high precision, ease of qualitative and quantitative analysis, as well as convenient observability, presenting a comprehensive set.
Chai et al. [
160] proposed an active transient thermography technique for detecting inverted solder balls. When a solder ball is defective, its resistance is significantly higher than that of a normal solder ball, leading to an abnormal temperature. Hence, using thermal image contrast from an infrared sensor, this method detects the presence and location of defective solder balls, primarily void defects and localized cracks. Lu et al. [
161] investigated a pulse-phase thermography-based method for identifying solder joint defects. In this approach, the test chip is stimulated with a thermal pulse, and the subsequent transient response is captured using a commercial thermal imaging camera. Thethermal imager was employed to measure the transient response of the test chip under infrared photothermal excitation. The thermal imager, equipped with a micro-lens with a pixel resolution of 25 μm, enhances spatial resolution. The temperature resolution of the thermal imager, utilizing a microbolometer detector, is superior to 80 mK, with a spectral response range of 7.5 to 14 μm, and a frame size of 640 × 480 pixels. Wei et al. [
162] developed an intelligent system for detecting solder joint defects using active thermography.
Figure 15b illustrates the experimental setup, employing a fiber-coupled semiconductor laser with a central wavelength of 808 nm as the heat source, monitored by the thermal imager. Statistical features were extracted and classified using the M-SVM algorithm. All missing protrusions were identified, achieving the highest recognition accuracy. The results demonstrate that the combination of active thermography and M-SVM is an effective method for intelligent diagnosis of microelectronic packaging solder material defects. He et al. [
163] utilized a pulsed laser with a central wavelength of 808 nm to heat the substrate of the test sample SFA1. The sample consisted of 25 solder balls arranged in a 5 × 5 pattern, with protrusion diameters and pitch distances of 500 μm and 1000 μm, respectively. Thermal images of the SFA1 package were acquired using the VH680 infrared imager. The experimental setup is depicted in
Figure 15c, while
Figure 15d shows the thermal image of the experimental sample SFA1. The matrix was used as the desired output vector, and a transformation function was applied to convert the desired output vector from an index to a vector. A PNN was then established with input vectors, output vectors, and propagation speed as parameters. The results indicate that the infrared detection system based on PNN is effective for defect detection in high-density packaging.
Figure 15.
Schematic diagram of welding defect detection: types of weld defects (
a) [
159]; (
b) schematic of experimental setup and distribution of welds in test samples [
162]; (
c) experimental setup and distribution diagram [
161]; (
d) infrared thermal images of weld defects [
163].
Figure 15.
Schematic diagram of welding defect detection: types of weld defects (
a) [
159]; (
b) schematic of experimental setup and distribution of welds in test samples [
162]; (
c) experimental setup and distribution diagram [
161]; (
d) infrared thermal images of weld defects [
163].
4.4. Others
Glass fibers are extensively utilized as reinforcement materials, with glass-fiber-reinforced polymers (GFRP) commonly found in electrical and electronic devices, as well as in numerous components used in our daily lives [
12]. Glass fibers present a competitive edge due to their lightweight nature and lower cost compared to other reinforcement materials like carbon fibers [
164], showcasing superior properties within composite materials [
165]. However, the manufacturing process may incur defects, especially the formation of voids. Fuel cells are essential components in emission-free energy conversion, directly converting chemical energy into electricity. The critical aspect of fuel cell functionality lies in the necessity for all distinct sealing layers to be both electrically insulating and hermetic. The material connecting the two steel interconnect sections of the cells is the glass solder layer, which incorporates artificially induced defects in the form of missing solder of varying diameters.
Meola et al. [
166] conducted an assessment of GFRP under low-energy, low-velocity impact using IRT. They employed a equipped with a quantum well infrared photodetector (QWIP) operating in the 8–9 μm range, with a spatial resolution of 640 × 512 pixels at full frame. For the purpose of comparison, thermal imaging and visible light images of the same sample are presented in
Figure 16a. The results demonstrated that non-destructive testing utilizing lock-in thermography could detect manufacturing defects such as uneven resin distribution, porosity, fiber misalignment, and impact damage. Dua et al. [
167] introduced a high-depth resolution frequency-modulated thermal wave imaging technique for infrared characterization of GFRP laminates. Each GFRP sample comprised five patches, with a thickness of 2 mm. The selected samples were subjected to experiments using two 1 kW halogen lamps. Thermal distributions of the samples were recorded by an infrared camera at a frame rate of 25 Hz. The results indicated that the layer-wise detection capability of time-correlated coefficient images significantly outperformed the widely used phase-based post-processing methods. Muzaffar et al. [
168] proposed a rapid and straightforward method for detecting faults in antenna arrays using infrared thermal imaging. The thermal imager employed was a 14-bit, 320 × 240 resolution mid-wave infrared (MWIR) camera from FLIR. The study demonstrated that IRT could be applied for detecting faulty elements in antenna arrays, with the variation of temperature rise on the absorptive screen being crucial for identifying the faulty components.
Figure 16d,e respectively present the sample image and the corresponding thermal imaging of defects. Wei et al. [
169] advocated the application of artificial intelligence techniques for automatic processing of infrared images to detect defects within the glass seal layer of solid oxide fuel cells. Three methods were investigated: (1) support vector machine, (2) adaptive enhancement, and (3) U-Net. The results indicated that features extracted from individual thermal profiles might be insufficient for defect identification, while U-Net displayed significant potential in thermal image segmentation. Wang et al. [
170] conducted experimental studies on the detection of impact damage in GFRP using pulse radar thermal wave imaging technology. They utilized a high-performance, cooled focal plane infrared imager with a response wavelength of 3.6–5.2 μm and pixel dimensions of 640 × 512. An 808 nm semiconductor laser was used, and various time/frequency domain analysis algorithms were applied to extract features from the thermal image sequences. The thermal image sequence was acquired using an IRT camera, The results showed that the dual-channel orthogonal demodulation algorithm exhibited excellent recognition capabilities for delamination defects in GFRP. Within the specified defect diameter and depth range, it could identify delamination defects with a depth ≥1.70 mm and a diameter-depth ratio (D/H) ≥2.35. By analyzing the signal-to-noise ratio (SNR) of feature images, gong et al. [
171] quantitatively evaluated the detection ability of laser bidirectional thermal wave radar imaging (BTWRI) to detect defects of carbon/glass fiber reinforced polymer (C/GFRP).
Figure 16b is the sample used in the experiment. By comparing the signal-to-noise ratio of feature images on a frame-by-frame basis, the optimal ACC detection image was obtained.
Figure 16c shows the defect phase diagram and amplitude diagram of the sample.
4.5. Summary
The advancement of technology has led to increasingly stringent requirements for the quality of electronic components [
11]. This chapter provides an overview of the application of IRT in electronic component defect detection from four aspects. Firstly, it introduces the application of IRT in semiconductor chip defect detection. Laser is commonly used as the excitation source, but not all thermal imaging techniques are suitable for detecting defects within semiconductor chip encapsulation. To address this, phase-locked thermography has been developed, which can overcome two limitations of IRT: the inability to differentiate surface and sub-surface features, and the lack of sensitivity. Next, it discusses the application of IRT in PCBs. The structure of PCBs and their relative positions on components are generally fixed. Defect detection in PCBs involves feature matching, and the accuracy of results varies with different parameters. Establishing a neural network in infrared non-destructive defect detection during soldering can enhance the feasibility of defect detection in soldering. In summary, IRT technology, by observing thermal distribution, can identify and address potential thermal issues, faults, or deficiencies in electronic components such as PCBs, chips, soldering, and GFRP.
Table 8 provides a summary of the applications of IRT excitation sources in electronic component defect detection and diagnosis.