Design of Artificial Intelligence-Based Novel Device for Fault Diagnosis of Integrated Circuits ^†

Abstract

The rapid advancement of integrated circuit (IC) technology has revolutionized various industries, but it has also introduced challenges in detecting faulty ICs. Traditional testing methods often rely on manual inspection or complex equipment, resulting in time-consuming and costly processes. In this work, a novel approach is proposed which uses a thermal camera and an Internet of Things (IoT) physical device, namely a Raspberry PI microcontroller, for the detection of faulty and non-faulty ICs. Further, a deep learning algorithm, namely You Only Look Once (YOLO), is coded inside the Raspberry PI controller using Python programming software to detect faulty ICs efficiently and accurately. Also, the various images of faulty and non-faulty ICs are used to train the algorithm and once the algorithm is trained, the thermal camera along with the Raspberry PI microcontroller is used for the real-time detection of faulty ICs and the YOLO algorithm analyzes the thermal images to identify regions with abnormal temperature patterns, indicating potential faults. The proposed approach offers several advantages over traditional methods, including increased efficiency and improved accuracy.

1. Introduction

Recent advancements in integrated circuits (ICs) and system on chip (soc) technology have significantly transformed numerous industries, including electronics, telecommunications, and automotive sectors. However, this progress has also presented significant challenges in detecting and ensuring the reliability of integrated circuits (ICs). The system faults can lead to critical system failures, reduced performance, the loss of production, and substantial financial losses. However, it is essential to locate/identify the fault and to isolate it in order to ensure recovery and safe mode operation [1].

Due to design or manufacturing defects and normal wear and tear, faults are developed [2]. The traditional methods of detecting faulty ICs often rely on manual visual inspection or complex and expensive testing equipment. Contact methods are used to identify the discontinuity in connections to ICs. Furthermore, non-contact methods such as X-ray, ultrasound, optical comparators, vision systems, computerized tomography (CT) scanning, long range, laser radar, thermal imaging, etc., are used for fault detection [3]. These approaches are not only time-consuming but also prone to human error and subjective interpretations. Also, it is essential to develop a module to diagnose a fault (fault detection) that may affect these system operations and to locate their root causes (fault isolation) [4]. As a result, there is a growing need for automated techniques that can streamline the detection process, improve efficiency, and enhance accuracy.

Thermal imaging has been gaining focus for its efficiency and reliability [3,5,6,7,8,9]. All objects in nature, as long as their temperature is not higher than the absolute temperature (−273 °C), display the irregular movement of molecules and atoms, which causes their surface to continuously radiate infrared light. In general, thermal imaging collects infrared light in the thermal infrared band between 8 μm and 14 μm, which lies in the electromagnetic spectrum between visible and microwave regions [9]. However, humans are not capable of visualizing these thermal radiations, the thermal cameras are utilized to visualize the thermal radiation emitted by the object. Once the thermal radiation emitted by the object is detected, the data are converted into gray value and the differences in gray value of each object are used for imaging. Furthermore, the thermal profiles of each object can be assessed in order to detect various parameters such as hotspots and the extent of heat spread and its location [3,9,10,11]. Thermal infrared imaging has proven its significance across various industries, including the medical, building and construction, agriculture, automotive, etc., industries. Furthermore, these industries showed improved efficiency, safety, and decision-making by making use of thermal imaging [12].

Non-contact thermal imaging combined with computer vision and machine learning are accurate, fast, and non-destructive to detect faults as in recent years, computer vision and machine learning algorithms have emerged as powerful tools for automated defect detection in various domains, and the fault diagnosis for integrated circuits with the help of automated defect detection can be generally classified into two stages, namely data acquisition and image classification [3,5,6,7,8,9,12].

Data acquisition is the process of collecting data, while image classification involves categorizing images based on their content. Further, the data acquisition process includes data acquisition, data pre-processing, the extraction of features, the training of learning models, and inference output. Common approaches for model training in image classification are support vector machines (SVMs), random forests, and deep learning, namely convolutional neural networks (CNNs), which can be time-consuming and resource-intensive [13,14]. These methods often require powerful hardware, such as graphical processing units (GPUs) or tensor processing units (TPUs), to train models effectively [14]. Also, real-time monitoring with these approaches can be challenging due to the computational demands involved in processing images in real-time. So, there is an increasing demand for a real-time approach that does not rely on high computational power, aiming to simplify the detection process.

2. Literature Survey

Over the last few years, many contact and non-contact methods have been used for the detection of fault diagnosis of ICs. Nowadays, non-contact methods are widely used for better performance and the detection of faults and are faster than contact methods. Further, non-contact methods are mainly carried out using X-ray, ultrasound, vision systems, computerized tomography (CT) scanning, long range lasers, laser-based radar, structured light, thermal imaging, etc. Of these mentioned techniques, thermal imaging is considered the best for the process.

Silva et al. (2013) proposed a technique using machine learning methods. The proposed method comprised three common steps, namely the extraction of features using principal component analysis (PCA), classification using the nearest neighbor (k-NN) and other methods, and the evaluation of the classifier’s performance using cross-validation (CV) technique [1]. Lo et al. (2019) presented a review on the diagnosis of systems using artificial intelligence (AI) approaches. Further, the authors discussed its applications, especially in the field of the diagnosis of complex systems [5]. Al-Obaidy et al. (2017) compared various soft computing methods, which were utilized for the fault detection of ICs. Also, the histogram thresholding is used to extract features that can be further reduced through principal component analysis. Furthermore, these minimized features can be given as input to the classifier which enables the classification of defects in PCB at IC level [3].

Redon et al. (2020) proposed a condition monitoring system via thermal image using a denoising technique for reducing noise. Denoising methods comprise two types, namely continuous wavelet transform and stationary wavelet transform [15]. Huo et al. (2017) proposed a self-adaptive fault diagnosis of roller bearings using infrared thermal images. In stage one, the authors decomposed the images using two-dimensional discrete wavelet transform (2D-DWT) and Shannon entropy. Furthermore, the authors utilized the histograms of selected coefficients as the input of the feature space selection method by using the genetic algorithm (GA) and nearest neighbor (NN) for the purpose of selecting two salient features that exhibit the highest classification accuracy [16].

The objective of this work is to combine thermal imaging with the YOLO algorithm and to develop an efficient and accurate system for the real-time monitoring and detection of faulty ICs based on their thermal characteristics and to overcome the limitations of existing methods.

3. Materials and Methods

The proposed device comprises components such as a thermal sensor, a Raspberry PI 4 Model B controller, and a battery. Figure 1 shows the overall block diagram of a proposed device. Further, the AMG8833-based thermal sensor is an 8 × 8 (64 pixels) two-dimensional non-contact type temperature detection module. Also, the thermal sensor is capable of transmitting infrared temperature readings through the inter-integrated circuits (I2C) protocol to the utilized Raspberry PI microcontroller.

Figure 2 shows the connection diagram of AMG8833 thermal sensor module with Raspberry PI microcontroller. The I2C utilizes two different pins, namely serial data (SDA) and serial clock (SCL). Further, the SDA and SCL of AMG8833 are connected to the SDA and SCL pins of the PI controller, respectively, which is shown in Figure 2. Also, the AMG8833 thermal sensor requires 3.3 volts for its operation, and it is fed using the Raspberry PI controller. In this work, a fast and compact module is proposed, which can be used to identify the IC fault conditions based on thermal profiles. Further, the short circuit faults based on electrical over stress are detected. Also, electrical overstress can be caused because of high voltage.

The proposed device is a simple handy device shown in Figure 2 and can be moved over any integrated circuits through non-contact types.

3.1. Proposed Approach for Fault Diagnosis

Figure 3 shows the proposed approach for fault diagnosis, which is composed of various stages, such as the preprocessing of input images and the training and testing phases of the YOLOV7 model.

3.1.1. Preprocessing of Images

Data preprocessing is a crucial step before feeding data to a model. Labeling is used to annotate faulty and unfaulty IC in images by drawing bounding boxes around them. These bounding boxes help the YOLO model to look for objects during training and testing. After labeling images, these data are converted into the .txt file format, which YOLO understands. This format includes details like the positions of the objects and their class labels.

3.1.2. YOLOV7 Algorithm

YOLO stands for “You Only Look Once”, and it is one of the most effective object identification methods for partitioning images into a grid system. Each divided cell within the grid is in charge of detecting objects on its own. Because of its precision and quickness, YOLO (you only look once) is one of the most well-known object identification techniques. When comparing YOLOV5 and YOLOV6 in terms of accuracy, YOLOV7 has a 100% accuracy rate. As a result, the algorithm used in this proposed work is YOLOV7. The design and development of YOLOV7 involves two stages, namely the training phase and testing phase.

In the training phase, 80% of the total input preprocessed faulty and unfaulty thermal images are given to the proposed YOLOV7 algorithm for training purposes. Once the YOLOV7 model is trained, the testing process is carried out. In the testing phase, 20% of the total input preprocessed faulty and unfaulty thermal images are given to the proposed YOLOV7 algorithm for testing purposes. Further, the performance metrics are evaluated to determine the efficacy of the proposed YOLOV7 model. The entire algorithm is coded using Python programming software and is executed using the Raspberry PI controller. Furthermore, the Raspberry PI, along with the thermal sensor module, acts as a handy device integrated with YOLOV7 and provides decision support regardless of whether the IC is faulty or unfaulty.

4. Results and Discussion

The normal circuit boards, especially ICs and the boards with faulty ICs, were considered for this study. For both faulty and unfaulty ICs, the required power supply was applied and the thermal images were obtained using AMG8833 thermal sensor module. Also, a total of 720 images, encompassing 360 faulty and 360 unfaulty thermal images, were acquired and stored. Further, these 720 faulty and unfaulty thermal images were utilized in this work to train and test the proposed YOLOV7 model. Out of 720 images, 576 images were used for training phase and the remaining 144 images were used for the testing phase of the YOLOV7 model. Further, the 576 faulty and unfaulty images were annotated using labeling software. Figure 4 shows the faulty and unfaulty image preprocessing using labeling software. Further, the faulty and unfaulty IC images were annotated by drawing bounding boxes around them and was given to the YOLOV7 model for training images.

Once the faulty and unfaulty thermal images were acquired and processed, the fault diagnosis was carried out. Further, 144 preprocessed faulty and unfaulty thermal images were utilized to carry out the performance of the proposed YOLOV7 model. Also, these 144 preprocessed faulty and unfaulty thermal images were fed to the proposed YOLOV7 model as the test images and the output prediction was obtained.

Figure 5 shows the fault diagnosis of ICs using YOLOV7 model. Further, it is seen that the prediction output of the YOLOV7 is given in terms of a bounding box. Also, the faulty and unfaulty ICs are predicted, and the prediction output is given as a caption at the top of the bounding box. Figure 6 shows the confusion matrix for faulty and unfaulty prediction using the YOLOV7 model. Further, the confusion matrix given in Figure 6 was generated after the testing process.

The four different performance metrics of the YOLOV7 model are evaluated and presented in

Table 1. Performance metrics of YOLOV7 model.

Performance Metrics	Percentage (%)
Accuracy	97
Precision	97
Recall	98
F1_Score	98

. Also, the performance metrics are expressed in terms of %. Further, it can be observed that the proposed YOLOV7 model has an accuracy of 97%. Also, the precision and recall of the proposed YOLOV7 model on faulty and unfaulty IC images are 97% and 98%, respectively. Also, the F1_Score of the proposed YOLOV7 model is around 98%. From the performance measures, it is evident that the proposed device integrated with the YOLOV7 model is highly efficient for classifying faulty and unfaulty ICs, which gains more prominence in IC fault diagnosis applications.

5. Conclusions

In this study, an efficient technique using thermal image processing was proposed to inspect faulty ICs located on PCB. Further, the thermal images were collected from AMG8833 thermal sensor module and the acquired images were preprocessed. These preprocessed images were given to the YOLOV7 algorithm for image classification. Results demonstrate that the proposed method provides 97 percent accuracy in detecting faulty and unfaulty images with less computational time. Further, by integrating the capabilities of thermal camera and the YOLOV7 algorithm, the user can be alerted regarding the fault conditions of the circuit. In the near future, the proposed method could be automatized and the faults could be identified and monitored remotely using the Internet of Things (IoT).