1. Introduction
Crystal structure erosions in Silicon nitride can yield defect products in many areas such as drugs and chemical industrial products. Defect crystal structures that are caused by the erosion of the surface can be detected from diffraction images. Detection of crystal structures at their early industrial stages decrease the risk of defective products greatly. With the development of cobalt blue light photography, crystal structures can be detected at their early industrial stages in a low-cost method [
1,
2,
3,
4].
Most cases of crystal structures, especially the defective 25% vacancies one, need to be detected for getting rid of them. There are two major types or classes of crystal structures: non-proliferating basal crystal structures (NP-crystal structures) and proliferating basal crystal structures (P-crystal structures) [
2]. NP-crystal structures are classified into three types: (i) pristine NP-crystal structures, which appear very early, and defective random NP-crystal structures. P-crystal structures are the defective 25% vacancies phase of crystal structures. Thus, crystal structures are ranked into four types, namely: No-defect structures, pristine crystal structures, defective random displacement crystal structures, and defective 25% vacancies crystal structures [
5,
6].
Figure 1 shows images of the normal surface and different severity of defected cases of crystal structures. The main issue of crystal structure defects includes the difficulty of recognizing light-defect types. There are lots of similarities among pristine and sometimes defective random displacement cases in the diffraction images. If crystal structures are left to proceed to the defective 25% vacancy types, industrial defects can occur [
7].
Many models have been proposed in the literature for the computerized classification of crystal structures. In the previous models, automatic identification of defects of crystal structure surfaces have been proposed. Detection of irregularities and rupture of crystals in images was presented in [
6,
7]. In incidents of micro-defect recognition, computerized image processing techniques were presented in [
8,
9,
10,
11]. Several models for vacancy classes in phosphorus light images were proposed in [
12,
13,
14,
15,
16], and the detection of crystal structures lesions was accomplished for the crystal structures image dataset in [
17].
Deep learning architectures and image processing were used for crystal structure automated detection [
17,
18,
19,
20]. In [
17], a region convolutional network (R-CNN) for crystal structure detection and classifying into binary cases was established. In [
18], a fusion deep learning model was developed to detect damage in images from the dataset in [
21].
Previous works developed the detection and classification of crystal structure types. These types of work introduce classification as a conventional image processing architecture where experts selected features are deliberated [
22,
23,
24,
25,
26]. In [
27], the authors proposed a technique using random forest on selected handcrafted features to detect the severity of defects in crystal structures. In [
28], a crystal structures taxonomy was implemented utilizing BossaNova vector mid-level that represents the Bags of Visual features. In [
29], a dual-stage handcrafted feature selection was used: one stage for identifying the existence of defects in the crystal structures and then detecting the severity phase. A bag of features method was established for crystal structure classes by using the orientated histogram gradients in. In [
30], both dual and multi-classification were attained by using multiresolution Horlick features.
Deep learning methods and neural networks have produced much success in the classification challenge due to their learning abilities without prior feature knowledge [
31,
32,
33]. In [
31], a CNN model was used for the dual classification of crystal structures from diffraction surface images, that focus on binary crystal structures output as yes or no defect-existence. In [
32], the Efficient-B5 CNN was utilized for classification, and binocular images for both surfaces (up and down surfaces) were used as inputs to the transfer learning architecture. In [
33], a Deep CNN for distinguishing two cases of normal crystal structures and No-defect structures was perceived. In [
34], the surface images were entered into two Deep CNN architectures with each one performing a binary grouping of crystal structures. In [
35], a CNN smartphone model was built for the classification of crystal structures at enhanced speed.
Identification of several cases of crystal structures was employed in [
36,
37,
38,
39,
40]. In [
40], a Deep-CNN for four-cases identification of crystal structures was employed. A hyper-parameter was performed in the V4 model to acquire four cases of crystal structures in [
36]. A deep CNN was employed to detect the four cases of crystal structures in [
37], where a Feed Forward CNN and a deep CNN were employed in the CrysPACS database for crystal structure identification. In [
38], deep learning models (AleXNet and ResNet) were matched for crystal structure grouping using the Kaggle database with VggNet for the highest accuracy. A transfer learning app using a pre-trained model was constructed to identify four cases of crystal structures in [
39]. This app executes in real-time using crystal structures images that are taken via special lenses that are attached to the cameras.
Recent research utilized the fusion of two deep learning stages for the crystal structures classification. In [
39,
40], the incorporation of two deep learning stages was utilized to identify the presence or absence of crystal structures, referable crystal structures, and crystal structures-sight-threatening crystal structures. In [
39], a fusion of two pre-trained deep CNNs, namely Resnet50 and Dense121, were employed for crystal structure classification. All of this research considered the fusion of more than one classification CNN models, but none of them employ regression techniques that can determine which features have the higher impact on the classification process. It should be noted that the stated crystal structures classification research reported defective random displacement accuracy. Other papers suggested multitasking CNN for image analysis with lesion industrial tasks. In [
40], a semi-supervised CNN is proposed for multitasking segmentation with the simultaneous splitting of red abrasions in surface images. In [
41], an area multitask detection model was presented to detect various defects of the crystal structure types.
In this research, we proposed a multitasking fusion deep CNN for classifying diffraction images into the four-crystal structure types from no-defect structures to defective 25% vacancies crystal structures. There is a dependency across the four crystal structure types. This dependency across the types can be computed using a regression model in the direction of accurate classifying. We proposed a multitasking fusion model comprising a regression computational model and a classification model to detect the four crystal structure types. The classifier differentiates between the crystal structure classes utilizing a sole loss function. Our deep model utilizes a twofold loss function, the first fold is utilized in the classifier and the other fold is used in the regression model which increases accuracy to a great extent.
The contribution of this paper is as follows:
A densely connected CNN accompanied by a squeeze layer is proposed to build a multitasking fusion model.
The squeeze layers have the advantage of differentiating channel dependencies.
A Perceptron layer is utilized at the end to perceive the four crystal structure types of crystal structures from the features selected by a classifier and the regression phase.
We also incorporate transfer learning and a public dataset to assess the performance of the proposed model.
This paper is structured as follows:
Section 2 describes the methods and materials. In
Section 3, experimental results are demonstrated.
Section 4 depicts the conclusion and future work.