*4.2. Visualization Results of FLAIR3*

Figure 4 shows FLAIR, T2W, and FLAIR3 images of a TSC child and a healthy child. On three MRI images of the TSC child, it can be observed that the contrast between the lesions and brain tissue on FLAIR is not clear enough, there is a severe interference of cerebrospinal fluid on T2W, and the contrast and clarity of the lesions on the newly generated FLAIR3 image are significantly improved (TSC lesion as shown by the red arrow). In addition, FLAIR3 inhibits cerebrospinal fluid and can clearly locate the TSC lesion.

**Figure 4.** Representative MRI from a TSC child and a healthy child, including T2W, FLAIR, and the proposed FLAIR3 (the red arrow highlights the TSC lesion).

#### *4.3. Performance of the Models*

The performance of DWF-net varies with the weight of W1, W2, and W3 as shown in Figure 5. The feature extractor in Figure 5a is 3D-EfficientNet, and the best AUC performance of 3D-EfficientNet is 0.989 (W1 = 0.0, W2 = 0.3, W3 = 0.7). Among the models evaluated, Res\_DWF\_net (with weight parameters W1 = 0.2, W2 = 0.3, W3 = 0.5), which employs 3D-ResNet as a feature extractor and a late fusion strategy as depicted in Figure 5b, achieves the highest performance. This model has an accuracy of 0.985 and an AUC of 0.998, outperforming other models.

**Figure 5.** The performance of DWF-net with different weights. The feature extractor in (**a**) is 3D-EfficientNet, and the feature extractor in (**b**) is 3D-ResNet. The horizontal axis represents the weight of W1, W2, and W3, and the vertical axis represents the performance of AUC.

The results for all the compared models in the testing dataset are presented in Table 3. When using 3D-EfficientNet, FLAIR3 achieves an AUC performance of 0.987 and the AUC of Eff\_FLAIR\_T2W is 0.974, and the AUC of FLAIR3 is higher than Eff\_FLAIR\_T2W. FLAIR3 achieves an AUC performance of 0.997 when using 3D-ResNet as the feature extraction network. When the feature extraction network is 3D ResNet, the AUC of Res\_FLAIR\_T2W is 0.994, and the AUC of FLAIR3 is higher than Res\_FLAIR\_T2W.



When using the same single-modal MRI as inputs, 3D-ResNet outperforms 3D-EfficientNet. Additionally, the AUC performance of the FLAIR3 model outperforms the

T2W-only model and FLAIR-only model. The baseline network (InceptionV3) achieves an AUC performance of 0.952, and the performance of our all-3D network exceeds the AUC performance of the baseline network of InceptionV3.

ROC curves for all models of the testing cohort are shown in Figure 6a–c, and Figure 6d shows the classification performance for all models of the testing cohort.

**Figure 6.** (**a**–**c**) represent the ROC curves for all models of the testing cohort. (**d**) represents the classification performance for all models of the testing cohort. The horizontal axis shows the model name, while the vertical axis represents the performance regarding AUC, ACC, SEN, and SPE.

#### *4.4. Results of Skull Stripping*

The classification performance of FLAIR and T2W images, with or without skull dissection, is presented in Table 4. The table demonstrates that if the network structure and input modality remain constant and the skull dissection preprocessing is not carried out, the classification performance of 3D ResNet and 3D EfficientNet will show a decline.

**Table 4.** The results of with/without skull stripping in T2W and FLAIR.


#### *4.5. Comparison of Normalization Methods*

Table 5 and Figure 7 depict the classification performance of three normalization methods, including without normalization, Z-score normalization, and min–max normalization on FLAIR images and T2W images. The horizontal axis represents the different normalization techniques, while the vertical axis represents their corresponding performance. In instances where the input modality and network structure remain constant, it is worth noting that the withoutnormalization method has the poorest AUC performance. Furthermore, the AUC performance of the min–max normalization technique is better than the Z-score normalization technique.


**Table 5.** The classification performance of with/without skull stripping in FLAIR images and T2W images.

**Figure 7.** The classification performance of the without-normalization method, the Z-score normalization, and the min–max normalization in FLAIR images and T2W images. (**a**) 3D-EfficientNet as a network feature extractor, FLAIR as the network input. (**b**) 3D-ResNet as a network feature extractor, FLAIR as the network input. (**c**) 3D-EfficientNet as a network feature extractor, T2W as the network input. (**d**) 3D-ResNet as a network feature extractor, T2W as the network input.

#### **5. Discussion**

The main objective of the proposed approach is to identify TSC children at an early stage using a 3D CNN model in conjunction with multi-contrast MRI in an automated manner. Initially, the approach incorporates FLAIR3 as a novel modality for diagnosing pediatric TSC lesions and optimizes the T2W and FLAIR combination to enhance the lesion–brain contrast in a clinic. The findings indicate that FLAIR3 has the ability to enhance the prominence of TSC lesions, while also enhancing classification accuracy and providing a more intuitive understanding of our deep learning model. Otherwise, the proposed method used two networks as feature extractors; one is 3D-EfficientNet, which is a parameter-efficient deep convolutional neural network framework, and the other classification network is 3D-ResNet, which is a classical residual network. Previously, the FLAIR3 modality was only used in MS disease [13], but the proposed methods generalized it to pediatric TSC disease and demonstrated that FLAIR3 was able to better visualize TSC lesions. Furthermore, a multi-modal fusion network for multi-contrast MRI data was proposed, which can feed FLAIR3 as a new modality into the proposed DWF-net network, finally achieving a state-of-the-art classification performance in identifying children with pediatric TSC. And the dataset has no PET and EEG as input, and only has just the structural MRI that can be easily and wildly collected at any hospital, which helpfully maximizes the potential applicability of the proposed approach in clinical practice. In summary, the proposed method also has innovations in the following aspects: 1) the use of a weighted fusion algorithm to maximize the fusion multi-contrast MRI and optimize weights to improve performance; 2) firstly proposes to use a FLAIR3 image to position and visualize the lesions in a clinical diagnosis of TSC. 3) The utilization of FLAIR3 as the complementary imaging input to maximize the information extracted from the structure MRI.

In comparison to the 2D CNN model InceptionV3 discussed in [18], the proposed 3D CNN models exhibit an enhanced classification performance. Some previous studies are also consistent with our conclusion that 3D networks perform better than 2D networks [19,28]. We believe that the performance improvement of the 3D network is mainly due to the full use of the spatial features of MRI voxels, which can extract more information. In this study, the proposed late fusion method can improve the classification performance compared to a single modality using a 3D CNN approach, implying that combining multiple contrasting MRI can exploit complementary visual information between multiple sequences. This result is consistent with a recent study by Han Peng et al. [29], which demonstrated that combining models from diverse modalities with complementary information leads to a superior performance. The success of the ensemble strategy is not only attributed to the number of large models but also to independent information gathered from different modalities. Additionally, recent research has revealed that the late fusion method outperforms the early fusion technique [30,31]. In addition, Jonsson et al. used a majority voting strategy to form the final predictions and achieved performance gains with multimodal inputs [22]. In our experimental results, our findings indicate that when utilizing the same MRI modality as network inputs, all models with 3D-ResNet feature extractors outperform the 3D-EfficientNet model. One possible explanation is that 3D-ResNet has more network parameters than 3D-Effectient, and the network structure is more complicated. Therefore, 3D-ResNet can extract more high-level image feature information than 3D-EfficientNet.

Surprisingly, our experiments have successfully demonstrated the effectiveness of FLAIR3 in a pediatric TSC diagnosis, and the AUC performance of the FLAIR3-only model outperforms the T2W-only model and FLAIR-only model when using the same network. We found that the use of 3D-EfficientNet results in a better AUC score for the Eff\_FLAIR3 model compared to the Eff\_FLAIR\_T2W model and that the Res\_FLAIR3 model outperforms the Res\_FLAIR\_T2W model when using the feature extraction network 3D ResNet. This could imply that FLAIR3 can provide more information. When the late fusion strategy is used, the weight W3 of FLAIR3 is the largest. A reasonable note is that FLAIR3 can enhance the lesion-to-brain contrast and the TSC lesion is clearer in FLAIR3 than in T2W

and FLAIR, so FLAIR3 can offer more low-dimensional visual lesion information for deep learning during the feature extraction stage. Such low-dimensional visual information may be very helpful for our deep learning algorithms, which could increase the interpretability of our deep learning algorithms [32].

Moreover, skull stripping plays a crucial role in computational neuro-imaging by being a vital preprocessing step that has a direct impact on subsequent analyses [33–35]. In this study, we found that both the 3D-ResNet and 3D-EfficientNet models perform better when utilizing MRI with skull stripping applied as the input. This may be due to the fact that the pixel value of the skull is significantly higher than that of the brain tissue [30,36], which allows for more information to be extracted during the feature selection phase. However, it is important to note that such information may be irrelevant for our deep learning methods and may even reduce their performance [37].

Furthermore, image normalization is critical to develop powerful deep learning methods [38,39]. In this study, the experiments included normalization, no normalization, min–max normalization, and Z-score normalization. All of the results showed that the AUC performance without the normalization method is the worst; the AUC performance of the min–max normalization is better than the Z-score normalization when the input modality and network structure are the same. Therefore, we suggest that in future similar studies, the min–max normalization method can be used as a primary choice to normalize the MRI images.

Otherwise, many experts considered that tubers are stable in size and appearance after birth and that the proportion to the whole brain will not obviously change with age [40]. The myelination process in a clinic has three stages, namely before 7–8 months of age, 7–8 months to 2 years of age, and after 2 years of age. So, the TSC situation of MRI after 2 years of age should be the same as before, but myelination after 2 years of age may not have affected our MRI images [41]. But these are statistical results, and there are some different situations for different TSC patients. In a clinic, MRI should be scanned several times under the age of 2 to reflect dynamic changes in epileptic lesions. Here, we did not exclude children under 2 years of age for being close to real clinical situations. The deep learning method we proposed can be promoted in a clinic and only needs to collect FLAIR and T2W images of a patient. Our method is simple and effective in a clinic and can be used as a computer-aided tool to help doctors diagnose TSC patients. In the future, further situations of TSC patients should be evaluated.

#### **6. Conclusions**

In summary, a novel deep learning method of the weighted late fusion model was proposed to effectively diagnose pediatric TSC lesions with multi-contrast and synthesiscontrast FLAIR3 MRI. The collected dataset of pediatric TSC disease has a total of 680 children, including 331 healthy and 349 TSC children. The current testing results illustrated that the proposed approach can attain a state-of-the-art AUC of 0.998 and accuracy of 0.985. As such, this method can act as a robust foundation for future studies regarding pediatric TSC patients.

#### **7. Patents**

The work reported in this manuscript has resulted in a patent.

**Author Contributions:** Data curation, C.Z., X.Z., R.L., Y.Z. (Yihang Zhou) and Z.H.; Formal analysis, J.L., J.Y., Z.L., Y.Z. (Yihang Zhou) and D.L.; Investigation, Z.H., H.W. and D.L.; Methodology, D.J., J.L., D.L., Z.H. and H.W.; Resources, D.J. and R.L.; Software, J.Y., Y.Z. (Yanjie Zhu) and H.W.; Validation, D.J., C.Z., X.Z. and H.W.; Writing—original draft, D.J. and Z.L.; Writing—review and editing, Z.H., H.W. and D.L. All authors have read and agreed to the published version of the manuscript.

**Funding:** This study received support from various sources, including the Sanming Project of Medicine in Shenzhen (SZSM201812005), the Guangdong High-level Hospital Construction Fund (Shenzhen Children's Hospital, ynkt2021-zz11), the Pearl River Talent Recruitment Program of Guangdong Province (2019QN01Y986), the Key Laboratory for Magnetic Resonance and Multimodality Imaging of Guangdong Province (2020B1212060051), the National Natural Science Foundation of China (62271474, 6161871373, 81729003, and 81901736), the Strategic Priority Research Program of Chinese Academy of Sciences (XDB25000000 and XDC07040000), the Science and Technology Plan Program of Guangzhou (202007030002), the Key Field R&D Program of Guangdong Province (2018B030335001), and the Shenzhen Science and Technology Program (JCYJ20210324115810030, KQTD20180413181834876, and JCYJ20220530160005012).

**Institutional Review Board Statement:** This study was approved by the Ethics Committee of Shenzhen Children's Hospital (No.2019005).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in this study. Written informed consent was obtained from the patient(s) to publish this paper.

**Data Availability Statement:** All of our data is from Shenzhen Children's Hospital and the data are unavailable due to privacy or ethical restrictions.

**Acknowledgments:** We thank the Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences for providing experimental equipment.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

### **References**


**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
