5.1. Results on Benchmark Simulated fMRI Dataset
To evaluate the performance of the learning methods, we apply five baseline methods and AT-EC to the benchmark simulation datasets, the Smith simulated datasets have 50 subjects and the simple_network simulated datasets have 60 subjects. we run the six methods on every subject to simulate a real scene. We report the mean and standard deviation of results across all subjects and assess the performance of these six methods based on Precision, Recall, , and . A method is considered to perform well if it achieves higher precision, recall, and F1 scores, with lower values.
Results on simulation datasets are shown in
Table 5. The bold values indicate that the method achieved the best results. Sim2 has the same ground truth as Sim1, but the external neuronal noises are mixed into the nodes. From the results of Sim2, we can notice that traditional machine learning algorithms are more affected by external noise inputs, the precision, recall, and
values of the IsGC approach have declined somewhat. The ACOCTE approach is a score-based search algorithm that has stronger noise resistance. The deep learning methods are less affected, the EC-GAN, CR-VAE, CVAEEC methods achieve better performance despite the influence of noise. Moreover, compared with other algorithms, AT-EC still maintains its relative advantages over other algorithms.
Sim3 includes global mean confounding factors into the fMRI time series of brain regions, which are variables that are not measured or controlled in a causal relationship. Many causal discovery methods will identify spurious effective connectivity under the influence of confounding factors. The results of Sim3 indicate that most algorithms are robust to global confounding factors. All other methods outperform or equal the results of Sim1, except the isGC approach. AT-EC achieves the best recall and was second only to with EC-GAN, which means AT-EC estimates almost all sides but identifies too many two-way connections.
Sim4 reduces the original non-Gaussian of the BOLD data. From the results of sim4, we can notice that all methods traditional machine learning methods show improved performance, however, the deep learning methods perform worse. Theoretically speaking, deep learning methods have stronger expressive power and adaptability, and they should achieve better performance. We believe that the threshold for the set sparse penalty is too small (0.3), making it difficult to filter out invalid edges. This is confirmed by the small value of precision and the recall of one in the results. Compared to other methods, the AT-EC method still achieves the best results in terms of precision, recall, , and .
Sim5 has two additional bidirectional structures, which are similar to the real scene since the bidirectional connection structures are common in the brain effective connectivity network. From the results of sim5, we find that ACOCTE and CVAEEC have high precision and a low recall, which means both methods learn brain EC with high accuracy and rarely learn the wrong EC, but miss some ECs. Conversely, EC-GAN and CR-VAE achieve high recall and low precision, indicating that the ECs were basically learned but many are learned incorrectly, probably because these methods identify too many two-way connections. Our method has a high accuracy and recall values, indicating that the learned ECs have few errors and missing.
Sim6 has more nodes and edges, which also are similar to the real scene, since the complex human brain can be divided into many brain regions, usually multiple brain regions collaborate to complete an action. From the results of sim6, we notice that the performance of most methods is declining, while CVAEEC and AT-EC methods are performing well. This may be because, for most methods, it is difficult to learn causal relationships for short-term time series. Thanks to our amortization transformer architecture and FC-guided EC learning mechanism, our approach still achieves the best performance ().
Overall, deep learning methods are capable of extracting deep features from fMRI data, allowing for more accurate and precise results. AT-EC achieves better performance than state-of-the-art deep learning algorithms since it exploits correlation information across subjects.
5.2. Results on Real Resting-State fMRI Dataset
Unlike simulated data, it is not possible to evaluate the performance of causal search algorithms on real fMRI data using fully defined ground truth. Instead, our evaluation relies on partial knowledge about the structural connections between brain regions in the medial temporal lobe based on existing studies [
34,
38].
For the real fMRI data, we run AT-EC on every individual subject (each subject has 421 time points) for the seven medial temporal lobe regions of interest of the left and right hemispheres separately. We also do the same on each baseline algorithm for each hemisphere and for individual cases the list of directed edges and their frequency of appearance across the 23 individual subjects. The EC between two brain regions is estimated when we consider edges that appear in 40% of the 23 individual subjects.
Figure 2 illustrates the EC networks estimated by AT-EC and the baseline methods from the left hemisphere medial temporal lobe and right hemisphere medial temporal lobe.
In
Figure 2f. The effective connectivity networks of the left and right hemisphere medial temporal lobes are similar, but exhibit some differences. These differences are mainly caused by the connections of
,
,
,
,
, and
. Effective connectivities
,
,
,
, and
are in the left hemisphere while
and
are in the right hemisphere.
Compared with the other methods, AT-EC learns the most correct edges (red line) with few missed edges (blue line) in both the left and right hemispheres. As is suggested by Lavenex and Amaral [
38], the flow of information from the medial temporal lobe cortices (
) directly into the entorhinal cortex (
) and travel to
to
, this is the main pathway connecting the medial temporal lobe cortices with the hippocampus. We can find that those important effective connectivities (
,
,
,
and
) are both learned in the left and right hemispheres, and other methods cannot identify them accurately. We also missed some important brain effective connectivities, such as the one-way connection between
and
(
), which is an important connection in the main pathway connecting the medial temporal lobe cortices with the hippocampus. Overall, the new method AT-EC performs better than the state-of-the-art methods and could provide a reliable perspective for the analysis of brain effective connectivity networks.