*3.1. Local Epigenomic Features Were more Informative than Local Sequence Data in Predicting EPIs*

Although previous studies found using sequence data yielded as good or even better prediction performance as/than using epigenomic data [13,14], this trend was reversed after a valid data splitting scheme was applied (Methods). Figure 5a shows that using (local) epigenomic data outperformed using (local) sequence data across all test chromosomes for each of multiple prediction models. We also compared the performance of the two data sources with similar models side by side in Figure 5b, where the basic CNN, ResNet CNN and gradient boosting were customized to the two data sources during the training process. *p*-values of the paired *t*-test (for each chromosome) to compare model weighted average AUROC were all <0.0002, suggesting that the local epigenomics data gave a statistically significant and stronger performance than the local sequence data.

**Figure 5.** Comparison on the weighted average Area Under Receiver Operating Characteristic (AUROC) between the sequence and epigenomics models. (**a**) Boxplot of test AUROCs for all sequence and epigenomic models across 21 chromosomes; (**b**) Mean test AUROC comparison between the sequence and epigenomics models. The bars are ±1 weighted standard deviation around the weighted mean AUROC. *p* values are from the paired *t*-test with *H*0: The test AUROCs are the same for the sequence and epigenomics models. *p* values are not adjusted for multiple comparison but remained significant after the Bonferroni adjustment.

Interestingly, while the test AUROC's for the sequence models were very close to 0.5, corresponding to random guessing, the epigenomic models achieved a notable difference from 0.5 (Figure 5a,b). This again suggests a better performance of using the epigenomics data among all prediction models implemented here. Furthermore, the AUROC's of the epigenomics models had similar standard deviations across chromosomes to those of the sequence models, and even minus one standard deviation of the mean AUROC of the epigenomics models was above the sequence models' average AUROC for each of the 3 scenarios in Figure 5b. Since the epigenomics models' AUROCs were significantly different from that of the sequence models, which were comparable to random guessing (0.5), local epigenomics features were still predictive of EPIs, though the performance was much inflated in previous studies [11,19,20]. Our finding also demonstrated that without appropriate data splitting (or generally valid experimental design), any EPI prediction results and downstream motif analyses with DNA sequence data should be interpreted with caution.
