Understanding and Therapeutic Application of Immune Response in Major Histocompatibility Complex (MHC) Diversity Using Multimodal Artificial Intelligence

Abstract

Human Leukocyte Antigen (HLA) is like a device that monitors the internal environment of the body. T lymphocytes immediately recognize the HLA molecules that are expressed on the surface of the cells of the different individual, attacking it defeats microorganisms that is one of the causes of rejection in organ transplants performed between people with unmatched HLA types. Over 2850 and 3580 different polymorphisms have been reported for HLA-A and HLA-B respectively, around the world. HLA genes are associated with the risk of developing a variety of diseases, including autoimmune diseases, and play an important role in pathological conditions. By using a deep learning method called multi-task learning to simultaneously predict the gene sequences of multiple HLA genes, it is possible to improve accuracy and shorten execution time. Some new systems use a model called convolutional neural network (CNNs) in deep learning, which uses neural networks consisting of many layers and can learn complex correlations between SNP information and HLA gene sequences based on reference data for HLA imputation, which serves as training data. The learned model can output predicted values of HLA gene sequences with high accuracy using SNP information as input. To investigate which part of the input information surrounding the HLA gene is used to make learning predictions, predictions were made using not only a small number of nearby SNP information but also many SNP information distributed over a wider area by visualizing the learning information of the model. While conventional methods are strong at learning using nearly SNP information and not good at learning using SNP information located at distant locations, some new systems are thought that prediction accuracy may have improved because this problem was overcome. HLA genes are involved in the onset of a variety of diseases and are attracting attention. As an important area from the perspective of elucidating pathological conditions and realizing personalized medicine. The applied multi-task learning to two different HLA imputation reference panels—a Japanese panel (n = 1118) and type I diabetes genetics consortium panel (n = 5122). Through 10-fold cross-validation on these panels, the multi-task learning achieved higher imputation accuracy than conventional methods, especially for imputing low-frequency and rare HLA alleles. The increased prediction accuracy of HLA gene sequences is expected to increase the reliability of HLA analysis, including integrated analysis between different racial populations, and is expected to greatly contribute to the identification of HLA gene sequences associated with diseases and further elucidation of pathological conditions.

1. Introduction

Human Leukocyte Antigen (HLA) are proteins expressed on the surface of cells. When foreign substances such as microorganisms invade inside or outside the cell, it has the function of notifying T lymphocytes, which are one of the components of white blood cells (WBCs) contained in the blood, of the presence of foreign substances and instructing them to eliminate them by expressing on the cell surface to combine the fragments created by the decomposition of foreign substances [1]. Thus, HLA is like a device that monitors the internal environment of the body, and T lymphocytes are constantly monitoring whether there are any strange foreign particles bound to HLA molecules, by which protects our bodies from attack by foreign substances such as microorganisms. When cells from a different individual invade our body, T lymphocytes immediately recognize the HLA molecules that are expressed on the surface of the cells of the different individual, attacking in the same way it defeats microorganisms that is one of the causes of rejection in organ transplants performed between people with unmatched HLA types [2].

Regarding HLA, which is human major histocompatibility complex (MHC), HLA antigens, which are human MHC antigens, were discovered in 1952 by Dausset in France, who discovered antibodies that reacted with leukocyte agglutination tests in the serum of patients with frequent blood transfusions, that began with the naming of the corresponding antigen Mac antigen, currently HLA-A2 antigen. However, more than half a century after its discovery, HLA is only found in WBCs, but is distributed in almost all cells and body fluids. Then, it has been revealed that it functions as a histocompatibility antigen, an important molecular marker involved in human immunity that the immune system to distinguish between “self” and “non-self”. The more HLA mismatches between the patients undergoing allogeneic hematopoietic stem cell transplantation and the donor, the higher the incidence of adverse complications, such as engraftment failure and graft-versus-host disease (GVHD) [3]. There are six types of HLA proteins, HLA-A, HLA-B, and HLA-C, which are class I molecules presented as transmembrane glycoproteins on the surface of all nucleated cells and HLA-DR, HLA-DQ, and HLA-DP, which are class II molecules usually expressed only by professional antigen-presenting cells, including B cells, macrophages, dendritic cells, Langerhans cells, thymic epithelium, and activated but not quiescent T cells due to their different functions and structures (Figure 1). Additionally, each of these molecules has structural variations called “polymorphisms”. For example, over 2850 and 3580 different polymorphisms have been reported for HLA-A and HLA-B respectively, around the world [4]. A test that examines the types of polymorphisms and determines the combination of HLA molecules possessed by an individual is called “HLA typing”. There are various methods for HLA typing, but recently DNA typing is commonly used to estimate the gene sequence that determines polymorphisms in HLA molecules [5]. Donor selection for hematopoietic cell transplantation is based on the degree of concordance of HLA-A, -B, -C, -DR molecule polymorphisms inferred from HLA typing results [6,7].

The original class I molecules consists of an alpha heavy chain attached to a beta2 microglobulin molecules, which as two peptide-binding domains, one immunoglobulin (Ig)-like domain, and one transmembrane region with a cytoplasmic tail. The heavy chains of class I molecules are encoded by genes at the HLA-A, HLA-B, and HLA-C loci. T cells that express CD8 molecules respond to MHC class I molecules. These lymphocytes often have cytoplasmic functions and must be able to recognize any infected cells. Because all nucleated cells express MHC class I molecules, all infected cells act as antigen-presenting cells to CD8-positive T cells that CD8 binds to the non-polymorphic portion of the heavy chains of class I molecules. Some MHC class I genes directs peptides to specific receptors on HLA-G, which play a role in protecting the fetus from maternal immune responses, and HLA-E, which presents peptides to specific receptors on NK cells, encodes a non-classical MHC molecule [8,9]. MHC class II molecules, on the other hand, consist of wo polypeptide (alpha and beta) chains; each chains have a peptide-binding domain, an Ig-like domain, and a transmembrane region with a cytoplasmic tail. Both polypeptides chains are encoded by genes in the HLA-DP, HLA-DQ, or HLA-DR regions of chromosome 6 (Figure 1). Most nucleated cells can be induced to express MHC class II molecules by interferon (IFN)-γ [10]. T cells that respond to class II molecules express CD4 and are often helper cells. Additionally, the MHC class III region of the genome encodes several molecules important in inflammation [11]. These include the complement (C2, C4, and factor B), tumor necrosis factor (TNF) alpha, lymphotoxin, and three heat shock proteins.

Additionally, by changing specific parts of the foreign body, the immune response of T lymphocytes to the foreign body can be significantly enhanced or suppressed, suggesting that this can be applied to further enhance the T lymphocyte response, immune response to a specific foreign substance when it is useful for the body [12,13]. For example, it could be applied to the development of powerful vaccines to increase resistance, immunity against pathogenic bacteria and viruses. Some diseases, such as rheumatism, develop when T lymphocytes react to components of the body that T lymphocytes should not normally react to [14]. In this case, it is possible to suppress the immune response of T lymphocytes that react to self by using a self-imitation product that has slightly changed the structure of self-components that T lymphocytes mistake for foreign substances [15]. In this review, we summarized the advance in prediction models of sequence validation of HLA using deep learning and their clinical application.

2. Major Histocompatibility Complex (MHC)

Research on the MHC began in 1936 when Gorer in England and Snell in the US discovered it as an antigen related to the blood type of mice. They found that the genetic region governing this antigen strongly influences the rejection response to skin grafts and name its Histocompatibility-2 antigen (H-2 antigen). Subsequently, analysis using inbred mice revealed that the gene region controlling the H-2 antigen is a gene complex consisting of multiple regions located on chromosome 17. In the early 1960s, Benacerraf, McDevitt and colleagues revealed that the MHC region contains a gene region that controls the mouse immune response and named this region the I region. T cells contacts MHC molecules with the T cell receptor (TCR) as well as co-receptors CD8 and CD4. Cytotoxic T cells (CTLs) express CD8 on their surface and bind to MHC class I molecules. CTLs release perforin and granzyme by recognizing target cells such as virus-infected cells, directly induce apoptosis and damage them [16,17]. Helper T cells (Th cells) express CD4 on their surface and bind to cells that express MHC class II molecules on their surface. Th1 cells produce IL-2 and IFN-γ, which activate CTLs, macrophages, and NK cells. Th2 cells produce cytokines such as IL-4 and activate B cells. Th17 cells produce IL-17 and are involved in inflammation [18]. Regulatory T cells (Treg) produce TGF-beta and IL-10 and have the function of actively terminating an immune response once it has started.

2.1. Vaccine and MHC

Thus, relationship between MHC region and immune response are critical. In cancer cells, cancer-specific proteins are broken down into peptides and presented as cancer antigen peptides along with MHC class I molecules on the cancer cell surface [19]. Cancer antigen specific CTLs that recognize this are activated and attack cancer cells [20,21,22]. However, if the number and strength of CTLs, immunity are insufficient, cancer will continue to proliferate and metastasize, becoming malignant [23]. Cancer vaccine therapy is a method of treating cancer by administering antigenic peptides into the patient’s blood vessels to strongly induce cancer specific CTLs [24,25,26]. To further enhance the effectiveness of cancer vaccine therapy, research is underway on ways to use antigen-presenting cells such as dendritic cells that present cancer antigen peptides, and on combinations with gene therapy. To further enhance the effectiveness of cancer vaccine therapy, research is underway on ways to use antigen-presenting cells (APCs) such as dendritic cells that present cancer antigen peptides, and on combinations with gene therapy. Among these, cancer vaccine therapy is a treatment method that is expected to exhibit cancer cell-specific antitumor effects by directly inoculating peptides and patients derived from tumor associated antigens (TAAs), which are highly expressed only in cancer cells and rarely expressed in normal cells, or mRNAs and genes derived from TAAs, and DCs loaded with or introduced with these to cancer patients [27]. In applying tumor-specific cancer immune responses to cancer immunotherapy, it is important to first identify the target TAA [28]. Various TAAs, such as cancer testis antigens and carcinoembryonic antigens, have been identified through comprehensive cDNA microarray analysis of cancer and normal tissues, and have been applied to cancer immunotherapy. These TAAs are released into the tissue fluid when some cancer cells die due to apoptosis, etc. [22].

A long peptide (LP) consisting of 10 to 20 amino acids that is taken up and degraded by DC endosome binds to HLA class II molecules and enters the cell [29,30]. It is expressed on the surface and is recognized by the T cell receptor (TCR) of CD4⁺ Th cells, which activates the Th cells [31,32]. At the same time, DCs transport TAA to the cytoplasm, where it is degrades by proteasomes, resulting in a short peptide (SP) consisting of 9 to 11 amino acids, which is bound to HLA class I molecules and attached to the cell surface, which is recognized by the TCR of CD8⁺ CTLs, cross-presentation pathway, and the CTLs are activated [33]. DC express the costimulatory molecules CD80/86, and when this binds to CD28 of naïve T cells, T cells are strongly activated and differentiate into effector T cells. Effector T cells do not require costimulatory molecules; they recognize HLA-TAA peptide complexes expressed on the surface of cancer cells and exhibit an immune response against cancer cells. CD4⁺ Th1 cells produce Th1 cytokines, which promote CTL induction and activation. Thus, cancer antigen vaccine therapy is a treatment method that actively immunizes cancer patients with TAAs from the outside to induce and enhance antitumor immunity, with the aim of inducing T cell immune responses against TAAs [25,26].

2.2. SP Vaccine Therapy

Thus, Th1 cytokines promote CTL induction and activation. CTLs are very important effector cells that kill cancer cells [21,34]. When TAA-derived SP is administered to cancer patients, it binds to HLA-I on DCs and is presented on the cell surface [35,36,37]. By inducing and activating peptide-specific CTLs and attacking cancer cells that express the same TAA-SP recognized by these CTLs, it is expected that a cancer-specific antitumor effect will occur [38,39]. Methods for identifying new TAAs include the Serex method and cDNA microarray analysis [40]. Furthermore, using algorithm analysis and HLA-expressing transgenic mice based on the amino acid sequence of the newly identified TAA, many SPs that induce HLA-I-restricted CTLs have been identified and have been applied to cancer vaccine therapy [25,41,42]. Furthermore, using proteome analysis techniques such as mass spectrometry, attempts have been made to identify TAAs and TAA-derived T cell epitopes by determining the amino acid sequences of peptides bound to HLA molecules on the surfaces of cancer cells and antigen-presenting cells [43,44]. The advantage of identifying TAA and T cell epitopes using such proteome analysis technology is that TAA peptides presented by HLA expressed on the surface of cancer tissues can be directly identified. Using methods such as proteosome analysis such as mass spectrometry and genetic analysis, multiple TAA-SPs that bind to HLA-I expressed in renal cell carcinoma have been identified and are being applied to cancer peptide vaccine therapy [45,46,47,48]. Clinical research on TAA peptide vaccine therapy using SP has been conducted for a variety of cancers, including esophageal cancer, head and neck cancer, lung cancer, and bladder cancer [25,41,49,50,51]. Furthermore, no serious adverse events were observed in either case, and tumor shrinkage and survival effects have been observed in some patients with advanced cancer [52,53]. SPs recognized by HLA-A24-restricted CTLs for novel cancer testis antigens LY6K, CDCA1, and IMP3, which are frequently expressed in esophageal, oral cavity, and lung cancers, identified by genome-wide cDNA. Furthermore, in vaccinated patients, an increase in overall survival was observed in direct proportion to the number of cancer antigen peptides to which CTLs responded [50]. However, monotherapy with cancer antigen SP peptide vaccines for patients with advanced cancer has a low response rate of less than 5% in most cases, and satisfactory has not yet been demonstrated [25,49,51].

2.3. LP Vaccine Therapy

Thus, monotherapy with cancer antigen SP peptide vaccines for patients with advanced cancer has a low response rate. To induce a stranger antitumor immune response, the presence of TAA-specific Th1 cells as well as is important [54,55]. Furthermore, the TAA-SP mentioned above induces unresponsiveness in CTLs by binding to HLA-I in somatic cells other than antigen-presenting cells that do not express costimulatory molecules such as CD80/86. In addition, in immunization with TAA-SP and incomplete Freund’s adjuvant (IFA), CTLs accumulate at the immunized area for a long time and do not migrate to the tumor area [25,56]. On the other hand, LP cannot directly bind to HLA-I, and after being taken up by antigen-presenting cells and processed within the cells, it is presented by HLA-II and not only induces Th cells, but also induce Th cells [57,58]. SP is produced through the cross-presentation pathway, which can be presented by HLA-I and induce TAA-specific CTL [51,59]. Therefore, LP is useful not only in inducing stronger antitumor effects but also in preventing the induction of unresponsiveness [60,61]. Clinical trials using LP are currently being conducted on a variety of cancers, and some have even reported survival benefits. Furthermore, it was confirmed that the identified LP-specific Th cells exist in the peripheral blood of cancer patients. Furthermore, because of cDNA analysis of TCR genes expressed in LP-specific Th cell clones in DEPDC1 and MPHOSPH1, which are cancer antigens that are frequently and highly expressed in bladder cancer, both bulk Th cell lines and clones expressed a single TCR alpha and beta chain gene, in which this pair was responsible for cancer antigen specificity and HLA restriction. By administering such LPs as a vaccine to cancer patients, it is expected that TAA peptide vaccine therapy with even stronger antitumor effects will be developed [25,26,41]. Furthermore, the TCR gene can be used for monitoring Th cells induced by LP vaccine therapy and for adoptive immunotherapy for cancer using peripheral blood T cells that have been forced to express the TCR gene [62,63,64,65,66]. When applying such peptide vaccine therapy clinically, there are some points to be considered regarding HLA-I, which presents SP contained in LP, that is a loss of heterozygosity (LOH) of the HLA-I gene in tumor [67,68,69]. Tumor cells expressing HLA-I and TAA are eliminated by CTL induced by peptide vaccine administration [70]. However, tumor cells that have undergone LOH escape attack by CTL and proliferate and metastasize [71,72]. To avoid this phenomenon, first, select an LP containing an SP that can induce CTLs restricted to HLA, such as HLA-A24, which has Bw4, a ligand for KIR, an inhibitory receptor of NK cells. Thus, even if HLA-A24 is deleted due to LOH, tumor cells can be eliminated by NK cells [73,74]. Furthermore, due to this mechanism, LOH of HLA-A24 in tumor cells is less likely to occur. Based on the same idea, the target patients are those who are HLA-A24 homozygote or whose opposite haplotype is Bw4 negative, or HLA-A2 homozygote; patients with B*40:02 are excluded. Furthermore, to prevent tumor escape from CTL due to LOH, LP containing multiple HLA-restricted CTL epitopes SP is utilized [75].

2.4. Cancer Antigen Vaccine Therapy Using DC and Neoantigen

Thus, tumor cells that have undergone LOH escape attack by CTL. Also, DC vaccine therapy is a treatment method that utilizes the powerful antigen-presenting ability of DCs [76,77]. There are methods such as loading DC with TAA peptide or cancer cell lysate and introducing mRNA or DNA encoding TAA into DC [78,79,80]. By inoculating cancer patients with these DCs as a vaccine, cancer cell-specific T cells are induced and activated. Immature DCs are induced by culturing monocytes collected from peripheral blood in the presence of GM-CSF and IL-4, and these immature DCs are loaded with TAA-derived peptides and mRNA or DNA encoding TAA [81,82]. A common method is to further stimulate DCs to mature and activate them, inducing them to have strong antigen-presenting ability, and then using them as a vaccine [81,83]. Cancer develops through the accumulation of genetic mutations in multiple stages, and therefore has many somatic mutations. As a result, when a peptide containing an amino acid mutation is presented by HLA-I, it is expected to be recognized as a “non-self-antigen”. [57,84,85] Cancer antigen vaccine therapy using mutated peptides derived from antigens, neoantigens encoded by somatic missense mutant genes that occur in cancer cells has attracted attention [69,86,87]. An advantage of applying neoantigens to cancer vaccine therapy is that some T cells that exhibit high affinity for TAA may be eliminated from the body through negative selection in the thymus [26,69,88]. Because these T cells have not undergone selection in the thymus, they are less susceptible to immune tolerance, and specific T cells are more likely to be efficiently induced [89]. Comprehensive genetic analysis of cancer cells, mainly malignant melanoma, has shown that mutated peptides caused by somatic mutations in the genes of cancer cells have an immunogenic effect on CTL and Th1 cells within the tumor [69,90,91,92]. This immune response plays an important role in antitumor immunity. Mutations that produce neoantigens are divided into driver mutations and passenger mutations [93,94]. Driver mutations are genetic mutations that promote the transformation of malignant tumor cells, and passenger mutations are random mutations that are not directly involved in cancer formation [90,93,95]. Most genetic mutations that supply neoantigens originate from passenger mutations rather than driver mutations. In addition, neoantigens generated by driver mutations are expressed from the early stage of tumor development, and T cells undergo epigenetic changes due to long-term exposure to neoantigens, making them unresponsive [88]. Passenger mutations differ among individual patients even when tumors arise from the same organ, and they are numerous and highly immunogenic, making them suitable targets for cancer vaccine therapy using neoantigens peptides. Molecular targeted therapy is the best cancer treatment that targets driver mutations [96,97]. Comprehensive exon and RNA sequence analysis of relatively early-stage lung cancer tissues revealed that LOH is frequently detected in patients with many neoantigen peptides, and immune escape is more likely to occur [69,98,99,100]. HLA-I allele of cancer patients constrain the types of mutations in driver genes that play a direct and important role in the development and progression of cancer cells [101,102,103,104]. In patients with specific HLA alleles, neoantigen peptides generated by these driver mutations are presented to T cells by the HLA and inducer tumor immunity, thereby eliminating tumor cells [105,106,107,108,109].

Antibody therapy that lifts the immunosuppressive state of T cells characteristic of cancer patients has shown remarkable results by inhibiting immune checkpoint molecules such as anti-CTLA-4, CTL antigen 4 antibodies and anti-PD-1, programmed cell death-1/PD-1 antibodies [110]. Neoantigen-specific T cells already exist in the tumor tissue before treatment, and these T cells are reactivated by anti-CTLA-4 antibody therapy or anti-PD-1 antibody therapy and play an important role in antitumor effects [111,112,113,114]. In addition, in patients with a good response to immune checkpoint inhibition therapy, more nonsynonymous substitution mutant genes with amino acid substitutions are observed, and neoantigen-specific T cells are detected in tumor tissue before treatment [69,111,114]. These results suggest that enhancement of the immune response to neoantigens may be the mechanism by which immune checkpoint blockade therapy exerts its effectiveness [69,115,116]. It is expected that the combination of cancer antigen vaccines containing neoantigens and immune checkpoint inhibition point inhibition therapy will produce strong antitumor effects [26,117]. An analysis of many patients with advanced cancer treated with anti-PD-1 antibodies or anti-CTLA-4 antibodies showed that HLA-B44 supertype-positive, a group of HLA antigens that have common motifs in the amino acids sequence of their binding peptides, based on the common attributes of the pocket in the HLA peptide-accommodating groove and HLA heterozygous patients show a good response, while HLA-B62 supertype-positive patients, HLA homozygous patients, and HLA LOH patients show a poor response [118]. Thus, there is a correlation between the effectiveness of immune checkpoint inhibitor therapy and the type and number of HLA alleles [119,120]. This phenomenon also indicates that T cells that recognize neoantigen peptides presented by HLA are responsible for the immune response that destroys tumor cells [69,106,121].

As clinical trials progress, cancer antigen vaccine therapy has few side effects than conventional cancer treatments and is effective in prolonging survival while maintain patients’ quality of life. However, the clinical effect of cancer antigen vaccine therapy alone on advanced cancer remains weak, but there are great expectations for vaccine therapy that targets multiple neoantigens [42,69,122,123]. Furthermore, the development of more potent cancer antigen will require more complex cancer immunotherapy that combine treatments that control immunosuppressive mechanisms in the tumor microenvironment [124]. Additionally, advances in genetic analysis technology have made it possible to evaluate the cancer and immune status of individual cancer patients using biomarkers and other methods [125,126,127]. Future efforts will also be focused on developing personalized cancer immunotherapy based on this information.

3. Computer-Based Inference of HLA Gene Sequences Related to Immune Function

Thus, there are great expectations for vaccine therapy that targets multiple neoantigens. WBC, one of the blood cells contained in human blood, are cells responsible for the immune response that recognize self and non-self and attacks foreign antigens. Like blood types for red blood cells (RBCs), blood types for WBCs also exist, and are determined by individual differences in the genomic sequence of the HLA gene. HLA genes are associated with the risk of developing a variety of diseases, including autoimmune diseases, and play an important role in pathological conditions [128]. Since HLA gene sequences are complex and require high costs to decipher, the HLA gene sequence is comprehensively predicted and analyzed on a computer from information on multiple single nucleotide polymorphisms (SNPs) located around the genome sequence [129]. HLA gene sequences associated with the risk of developing various diseases have been elucidated, but the reported risk sequences often differ between races [130]. For example, in type 1 diabetes in Westerners, amino acid other than aspartic acid at position 57 of the HLA-DQB1 molecule was known to be the most strongly associated risk sequence, but a similar association was not observed in Japanese. It is expected that further knowledge will be obtained by performing integrated analysis among different ethnic groups, but the prediction accuracy for low-frequency HLA gene sequences decreases significantly with conventional HLA imputation methods [130,131]. There was a problem in that it was difficult to obtain highly reliable results in integrated analysis between groups that required sequences.

HLA gene complex plays a crucial role in the development of autoimmune diseases. Certain HLA alleles have been identified as risk factors for various autoimmune conditions. Some of the key HLA gene sequences associated with an increased risk of developing autoimmune diseases are as follows; in rheumatoid arthritis (RA), HLA-DRB1*04:01, HLA-DRB1*04:04, and HLA-DRB1*04:05 (shared epitope alleles) [132], in type 1 diabetes mellitus (DM),HLA-DRB1*03:01, HLA-DRB1*04:01, and HLA-DRB1*04:05 and HLA-DQB1*02:01 and HLA-DQB1*03:02 [133], in multiple sclerosis, HLA-DRB1*15:01 and HLA-A*02:01 (protective allele) [134], in systemic lupus erythematosus (SLE), HLA-DRB1*03:01 and HLA-DRB1*15:01 [135,136], in ankylosing spondylitis, HLA-B*27 [137], in celiac disease, HLA-DQA1*05 and HLA-DQB1*02 (encoding for DQ2.5 heterodimer) and HLA-DQA1*03 and HLA-DQB1*03:02 (encoding for DQ8 heterodimer) [138], in psoriasis, HLA-C*06:02 [139], and in inflammatory bowel diseases, HLA-DRB1*01:03 and HLA-DRB1*07:01 (Crohn’s disease) and HLA-DRB1*01:02 and HLA-DRB1*15:01 (ulcerative disease) [140,141,142]. It’s important to note that the presence of these HLA alleles does not necessarily mean an individual will develop an autoimmune disease, as other genetic and environmental factors also contribute to the risk.

Additionally, the specific HLA associations may vary across different populations and ethnic groups. For instance, in Caucasian populations, certain HLA alleles like HLA-DRB1*03:01, HLA-DRB1*04:01, and HLA-DRB1*04:04 are strongly linked to an increased risk of RA [143,144]. In contrast, studies in Arab populations have reported inconsistent associations, with HLA-DRB1*04:01 and HLA-DRB1*10 being implicated in some cases [145,146,147]. Similarly, for type 1 DM, HLA-DRB1*03:01 and HLA-DRB1*04:01 alleles confer high risk in Caucasians, while HLA-DRB1*04:05 and HLA-DRB1*04:06 are associated with increased risk in Arab populations [148]. In SLE, HLA-DRB1*03:01 and HLA-DRB1*15:01 are risk factors in Caucasians, but HLA-DRB1*16:02 and HLA-DRB1*15:01 are implicated in Arab SLE patients [149]. These differences likely arise from the unique genetic backgrounds and evolutionary histories of different ethnic groups, leading to variations in the HLA allele frequencies and linkage disequilibrium patterns. Additionally, environmental factors and gene-environment interactions may also contribute to the observed ethnic disparities in HLA associations with autoimmune diseases.

In recently, artificial intelligence (AI) technologies, including machine learning methods, have been attracting attention to solve problems that are difficult to solve with conventional analysis methods. Deep learning is a machine learning method that uses multilayer neural networks. This technology has been attracting attention since the mid-2010s, mainly because it shows predictive performance that far exceeds existing methods in fields such as image recognition and natural language processing [150,151]. In deep learning models used in protein engineering, the input is often the amino acid sequence of the protein, and the output is the functional value doe prediction [152,153,154]. It has the characteristic of being able to learn complex features that cannot be captured by general statistical and machine learning analysis methods. Some new systems use a model called convolutional neural network (CNNs) in deep learning, which uses neural networks consisting of many layers and can learn complex correlations between SNP information and HLA gene sequences based on reference data for HLA imputation, which serves as training data [155]. The learned model can output predicted values of HLA gene sequences with high accuracy using SNP information as input [156].

By using a deep learning method called multi-task learning to simultaneously predict the gene sequences of multiple HLA genes, it is possible to improve accuracy and shorten execution time [130]. To investigate which part of the input information surrounding the HLA gene is used to make learning predictions, predictions were made using not only a small number of nearby SNP information but also many SNP information distributed over a wider area by visualizing the learning information of the model [29,129,130]. While conventional methods are strong at learning using nearly SNP information and not good at learning using SNP information located at distant locations, some new systems are thought that prediction accuracy may have improved because this problem was overcome. HLA genes are involved in the onset of a variety of diseases and are attracting attention [128,157]. As an important area from the perspective of elucidating pathological conditions and realizing personalized medicine. The increased prediction accuracy of HLA gene sequences is expected to increase the reliability of HLA analysis, including integrated analysis between different racial populations, and is expected to greatly contribute to the identification of HLA gene sequences associated with diseases and further elucidation of pathological conditions [129,158,159].

On the multi-task learning approach used for predicting HLA gene sequences, model architecture is a multitask convolutional neural network with a shared part of two convolutional layers and a fully connected layer, followed by individual fully connected layers for each HLA gene group [130]. The HLA genes are grouped into 4 groups: (1) Class I, (2) Class II, (3) Class III, and (4) Other genes like TAP. The shared convolutional layers extract common features across HLA genes in a group, while the individual fully connected layers predict allelic dosages for each specific HLA gene. SoftMax activation is used to output imputation dosages ranging from 0.0 to 1.0 for each allele. Dropout and batch normalization are applied for regularization. As for training data, Two HLA reference panels are used for training: Japanese (n = 1118) and Type 1 Diabetes Genetics Consortium (T1DGC) (n = 5122). The input is pre-phased SNV genotypes encoded as one-hot vectors based on reference/alternate alleles. Hierarchical fine-tuning is used by transferring parameters from higher (2-digit) to lower (4/6-digit) allele resolution models during training. As for validation, 10-fold cross-validation is performed on the reference panels to evaluate imputation accuracy. An independent Japanese HLA dataset (n = 908) is also used for validation. Multi-ethnic validation is done using 1000 Genomes Project data. Accuracy metrics used are sensitivity, positive predictive value (PPV), r2 for dosage, and concordance rate for best-guess genotypes. Thus, the multi-task deep learning approach allows leveraging shared features across HLA genes while still predicting individual gene sequences, outperforming conventional methods, especially for low-frequency alleles.

As for limitations of methodology of fixed prediction targets, unlike genotype imputation methods that can handle missing genotypes, the multi-task learning approach treats HLA allele imputation as a fixed classification problem, which may limit its flexibility [130]. For dependence on linkage disequilibrium (LD), while the multi-task learning is less dependent on distance-based LD decay patterns compared to conventional methods, its performance may still be influenced by complex LD structures in the MHC region [130]. For hierarchical fine-tuning assumptions, the multi-task learning approach assumes a hierarchical structure of HLA alleles (2-digit, 4-digit, 6-digit) and transfers parameters across these levels. However, some alleles in the reference panel may not follow this hierarchy, potentially affecting the model’s performance [130]. As for broader limitations of the multi-task learning approach for HLA imputation in limited applications in population genetics, the application of deep neural networks, particularly multi-task learning, to population genetics data has been limited so far, highlighting the need for further exploration and validation [130]. For computational complexity, while multi-task learning aims to reduce processing time by grouping tasks, the computational complexity of deep learning models can still be a limitation, especially for large-scale biobank data [130]. As for interpretability, deep learning models, including multi-task architectures, are often criticized for their lack of interpretability, making it challenging to understand the underlying patterns learned by the model. As for data requirements, multi-task learning approaches may require large and diverse reference panels to effectively capture the complex patterns and dependencies across HLA genes, which can be challenging to obtain [130]. It’s important to note that the authors acknowledge the need for further research and validation to address these limitations and explore the full potential of multi-task learning for HLA imputation and other applications in population genetics.

The studies provide several empirical examples demonstrating the effectiveness of the proposed method for multi-task learning to predict HLA gene sequences. The applied multi-task learning to two different HLA imputation reference panels—a Japanese panel (n = 1118) and type I diabetes genetics consortium panel (n = 5122). Through 10-fold cross-validation on these panels, the multi-task learning achieved higher imputation accuracy than conventional methods, especially for imputing low-frequency and rare HLA alleles [130]. They further evaluated multi-task learning on an independent HLA dataset (n = 908) and found it outperformed other methods [130]. Simulation studies on multi-ethnic individuals from the 1000 Genomes Project demonstrated the multi-task learning’s ability to accurately impute HLA alleles across diverse populations [130]. The authors applied multi-task learning to impute HLA alleles in large genome-wide association studies from the BioBank Japan (BBJ) and UK Biobank (UKB) cohorts. They then performed trans-ethnic fine-mapping analysis in the MHC region using the imputed HLA data, highlighting a practical application [130]. In a preprint, Naito et al. reported results from applying their similar multi-task CNN architecture to the type I diabetes genetics consortium dataset, achieving 97.6% imputation accuracy, superior to SNP2HLA and comparable to HIBAG. The studies provide concrete empirical evidence across multiple reference panels, independent datasets, simulations, and real data applications demonstrating the effectiveness of the multi-task deep learning approach for accurate HLA gene imputation.

Multi-task learning can contribute to improving accuracy and reducing execution time compared to training separate models for each task by leveraging shared representations and features across related tasks [160,161]. As for improved accuracy, multi-task learning allows the model to learn more robust and generalizable feature representations by sharing knowledge across related tasks. This helps capture underlying patterns that are useful for multiple tasks, leading to better generalization. By jointly optimizing for multiple objectives, multi-task learning can regularize the model and prevent overfitting to any single task, resulting in improved overall accuracy. Multi-task learning enables transfer learning, where knowledge gained from one task can be transferred to related tasks, boosting their performance. As for reduced execution time, with multi-task learning, a single model is trained to perform multiple tasks simultaneously, eliminating the need to train and deploy separate models for each task. This reduces the overall training and inference time. Shared representations and parameters across tasks in multi-task learning architectures like Hard Parameter Sharing (HPS) and Cross-Stitch Networks lead to fewer parameters compared to training individual models, resulting in faster execution. Efficient multi-task learning methods like FastCAR optimize the network architecture and loss function weighting schemes, allowing for faster convergence and reduced training time while maintaining accuracy [162]. Techniques like dynamic task weighting and gradient modulation in multi-task learning L can help balance the learning of different tasks, leading to faster convergence and lower execution times. Thus, multi-task learning improves accuracy by leveraging shared representations and knowledge transfer across related tasks, while reducing execution time by eliminating redundant computations and optimizing architectures and loss functions for efficient multi-task training and inference.

Leveraging insights into HLA diversity holds significant potential for personalized medicine and disease treatment in several key areas. In organ transplantation, HLA typing plays a crucial role in matching donors and recipients for organ transplantation. By utilizing AI and machine learning to analyze complex HLA data, we can optimize donor-recipient matching, reducing the risk of organ rejection and increasing the chances of successful transplantation. This personalized approach improves transplant outcomes. In disease susceptibility, certain HLA alleles are linked to an increased risk of developing complex diseases like type 1 diabetes, RA, and some cancers. AI and machine learning can identify patterns in massive HLA datasets, pinpointing specific disease-associated alleles. This enables early detection, risk stratification, and personalized treatment plans tailored to an individual’s HLA profile. In pharmacogenomics, HLA typing is valuable in pharmacogenomics, the study of how genes influence drug responses. By leveraging AI to analyze HLA data, we can predict an individual’s potential reaction to specific drugs, avoiding adverse effects and optimizing therapeutic efficacy. This personalized approach enhances drug safety and treatment outcomes. In vaccine development, understanding HLA diversity across populations can inform vaccine design and deployment strategies. Tailoring vaccines to account for HLA variation within target populations may maximize their effectiveness and enable a more precise, personalized approach to vaccination programs [163]. Thus, harnessing HLA diversity through AI and machine learning offers opportunities for personalized medicine across organ transplantation, disease risk assessment, pharmacogenomics, and vaccine development. This approach has the potential to revolutions healthcare by enabling early intervention, optimized treatments, and improved patient outcomes tailored to an individual’s unique genetic profile.

Multi-task learning for predicting HLA gene sequences has several potential real-world applications in the context of HLA diversity and immune response. In disease association studies, HLA genes play a crucial role in the immune system and are associated with various autoimmune diseases, infectious diseases, and adverse drug reactions. Accurate imputation of HLA alleles can facilitate large-scale association studies, leading to a better understanding of disease mechanisms and personalized medicine approaches [164]. In transplantation medicine, HLA matching is critical for successful organ and stem cell transplantation. Precise HLA typing of donors and recipients can improve transplant outcomes by reducing the risk of rejection and graft-versus-host disease [164]. HLA diversity influences the immune response to pathogens and vaccines in vaccine development. Accurate HLA imputation can help identify population-specific HLA alleles associated with vaccine efficacy or adverse reactions, informing the design of more effective and safer vaccines [163]. HLA molecules present antigenic peptides to T cells in immunotherapy, playing a crucial role in cancer immunotherapy. Precise HLA typing can aid in developing personalized cancer immunotherapies, such as neoantigen-based vaccines and adoptive T-cell therapies [163]. In population genetics studies, HLA genes are among the most polymorphic in the human genome, making them valuable markers for studying human evolution, migration patterns, and population structure [163]. By leveraging multi-task learning and deep neural networks, the proposed methodology can accurately impute HLA alleles, including low-frequency and rare variants, from SNP data across diverse populations. This can facilitate large-scale studies and applications in precision medicine, transplantation, vaccine development, and population genetics, ultimately leading to a better understanding and management of immune-related diseases and conditions.

4. Multimodal AI

Thus, harnessing HLA diversity through AI and machine learning offers opportunities for personalized medicine across organ transplantation, disease risk assessment, pharmacogenomics, and vaccine development. Transformer brought about such a dramatic change in the field of AI that have “changed everything about deep learning.” Natural language models using Transformer, BERT (bidirectional encoder representations from Transformer) (Figure 2), GPT-3, PaLM are rapidly improving their performance and are becoming common knowledge. Transformer, which is a neural network consisting only of attention mechanisms, is used not only in the field of natural language but also in a variety of other fields, including Vision Transformer for image recognition, Alphafold2 for protein 3-dimension (3D) structure analysis, CLIP that combines language and images, and Gato, which performs over 600 tasks, is developed using Transformer. With Gato, data such as images and language are essentially the same regardless of modality, making it possible to create multiple AI.Large language model (LLM) is capable of multimodal learning, which means learning data with different properties, modes at the same time [164]. In other words, multimodal AI is a deep learning method that inputs multiple types of data and processes it in an integrated manner. CNNs create decision machines and process data from a single type of information input.

On the other hand, multimodal AI refers to a deep learning method that takes multiple types of data as input and processes them in an integrated manner. When humans process information, they combine multiple sensory information that comes in from the outside, such as visual information, smell, touch, taste, and hearing, represented by the five senses. Multimodal AI uses a method of creating a deep learning model from multiple data like the information processing performed by the human brain and creating a judger. For example, when determining an object in a certain image, in most cases an AI model is created by learning only the image, visual information. This type of learning and making decisions based on only one piece of information is called “single modal”. On the other hand, in the case of multimodal AI, it becomes possible to judge objects based on multiple modals of the five human senses, such as visual information, auditory information, and olfactory information. However, this is a future vision of multimodal AI that may be realize in the future, and the multimodal learning that is used has not reached that level. One possibility is to use the meta information of the image to improve the accuracy of discrimination. As multimodal learning progresses, a single AI model will be able to determine multiple factors [165]. This increases the possibility of being able to deal with unpredictable abnormal patterns.

Advances in the field of machine learning, represented by natural language processing (NLP), and the increasing functionality of computers have established a foundation for efficiently extracting information from the vast amount of sequence data that has been accumulated through the development of sequencing technology. Language mode-related research is rapidly developing in protein research using machine learning. A language model is a representation learning technology that learns the characteristics of the data itself from large-scale data [166]. It was originally proposed as a statistical model that learns the feature representation of each word that makes up a natural language sentence generation probability function. Protein language models are language models that are trained by treating the primary sequence of a protein as a “sentence” and each amino acid as a “word” [167,168,169,170]. Broadly divided by architecture, there are word2vec, CNNs, recurrent neural networks (RNNs), and those based on Transformer, which is a neural network proposed in connection with machine translation. Unlike conventional CNNs and RNNs, Transformer is unique in that it can incorporate dependencies between distant tokens through its self-attention mechanism. Here, the words (processing units) processed by the language model are called tokens, which in this paper correspond to each amino acid included in a protein sequence [171]. When using Transformer, information is processed using a three-layered structure: (i) vector representation (token embedding) specific to each amino acid, (ii) incorporation of context information via self-attention, and (iii) calculation of amino acid appearance probability for each residue position. The amino acid representation finally obtained by Transformer is called an embedding and corresponds to the output of (ii). An architecture has been proposed that combines an encoder to obtain embeddings and a decoder to generate arrays.

To treat biological sequences with machine learning, it is first necessary to obtain their feature representation [172,173]. For proteins, classically, position-specific vectors expressions based on so-called descriptors, including AAindex, ST-scale, Z-scale, T-scale, FASGAI, MS-WHIM, ProtFP, VHSE etc., or MSA that reflect the physicochemical and structural properties of amino acids in a sequence scoring matrix (PSSM), etc. have been used [174,175]. A language model in natural language processing applied to proteins, amino acid sequences is called a protein language model [176,177,178,179]. Since there are many unknowns about how proteins function in living organisms, it is hoped that new knowledge will be obtained by using large amounts of data and pre-learning methods such as BERT [171,180] (Figure 2). The Transformer used in BERT is an architecture widely used in natural language processing (Figure 2). There are increasing attempts to improve the interpretability of models by analyzing the attention mechanism in this Transformer. In particular, the one that interprets BERT is called BERTology. There are two main analysis methods for Transformer-based pre-trained models, which are effective for the target task by performing learning on a different task before the target task to be solved for acquiring features. To make robust predictions possible even when there is little training data, it is possible to solve this problem by pre-training BERT on a large-scale protein database [171,180,181]. Analysis of attention mechanism: among the amino acid residues whose attention weights exceed a threshold, it checks how many combinations have biological characteristics [182]. Specifically, it is calculated using the following formula, which is an evaluation index close to precision.

$$\mathrm{f}\mathrm{a}\left(\mathrm{x}\right)={\displaystyle \frac{{\sum }_{\mathbf{x}\in \mathbf{X}}^{ } {\sum }_{\mathbf{i}=\mathbf{1}}^{\left|\mathbf{x}\right|} {\sum }_{\mathbf{j}=\mathbf{1}}^{\left|\mathbf{x}\right|}\mathbf{f}\left(\mathbf{i},\mathbf{j}\right)·\mathbf{L}\mathbf{a}\mathbf{i},\mathbf{j}>\mathsf{\theta }}{{\sum }_{\mathbf{x}\in \mathbf{X}}^{ } {\sum }_{\mathbf{i}=\mathbf{1}}^{\left|\mathbf{x}\right|} {\sum }_{\mathbf{j}=\mathbf{1}}^{\left|\mathbf{x}\right|} \mathbf{L}\mathbf{a}\mathbf{i},\mathbf{j}>\mathsf{\theta }}}$$

(1)

A probe task is an auxiliary task used to improve the interpretability of pre-trained models in natural language processing, which checks what kind of information the internal representation acquired in the pre-trained model contains. Specially, it input the representation obtained from the model into a classifier, have it solved some tasks, and check whether useful information is encoded for that task. There are two types of probe tasks for Transformer-based models: embedded probes and attention probes. Embedded probes target the output of each layer, and attention probes target attention weight [183,184]. In the analysis of attention mechanisms, contact maps are important features in protein secondary structures [176,185,186]. A contact map is a combination of spatially close to amino acids in protein folding. Using this contact map as a feature for amino acid pairs and applying the attention analysis method described above, fa(x) for each pre-trained model ranged from 44.5% to 63.2%. In addition, the attention heads that most reflected the contact map were in deeper layers. Considering that the background probability in the contact map is 1.3%, the pre-learning in the protein language model has acquired a high-order representation that reflects the contact map [173,178,187]. Binding sites are the sites where proteins interact with other molecules and are very important features for protein function [188]. The proportion of attentions that refer to this binding site, fa(x) is between 45.6% and 50.7%, which is a very high value considering that the background probability of the binding site is 4.8%. Additionally, most attention heads refer to binding sites at a high rate [189]. The reason why the binding site is given such importance in the model even though it is a feature that reflects interaction with external molecules is that it has been suggested that structural motifs may be highly conserved because they are features directly related to protein function [188,190,191]. Post-translational modifications are changes made to proteins after they are translated from mRNA, and phosphorylation is a typical post-translational modification, which play a major role in protein structure and function [192]. In post-translational modification, fa(x) is 64%, which is a very high value considering that the background probability of post-translational modification is 0.8% [193]. However, the number of attention heads that refer to post-translational modification site is small [194]. As a result of performing embedding probes and attention probes in each layer of the pre-learning model, the F1 score is used for secondary structure prediction, and the precision is used for binding site prediction and contact prediction. Prediction of secondary structures such as helices, turns/bends, and strands show prediction accuracy even using outputs at relatively low layers [195].

One of the advantages of using language models is that their pre-training can be performed unsupervised, which is also called self-supervised learning because it learns using the input array itself as a teacher. The specific method differs depending on the architecture of the language model. Herein, we will focus on masked language modeling (MLM), which was introduced in BERT and is widely used. With this method, stochastically mask some tokens in the input array. By inputting this masked sequence into Transformer, the likelihood of each amino acid at the masked position can be calculated. Therefore, in MLM, the negative log likelihood L (2) for the original amino acid at all mask positions M is minimized as a loss function. Thus, by inferring the excluded residue from information on surrounding residues (context), it is possible to learn the appearance pattern of amino acids in the sequence [196,197].

$$\mathrm{L}=-\sum _{\mathrm{i}\in \mathrm{M}}^{ }\mathrm{l}\mathrm{o}\mathrm{g}\left(\mathrm{p}\mathrm{i},\mathrm{a}\right)$$

(2)

In a method using a BERT model with a self-attention mechanism pre-trained using a large-scale protein database, it is possible to predict B cell epitopes and peptide binding to MHC class II [171,180,198]. A vaccine is a substance, such as a weakened pathogen, that is artificially administered to create immunity against an infectious disease. Historically, direct ingestion of live, attenuated pathogens has triggered an immune response within the body, but later dead pathogens, their antigens, and toxins alone are used as vaccines. Lymphocytes, especially B cells and T cells, are involved in acquiring immunity against infectious diseases. B cells that induce antigens-specific immune response in vivo express B cell receptors (BCRs) on their cell membranes, which directly bind to antigen proteins and recognize their epitope regions, which allows the production of large amounts of antigen-specific antibodies. On the other hand, the T antigen peptide produced when the pathogen protein is phagocytosed by antigen-presenting cells as an immunogen must be used as an immunogen for cells to recognize pathogen proteins, needs to be presented on the cell surface by binding to MHC molecules. The T cell then recognizes the presented peptide and responds and activates B cells that produce antibodies or directly destroys infected cells [199]. Based on these properties, research has recently begun on peptide vaccines, which attempt to acquire immunity by administering small peptide fragments that have a structure accepted by B cells and T cells [200].

By administering a vaccine that mimics the structure of a specific antigen epitope region, it is possible to induce the production of monoclonal antibodies specific to that antigen [201]. To search for peptide vaccine candidates that act against B cells and T cells, it is necessary to predict the peptides that bind to BCRs and the peptides that bind to MHC molecules. It has traditionally been thought that this prediction requires precise clarification of the steric bounds between proteins [202]. Experimentally determining the 3D structure of an antibody-antigen complex is extremely time-consuming and labor-intensive, making it unsuitable for assaying large numbers of candidate peptides, such as in vaccine development [203]. When predicting bonds using computer simulations, there are still issues in which bonds can only be investigated when proteins are considered as rigid bodies, and issues that require enormous amounts of time and computational resources even when using molecular dynamics-based methods. On the other hand, in recent years, research has been progressing on binding prediction using machine learning that does not explicitly treat binding between proteins. However, high accuracy has not yet been achieved due to challenges in B cell epitope prediction and prediction of peptide binding to MHC molecules, especially MHC class II molecules, using machine learning [204,205]. The first problem is that it is difficult to learn long-distance dependencies. In recent years, long-distance dependency learning has been attempted using LSTM (Long Short-Term Memory), a type of deep learning, for sequence data of amino acid sequences. LSTM is a model that deals with the problem of long-distance dependencies using storage parts called cells [206,207]. However, it still needs to go through a long-distance network between amino acids, and information may be lost [208,209].

Furthermore, when applied to proteins, understanding the dependence relationships caused by the higher-order structure of the protein remains an issue [210,211,212]. In the task of predicating peptides that bind to MHC class II, it is necessary to capture the complex interactions between amino acids in both MHC class II and the peptide, which is difficult to capture such interactions with LSTM [207,213,214]. In predicting peptides that bind to MHC class I molecules, one of the two major classes of MHC molecules, there are fewer problems learning long-distance dependencies, and predictions using machine learning can achieve higher accuracy, because the amino acid sequence is shorter than MHC class II [214,215,216]. The second issue is that sufficient generalization performance cannot be obtained when supervised learning data is small [217]. The simplest approach to improving generalization performance is to prepare more supervised learning data, but this is not easy because biological experiments to increase training data require a large amount of cost. To solve these problems, BERT, which is based on a large-scale protein database, does not store sequence information as an intermediate vector like LSTM, but can model long sequences with variable lengths by using only an attention mechanism [218]. In LSTM, the farther the distance between amino acids, the more networks it must pass through, and as a result, there is a risk of losing information about amino acids that are far apart. The shorter this distance, the easier it is to learn long-distance dependencies, but the attention mechanism, which also allows to capture complex interactions between two series data, has the advantage of being able to directly model relationships between amino acids, regardless of their positions in the sequence [217,219]. Another great feature of BERT is that it can use large amounts of unsupervised data for pre-training [220]. By pre-training BERT based on a large amount of protein data, generalization performance can be improved even when training data is small [172,182]. One of the main features of BERT is that it uses a bidirectional Transformer in its architecture, which allows it to learn long-distance dependencies better than LSTM [221]. Transformer is a neural network consisting only of attention mechanisms, and Transformer can capture long-distance dependencies better than LSTM. Model with multiple layers of unidirectional Transformers perform better than LSTM in various NLP tasks. BERT furthermore makes the Transformer bidirectional so that sequence information from the opposite direction is also used to improve performance. In this task, it is important to understand the long-range dependence of each amino acid.

When predicting the binding of peptides to MHC class II, it is important to learn the complex interactions between MHC class II and peptides [222,223]. BERT can capture such long-distance dependencies and can be expected to learn complex interactions by adding multiple layers of attention mechanisms. Another main feature of BERT is that generalizations performance can be improved by pretraining with MLM on large datasets, which in MLM is a task in which some words are deleted from an input sentence and the deleted words are predicted from the surrounding context. This task allows to learn the features of the word itself, considering the context. MLM can use large amounts of unsupervised text data because the words themselves functions as a type of supervised data [224]. Through this pre-training, BERT has further improved its performance in various NLP tasks. In addition, pre-training can learn more efficiently than other methods even when there is little data.

In the early stages of research, B cell epitope prediction used only the physicochemical properties of the amino acids that make up proteins as features [225,226]. Since then, a method based on machine learning that incorporates information on the amino acid sequence itself has achieved relatively high accuracy, and many methods are used, including those using support vector machines, Random Forests, forward propagation neural networks, RNNs, and LSTM with attention mechanisms [227]. Although Lbtope and BepiPred-2.0 use features of dipeptide composition, long-range features of antigen proteins are not incorporated into the model [228]. Because DLBepitope uses a simple forward propagation neural network, it is difficult to learn complex dependencies between amino acids compared to models with sequence information and attention mechanisms. ABCpred attempts to learn sequence information using RNN, but RNN cannot capture long-distance dependencies. The method using LSTM with attention mechanism achieves higher accuracy than BepiPred-2.0 by combining features that take such long-distance dependencies into account and structural and chemical features of the entire antigen protein [229,230,231]. However, since LSTM is used, information between amino acids that are far apart may be lost. Many machine learning methods have been proposed for predicting peptide binding to MHC class II, but the following two issues need to be appropriately addressed [227,232,233]. The first issue is the problem of vanishing gradients and long-range dependence problems that occur in algorithms for long sequence data such as amino acid sequences [234]. DeepSeqPanll deals with this by using a model that combines LSTM with an attention mechanism and CNN, but this problem remains because it uses LSTM in the model [235]. The second issue is the need to consider the interaction between peptides and MHC class II [222]. As input a graph representation and amino acid sequence of the protein, graph representation of compound and amino acid sequence project onto a low-dimensional vector using a graph neural network (GNNs) and CNNs, respectively, and the interaction between these vectors is captured using an attention mechanism, showing higher performance than existing methods [236,237,238]. However, since GNNs require structural information as input, they cannot be trained in advance using a large-scale protein database, which will be described later, and CNNs cannot capture long-distance dependencies [239,240]. By using only attention mechanism in the model, simultaneously solve the problem of long-range dependence and the problem of considering the interaction between peptide and MHC class II, while improving generalization performance by performing pre-learning on a large-scale protein database [171,178,241,242]. Furthermore, machine learning in general has the problem of not being able to sufficiently learn features from the training data when the training data is small [243]. Simply increasing the training data is difficult because it requires a large amount of cost, and in recent years, methods such as data augmentation and pre-learning have been used. Data augmentation is a method of extending learning data by making predictions using a machine learning model for similar data and adding temporary labels. In predicting peptide binding to MHC class II, the performance of the model is improved by expanding the training data for peptide binding prediction using ligand data obtained from mass spectrometry [57,244].

In addition, there are two issues: it is difficult to learn complex dependencies between distant amino acids, and accuracy is low when there is insufficient training data [206,208,245]. To solve these issues, there is a method for predicting peptide binding using BERT, in which the advantages are (1) the attention mechanism allows learning long-range dependencies and interactions between amino acids in MHC class II and peptide bounds; (2) the generalization performance of the model can be improved by using a large-scale unsupervised database for pre-training [246,247,248,249]. One of the main features of BERT is that it can learn long-distance dependencies better than LSTM by adopting a bidirectional Transformer in its architecture, which can capture long-distance dependencies better than LSTM. Models with multiple layers of unidirectional Transformer perform better than LSTM in various NLP tasks. BERT further improves performance by making the Transformer bidirectional and using sequence information from the opposite direction. In this task, it is important to capture the long-range dependencies of each amino acid [250]. In this task, it is important to capture the long-range dependencies of each amino acid [206,208,209]. When predicting the binding of peptides to MHC class II, it is important to learn the complex interactions between MHC class II and peptides. BERT can capture such long-distance dependencies and is expected to learn complex interactions by adding multiple layers of attention mechanisms. Another main feature of BERT is that it can improve generalization performance by pre-training large-scale datasets using MLM in NLP, in which some words are deleted from an input sentence and the deleted words are predicted from the surrounding context and the words themselves function as a type of supervised data, so a large amount of unsupervised text data can be used. Through this pre-training, BERT can further improve its performance in various NLP tasks [251,252]. In addition, pre-training can learn more efficiently than other methods even when there is little data [253]. Therefore, BERT is applied to peptide bond prediction to capture long-distance dependencies between amino acids.

B-cell epitope prediction requires only a single peptide sequence as input, so BERT can be used directly by inputting the peptide sequence directly to BERT [254,255]. In predicting peptide binding of MHC class II molecules, it is not obvious how to apply BERT because it handles as input the triple of a peptide sequence, the amino acid sequence of the alpha chain of the MHC class II molecules, and the amino acid sequence of the beta chain of the MHC class II molecule. Traditionally, models before BERT project multiple inputs into vectors independently and then apply a bidirectional attention mechanism [256]. It is possible to effectively model relationships between multiple inputs by concatenating inputs with a special token called a separate token and applying a bidirectional self-attention mechanism. As a result, it is expected that the model can be modeled in the same way regardless of whether the input is a single amino acid sequence or multiple amino acid sequence. However, when BERT is trained only on supervised learning data, that is, when no prior training is performed, BERT cannot learn well. This is because there is little training data used to predict B cell epitopes and peptide binding to MHC class II, making it difficult to adjust the huge parameters of BERT. Therefore, when pre-training BERT using the large-scale protein database Pfam, the amount of supervised learning data used to predict B cell epitopes is B cell epitopes is approximately 200,000, and the amount of supervised learning data used to predict peptide binding to MHC class II is approximately 50,000, Pfam is a database of approximately 31 million proteins [171,180,181]. Therefore, by pre-training Pfam using MLM, it is possible to adjust the huge parameters of BERT [257]. In addition, BERT pretrained with Pfam can improve generalization performance for tasks such as learning protein structure and function, secondary structure prediction, and contact prediction [171]. Regarding the architecture of the model, after receiving as input a sequence of N characters of amino acids x = (x1, x2, x3,……..xN), the output is a sequence of H-dimensional embedding vector Z = (z1, x2, x3,….. zN) in the l layer. When the average vector S of the output Z = (z1, x2, x3,….. zN) of the final layer of BERT is an aggregate representation of the input sequence, the vector S∈RH takes the inner product with the weight W∈RH of the classification layer. As a result, it is projected into one dimension, and the final output o is activated by the sigmoid function. Cross entropy error is used for optimization.

$$\mathrm{S}=1/\mathrm{n}{\sum }_{\mathrm{i}}^{\mathrm{n}}\mathrm{Z}\mathrm{i}$$

(3)

O = sigmoid (SW^T)

(4)

In addition, a comparison of AUC and SECC on test data for BERT with pre-training and BERT without pre-training shows that pre-training contributes to improving generalization performance, but this is because the parameters of BERT, which has a high degree of freedom, cannot be learned sufficiently unless pre-training is performed using a large-scale database. One way to improve model performance is to perform data augmentation separately from pre-learning. In addition, by visualizing attention, BERT can see where the learned model is paying attention when making predictions. BERT focuses on amino acids in close positions in the 3D structure and on binding sites [256].

Med-UniC is a novel framework designed to integrate multimodal medical data from English and Spanish for cross-lingual medical vision-language pre-training. It addresses the challenge of community bias caused by different languages by proposing Cross-lingual Text Alignment Regularization (CTR). CTR explicitly unifies cross-lingual semantic representations of medical reports from diverse language communities through latent language disentanglement, mitigating bias from determining positive-negative sample pairs within analogous reports [258]. In cross-lingual medical language model, constructs a cross-lingual vocabulary by combining English and Spanish medical terms. Pre-trains on mixed English and Spanish medical reports using masked language modeling. Selectively updates high layers to alleviate catastrophic forgetting during vision- language pre-training. In vision-language alignment, incorporates contrastive learning to predict matched image-text pairs while mapping negative pairs far apart. Uses non-linear visual and linguistic projectors to map images and texts to a joint embedding space. In Cross-Lingual Text Alignment Regularization (CTR), Optimizes through latent language disentanglement, not depending on negative samples. Mitigates bias from determining positive-negative sample pairs within analogous reports. Ensures cross-lingual representation is not biased toward any specific language community. Med-UniC achieves superior performance across 5 medical image tasks and 10 datasets covering over 30 diseases, offering a versatile framework for unifying multimodal medical data from diverse linguistic communities. Experimental results highlight the presence of community bias in cross-lingual vision-language pre-training and show that reducing this bias enhances performance in both vision-language and uni-modal visual task [258].

ETP (ECG-Text Pre-training) is an innovative approach that leverages both electrocardiogram (ECG) data and corresponding textual reports within a cross-modal learning paradigm to learn transferable ECG representations [258]. The key idea is to use a trainable ECG encoder and a frozen language model to embed paired ECG signals and automatically generated clinical reports separately [259]. The self-supervised pre- training objective aims to maximize the similarity between paired ECG and report embeddings while minimizing the similarity between ECG and other mismatched reports [259]. This multimodal ECG-text pre-training approach enables zero-shot learning for ECG classification tasks, achieving around 10% performance improvement without using any annotated data compared to supervised and self-supervised baselines that rely on annotated data [259]. Notably, ETP demonstrates high generalizability, effectiveness, and efficiency, attaining the highest recall and F1 scores on the MIT-BIH dataset despite containing different ECG classes from the pre-training dataset [259]. The frozen language model component plays a crucial role in helping ECG zero-shot learning by providing rich semantic representations from the textual reports, which guide the ECG encoder to learn transferable representations that can generalize to unseen classes during inference [259].

The integration of genomic data, AI technologies, and clinical outcomes significantly contributes to a better understanding of HLA diversity and its role in various pathological conditions. HLA genes are highly polymorphic and play a crucial role in immune response, making them important for understanding disease susceptibility and outcomes. Advanced genomic technologies like next-generation sequencing and imputation methods enable high-resolution characterization of HLA allelic variations [260]. AI and computational approaches facilitate analysis of large-scale HLA genomic data and identification of disease associations [261]. Genome-wide association studies consistently show strong associations between HLA variations and immune-mediated diseases like autoimmune disorders, but conventional methods are limited in fine-mapping these highly polymorphic regions. High-resolution HLA sequencing and AI-driven analysis enable unambiguous identification of disease-associated polymorphisms. Integrating HLA genomics with clinical data has revealed associations between specific HLA-KIR combinations and transplantation outcomes like graft rejection and graft-versus-host disease [261]. This aids in donor selection and personalized transplantation strategies. Studies combining HLA sequencing, AI algorithms, and phenotypic data from large biobanks like UK Biobank have uncovered novel HLA associations with cancer risk and heterogeneity across subtypes [260,262]. This highlights HLA diversity’s role in tumor immunology and immunotherapy response. AI-driven analysis of HLA diversity along with KIR genomic regions provides insights into innate immunity mechanisms relevant for infectious diseases and cancer immunotherapies [261]. Therefore, the synergy between high-throughput genomics, AI-powered bioinformatics, and deep phenotyping through biobanks enables comprehensive decoding of HLA polymorphisms and their intricate connections to various pathological conditions, paving the way for improved diagnostics and therapeutics [260,261,262].

Recent developments in multimodal AI have significant implications for cancer immunotherapy. Multimodal AI models can integrate diverse data sources like radiology, histology, genomics, and electronic health records to provide a comprehensive view of the patient’s immune response and tumor microenvironment. This holistic assessment enables better identification of biomarkers and predictors for immunotherapy response and resistance mechanisms [263,264]. As for improved patient stratification, Multimodal AI can capture the complexity of antitumor immune responses by combining multiple modalities like tumor genomics, immune cell profiling, and imaging data. This allows for more accurate stratification of patients who are likely to respond to immunotherapies like immune checkpoint blockade (ICB) [264]. In identification of multimodal biomarkers, Single biomarkers often have limited predictive power for immunotherapy response. Multimodal AI can discover novel multimodal biomarkers by integrating complementary information across modalities, providing a more robust predictor of treatment outcomes [262,264]. In precision immunotherapy selection, by profiling the patient’s immune status holistically, multimodal AI can identify the key rate-limiting factors hindering antitumor immunity. This enables selection of optimal immunotherapy strategies or combinations tailored to overcome the specific resistance mechanisms in each patient [264]. As for monitoring treatment response, longitudinal multimodal data can be leveraged by AI to monitor dynamic changes in the tumor microenvironment during immunotherapy. This allows early prediction of treatment response or resistance, enabling timely adaptation of therapeutic strategies [262,264]. In novel biomarker discovery, AI-driven exploration of multimodal data associations can uncover previously unknown patterns and relationships between different modalities. These insights can guide the discovery of novel biomarkers and therapeutic targets for cancer immunotherapy [262,264]. Therefore, multimodal AI approaches provide a powerful framework to dissect the complexities of antitumor immunity, enabling more precise and personalized cancer immunotherapy strategies.

5. Conclusions

Cancer antigen vaccine therapy is a treatment method that actively immunizes cancer patients with TAAs from the outside to induce and enhance antitumor immunity, with the aim of inducing T cell immune responses against TAAs. As clinical trials progress, cancer antigen vaccine therapy has few side effects than conventional cancer treatments and is effective in prolonging survival while maintain patients’ quality of life. Furthermore, the development of more potent cancer antigen will require more complex cancer immunotherapy that combine treatments that control immunosuppressive mechanisms in the tumor microenvironment. In patients with specific HLA alleles, neoantigen peptides generated by these driver mutations are presented to T cells by the HLA and inducer tumor immunity, thereby eliminating tumor cells. Additionally, advances in genetic analysis technology have made it possible to evaluate the cancer and immune status of individual cancer patients using biomarkers and other methods. When predicting the binding of peptides to MHC class II, it is important to learn the complex interactions between MHC class II and peptides. BERT can capture such long-distance dependencies and can be expected to learn complex interactions by adding multiple layers of attention mechanisms. BERT is applied to peptide bond prediction to capture long-distance dependencies between amino acids. Another great feature of BERT is that it can use large amounts of unsupervised data for pre-training. BERT pretrained with Pfam can improve generalization performance for tasks such as learning protein structure and function, secondary structure prediction, and contact prediction. In addition, by visualizing attention, BERT can see where the learned model is paying attention when making predictions. The prediction tools will be useful for the novel therapeutic approaches for immune diseases via HLA immune responses.