Next Article in Journal
Drought-Induced Leaf Proteome Changes in Switchgrass Seedlings
Next Article in Special Issue
Probiotics Differently Affect Gut-Associated Lymphoid Tissue Indolamine-2,3-Dioxygenase mRNA and Cerebrospinal Fluid Neopterin Levels in Antiretroviral-Treated HIV-1 Infected Patients: A Pilot Study
Previous Article in Journal
Evaluation of Pulmonary Toxicity of Zinc Oxide Nanoparticles Following Inhalation and Intratracheal Instillation
Previous Article in Special Issue
Prolonged Morphine Exposure Induces Increased Firm Adhesion in an in Vitro Model of the Blood–Brain Barrier
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Impact of HIV Infection and Anti-Retroviral Therapy on the Immune Profile of and Microbial Translocation in HIV-Infected Children in Vietnam

1
Department of Viral Infection and International Health, Graduate School of Medical Sciences, Kanazawa University, Kanazawa 920-8640, Japan
2
National Hospital of Pediatrics, Hanoi 100-000, Vietnam
3
Yakult Central Institute, Tokyo 186-8650, Japan
4
Department of Parasitology, Graduate School of Medical Sciences, Kanazawa University, Kanazawa 920-8640, Japan
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Int. J. Mol. Sci. 2016, 17(8), 1245; https://doi.org/10.3390/ijms17081245
Submission received: 3 July 2016 / Revised: 26 July 2016 / Accepted: 28 July 2016 / Published: 2 August 2016

Abstract

:
CD4+ T-lymphocyte destruction, microbial translocation, and systemic immune activation are the main mechanisms of the pathogenesis of human immunodeficiency virus type 1 (HIV) infection. To investigate the impact of HIV infection and antiretroviral therapy (ART) on the immune profile of and microbial translocation in HIV-infected children, 60 HIV vertically infected children (31 without ART: HIV(+) and 29 with ART: ART(+)) and 20 HIV-uninfected children (HIV(−)) aged 2–12 years were recruited in Vietnam, and their blood samples were immunologically and bacteriologically analyzed. Among the HIV(+) children, the total CD4+-cell and their subset (type 1 helper T-cell (Th1)/Th2/Th17) counts were inversely correlated with age (all p < 0.05), whereas regulatory T-cell (Treg) counts and CD4/CD8 ratios had become lower, and the CD38+HLA (human leukocyte antigen)-DR+CD8+- (activated CD8+) cell percentage and plasma soluble CD14 (sCD14, a monocyte activation marker) levels had become higher than those of HIV(−) children by the age of 2 years; the CD4/CD8 ratio was inversely correlated with the plasma HIV RNA load and CD8+-cell activation status. Among the ART(+) children, the total CD4+-cell and Th2/Th17/Treg-subset counts and the CD4/CD8 ratio gradually increased, with estimated ART periods of normalization being 4.8–8.3 years, whereas Th1 counts and the CD8+-cell activation status normalized within 1 year of ART initiation. sCD14 levels remained high even after ART initiation. The detection frequency of bacterial 16S/23S ribosomal DNA/RNA in blood did not differ between HIV-infected and -uninfected children. Thus, in children, HIV infection caused a rapid decrease in Treg counts and the early activation of CD8+ cells and monocytes, and ART induced rapid Th1 recovery and early CD8+-cell activation normalization but had little effect on monocyte activation. The CD4/CD8 ratio could therefore be an additional marker for ART monitoring.

Graphical Abstract

1. Introduction

Mucosa-associated lymphoid tissues, such as the gut-associated lymphoid tissue (GALT), harbor approximately 40%–60% of lymphocytes in the human body. GALT is the largest replication site and reservoir of human immunodeficiency virus type 1 (HIV) among HIV-infected individuals. In the early stages of HIV infection, the remarkable destruction of CD4+ T (CD4+) cells, particularly the type 17 helper T-cell (Th17) subset, occurs in GALT [1,2,3], resulting in the decline in the immune and mechanical barrier functions of the gut mucosa. Subsequently, the translocation of microbial products, such as lipopolysaccharides from gram-negative bacteria and bacterial DNA, from the gastrointestinal tract to systemic circulation occurs, which induces systemic immune activation, disrupts immune balance, and causes further loss of CD4+ cells [3,4,5,6]. CD4+ cells in the peripheral blood recover 1–3 years after antiretroviral therapy (ART) initiation, whereas the recovery of CD4+ cells in GALT is much slower. The barrier function of the gut mucosa therefore remains impaired, and the immune activation continues for many years after ART initiation [3,7,8].
Children who have contracted HIV from their mothers (vertical infection) progress to acquired immunodeficiency syndrome (AIDS) more rapidly than HIV-infected adults, showing a more rapid decline in CD4+-cell counts and high plasma HIV viral load (VL). Without ART, more than 50% of these children die by the age of 2 years [9,10]. In children, there is an abundance of lymphoid aggregates in the gut submucosa and memory CD4+CCR5+ T cells in gut epithelial cells [11]. Because these cells are highly susceptible to HIV infection even without activation, they are likely to be the prime site of HIV infection and replication [9,11]. Furthermore, a relatively low HIV-specific Th1 response coinciding with a low Th1/Th2 ratio and an expanded population of regulatory T cells (Tregs) are observed among HIV-infected children, particularly among infants [9,12,13]. HIV-infected children also exhibit a decrease in CD4+-cell counts, CD8+-cell activation, and intestinal microbial translocation [14,15], and ART effectively restores CD4+-cell populations and suppresses CD8+-cell activation [14,16,17], as is the case among HIV-infected adults. However, the impact of HIV infection and ART on different CD4+-cell subsets (Th1, Th2, Th17, and Treg) in children remains poorly understood.
Since 2008, we have followed up with HIV-infected children in Hanoi, Vietnam to find an efficient and cost-effective method and immunological markers for monitoring ART in resource-limited settings [18]. Based on the results of that field study, we conducted this cross-sectional study to clarify the impact of HIV infection and ART on the immune profile and microbial translocation status of children aged over 2 years who have an immune status considered to be relatively mature and stable [9,19,20].

2. Results

2.1. Characteristics of the Subjects

This study included 60 HIV vertically infected children (31 without ART (HIV(+)) and 29 with ART (ART(+)) and 20 HIV-uninfected healthy children (HIV(−)). The characteristics of the study subjects are shown in Table 1. The ART(+) children had received ART for a median period of 3.5 years, and 22 (75.9%) had an undetectable plasma VL (< 220 copies/mL). More ART(+) (5/29, 17.2%) than HIV(+) (1/31, 3.2%) children were at WHO clinical stage 2, although the difference was not statistically significant (p = 0.098). The HIV(+) and ART(+) children were 2 years older than the HIV(−) children (HIV(+) vs. HIV(−): median age 6.2 vs. 4.1 years, respectively, p = 0.034; ART(+) vs. HIV(−): median age 6.1 vs. 4.1 years, respectively, p = 0.009), although height and body weight did not significantly differ among the three groups (all p > 0.1).

2.2. Immune Status of HIV-Infected Children

The immune statuses of the children in the three groups are shown in Table 1. Compared with the HIV(−) children, the HIV(+) children had significantly lower total CD4+-cell (p = 0.003) and Th1/Th2/Th17/Treg-subset counts (p = 0.003/0.016/<0.001/<0.001, respectively), a lower percentage of CD4+ cells in lymphocytes (p = 0.001), a higher percentage of CD8+ cells in lymphocytes (p < 0.001), a lower CD4/CD8 ratio (p < 0.001), and a higher percentage of CD38+HLA (human leukocyte antigen)-DR+CD8+ cells in CD8+ cells (activated CD8+ cells, p < 0.001), although CD8+-cell counts and the CD4+-cell activation status (percentage of CD38+HLA-DR+CD4+ cells in CD4+ cells) did not significantly differ between the two groups (p = 0.239 and 0.969, respectively). The HIV(+) children had higher plasma soluble CD14 (sCD14; a monocyte activation marker) levels than the HIV(−) children (p = 0.009).
The ART(+) children had significantly lower total CD4+-cell (p = 0.018) and Th2/Th17/Treg-subset counts (p = 0.009/<0.001/0.004), a lower CD4/CD8 ratio (p = 0.001), and higher sCD14 levels (p < 0.001) than the HIV(−) children. However, the CD4+-cell percentage in lymphocytes (p = 0.404), Th1 count (p = 0.611), and activated CD8+-cell percentage (p = 0.329) did not significantly differ between the two groups.
The ART(+) children had significantly higher Th1//Th17/Treg-subset counts (p = 0.002/0.016/<0.001), higher sCD14s levels (p < 0.001), and lower activated CD8+ percentages (p < 0.001) than the HIV(+) children. However, the total CD4+-cell (p = 0.429) and Th2-subset counts (p = 0.970) and the CD4/CD8 ratio (p = 0.181) did not significantly differ between the two groups.

2.3. Impact of HIV Infection on Immune Profile

Among the HIV(+) children, the total CD4+-cell counts (Figure 1A, p = 0.007), Th1/Th2/Th17-subset counts (Figure 1B–D, p = 0.040/0.012/0.024, respectively), CD4+-cell percentage in lymphocytes (Figure 1F, p = 0.033), and CD8+-cell counts (Figure 1G, p = 0.060) were inversely correlated with age (that is, nearly equal to their HIV-infection period). On the other hand, Treg counts (Figure 1E) and the CD4/CD8 ratio (Figure 1J) were lower and the CD38+HLA-DR+CD8+- (activated CD8+) cell percentage (Figure 1I) and plasma sCD14 levels (Figure 1K) were higher than those of the HIV(−) children by the age of 2 years. The CD38+HLA-DR+CD4+- (activated CD4+) cell percentage was not significantly correlated with age (Figure 1H). The activated CD8+-cell percentage (Figure 1L, p < 0.001) and sCD14 levels (Figure 1M, p = 0.030) were positively correlated with the plasma VL. The CD4/CD8 ratio was inversely correlated with the plasma VL (Figure 1N, p = 0.045) and CD8 activation status (Figure 1O, p = 0.003).

2.4. Impact of ART on the HIV-Induced Immune Profile

Among the ART(+) children, the total CD4+-cell counts (Figure 2A, p = 0.001), Th2/Th17/Treg-subset counts (Figure 2C–E, p = 0.001/0.001/0.030, respectively), CD4+-cell percentage in lymphocytes (Figure 2F, p = 0.001), and CD4/CD8 ratio (Figure 2J, p = 0.001) gradually, but significantly, increased during ART, whereas the Th1 counts had increased to the range of those of the HIV(–) children by around 1 year of ART initiation (Figure 2B). The CD8+-cell counts were not significantly correlated with ART duration. None of these markers were significantly correlated with age (data not shown).
Linear regression analyses were conducted to estimate the ART duration required for the immunological markers to reach the median values of the HIV(−) children. The estimations were as follows: 4.8 years (95% confidence interval (CI): 2.7–7.0 years) of ART for the total CD4+-cell counts, 3.6 years (95% CI: 1.4–5.7) for the CD4+-cell percentage in lymphocytes, 5.6 years (95% CI: 3.4–7.9) for the Th2 counts, 8.2 years (95% CI: 6.2–10.2) for the Th17 counts, 8.3 years (95% CI: 3.4–12.2) for the Treg counts, and 6.6 years (95% CI: 4.5–8.7) for the CD4/CD8 ratio.
During the first year of ART initiation, the percentage of activated CD8+ cells rapidly decreased to the upper range of that among the HIV(–) children and then continued to slowly, but significantly, decrease (Figure 2I, p = 0.010). In contrast, the percentage of activated CD4+ cells marginally decreased with ART duration (Figure 2H, p = 0.053), but the percentage of activated CD4+ cells even among the HIV(+) children did not significantly differ from that among the HIV(−) children (5.6% vs. 6.4%, p = 0.969, Table 1). The plasma sCD14 levels showed no significant changes during almost 6 years of ART (Figure 2K).
The CD4/CD8 ratio was inversely correlated with the percentage of activated CD8+ cells (p = 0.003, Figure 2L).

2.5. Physiological Change in Immunological Markers with Age among the HIV(−) Children

Among the HIV(−) children, only the percentage of activated CD8+ cells significantly decreased with age (p = 0.023, Figure 3I), and plasma sCD14 levels marginally decreased with time by 4 years of age and remained stable thereafter (p = 0.066, Figure 3K). The other immunological markers did not show significant relationships with age (Figure 3A–H,J,L).

2.6. Microbial Translocation Status

The impact of HIV infection and ART on the microbial translocation status of the children was determined by the detection of 12 bacterial 16S/23S ribosomal RNA genes (rDNA) in plasma and 16S/23S ribosomal RNA molecules (rRNA) in whole blood using quantitative PCR (qPCR) and reverse transcription (RT)-qPCR, respectively. Bacterial 16S/23S rDNA from Staphylococcus, Streptococcus, and/or Pseudomonas species was detected in 25.8% (8/31) of the HIV(+) children compared with 15.0% (3/20) of the HIV(−) children (Table 2). The detection frequency of Staphylococcus rDNA was significantly higher among the HIV(+) children than among the ART(+) children (22.6% vs. 0%, respectively, p = 0.011), whereas there was no significant difference between the HIV(+) and HIV(−) children in terms of the frequency of each bacterial rDNA (p = 0.169) and the number of bacterial DNA copies (data not shown).
No target bacterial 16S/23S rRNA was detected in children, regardless of the HIV status (data not shown).

3. Discussion

In the current study, we investigated the impact of HIV infection and ART on the immune profile of children aged over 2 years. Among the HIV(+) children, the total counts of CD4+ cells and their subsets (Th1/Th2/Th17) were inversely correlated with age, whereas Treg counts, CD38+HLA (human leukocyte antigen)-DR+CD8+- (activated CD8+) cell percentage, and plasma sCD14 levels (activated monocyte level) were not. By the age of 2 years, Treg counts had become lower and the activated CD8+ cell and monocyte levels had become higher than those of the HIV(−) children. In addition, activated CD8+ cell and monocyte levels were positively correlated with VL, and the CD8+-cell activation status was inversely, although marginally, correlated with Treg counts, (p = 0.0575). These findings indicate that vertical HIV infection induces a rapid decrease in the Treg subset and the early activation of CD8+ cells and monocytes. These data suggest that the early decline in Tregs contributes to CD8+-cell activation in the early phase of HIV infection in children, as previously reported in adults [21]. The early activation of CD8+ cells after HIV infection [15,16] and a correlation between CD8+ cell activation and VL [22] have been previously reported, whereas a rapid decrease in Treg subset has not been previously reported in children, although it has been reported in adults with primary HIV infection [23].
Among the ART(+) children, Th2/Th17/Treg-subset counts were positively correlated with ART duration and were estimated to reach the levels of HIV(−) children after 5.6, 8.2, and 8.3 years of ART initiation, respectively, compared with an estimate of only 1–2 years for Th1 counts. The activated CD8+ cell percentage decreased to the level of the HIV(−) children during the first year of ART initiation, whereas the plasma sCD14 level (activated monocytes level) was maintained at a level higher than that of the HIV(−) children even after VL was controlled and CD8+-cell activation was normalized. These findings indicate that in children, ART induced a more rapid Th1 recovery than the Th2/Th17/Treg subsets and a rapid normalization of CD8+-cell activation but had little effect on monocyte activation. Although previous studies have reported the early normalization of CD8+-cell activation after ART initiation [16,24], the recovery profile of CD4+-cell subsets after ART initiation, particularly the rapid recovery of Th1 and the slower recovery of Th2/Th17/Treg, has not been reported before in children. Incidentally, the rapid recovery of Th1 may play a role in the control of HIV infection together with ART. Similar to our findings, a previous study has reported that the plasma sCD14 level is not normalized even after 2 years of ART initiation in children, although it is significantly reduced after ART initiation [25]. Because the sCD14 level has been proposed as an independent predictor of non-AIDS-defining morbidity events even during suppressive ART in HIV-infected adults [26,27], further longitudinal studies may be needed to elucidate the implication of the high monocyte activation status in HIV-infected children under ART.
Among the HIV(+) children, the CD4/CD8 ratio was inversely correlated with the plasma VL and the activated CD8+-cell percentage. Among the ART(+) children, the CD4/CD8 ratio was positively correlated with ART duration, as were the total CD4+-cell and Th2/Th17/Treg-subset counts, and inversely correlated with the percentage of activated CD8+ cells. Recent studies have suggested that the CD4/CD8 ratio is a marker of T-cell activation, senescence, and activation/exhaustion in treated HIV-infected children and young adults, and that it could be independently associated with the risk of non-AIDS-related morbidity and mortality [28,29]. Given the cost effectiveness of measuring CD4+ and CD8+ cell levels in clinical laboratories, the CD4/CD8 ratio could be used as an additional immunological marker to monitor ART outcomes, particularly in resource-limited settings.
Intestinal microbial translocation plays a role in the pathogenesis of HIV infection [6]. Furthermore, immune activation is driven by intestinal microbial translocation in HIV-infected children both before and after ART [22,30]. However, in this study, none of the targeted bacterial 16S/23S rRNA fragments were detected in whole blood samples of the 80 study participants, indicating that live bacteria (bacteremia) are rarely detected in HIV-infected and -uninfected children. In contrast, bacterial rDNA was detected in 8 HIV(+) children and 3 HIV(−) children, and the detection frequency of staphylococcal rDNA was significantly higher among the HIV(+) children than among the ART(+) children (22.6% vs. 0%, respectively). However, there was no significant association of the copy number of bacterial 16S/23S rDNA with CD8+-cell or monocyte activation and no significant difference in the frequency of bacterial rDNA between the HIV(+) and HIV(−) children. These data suggest that ART decreases the occurrence of intestinal microbial translocation in HIV-infected children, although the possibility of contamination with Staphylococcus epidermidis cannot be excluded. Thus, further studies are needed to elucidate the reasons for the discrepancies between our data and previous findings [22,30] in relation to microbial translocation in HIV-infected children.
In this study, we recruited children who were over 2 years of age because their immune systems are considered to be relatively mature and stable [9,19,20]. We showed that all targeted immunological markers, except CD8+-cell activation, did not significantly change with age among the HIV(–) children (Figure 3). These data demonstrate that the results presented here on the impact of HIV infection and ART on the immune profile of children were not influenced by growth-related physiological changes.
There are some limitations to this study. First, this was a cross-sectional study rather than a longitudinal study; Second, the number of study subjects was relatively small, which limited our findings in relation to microbial translocation; Third, the HIV(+) and ART(+) children were 2 years older than the HIV(−) children; Finally, only cell-surface markers were used to identify the CD4+-subsets, particularly Treg. Therefore, these findings should be confirmed by longitudinal studies following HIV-infected children before and during ART with age-matched controls.
To our knowledge, this is the first study to investigate the impact of HIV infection and ART on the immune profile of CD4+-cell subsets (Th1/Th2/Th17/Tregs) concurrently with the activation of T cells and monocytes in children. We found that HIV infection induced a more rapid decline in the Treg subset than in the Th1/Th2/Th17 subsets and the early activation of CD8+ cells and monocytes. We also found that ART induced a more rapid recovery of the Th1 subset than the Th2/Th17/Treg subsets and the early normalization of CD8+-cell activation but that it had little effect on the activation of monocytes in children. Finally, we have provided evidence that the CD4/CD8 ratio is an additional marker for ART outcomes.

4. Materials and Methods

4.1. Subjects and Study Design

Sixty Vietnamese children vertically infected with HIV were recruited in May 2012. These children were assigned to one of two groups: the HIV(+) group and the ART(+) group. The HIV(+) group comprised 31 children with HIV who did not receive ART; the female/male ratio was 14/17, and the median age was 6.2 years (2.0–11.0 years). The ART(+) group comprised 29 children with HIV who had been treated with ART; the female/male ratio was 12/17, and the median age was 6.1 years (3.6–8.6 years). The study inclusion criteria for these HIV-1-infected groups were as follows: the children (1) had been followed at the National Hospital of Pediatrics (NHP) in Hanoi, Vietnam; and (2) were more than 2 years of age because at this point the immune system is relatively mature and stable [9,19,20]. The exclusion criteria were as follows: children who (1) had progressed to AIDS; (2) had received any treatment within the prior 8 weeks that might influence the immune system; and (3) had symptoms of gastrointestinal infections at the time of recruitment. The children in the ART(+) group resided at an orphanage center near Hanoi; the children in the HIV(+) group were followed at the outpatient department of NHP. A third control group, the HIV(−) group, was also included and comprised 20 healthy Vietnamese children without HIV infections. The female/male ratio was 8/12, and the median age was 4.1 years (2.0–8.3 years). The children in this group resided at an orphanage center near Hanoi. The characteristics of the study subjects are shown in Table 1.
At recruitment, the median ART duration in the ART(+) group was 3.5 years (0.8–5.8) years. Of the 29 children, 8 received zidovudine (AZT)/lamivudine (3TC)/nevirapine (NVP), 7 received stavudine (d4T)/3TC/NVP, 6 received AZT/3TC/efavirenz (EFV), 4 received d4T/3TC/EFV, 2 received AZT/3TC/lopinavir boosted with ritonavir (LPV/r), 1 received abacavir (ABC)/3TC/LPV/r, and 1 received ABC/didanosine/LPV/r.
The protocol of this cross-sectional study was approved by the Ethics Committee of Kanazawa University in Japan and the Ethics Committee of NHP in Vietnam. All of the HIV-infected children who met the inclusion criteria and did not correspond to the exclusion criteria, and all of the HIV-uninfected children who resided in the orphanage and were eligible for the criteria, were invited to join this study. The family or guardian of each subject was informed, and only those who voluntarily consented to participate were recruited. Written consent was obtained from all participants.

4.2. Plasma HIV VL and sCD14 Concentration

Plasma HIV VL was measured using a Cobas Taqman HIV-1 Test Kit version 1.0 (Roche Molecular Systems, Inc., Branchburg, NJ, USA), following the manufacturer’s instructions (detection limit: 40 copies/mL). Plasma samples were diluted to 1:5.5 for measurements, which resulted in a final detection limit of 220 copies/mL.
Plasma sCD14 concentration was measured with a Human sCD14 Immunoassay Kit (R&D Systems, Minneapolis, MN, USA), following the manufacturer’s instructions.

4.3. Immunological Analysis

Immune activation was evaluated based on the percentage of CD4+ and CD8+ lymphocytes expressing CD38 and major histocompatibility complex class II (HLA-DR) molecules [31]. The different CD4+-cell subsets were identified using cell-surface markers: CXCR3+CCR6CD4+ (“Th1”), CXCR3CCR6CD4+ (“Th2”), CXCR3CCR6+CD4+ (“Th17”) [32], and CD25highCD4+ (“Treg”) [33].
Blood samples were processed for cell staining within 6 h after collection. Whole blood samples (50 µL) were stained for 15 min at 4 °C with a combination of four monoclonal antibodies: anti-CD4 PerCP and CD8 PE (BD Biosciences, San Jose, CA, USA), CD38 FITC (Miltenyi Biotec, Auburn, AL, USA), and HLA-DR PE-Cy7 (Biolegend, San Diego, CA, USA). Alternatively, samples were stained with a combination of four anti-CD4 monoclonal antibodies: anti-CD4 PerCP and CD25 PE (BD Biosciences), and CXCR3 FITC and CCR6 PE-Cy7 (Biolegend). After red blood cells were lysed with a lysing buffer (BD Biosciences), the remaining cells were washed once with 1 mL of phosphate buffered saline (PBS), kept at 4 °C in PBS with 1% paraformaldehyde and 0.5% bovine serum albumin, and analyzed within 48 h after staining at Kanazawa University with a JSAN flow cytometer (Bay Bioscience, Kobe, Japan). The data were analyzed with Flowjo V.7.5.5 (FLOWJO, OR, USA).

4.4. Detection of Bacterial Ribosomal RNA Genes (rDNA) in Plasma

Isolated plasma (100 µL) was used for DNA extraction with the SMI TEST EX R&D (Medical and Biological Laboratories Co., Ltd., Aichi, Japan), following the manufacturer’s instructions. qPCR was then performed with a TaKaRa Taq kit (TaKaRa Bio Inc., Shiga, Japan). The reaction mixture (10 μL) contained 5 μL of template DNA and 0.2 μM of each specific primer set, except for the primer targeting g-Bfra-F2/g-Bfra-R (0.4 μM). The primers were designed to target bacterial 16S or 23S rRNA genes (Table S1). The amplification program comprised one cycle at 94 °C for 5 min and 45 cycles at 94 °C for 20 s, 55 °C or 60 °C for 20 s, and 72 °C for 50 s. A standard curve was generated with qPCR data and cycle threshold (Ct) values. The target gene copy number in the plasma samples was determined in a sample of the extracted DNA (1/20 of the DNA extracted from 100 µL of plasma). This DNA was subjected to qPCR, and the Ct value was applied to the standard curve to obtain the corresponding bacterial rDNA copy number/µL of plasma. With this procedure, the lower detection limit for the targeted gene was 2 copies/µL of plasma.

4.5. Detection of Bacterial rRNA in Blood

Peripheral blood (1 mL) was added to two volumes of the RNA Protect bacterial reagent (QIAGEN GmbH, Hilden, Germany). After centrifugation of the mixture at 14,000× g for 10 min, the supernatant was discarded and the pellet was stored at −80 °C until further use. Total RNA extraction was followed by RT-qPCR as previously described [34]. Each RNA sample was diluted; then, diluted samples (corresponding to amounts of 1/200 and 1/20 of the extracted RNA from 1 mL of blood) were subjected to RT-qPCR with specific primer sets that targeted bacterial 16S/23S rRNA [34,35] to investigate if live bacteria existed in the blood stream of the study subjects (Table S1). With this procedure, the lower detection limit for the targeted bacteria was 4 cells/mL of blood.

4.6. Statistical Analysis

Statistical analyses were performed with the SPSS programs (IBMSPSS statistics 19, IBM Corporation, NY, USA). The Mann–Whitney U test was used to compare the markers among the three groups (HIV(−), HIV(+),and ART(+)). Fisher’s exact test or the chi square test was used to compare the detection frequency of bacterial rDNA in plasma, sex distribution, and the WHO clinical stage across the groups. Spearman’s rank correlation was used to analyze the correlation among the biological markers in each group. p-values < 0.05 were considered statistically significant.

5. Conclusions

In children, HIV infection caused a rapid decrease in Treg counts and the early activation of CD8+ cells and monocytes, and ART induced rapid Th1 recovery and early CD8+-cell activation normalization but had little effect on monocyte activation. The CD4/CD8 ratio could be an additional marker for ART monitoring.

Supplementary Materials

Supplementary materials can be found at https://www.mdpi.com/1422-0067/17/8/1245/s1.

Acknowledgments

This study was supported in part by the Ministry of Education, Culture, Sports, Science and Technology (MEXT) in Japan (the Program of Founding Research Centers for Emerging and Reemerging Infectious Disease) and the 2012 Kanazawa University President Strategic Research fund. No potential conflict of interests relevant to this article exists. The authors are grateful to the children who participated in this study for their invaluable support throughout the use of their samples, and to the staff of the Departments of Infectious Diseases and Molecular Laboratory of NHP who made enormous contributions to this work.

Author Contributions

Azumi Ishizaki, Xiuqiong Bi, and Hiroshi Ichimura conceived and designed the study; Lam Van Nguyen, Hung Viet Pham, Chung Thi Thu Phan, Azumi Ishizaki, Thuy Thi Thanh Giang, Thuy Thi Bich Phung, and Tuyen Thi Nguyen collected the blood samples and clinical data; An Nhat Pham and Dung Thi Khanh Khu coordinated the groups of patients; Azumi Ishizaki, Kazunori Matsuda, Hung Viet Pham, and Chung Thi Thu Phan processed the samples; Xiuqiong Bi, Hung Viet Pham, Kazunori Matsuda, and Kiyohito Ogata performed the experiments; Azumi Ishizaki, Xiuqiong Bi, Masaharu Tokoro, and Hiroshi Ichimura analyzed the data; An Nhat Pham, Dung Thi Khanh Khu, and Hiroshi Ichimura supplied the laboratory materials; Xiuqiong Bi, Azumi Ishizaki, and Hiroshi Ichimura wrote the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Mehandru, S.; Poles, M.A.; Tenner-Racz, K.; Horowitz, A.; Hurley, A.; Hogan, C.; Boden, D.; Racz, P.; Markowitz, M. Primary HIV-1 infection is associated with preferential depletion of CD4+ T lymphocytes from effector sites in the gastrointestinal tract. J. Exp. Med. 2004, 200, 761–770. [Google Scholar] [CrossRef] [PubMed]
  2. Brenchley, J.M.; Paiardini, M.; Knox, K.S.; Asher, A.I.; Cervasi, B.; Asher, T.E.; Scheinberg, P.; Price, D.A.; Hage, C.A.; Kholi, L.M.; et al. Differential Th17 CD4 T-cell depletion in pathogenic and nonpathogenic lentiviral infections. Blood 2008, 112, 2826–2835. [Google Scholar] [CrossRef] [PubMed]
  3. Hunt, P.W. Th17, gut, and HIV: Therapeutic implications. Curr. Opin. HIV AIDS 2010, 5, 189–193. [Google Scholar] [CrossRef] [PubMed]
  4. Gordon, S.N.; Cervasi, B.; Odorizzi, P.; Silverman, R.; Aberra, F.; Ginsberg, G.; Estes, J.D.; Paiardini, M.; Frank, I.; Silvestri, G. Disruption of intestinal CD4+ T cell homeostasis is a key marker of systemic CD4+ T cell activation in HIV-infected individuals. J. Immunol. 2010, 185, 5169–5179. [Google Scholar] [CrossRef] [PubMed]
  5. Estes, J.D.; Harris, L.D.; Klatt, N.R.; Tabb, B.; Pittaluga, S.; Paiardini, M.; Barclay, G.R.; Smedley, J.; Pung, R.; Oliveira, K.M.; et al. Damaged intestinal epithelial integrity linked to microbial translocation in pathogenic simian immunodeficiency virus infections. PLoS Pathog. 2010, 6, e1001052. [Google Scholar] [CrossRef] [PubMed]
  6. Brenchley, J.M.; Price, D.A.; Schacker, T.W.; Asher, T.E.; Silvestri, G.; Rao, S.; Kazzaz, Z.; Bornstein, E.; Lambotte, O.; Altmann, D.; et al. Microbial translocation is a cause of systemic immune activation in chronic HIV infection. Nat. Med. 2006, 12, 1365–1371. [Google Scholar] [CrossRef] [PubMed]
  7. Mehandru, S.; Poles, M.A.; Tenner-Racz, K.; Jean-Pierre, P.; Manuelli, V.; Lopez, P.; Shet, A.; Low, A.; Mohri, H.; Boden, D.; et al. Lack of mucosal immune reconstitution during prolonged treatment of acute and early HIV-1 infection. PLoS Med. 2006, 3, e484. [Google Scholar]
  8. Bi, X.; Suzuki, Y.; Gatanaga, H.; Oka, S. High frequency and proliferation of CD4+ FOXP3+ Treg in HIV-1-infected patients with low CD4 counts. Eur. J. Immunol. 2009, 39, 301–309. [Google Scholar] [CrossRef] [PubMed]
  9. Tobin, N.H.; Aldrovandi, G.M. Immunology of pediatric HIV infection. Immunol. Rev. 2013, 254, 143–169. [Google Scholar] [CrossRef] [PubMed]
  10. Prendergast, A.J.; Klenerman, P.; Goulder, P.J. The impact of differential antiviral immunity in children and adults. Nat. Rev. Immunol. 2012, 12, 636–648. [Google Scholar] [CrossRef] [PubMed]
  11. Bunders, M.J.; van der Loos, C.M.; Klarenbeek, P.L.; van Hamme, J.L.; Boer, K.; Wilde, J.C.; de Vries, N.; van Lier, R.A.; Kootstra, N.; Pals, S.T.; et al. Memory CD4+CCR5+ T cells are abundantly present in the gut of newborn infants to facilitate mother-to-child transmission of HIV-1. Blood 2012, 120, 4383–4390. [Google Scholar] [CrossRef] [PubMed]
  12. Muenchhoff, M.; Prendergast, A.J.; Goulder, P.J. Immunity to HIV in early life. Front. Immunol. 2014, 5, 391. [Google Scholar] [CrossRef] [PubMed]
  13. Freguja, R.; Gianesin, K.; Mosconi, I.; Zanchetta, M.; Carmona, F.; Rampon, O.; Giaquinto, C.; de Rossi, A. Regulatory T cells and chronic immune activation in human immunodeficiency virus 1 (HIV-1)-infected children. Clin. Exp. Immunol. 2011, 164, 373–380. [Google Scholar] [CrossRef] [PubMed]
  14. Patel, K.; Hernan, M.A.; Williams, P.L.; Seeger, J.D.; McIntosh, K.; Dyke, R.B.; Seage, G.R., 3rd. Long-term effects of highly active antiretroviral therapy on CD4+ cell evolution among children and adolescents infected with HIV: 5 years and counting. Clin. Infect. Dis. 2008, 46, 1751–1760. [Google Scholar] [CrossRef] [PubMed]
  15. Ananworanich, J.; Apornpong, T.; Kosalaraksa, P.; Jaimulwong, T.; Hansudewechakul, R.; Pancharoen, C.; Bunupuradah, T.; Chandara, M.; Puthanakit, T.; Ngampiyasakul, C.; et al. Characteristics of lymphocyte subsets in HIV-infected, long-term nonprogressor, and healthy Asian children through 12 years of age. J. Allergy Clin. Immunol. 2010, 126, 1294–1301. [Google Scholar] [CrossRef] [PubMed]
  16. Rosenblatt, H.M.; Stanley, K.E.; Song, L.Y.; Johnson, G.M.; Wiznia, A.A.; Nachman, S.A.; Krogstad, P.A. Immunological response to highly active antiretroviral therapy in children with clinically stable HIV-1 infection. J. Infect. Dis. 2005, 192, 445–455. [Google Scholar] [CrossRef] [PubMed]
  17. Lederman, M.M.; Calabrese, L.; Funderburg, N.T.; Clagett, B.; Medvik, K.; Bonilla, H.; Gripshover, B.; Salata, R.A.; Taege, A.; Lisgaris, M.; et al. Immunologic failure despite suppressive antiretroviral therapy is related to activation and turnover of memory CD4 cells. J. Infect. Dis. 2011, 204, 1217–1226. [Google Scholar] [CrossRef] [PubMed]
  18. Pham, H.V.; Ishizaki, A.; Nguyen, L.V.; Phan, C.T.; Phung, T.T.; Takemoto, K.; Pham, A.N.; Bi, X.; Khu, D.T.; Ichimura, H. Two-year outcome of first-line antiretroviral therapy among HIV-1 vertically-infected children in Hanoi, Vietnam. Int. J. STD AIDS 2015, 26, 821–830. [Google Scholar] [CrossRef] [PubMed]
  19. Shearer, W.T.; Rosenblatt, H.M.; Gelman, R.S.; Oyomopito, R.; Plaeger, S.; Stiehm, E.R.; Wara, D.W.; Douglas, S.D.; Luzuriaga, K.; McFarland, E.J.; et al. Lymphocyte subsets in healthy children from birth through 18 years of age: The pediatric AIDS clinical trials group p1009 study. J. Allergy Clin. Immunol. 2003, 112, 973–980. [Google Scholar] [CrossRef] [PubMed]
  20. Schatorje, E.J.; Gemen, E.F.; Driessen, G.J.; Leuvenink, J.; van Hout, R.W.; de Vries, E. Paediatric reference values for the peripheral T cell compartment. Scand. J. Immunol. 2012, 75, 436–444. [Google Scholar] [CrossRef] [PubMed]
  21. Tenorio, A.R.; Martinson, J.; Pollard, D.; Baum, L.; Landay, A. The relationship of T-regulatory cell subsets to disease stage, immune activation, and pathogen-specific immunity in HIV infection. J. Acquir. Immune Defic. Syndr. 2008, 48, 577–580. [Google Scholar] [CrossRef] [PubMed]
  22. Pilakka-Kanthikeel, S.; Huang, S.; Fenton, T.; Borkowsky, W.; Cunningham, C.K.; Pahwa, S. Increased gut microbial translocation in HIV-infected children persists in virologic responders and virologic failures after antiretroviral therapy. Pediatr. Infect. Dis. J. 2012, 31, 583–591. [Google Scholar] [CrossRef] [PubMed]
  23. Simonetta, F.; Lecuroux, C.; Girault, I.; Goujard, C.; Sinet, M.; Lambotte, O.; Venet, A.; Bourgeois, C. Early and long-lasting alteration of effector CD45RAFoxp3high regulatory T-cell homeostasis during HIV infection. J. Infect. Dis. 2012, 205, 1510–1519. [Google Scholar] [CrossRef] [PubMed]
  24. Rainwater-Lovett, K.; Nkamba, H.; Mubiana-Mbewe, M.; Moore, C.B.; Margolick, J.; Moss, W.J. Changes in cellular immune activation and memory T-cell subsets in HIV-infected Zambian children receiving HAART. J. Acquir. Immune Defic. Syndr. 2014, 67, 455–462. [Google Scholar] [CrossRef] [PubMed]
  25. Kourtis, A.P.; Ibegbu, C.C.; Wiener, J.; King, C.C.; Tegha, G.; Kamwendo, D.; Kumwenda, J.; Kaur, S.P.; Flax, V.; Ellington, S.; et al. Role of intestinal mucosal integrity in HIV transmission to infants through breast-feeding: The BAN Study. J. Infect. Dis. 2013, 208, 653–661. [Google Scholar] [CrossRef] [PubMed]
  26. Sandler, N.G.; Wand, H.; Roque, A.; Law, M.; Nason, M.C.; Nixon, D.E.; Pedersen, C.; Ruxrungtham, K.; Lewin, S.R.; Emery, S.; et al. Plasma levels of soluble CD14 independently predict mortality in HIV infection. J. Infect. Dis. 2011, 203, 780–790. [Google Scholar] [CrossRef] [PubMed]
  27. Tenorio, A.R.; Zheng, Y.; Bosch, R.J.; Krishnan, S.; Rodriguez, B.; Hunt, P.W.; Plants, J.; Seth, A.; Wilson, C.C.; Deeks, S.G.; et al. Soluble markers of inflammation and coagulation but not T-cell activation predict non-AIDS-defining morbid events during suppressive antiretroviral treatment. J. Infect. Dis. 2014, 210, 1248–1259. [Google Scholar] [CrossRef] [PubMed]
  28. Sainz, T.; Serrano-Villar, S.; Diaz, L.; Gonzalez Tome, M.I.; Gurbindo, M.D.; de Jose, M.I.; Mellado, M.J.; Ramos, J.T.; Zamora, J.; Moreno, S.; et al. The CD4/CD8 ratio as a marker T-cell activation, senescence and activation/exhaustion in treated HIV-infected children and young adults. AIDS 2013, 27, 1513–1516. [Google Scholar] [CrossRef] [PubMed]
  29. Serrano-Villar, S.; Perez-Elias, M.J.; Dronda, F.; Casado, J.L.; Moreno, A.; Royuela, A.; Perez-Molina, J.A.; Sainz, T.; Navas, E.; Hermida, J.M.; et al. Increased risk of serious non-AIDS-related events in HIV-infected subjects on antiretroviral therapy associated with a low CD4/CD8 ratio. PLoS ONE 2014, 9, e85798. [Google Scholar] [CrossRef] [PubMed]
  30. Pilakka-Kanthikeel, S.; Kris, A.; Selvaraj, A.; Swaminathan, S.; Pahwa, S. Immune activation is associated with increased gut microbial translocation in treatment-naive, HIV-infected children in a resource-limited setting. J. Acquir. Immune Defic. Syndr. 2014, 66, 16–24. [Google Scholar] [CrossRef] [PubMed]
  31. Hunt, P.W.; Martin, J.N.; Sinclair, E.; Bredt, B.; Hagos, E.; Lampiris, H.; Deeks, S.G. T cell activation is associated with lower CD4+ T cell gains in human immunodeficiency virus-infected patients with sustained viral suppression during antiretroviral therapy. J. Infect. Dis. 2003, 187, 1534–1543. [Google Scholar] [CrossRef] [PubMed]
  32. Maecker, H.T.; McCoy, J.P.; Nussenblatt, R. Standardizing immunophenotyping for the human immunology project. Nat. Rev. Immunol. 2012, 12, 191–200. [Google Scholar] [CrossRef] [PubMed]
  33. Kinter, A.; McNally, J.; Riggin, L.; Jackson, R.; Roby, G.; Fauci, A.S. Suppression of HIV-specific T cell activity by lymph node CD25+ regulatory T cells from HIV-infected individuals. Proc. Natl. Acad. Sci. USA 2007, 104, 3390–3395. [Google Scholar] [CrossRef] [PubMed]
  34. Sakaguchi, S.; Saito, M.; Tsuji, H.; Asahara, T.; Takata, O.; Fujimura, J.; Nagata, S.; Nomoto, K.; Shimizu, T. Bacterial rRNA-targeted reverse transcription-PCR used to identify pathogens responsible for fever with neutropenia. J. Clin. Microbiol. 2010, 48, 1624–1628. [Google Scholar] [CrossRef] [PubMed]
  35. Matsuda, K.; Tsuji, H.; Asahara, T.; Matsumoto, K.; Takada, T.; Nomoto, K. Establishment of an analytical system for the human fecal microbiota, based on reverse transcription-quantitative PCR targeting of multicopy rRNA molecules. Appl. Environ. Microbiol. 2009, 75, 1961–1969. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Correlation between immunological markers, age, and/or plasma viral load among HIV(+) children (AO). This bivariate correlation was estimated on the basis of Spearman’s rank correlation analysis. Regression lines are shown only for significantly correlated bivariates. Th: helper T-cell, Treg: regulatory T-cell, VL: viral load. * Median and interquartile range of the HIV(−) group.
Figure 1. Correlation between immunological markers, age, and/or plasma viral load among HIV(+) children (AO). This bivariate correlation was estimated on the basis of Spearman’s rank correlation analysis. Regression lines are shown only for significantly correlated bivariates. Th: helper T-cell, Treg: regulatory T-cell, VL: viral load. * Median and interquartile range of the HIV(−) group.
Ijms 17 01245 g001
Figure 2. Correlations between ART duration and immunological markers among ART(+) children (AL). * Median and interquartile range (IQR) of the HIV(−) group; # Median and IQR of the HIV(+) group.
Figure 2. Correlations between ART duration and immunological markers among ART(+) children (AL). * Median and interquartile range (IQR) of the HIV(−) group; # Median and IQR of the HIV(+) group.
Ijms 17 01245 g002
Figure 3. Correlation between immunological markers and age among HIV(−) children (AL). This bivariate correlation was estimated on the basis of Spearman’s rank correlation analysis. Regression lines are shown only for significantly correlated bivariates.
Figure 3. Correlation between immunological markers and age among HIV(−) children (AL). This bivariate correlation was estimated on the basis of Spearman’s rank correlation analysis. Regression lines are shown only for significantly correlated bivariates.
Ijms 17 01245 g003aIjms 17 01245 g003b
Table 1. Characteristics and immune status of each group.
Table 1. Characteristics and immune status of each group.
ItemsHIV(+) (n = 31)HIV(−) (n = 20)ART(+) (n = 29)p Values
HIV(+) vs. HIV(−)ART(+) vs. HIV(−)HIV(+) vs. ART(+)
Age (years) *6.2 (2.0–11.0)4.1 (2.0–8.3)6.1 (3.6–8.6)0.0340.0090.584
Gender, female (n)/male (n) *14/178/1212/170.7180.9200.764
Height (cm) *109.0 (77.0–129.5)110.0 (80.0–130.0)110.0 (90.0–130.0)0.7710.4180.539
Body weight (kg) *17.5 (10.0–27.0)16.0 (9.0–35.0)19.8 (12.0–32.8)0.3950.1400.192
WHO clinical stage:
2 (n)/1 (n)
1/30 5/24--0.098
ART duration (years) * 3.5 (0.8–5.8)---
Viral load (log10 copies/mL) *5.0 (3.2–6.5) **--< 0.001
% CD4+ *22.1 (3.6–44.5)29.5 (19.6–54.9)28.8 (12.4–47.1)0.0010.4040.010
CD4+-cell counts (cells/μL) *698 (97–1784)1050 (693–2688)894 (244–1711)0.0030.0180.429
Th1 counts (cells/μL) *80 (25–227)136 (74–220)147 (49–211)0.0030.6110.002
Th2 counts (cells/μL) *537 (63–1375)822 (413–2196)553 (119–1369)0.0160.0090.970
Th17 counts (cells/μL) *45 (6–116)109 (51–192)58 (23–144)<0.001<0.0010.016
Treg counts (cells/μL) *14 (0–133)48 (16–94)30 (9–71)<0.0010.004<0.001
%CD38+HLA-DR+/CD4 *5.6 (2.2–17.3)6.4 (2.7–27.6)4.3 (2.0–15.6)0.9690.0360.003
% CD8+ *43.4 (29.5–61.4)31.4 (23.5–43.3)44.7 (31.1–61.4)<0.001<0.0010.464
CD8+-cell counts (cells/μL) *1368 (470–3127)1101 (634–2874)1212 (769–2064)0.2390.2900.631
%CD38+HLA-DR+/CD8 *27.5 (12.2–53.3)12.9 (5.8–38.6)10.2 (5.0–27.7)<0.0010.329<0.001
CD4/CD8 *0.50 (0.06–1.19)1.03 (0.45–2.34)0.66 (0.20–1.42)<0.0010.0010.181
sCD14 (ng/mL) *1637 (1049–3003)1413 (944–2580)1964 (1281–3169)0.009<0.001<0.001
HIV(+): Children infected with HIV and without ART; ART(+): Children infected with HIV and on ART; HIV(−): Children not infected with HIV. P values are from the Man-Whitney U test, except the p values for sex and WHO clinical stage comparison, which are from the chi square test or Fisher’s exact test. * Median (range); ** 22 ART(+) children with undetectable viral load.
Table 2. Bacterial 16S/23S ribosomal RNA gene (rDNA) detection in plasma.
Table 2. Bacterial 16S/23S ribosomal RNA gene (rDNA) detection in plasma.
Target BacteriaHIV(+) (n = 31)HIV(−) (n = 20)ART(+) (n = 29)p Values
HIV(+) vs. HIV(−)ART(+) vs. HIV(−)HIV(+) vs. ART(+)
C. coccoides group000---
C. leptum subgroup000---
B. fragilis group000---
Bifidobacterium000---
Atopobium cluster000---
Prevotella000---
Enterobacteriaceae000---
Streptococcus1 (3.2%)001-1
Enterococcus000---
Staphylococcus7 (22.6%)1 (5.0%)00.130.410.011
Pseudomonas1 (3.2%)2 (10.0%)00.550.161
L. casei subgroup000---
HIV(+): Children infected with HIV and without ART; ART(+): Children infected with HIV and on ART; HIV(−): Children not infected with HIV; C. coccoide: Clostridium coccoide; C. leptum; Clostridium leptum; B. fragilis: Bacteroides fragilis. p values: Fisher’s exact probability test.

Share and Cite

MDPI and ACS Style

Bi, X.; Ishizaki, A.; Nguyen, L.V.; Matsuda, K.; Pham, H.V.; Phan, C.T.T.; Ogata, K.; Giang, T.T.T.; Phung, T.T.B.; Nguyen, T.T.; et al. Impact of HIV Infection and Anti-Retroviral Therapy on the Immune Profile of and Microbial Translocation in HIV-Infected Children in Vietnam. Int. J. Mol. Sci. 2016, 17, 1245. https://doi.org/10.3390/ijms17081245

AMA Style

Bi X, Ishizaki A, Nguyen LV, Matsuda K, Pham HV, Phan CTT, Ogata K, Giang TTT, Phung TTB, Nguyen TT, et al. Impact of HIV Infection and Anti-Retroviral Therapy on the Immune Profile of and Microbial Translocation in HIV-Infected Children in Vietnam. International Journal of Molecular Sciences. 2016; 17(8):1245. https://doi.org/10.3390/ijms17081245

Chicago/Turabian Style

Bi, Xiuqiong, Azumi Ishizaki, Lam Van Nguyen, Kazunori Matsuda, Hung Viet Pham, Chung Thi Thu Phan, Kiyohito Ogata, Thuy Thi Thanh Giang, Thuy Thi Bich Phung, Tuyen Thi Nguyen, and et al. 2016. "Impact of HIV Infection and Anti-Retroviral Therapy on the Immune Profile of and Microbial Translocation in HIV-Infected Children in Vietnam" International Journal of Molecular Sciences 17, no. 8: 1245. https://doi.org/10.3390/ijms17081245

APA Style

Bi, X., Ishizaki, A., Nguyen, L. V., Matsuda, K., Pham, H. V., Phan, C. T. T., Ogata, K., Giang, T. T. T., Phung, T. T. B., Nguyen, T. T., Tokoro, M., Pham, A. N., Khu, D. T. K., & Ichimura, H. (2016). Impact of HIV Infection and Anti-Retroviral Therapy on the Immune Profile of and Microbial Translocation in HIV-Infected Children in Vietnam. International Journal of Molecular Sciences, 17(8), 1245. https://doi.org/10.3390/ijms17081245

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop