Next Article in Journal
Aging of the Immune System: Focus on Natural Killer Cells Phenotype and Functions
Previous Article in Journal
Platelets, a Key Cell in Inflammation and Atherosclerosis Progression
Previous Article in Special Issue
Immunotherapy with Cell-Based Biological Drugs to Cure HIV-1 Infection
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Cells
Volume 11
Issue 6
10.3390/cells11061015
cells-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Impact of Anti PD-1 Immunotherapy on HIV Reservoir and Anti-Viral Immune Responses in People Living with HIV and Cancer

by
Marine Baron
1,*,
Cathia Soulié
2,
Armelle Lavolé
3,
Lambert Assoumou
4,
Baptiste Abbar
1,
Baptiste Fouquet
1,
Alice Rousseau
1,
Marianne Veyri
5,
Assia Samri
1,
Alain Makinson
6,
Sylvain Choquet
7,
Julien Mazières
8,
Solenn Brosseau
9,
Brigitte Autran
1,
Dominique Costagliola
4,
Christine Katlama
10,
Jacques Cadranel
3,
Anne-Geneviève Marcelin
2,
Olivier Lambotte
11,12,
Jean-Philippe Spano
5,
Amélie Guihot
1,
The French Cooperative Thoracic Intergroup (IFCT) CHIVA-2 Investigators
† and
The ANRS Co 24 OncoVIHAC Study Group
‡add Show full author list remove Hide full author list
1
INSERM U1135, CIMI, Département d’Immunologie, AP-HP, Hôpital Pitié-Salpêtrière, Sorbonne Université, F-75013 Paris, France
2
INSERM UMR_S 1136, Institut Pierre Louis d’Epidémiologie et de Santé Publique, Département de Virologie, AP-HP, Hôpital Pitié-Salpêtrière, Sorbonne Université, F-75013 Paris, France
3
GRC #04 Theranoscan, Département de Pneumologie et Oncologie Thoracique, AP-HP, Hôpital Tenon, Sorbonne Université, F-75020 Paris, France
4
INSERM UMR_S 1136, Institut Pierre Louis d’Epidémiologie et de Santé Publique, Sorbonne Université, F-75013 Paris, France
5
Département d’Oncologie Médicale, AP-HP, Hôpital Pitié-Salpêtrière, Sorbonne Université, F-75013 Paris, France
6
INSERM U1175, Département de Maladies Infectieuses, Centre Hospitalier Universitaire de Montpellier, Université de Montpellier, F-34090 Montpellier, France
7
Département d’Hématologie Clinique, AP-HP, Hôpital Pitié-Salpêtrière, Sorbonne Université, F-75013 Paris, France
8
Département de Pneumologie, Centre Hospitalier Universitaire de Toulouse, F-31000 Toulouse, France
9
Département de Pneumologie, AP-HP, Hôpital Bichat-Claude Bernard, F-75018 Paris, France
10
Département de Maladies Infectieuses, AP-HP, Hôpital Pitié-Salpêtrière, Sorbonne Université, F-75013 Paris, France
11
Département d’Immunologie Clinique, AP-HP, Hôpital Bicêtre, Université Paris-Saclay, F-94270 Le Kremlin Bicêtre, France
12
INSERM, CEA, Center for Immunology of Viral, Auto-immune, Hematological and Bacterial Diseases (IDMIT/IMVA-HB), UMR1184, Université Paris-Saclay, F-94270 Le Kremlin Bicêtre, France
 
add Show full affiliation list
 
remove Hide full affiliation list
*
Author to whom correspondence should be addressed.
†
The French Cooperative Thoracic Intergroup (IFCT) is an independent, non-profit academic research group of investigators. They are listed in Members of the French Cooperative Thoracic Intergroup (IFCT) CHIVA-2 investigators.
‡
The ANRS Co 24 OncoVIHAC Study group is listed in Members of the ANRS Oncovirim, Study Group.
Cells 2022, 11(6), 1015; https://doi.org/10.3390/cells11061015
Submission received: 1 February 2022 / Revised: 3 March 2022 / Accepted: 9 March 2022 / Published: 17 March 2022
(This article belongs to the Special Issue HIV and Host Interactions Ⅱ)
Browse Figures
Review Reports Versions Notes

Abstract

:
The role of immune checkpoints (ICPs) in both anti-HIV T cell exhaustion and HIV reservoir persistence, has suggested that an HIV cure therapeutic strategy could involve ICP blockade. We studied the impact of anti-PD-1 therapy on HIV reservoirs and anti-viral immune responses in people living with HIV and treated for cancer. At several timepoints, we monitored CD4 cell counts, plasma HIV-RNA, cell associated (CA) HIV-DNA, EBV, CMV, HBV, HCV, and HHV-8 viral loads, activation markers, ICP expression and virus-specific T cells. Thirty-two patients were included, with median follow-up of 5 months. The CA HIV-DNA tended to decrease before cycle 2 (p = 0.049). Six patients exhibited a ≥0.5 log10 HIV-DNA decrease at least once. Among those, HIV-DNA became undetectable for 10 months in one patient. Overall, no significant increase in HIV-specific immunity was observed. In contrast, we detected an early increase in CTLA-4 + CD4+ T cells in all patients (p = 0.004) and a greater increase in CTLA-4+ and TIM-3 + CD8+ T cells in patients without HIV-DNA reduction compared to the others (p ≤ 0.03). Our results suggest that ICP replacement compensatory mechanisms might limit the impact of anti-PD-1 monotherapy on HIV reservoirs, and pave the way for combination ICP blockade in HIV cure strategies.

1. Background

Immune checkpoints (ICPs), such as programmed cell death protein 1 (PD-1), cytotoxic T lymphocytes antigen 4 (CTLA-4), T-cell immunoreceptor with immunoglobulin and ITIM domain (TIGIT) and lymphocyte activation gene (LAG-3) are T cell inhibitory receptors that are activated on immune cells after TCR engagement to counterbalance chronic antigenic stimulation [1]. In cancers, ICPs are involved in immune control escape, promoting anti-tumor T-cell exhaustion. Nowadays, ICP blockade (ICB) by monoclonal antibodies is used for restoring anti-tumor immunity and is a major advance in cancer therapy [2].
During HIV-1 infection, ICPs are involved in both reservoir latency and anti-viral T cell exhaustion. On the one hand, PD-1, TIGIT and LAG-3 expression is positively associated with the frequency of CD4 + T cells harboring integrated HIV-DNA, and PD-1 engagement has been showed to inhibit HIV production and reactivation in latently infected cells ex vivo [3,4]. Accordingly, ICB reverses latency and reactivates viral production [3,5]. On the other hand, increased levels of PD-1 expression on total and HIV-specific CD8 + and CD4+ T cells in untreated HIV-1 infection are significantly correlated with both increased HIV viral load (VL) and with reduced capacity of cytokine production and of proliferation of HIV-specific T cells [6,7,8]. Anti-PD-1 and anti-PD-L1 antibodies demonstrated immune dysfunction reverse [8,9,10]. Taken together, these data suggest that ICB used as a “shock and kill” strategy could at the same time reverse latency and make the virus visible to the immune system, and restore anti-HIV immunity towards an HIV cure. The impact of ICB monotherapy on HIV-1 infection in vivo has been reported with controversies [11]. In our experience, one patient treated with nivolumab for cancer demonstrated a drastic and persistent decrease in cell-associated (CA) HIV-DNA [12]. Three other reports revealed latency reversal in patients [3,5,13]. A more recent study with very short follow-up including 33 people living with HIV (PLWH) with cancer showed no modification of the replication-competent reservoir [14].
During other co-infecting chronic viral infections, such as HBV, HCV, JC-virus or EBV infections, pathogens also exploit ICPs for immune evasion [15,16,17]. Several studies have suggested that ICB could be used as anti-infectious treatment for chronic HBV and HCV infection or for progressive multifocal leukoencephalopathy [18,19,20]. Finally, EBV-associated lymphomas and HHV-8- associated Kaposi sarcoma (KS) could benefit from ICB [21,22]. Of note, no predictive biomarker of ICB efficacy is available concerning chronic viral infections.
A biological sub-study was set-up, from both the ANRS CO24 OncoVIHAC prospective multicenter cohort (OncoVIRIM) and clinical trial IFCT-1602 CHIVA-2 (BIO-CHIVA-2), including PLWH with cancer treated with ICB. Here, we present the results of the largest cohort of patients assessing in vivo CA HIV-DNA, immunological changes and the evolution of virus-specific T cells among PLWH treated with ICB for cancer.

2. Methods

2.1. Study Design and Population

Patients were included from December 2017 to March 2020 from ten French centers on behalf the Agence nationale de recherche sur le SIDA et les hépatites virales (ANRS) and Intergroupe Francophone de cancérologie thoracique (IFCT) groups. Inclusion criteria: were age above 18 years old, proven HIV-1 or 2 infection, viral load below 50 cps/mL in OncoVIRIM or 200 cps/mL in BIO-CHIVA-2 under ART, ICB for cancer. Patients received nivolumab or cemiplimab 3 mg/kg every 2 weeks or pembrolizumab 2 mg/kg every 3 weeks until tumor progression, toxicity or patient decision to cease treatment. All patients signed a written informed consent form. The protocol was approved by an institutional review board (ANRS CO-24 OncoVIHAC: CPP17-020/2017-A00699-44; IFCT-1602 CHIVA2: CPP-Sud-Est V/17-IFCT-01) and was performed in accordance with the Helsinki declaration. Fresh blood samples were obtained at baseline and before cycles 2, 3 or 4, 9, 15 or 18, 27 or 36 and 51 and at the end of treatment.

2.2. HLA-Typing

HLA typing was performed using PCR-SSO hybridization methods in a Luminex FLEXMAP-3D® platform and analyzed on the HLA-FusionTM software (LABtype SSO class I/II tests: LABtype XR Class I locus A, B, C and Labtype SSO class II DRB1, DQA1/B1, OneLambda Inc., West Hills, CA, USA).

2.3. Viral Assays

Serologies for CMV and EBV (DiaSorin, Saluggia, Italy), HBV and HCV (Abbott, Chicago, IL, USA), HHV-8 [23] were analyzed at baseline. The HIV, HBV and HCV viral loads (VL) were analyzed in plasma using AmpliPrep/COBAS TaqMan (Roche Diagnostics, Basel, Switzerland) and the CMV and EBV loads were measured in whole blood (Qiagen, Hilden, Germany). HHV-8 was quantified by RT-PCR [24]. CA HIV-1 DNA was quantified by ultrasensitive RT-PCR (Biocentric, Bandol, France) [25].

2.4. Flow Cytometric Analysis

Flow cytometry testing was performed on fresh blood with two mixes of antibodies assessing first lymphocyte subsets and ICP: TIM3-BB515 (BD, Franklin Lakes, NJ, USA, 7D3), CXCR5-PE (BioLegend, San Diego, CA, USA, J252D4), CD45RA-ECD (BC, Fullerton, CA, USA, 2H4LDH11LDB9), CD27-PerCP-Cy5.5 (BD, Franklin Lakes, NJ, USA, L128), CCR7-Pe-Cy7 (BD, Franklin Lakes, NJ, USA, 3D12), CTLA4-APC (BD, Franklin Lakes, NJ, USA, BNI3), CD4-APC-R700 (BD, RPA-T4, Franklin Lakes, NJ, USA), CD3-APC-Cy7 (BD, SK7, Franklin Lakes, NJ, USA), PD1-BV421 (BD, EH12.1, Franklin Lakes, NJ, USA) and CD8-BV510 (BD, SK1, Franklin Lakes, NJ, USA), and intracellular staining with CTLA4-APC (BD, BNI3, Franklin Lakes, NJ, USA); second, activation markers were assessed:CD4-FITC (BD, RPA-T4, Franklin Lakes, NJ, USA), CD25-PE (BC, B1.49.9, Fullerton, CA, USA), CD69-PE-CF594 (BD, FN50, Franklin Lakes, NJ, USA), PD1-PE-Cy7 (BD, EH12.1, Franklin Lakes, NJ, USA), HLA-DR-AF700 (BioLegend, San Diego, CA, USA, L243), CD3-APC-AF750 (BC, Fullerton, CA, USA, UCHT1), CD38-BV421 (BioLegend, San Diego, CA, USA, HB7) and CD8 BV510 (BD, Franklin Lakes, NJ, USA, SK1), and intracellular staining with Ki67-APC (ThermoFisher, Waltham, MA, USA, 20Raj1). Cell permeabilization (Cytofix/Cytoperm, BD, Franklin Lakes, NJ, USA), fixation (CellFIX, BD, Franklin Lakes, NJ, USA), acquisition (10-colors Gallios) and analysis (FlowJo 10.5) were performed as previously described [26]. T cell subsets were defined as: naive T cells (TN): CD27 + 45RA + CCR7+, central memory T cells (TCM): CD27 + 45RA-CCR7+, transitional memory T cells (TTM): CD27 + 45RA-CCR7-, effector memory T cells (TEM): CD27-45RA-CCR7-, CD45RA re-expressing TEM (TEMRA): CD27-45RA + CCR7-. Boolean ICP analysis defined single, double and triple expressers displaying only one, two or three ICP.

2.5. Intracellular Cytokine Staining Assays

Virus-specific T cells were analyzed using an ICS assay [27]. Briefly, 1 × 106 thawed PBMC were stimulated overnight with each virus peptides pool (Supplementary Methods S1 and Supplementary Table S1) or staphylococcal enterotoxin B toxin as the positive control and medium alone as negative control. Cells were surface-stained with Live/Dead Fixable Aqua Dead Cell (Invitrogen, Carlsbad, CA, USA, L34957) and anti PD1-BV421 (BD, Franklin Lakes, NJ, USA, EH12.1), CD4-APC (BD, Franklin Lakes, NJ, USA, RPA-T4) and CD8-PerCP-Cy5.5 (BD, Franklin Lakes, NJ, USA, SK1); then intracellular staining was performed for interleukin-2 (IL2)-PE (BD, Franklin Lakes, NJ, USA, 5344.111), interferon-γ (IFN-γ)-FITC (BD, Franklin Lakes, NJ, USA, 25723.11), tumor necrosis factor-α (TNF-α)-PE-Cy7 (BD, Franklin Lakes, NJ, USA, Mab11) and CD3-APC-Cy7 (BD, Franklin Lakes, NJ, USA, SK7).
The HIV-stimulated cells from 18 patients were also analyzed for the expression of ICP and surface stained with anti CD4-ECD (BC, Fullerton, CA, USA, UCHT1, 7448079F), CD8-PerCP-Cy5.5 (BD, Franklin Lakes, NJ, USA, SK1), PD1-BV421 (BD, Franklin Lakes, NJ, USA, EH12.1), TIM3-PeCy7 (ThermoFisher; Waltham, MA, USA, F38-2E2), CTLA4-APC (BD, Franklin Lakes, NJ, USA, BNI3), LAG3-AlexaFluor700 (ThermoFisher, Waltham, MA, USA, 3DS223H) and IgG 4-PE (Southern Biotech, Birmingham, AL, USA, HP6025); then intracellular staining was performed for IFN-γ-FITC (BD, Franklin Lakes, NJ, USA, 25723.11), CD3-APC-Cy7 (BD, Franklin Lakes, NJ, USA, SK7) and CTLA4-APC (BD, Franklin Lakes, NJ, USA, BNI3).
Percentages of CD8+ and CD4 + T cells producing cytokines were determined after subtraction of negative controls and by adding the various peptide pools for each virus. Phenotypes of specific T cells were evaluated only on cytokine-producing cells with more than 50 events.

2.6. Statistical Analysis

The non-parametric Wilcoxon test was used to test differences between paired groups. The Mann–Whitney test was used to test differences between unpaired groups. Statistical significance was considered for p-values below 0.05. When multiple comparisons were performed, the Bonferroni correction was used to correct the significance level, as stated in the figure legends.

3. Results

3.1. Patients Characteristics

Thirty-two patients with HIV-1 infection were included: n = 22 in OncoVIRIM, n = 10 in BIO-CHIVA-2 (Table 1). The cancer types were non-small cell lung-cancer (n = 20), bladder cancer (n = 3), melanoma (n = 2), head and neck cancer (n = 2), Hodgkin lymphoma, KS, anal, oropharynx and orbit cancer (1 patient each). Viral co-infections included HHV-8 infection (53%), resolved HBV infection (38%), resolved HCV infection (38%) and chronic HBV infection (22%). At baseline, the median CA HIV-DNA was 184 cps/106 cells (range 40–1749) and the median HIV VL was 20 cps/mL (range <1–352). The median baseline CD4 cell count was 369/mm3 (range 45–915) and the CD4/CD8 ratio was 1 (range 0.2–2.1). All patients were treated with anti-PD-1: nivolumab (69%), pembrolizumab (28%) and cemiplimab (3%). Six of the patients received ICB as a first line treatment and the others had refractory or relapse diseases. The median follow-up duration was five months (range 1–30) and the median number of cycles received was six (range 2–36). At last follow-up, all but one patient had discontinued ICB. Eighteen patients died: 15 from tumor progression, one from an immune-related adverse event (myocarditis), one from COVID-19 infection and one from an unknown cause.

3.2. Early Two-Fold Decrease of CA-HIV-DNA following Anti-PD-1

Evaluating as a first step the total CA HIV-DNA at each time point, we observed a 1.9-fold decrease from a median 184 to 99 cps/106 cells at C2 when compared to baseline (p = 0.0499), without reaching the significance level after the Bonferroni correction (Figure 1). In addition, as 0.3 log10 physiological fluctuations are commonly described in this assay, we used a threshold of 0.5 log10 to further evaluate this decay. A more than 0.5 log10 decrease in HIV-DNA cps/106 cells was observed at once after anti-PD-1 initiation in six patients (Pt #7, #14, #17, #21, #30 and #31). We further identified these patients with reservoir size reduction as “RR” patients, and we studied if they exhibited specific immunological characteristics to better understand which factors could be involved. Baseline CA HIV DNA was higher in RR compared to those without reservoir size reduction (further identified as “NoRR” patients): 307 versus 166 cps/106 cells (p = 0.002) (Supplementary Figure S1). Amongst the six RR, patient #7 demonstrated a persistent and stable HIV-DNA decrease below the detection threshold at the three consecutive last samples. Among the 26 other patients, 4 had a transient increase of ≥0.5 log10 HIV-DNA cps/106 cells and there was no change below or above 0.5 log10 for the 22 other patients. Because age potentially impacts the expression of PD-1 on T lymphocytes, we looked at baseline HIV-DNA between younger (<60 years) and older (≥60 years) patients and found no difference between groups (median values 191 and 181 cps/106 cells, respectively, p = 0.9018). We also compared the median age in RR and in NR and found no significant difference (p = 0.6974).

3.3. CD4 Stability and Early T Cell Activation

There was no difference in baseline CD4 count and HIV-VL between RR and NoRR patients (Supplementary Figure S1). The HIV-VL decreased from a median baseline of 20 cp/mL to 1 cp/mL at cycle 9 (p = 0.0313) without reaching the significance level after the Bonferroni correction, and there was no change in the CD4 cell counts (Figure 1), nor in the CD4/CD8 ratios, in the CD3, CD8, NK and B cell counts or in lymphocyte differentiation (Supplementary Figure S2). Overall, the proportions of HLA-DR + CD4+ and CD8 + T cells and of CD38+ and HLA-DR + CD38 + CD8 + T cells significantly increased at C2 or C3: respectively 1.8, 1.6, 1.4 and 1.7 fold increase from baseline (p < 0.0001, Figure 2, Supplementary Figures S3 and S4). Similarly, there was an early increase in Ki67 + CD4+ and CD8 + T cells at C2 or C3: respectively 2.1 and 2.3 fold increase from baseline (p ≤ 0.0003). Among RR, the baseline proportions of CD25 + CD8+ T cells (33%) in patient #7 and of HLA-DR + CD8 + T cells (25%) in patient #21 were the highest of the cohort, while patient #14 displayed an early major increase of CD4 + Ki67+ and CD4 + HLA-DR T cells at C3. Nevertheless, there was no statistical difference between activation markers at baseline or kinetics among patients with and without reservoir size reduction (Supplementary Figure S5). These results confirm that ICB induces early immunological activation and show that some RR patients had high baseline levels of activation markers.

3.4. CTLA-4 Is Upregulated on CD4 Cells and ICP Compensatory Mechanisms Are Less Pronounced in Patients RR

Next, we assessed at each time point the co-expression of PD-1 together with CTLA-4 and TIM-3 on T cells to explore whether inhibitory compensation of ICP blockade occurred (Figure 3, Supplementary Figures S6–S8). At baseline, RR patients did not demonstrate any specific profile except patient RR #21 who showed the highest proportion of PD-1 + CD8 + T cells (99%) in the cohort. (Supplementary Figure S8). At C2 and overtime, the membrane PD-1 molecule became barely detectable on CD4+ and CD8 + T cells (p < 0.0001). In contrast, we observed a significant but transient increase in CTLA-4 + CD4 + T cells at C2 (fold change: 1.5, p = 0.004) and a trend for an increase of CTLA-4 + CD8+ T cells at C2 which did not reach statistical significance (p = 0.03). Of note, the proportions of CD8+ T cells displaying CTLA-4 and TIM-3 increased more at C2 or C3 in NoRR patients compared to RR patients (respectively 2.4 versus 1.1 for CTLA-4 + CD8 + fold change, and 1.4 versus 0.9 for TIM-3 + CD8 + fold change) (p ≤ 0.03) (Figure 3). Taken together, these results suggest that anti-PD-1 monotherapy could be associated with some compensatory increase of other ICP molecules on T cells which is less pronounced in the RR patients.

3.5. Stability of Peripheral HIV-Specific T Cells Despite PD-1 Overexpression at Baseline

We next evaluated whether ICB could enhance HIV-specific T cell responses in vivo, defined as IFN-γ producing CD8+ T cells. At baseline, the median proportion of HIV-specific CD8 + T cells was 0.48% (range 0–6.52) of total CD8+ T cells. Those frequencies did not differ in the RR patients compared to the NoRR patients, although the patient RR #21 had the third highest proportion of HIV-specific CD8+ T cells among the cohort (2.63%) (Supplementary Figure S9). Overall, the percentages of HIV-specific CD8+ T cells did not statistically change over time (Figure 4) and there was no difference in the kinetics of HIV-specific CD8+ T cells between the RR and NoRR patients (Supplementary Figure S9). The frequencies of IL-2 + CD8+ and TNF-α+ CD8+ T cells were much lower at baseline (0.02% each) than the IFN-γ + CD8+ T cells frequencies and did not change over time (Supplementary Figure S10). The cell poly-functionality defined as the co-production of IFN-γ, and/or IL-2 and/or TNF-α also remained stable over time without differences between RR patients and the others (Supplementary Figure S11). Similarly, CD4 HIV-specific T cell responses did not change over time (Supplementary Figure S10).
When analysing PD1 expression on HIV-specific T cells at baseline, we observed that HIV-specific T cells displayed higher percentages of PD-1+ cells and CTLA-4+ cells (Figure 5 and Figure S12) and higher mean fluorescence intensity (MFI) of PD-1 (Supplementary Figure S13) compared to non-HIV-specific CD8+ T cells (p < 0.001). PD-1 expression on HIV-specific CD8 + T cells was non significantly lower in RR compared to NoRR patients (Supplementary Figure S9). We then tested in 18 patients whether such a lack of immune enhancement could reflect an upregulation of other ICP on HIV-specific CD8 T cells. However, CTLA-4, TIM-3 and LAG-3 expression on HIV-specific T cells did not increase over time (Figure 5).
Taken together, these results suggest that ICB monotherapy is insufficient for enhancing an HIV-specific response despite high PD1-expression on HIV-specific CD8+ T cells at baseline.

3.6. Immunological and Virological Parameters of Other Viruses Are Not Modified

As ICB have been proposed to be used against other chronic viral co-infections such as EBV, CMV and HBV, we aimed at evaluating the effects of ICB on other viruses in these poly-infected patients, and whether anti-PD-1 could enhance these other virus-specific T cell responses. In patients seropositive for HHV-8, HCV, HBV and CMV, there was no corresponding viral replication at baseline, except in 2 patients with active HBV infection and for whom anti-HBV treatment was changed at the start of ICB. After anti-PD-1 initiation, there was no significant change in HHV-8, HBV, HCV, EBV and CMV loads over time (Supplementary Figure S14). There was no significant change in virus-specific CD8 + T cells over time despite highly heterogeneous profiles (Figure 6). When studying PD-1 expression on virus-specific CD8+ T cells, we observed at baseline higher percentages of PD-1 expression (Figure 5) and higher MFI of PD-1 (Supplementary Figure S13) on EBV- and HBV-specific CD8+ T cells (p < 0.007). Taken together, these results suggest that ICB can be safely used in PLWH with viral co-infections and that PD-1 overexpression on EBV and HBV-specific CD8+ T cells could be targeted, even though we failed to demonstrate specific invigoration in peripheral blood.

3.7. Immuno-Virological Profiling of RRs

The individual profiles (HIV-RNA, CA HIV-DNA and HIV-specific T cells) of RR patients are summarized in Figure 7. None of these patients with available HLA typing (n = 4) had the HLA-B27 or B57 protective alleles. Patient #7 had the longest follow-up of 14 months and his HIV-DNA decreased from 187 cps/106 cells to become persistently undetectable at the last three time points (C9, 15 and C27). Patient #14 displayed the highest HIV-DNA at baseline (1749 cps/106 cells) which decreased by 0.6 log10 down to 403 cps/106 cells six months later. Patient #17 experienced an early 0.5 log10 decrease in HIV-DNA from 620 to 178 cps/106 cells at C2. Patient #21 demonstrated a persistent CA HIV-DNA reduction from 231 to 66 cps/106 cells 4 months later (−0.5 log10). Patient #30 HIV-DNA decreased from 363 to 106 cps/106 cells (−0.5 log10) 4 months later. Finally, the HIV-DNA of patient #31 decreased gradually from 251 to 53 cps/106 cells 2 months later (−0.7 log10). ICB was stopped because of tumor progression in 4 cases and of cutaneous toxicity in 1 case, and monitoring was stopped in 1 case because of patient relocation.

4. Discussion

Here, we report the largest comprehensive and homogeneous immunological and virological assessment of the influence of immune check-point blockade on HIV reservoirs and T cell functions in a series of 32 PLWH treated with anti-PD-1 for cancer.
Despite some cancers heterogeneity, there was a majority of lung cancers and no evidence that the cancer type might influence the ICB action on the HIV reservoir and on HIV-specific T cells.
We chose to focus as a first step of reservoir size analysis on the measurement of total CA HIV-DNA since our final aim was to study the evolution of the HIV-reservoir as a result of both HIV latent cell reactivation and immune invigoration. We assume that it provides less understanding of the potential influence of ICB on the reservoirs, but such evaluation appeared to us to be more relevant on a clinical perspective. The CA HIV-DNA decrease observed after the cycle 1 was only transient and such ICB could have impacted more profoundly and durably the HIV-reservoir, though still transiently, in only 19% of cases. The threshold of CA HIV-DNA reduction of 0.5 log is above the usual 0.3 log10 fluctuations of the methods and was an acceptable cut-off in other studies [28,29]. This proportion of 19% RR could be mis-estimated given that CA HIV-DNA was undetectable at baseline in six patients. The 6 patients with a CA HIV-DNA decrease ≥ 0.5 log10 under ICB included one with CA HIV-DNA becoming persistently undetectable for 14 months and one with features of a “shock and kill”. Although follow-up was short due to frequent tumor progression, four patients with RR were followed during more than three months, which represents one of the longest descriptions of CA-HIV DNA under ICB. Of note, two out of six patients RR (patient #14 and #17) had a decrease of CD4+ T cells frequencies at the same time, which may have impacted the results of HIV reservoir. Overall, no significant immunological differences in terms of immune activation or HIV-specific T cell responses were observed between these rare RR and NoRR patients. The weak HIV-specific T cell responses observed in our patients and the lack of ICB effect might reflect the consequences of previous multiple chemotherapies in those PLWH with cancers
Our results suggest one mechanism to explain such limited ICB impact on HIV reservoirs is that the virus persists preferentially in CD4 + T cells expressing other ICP than PD-1. The greater increase in CTLA-4 and TIM-3 expression on CD8 + T cells in the NoRR compared to the RR patients reinforces this hypothesis. Such ICP compensation and upregulation on CD8 + T cells could have hampered the CTL response to the virus and prevent an efficient “shock and kill” mechanism. These observations are consistent with the recently published demonstration of ICP overexpression in Hodgkin lymphoma tumor microenvironment exposed to anti-PD-1 [30]. Thus, a combination of IC blockade could be necessary to bypass multiple ICP expression both on HIV reservoir cells and on HIV-specific T cells as suggested by others, although such combinations are limited by an increased and barely acceptable toxicity in PLWH [4,14,31,32,33]. Finally, the dramatic decrease in PD-1 expression on T cells reflects the antigenic occupancy with the monoclonal antibody of detection which recognizes the same epitope as the therapeutic one, as it has been demonstrated previously [34].
Finally, we provide insights into the virological safety of ICB and its impact on immune responses against other viruses. Indeed, ICB is safe and can be widely used in co-infected patients, although the great majority of patients had very low viral antigen loads with undetectable HBV, HCV, CMV and HHV-8 VL at baseline. Of note, the two patients with active HBV infection at baseline benefited from anti-HBV treatment at the same time, which also hinders the interpretation of VL decrease under ICB.
In conclusion, our study shows that ICB with anti-PD1 antibodies in PLWH with cancer had a very limited impact on HIV reservoirs and immunity to HIV which might be explained by an ICP compensatory phenomenon as assessed by the early increase in CTLA-4 and Tim-3 expression. Further studies are warranted to evaluate the impact of combined immune checkpoint blockade on the HIV reservoirs and immunity of PLWH suffering from cancers.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/cells11061015/s1, Supplementary Methods S1: Viral stimulations before ICS assays are detailed [26,27,35,36,37,38,39,40,41,42]; Figure S1: Cell-associated HIV DNA at baseline is higher in patients with virological reduction compared to the others; Figure S2: Lymphocytes’ subsets and subpopulations are not modified under anti-PD-1 therapy; Figure S3: Example dot plots of A, HLA-DR and of B, Ki-67 expression among CD4 and CD8 T cells at baseline and at cycle 2; of C, CD38 and of D, CD38+ HLA-DR+ expression among CD8 T cells at baseline and at cycle 2; Figure S4: CD69 and CD25 expression on CD4+ and CD8+ T cells are not modified under anti-PD-1 therapy; Figure S5: Activation markers expression at baseline and activation markers kinetics are not different between the RR patients and the others; Figure S6: PD-1, CTLA-4 and TIM-3 expression profile on total CD8+ and CD4+ T cells at baseline before anti-PD-1 therapy; Figure S7: Example dot plots of PD-1, CTLA-4 and TIM-3 expression among CD4+ and CD8+ T cells at baseline and at cycle 2; Figure S8: ICP expression at baseline are not different between the RR patients and the others; Figure S9: HIV-specific CD8+ T cells proportions at baseline and kinetics are not different between the RR patients and the others; Figure S10: Intra-cellular cytokine expression after HIV-peptide stimulation is not modified under anti-PD-1 therapy; Figure S11: Polyfunctionality profile amongst HIV-specific CD8+ T cells is not modified under anti-PD-1 therapy; Figure S12: Example dot plots of A, IFN-ɣ+ CD8+ T cells after stimulation with medium alone (negative control on the left) and with staphylococcal enterotoxin B toxin (positive control on the right); Figure S13: High levels of mean fluorescence intensity (MFI) are detected on HIV, EBV and HBV-specific CD8+ T cells baseline after viral stimulation; Figure S14: Viral loads under ICB; Table S1: Immunodominant HBV peptides sequences, genotype, HLA restriction and amino acids number.

Author Contributions

A.G., B.A. (Brigitte Autran), O.L., L.A., D.C., J.C., A.L. and J.-P.S. designed the study, A.L., B.A. (Baptiste Abbar), M.V., A.M., S.C., J.M., S.B., J.C., C.K., O.L. and J.-P.S. took care of the patients, M.B., A.R. and A.S. performed immunological analysis, M.B. performed biological analysis interpretations and assembly the data, C.S. and A.-G.M. performed virological analysis, B.F. performed HLA-typing, M.B., A.G. and B.A. (Brigitte Autran) wrote the manuscript. All authors had full access to all data in the study and had final responsibility for the decision to submit for publication. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the ANRS/Emerging Infectious Diseases for ANRS CO24 OncoVIHAC (Grant ANRS CO24); The Intergroupe Francophone de Cancérologie Thoracique (Grant IFCT-1602); and Bristol-Myers Squibb (Grant CA209-689) for IFCT-1602-CHIVA2.ANRS/Emerging Infectious Diseases played no further role in the collection, analysis or interpretation of the data following approval of the protocol, but reviewed the final manuscript. IFCT was involved in the study design, the data collection and in the decision to submit paper for publication. BMS played no role. “Vaincre le Cancer” and “Foundation SCOR pour la Science” funded part of the immunological assays.

Institutional Review Board Statement

The protocol was approved by an institutional review board (ANRS CO-24 OncoVIHAC: CPP17-020/2017-A00699-44; IFCT-1602 CHIVA2: CPP-Sud-Est V/17-IFCT-01) and was performed in accordance with the Helsinki declaration.

Informed Consent Statement

All patients signed a written informed consent form.

Data Availability Statement

All biological data were linked and shared at Department of Immunology, Pitié-Salpétrière Hospital. External users with a formal analysis plan can request access to the data to Department of Immunology.

Acknowledgments

We thank Joséphine-Anna-Tine and Michèle Génin for their work in the INSERM-UMRS-1136. We thank Ali Boudifa for HLA typing. We thank T4-T8 laboratory, Sophie Even and Christophe Parizot for cytometer support. We thank Elodie Couret and Basile Moambi for the technical support. We thank ANRS/Emerging infectious diseases, the IFCT group (in particular Franck Morin and Elodie Amour) and Bristol-Myers Squibb for their funding and their support. We thank the CancerVIH group for promoting the study and recruiting patients. Finally, we also deeply thank all the French clinicians for patients’ inclusions in the study. We thank Cecile Moins and Ventzislava Petrov-Sanchez from the ANRS/ Emerging infectious diseases for their support.

Conflicts of Interest

The authors have no conflict of interest to declare.

Members of the ANRS: Oncovirim, Study Group

Sophie Abgrall (France, [email protected]), Sylvain Choquet (France, [email protected]), Olivia Faucher-Zaegel (France, [email protected]), Laurent Greillier (France, [email protected]) Olivier Lambotte (France, [email protected]), Armelle Lavolé ([email protected]), Alain Makinson (France, [email protected]), Alexandra Picard (France, [email protected]), Andrei-Horia Seferian (France, [email protected]), Jean-Philippe Spano (France, [email protected]), Jacques Cadranel (France, [email protected]), Marine Baron ([email protected]), Amélie Guihot ([email protected]), Anne-Geneviève Marcelin ([email protected]), Dominique Costagliola (France, [email protected]), Lambert Assoumou (France, [email protected]), Christine Katlama (France, [email protected]), Brigitte Autran (France, [email protected]), Marianne Veyri (France, [email protected]).

Members of the French Cooperative Thoracic Intergroup (IFCT) CHIVA-2 Investigators

Armelle Lavole (France, [email protected]), Julien Mazières (France, [email protected]), Sophie Schneider (France, [email protected]), Solenn Brousseau ([email protected]), Lize Kiakouama (France, [email protected]), Laurent Greillier (France, [email protected]), Alain Makinson (France, [email protected]), Alexandra Langlais (France, [email protected]), Franck Morin (France, [email protected]), Jean-Philippe Spano (France, [email protected]), Jacqus Cadranel (France, [email protected]).

References

  1. Wherry, E.J. T Cell Exhaustion. Nat. Immunol. 2011, 12, 492–499. [Google Scholar] [CrossRef]
  2. Postow, M.A.; Callahan, M.K.; Wolchok, J.D. Immune Checkpoint Blockade in Cancer Therapy. J. Clin. Oncol. Off. J. Am. Soc. Clin. Oncol. 2015, 33, 1974–1982. [Google Scholar] [CrossRef]
  3. Fromentin, R.; DaFonseca, S.; Costiniuk, C.T.; El-Far, M.; Procopio, F.A.; Hecht, F.M.; Hoh, R.; Deeks, S.G.; Hazuda, D.J.; Lewin, S.R.; et al. PD-1 Blockade Potentiates HIV Latency Reversal Ex Vivo in CD4+ T Cells from ART-Suppressed Individuals. Nat. Commun. 2019, 10, 814. [Google Scholar] [CrossRef]
  4. Fromentin, R.; Bakeman, W.; Lawani, M.B.; Khoury, G.; Hartogensis, W.; DaFonseca, S.; Killian, M.; Epling, L.; Hoh, R.; Sinclair, E.; et al. CD4+ T Cells Expressing PD-1, TIGIT and LAG-3 Contribute to HIV Persistence during ART. PLOS Pathog. 2016, 12, e1005761. [Google Scholar] [CrossRef] [PubMed]
  5. Evans, V.A.; van der Sluis, R.M.; Solomon, A.; Dantanarayana, A.; McNeil, C.; Garsia, R.; Palmer, S.; Fromentin, R.; Chomont, N.; Sékaly, R.-P.; et al. Programmed Cell Death-1 Contributes to the Establishment and Maintenance of HIV-1 Latency. AIDS 2018, 32, 1491–1497. [Google Scholar] [CrossRef]
  6. Day, C.L.; Kaufmann, D.E.; Kiepiela, P.; Brown, J.A.; Moodley, E.S.; Reddy, S.; Mackey, E.W.; Miller, J.D.; Leslie, A.J.; DePierres, C.; et al. PD-1 Expression on HIV-Specific T Cells Is Associated with T-Cell Exhaustion and Disease Progression. Nature 2006, 443, 350–354. [Google Scholar] [CrossRef] [PubMed]
  7. D’Souza, M.; Fontenot, A.P.; Mack, D.G.; Lozupone, C.; Dillon, S.; Meditz, A.; Wilson, C.C.; Connick, E.; Palmer, B.E. Programmed Death 1 Expression on HIV-Specific CD4+ T Cells Is Driven by Viral Replication and Associated with T Cell Dysfunction. J. Immunol. 2007, 179, 1979–1987. [Google Scholar] [CrossRef]
  8. Trautmann, L.; Janbazian, L.; Chomont, N.; Said, E.A.; Gimmig, S.; Bessette, B.; Boulassel, M.-R.; Delwart, E.; Sepulveda, H.; Balderas, R.S.; et al. Upregulation of PD-1 Expression on HIV-Specific CD8+ T Cells Leads to Reversible Immune Dysfunction. Nat. Med. 2006, 12, 1198–1202. [Google Scholar] [CrossRef]
  9. Velu, V.; Titanji, K.; Zhu, B.; Husain, S.; Pladevega, A.; Lai, L.; Vanderford, T.H.; Chennareddi, L.; Silvestri, G.; Freeman, G.J.; et al. Enhancing SIV-Specific Immunity in Vivo by PD-1 Blockade. Nature 2009, 458, 206–210. [Google Scholar] [CrossRef] [PubMed]
  10. Gay, C.L.; Bosch, R.J.; Ritz, J.; Hataye, J.M.; Aga, E.; Tressler, R.L.; Mason, S.W.; Hwang, C.K.; Grasela, D.M.; Ray, N.; et al. Clinical Trial of the Anti-PD-L1 Antibody BMS-936559 in HIV-1 Infected Participants on Suppressive Antiretroviral Therapy. J. Infect. Dis. 2017, 215, 1725–1733. [Google Scholar] [CrossRef]
  11. Abbar, B.; Baron, M.; Katlama, C.; Marcelin, A.-G.; Veyri, M.; Autran, B.; Guihot, A.; Spano, J.-P. Immune Checkpoint Inhibitors in People Living with HIV: What about Anti-HIV Effects? AIDS Lond. Engl. 2019, 34, 167–175. [Google Scholar] [CrossRef] [PubMed]
  12. Guihot, A.; Marcelin, A.-G.; Massiani, M.-A.; Samri, A.; Soulié, C.; Autran, B.; Spano, J.-P. Drastic Decrease of the HIV Reservoir in a Patient Treated with Nivolumab for Lung Cancer. Ann. Oncol. 2018, 29, 517–518. [Google Scholar] [CrossRef]
  13. Wightman, F.; Solomon, A.; Kumar, S.S.; Urriola, N.; Gallagher, K.; Hiener, B.; Palmer, S.; Mcneil, C.; Garsia, R.; Lewin, S.R. Effect of Ipilimumab on the HIV Reservoir in an HIV-Infected Individual with Metastatic Melanoma. AIDS Lond. Engl. 2015, 29, 504–506. [Google Scholar] [CrossRef] [PubMed]
  14. Rasmussen, T.A.; Rajdev, L.; Rhodes, A.; Dantanarayana, A.; Tennakoon, S.; Chea, S.; Spelman, T.; Lensing, S.; Rutishauser, R.; Bakkour, S.; et al. Impact of Anti-PD-1 and Anti-CTLA-4 on the HIV Reservoir in People Living with HIV with Cancer on Antiretroviral Therapy: The AIDS Malignancy Consortium-095 Study. Clin. Infect. Dis. Off. Publ. Infect. Dis. Soc. Am. 2021, 73, e1973–e1981. [Google Scholar] [CrossRef] [PubMed]
  15. Macedo, C.; Webber, S.A.; Donnenberg, A.D.; Popescu, I.; Hua, Y.; Green, M.; Rowe, D.; Smith, L.; Brooks, M.M.; Metes, D. EBV-Specific CD8+ T Cells from Asymptomatic Pediatric Thoracic Transplant Patients Carrying Chronic High EBV Loads Display Contrasting Features: Activated Phenotype and Exhausted Function. J. Immunol. Baltim. Md 1950 2011, 186, 5854–5862. [Google Scholar] [CrossRef] [PubMed]
  16. Tan, C.S.; Bord, E.; Broge, T.A.; Glotzbecker, B.; Mills, H.; Gheuens, S.; Rosenblatt, J.; Avigan, D.; Koralnik, I.J. Increased Program Cell Death-1 (PD-1) Expression on T Lymphocytes of Patients with Progressive Multifocal Leukoencephalopathy (PML). J. Acquir. Immune Defic. Syndr. 2012, 60, 244–248. [Google Scholar] [CrossRef]
  17. Wykes, M.N.; Lewin, S.R. Immune Checkpoint Blockade in Infectious Diseases. Nat. Rev. Immunol. 2018, 18, 91–104. [Google Scholar] [CrossRef] [PubMed]
  18. Gane, E.; Verdon, D.J.; Brooks, A.E.; Gaggar, A.; Nguyen, A.H.; Subramanian, G.M.; Schwabe, C.; Dunbar, P.R. Anti-PD-1 Blockade with Nivolumab with and without Therapeutic Vaccination for Virally Suppressed Chronic Hepatitis B: A Pilot Study. J. Hepatol. 2019, 71, 900–907. [Google Scholar] [CrossRef]
  19. Gardiner, D.; Lalezari, J.; Lawitz, E.; DiMicco, M.; Ghalib, R.; Reddy, K.R.; Chang, K.-M.; Sulkowski, M.; Marro, S.O.; Anderson, J.; et al. A Randomized, Double-Blind, Placebo-Controlled Assessment of BMS-936558, a Fully Human Monoclonal Antibody to Programmed Death-1 (PD-1), in Patients with Chronic Hepatitis C Virus Infection. PLoS ONE 2013, 8, e63818. [Google Scholar] [CrossRef]
  20. Roos-Weil, D.; Weiss, N.; Guihot, A.; Uzunov, M.; Bellanger, A.; Eymard, B.; Saadoun, D.; Houillier, C.; Idbaih, A.; Demeret, S.; et al. Immune Checkpoint Inhibitors for Progressive Multifocal Leukoencephalopathy: A New Gold Standard? J. Neurol. 2021, 268, 2458–2465. [Google Scholar] [CrossRef]
  21. Chen, J.; Del Valle, L.; Lin, H.-Y.; Plaisance-Bonstaff, K.; Forrest, J.C.; Post, S.R.; Qin, Z. Expression of PD-1 and PD-Ls in Kaposi’s Sarcoma and Regulation by Oncogenic Herpesvirus Lytic Reactivation. Virology 2019, 536, 16–19. [Google Scholar] [CrossRef] [PubMed]
  22. Delyon, J.; Bizot, A.; Battistella, M.; Madelaine, I.; Vercellino, L.; Lebbé, C. PD-1 Blockade with Nivolumab in Endemic Kaposi Sarcoma. Ann. Oncol. Off. J. Eur. Soc. Med. Oncol. 2018, 29, 1067–1069. [Google Scholar] [CrossRef] [PubMed]
  23. Simpson, G.R.; Schulz, T.F.; Whitby, D.; Cook, P.M.; Boshoff, C.; Rainbow, L.; Howard, M.R.; Gao, S.J.; Bohenzky, R.A.; Simmonds, P.; et al. Prevalence of Kaposi’s Sarcoma Associated Herpesvirus Infection Measured by Antibodies to Recombinant Capsid Protein and Latent Immunofluorescence Antigen. Lancet Lond. Engl. 1996, 348, 1133–1138. [Google Scholar] [CrossRef]
  24. Lallemand, F.; Desire, N.; Rozenbaum, W.; Nicolas, J.C.; Marechal, V. Quantitative Analysis of Human Herpesvirus 8 Viral Load Using a Real-Time PCR Assay. J. Clin. Microbiol. 2000, 38, 1404–1408. [Google Scholar] [CrossRef] [PubMed]
  25. Avettand-Fènoël, V.; Chaix, M.-L.; Blanche, S.; Burgard, M.; Floch, C.; Toure, K.; Allemon, M.-C.; Warszawski, J.; Rouzioux, C.; French Pediatric Cohort Study ANRS-CO 01 Group. LTR Real-Time PCR for HIV-1 DNA Quantitation in Blood Cells for Early Diagnosis in Infants Born to Seropositive Mothers Treated in HAART Area (ANRS CO 01). J. Med. Virol. 2009, 81, 217–223. [Google Scholar] [CrossRef] [PubMed]
  26. Nakid-Cordero, C.; Arzouk, N.; Gauthier, N.; Tarantino, N.; Larsen, M.; Choquet, S.; Burrel, S.; Autran, B.; Vieillard, V.; Guihot, A. Skewed T Cell Responses to Epstein-Barr Virus in Long-Term Asymptomatic Kidney Transplant Recipients. PLoS ONE 2019, 14, e0224211. [Google Scholar] [CrossRef]
  27. Guihot, A.; Oksenhendler, E.; Galicier, L.; Marcelin, A.-G.; Papagno, L.; Bedin, A.-S.; Agbalika, F.; Dupin, N.; Cadranel, J.; Autran, B.; et al. Multicentric Castleman Disease Is Associated with Polyfunctional Effector Memory HHV-8-Specific CD8+ T Cells. Blood 2008, 111, 1387–1395. [Google Scholar] [CrossRef]
  28. Achenbach, C.J.; Assoumou, L.; Deeks, S.G.; Wilkin, T.J.; Berzins, B.; Casazza, J.P.; Lambert-Niclot, S.; Koup, R.A.; Costagliola, D.; Calvez, V.; et al. Effect of Therapeutic Intensification Followed by HIV DNA Prime and RAd5 Boost Vaccination on HIV-Specific Immunity and HIV Reservoir (EraMune 02): A Multicentre Randomised Clinical Trial. Lancet HIV 2015, 2, e82–e91. [Google Scholar] [CrossRef]
  29. Calin, R.; Hamimi, C.; Lambert-Niclot, S.; Carcelain, G.; Bellet, J.; Assoumou, L.; Tubiana, R.; Calvez, V.; Dudoit, Y.; Costagliola, D.; et al. Treatment Interruption in Chronically HIV-Infected Patients with an Ultralow HIV Reservoir. AIDS Lond. Engl. 2016, 30, 761–769. [Google Scholar] [CrossRef]
  30. Michot, J.-M.; Mouraud, S.; Adam, J.; Lazarovici, J.; Bigenwald, C.; Rigaud, C.; Tselikas, L.; Dartigues, P.; Danu, A.; Bigorgne, A.; et al. CD8+ T Lymphocytes Immune Depletion and LAG-3 Overexpression in Hodgkin Lymphoma Tumor Microenvironment Exposed to Anti-PD-1 Immunotherapy. Cancers 2021, 13, 5487. [Google Scholar] [CrossRef] [PubMed]
  31. McGary, C.S.; Deleage, C.; Harper, J.; Micci, L.; Ribeiro, S.P.; Paganini, S.; Kuri-Cervantes, L.; Benne, C.; Ryan, E.S.; Balderas, R.; et al. CTLA-4+PD-1− Memory CD4+ T Cells Critically Contribute to Viral Persistence in Antiretroviral Therapy-Suppressed, SIV-Infected Rhesus Macaques. Immunity 2017, 47, 776–788. [Google Scholar] [CrossRef] [PubMed]
  32. Van der Sluis, R.M.; Kumar, N.A.; Pascoe, R.D.; Zerbato, J.M.; Evans, V.A.; Dantanarayana, A.I.; Anderson, J.L.; Sékaly, R.P.; Fromentin, R.; Chomont, N.; et al. Combination Immune Checkpoint Blockade to Reverse HIV Latency. J. Immunol. 2020, 204, ji1901191. [Google Scholar] [CrossRef] [PubMed]
  33. Saleh, R.; Toor, S.M.; Khalaf, S.; Elkord, E. Breast Cancer Cells and PD-1/PD-L1 Blockade Upregulate the Expression of PD-1, CTLA-4, TIM-3 and LAG-3 Immune Checkpoints in CD4+ T Cells. Vaccines 2019, 7, 149. [Google Scholar] [CrossRef]
  34. Osa, A.; Uenami, T.; Koyama, S.; Fujimoto, K.; Okuzaki, D.; Takimoto, T.; Hirata, H.; Yano, Y.; Yokota, S.; Kinehara, Y.; et al. Clinical Implications of Monitoring Nivolumab Immunokinetics in Non–Small Cell Lung Cancer Patients. JCI Insight 2018, 3, 3. [Google Scholar] [CrossRef] [PubMed]
  35. Samri, A.; Durier, C.; Urrutia, A.; Sanchez, I.; Gahery-Segard, H.; Imbart, S.; Sinet, M.; Tartour, E.; Aboulker, J.P.; Autran, B.; et al. Evaluation of the interlaboratory concordance in quantification of human immunodeficiency virus-specific T cells with a gamma interferon enzyme-linked immunospot assay. Clin. Vaccine. Immunol. CVI 2006, 13, 684–697. [Google Scholar] [CrossRef]
  36. Sacre, K.; Carcelain, G.; Cassoux, N.; Fillet, A.-M.; Costagliola, D.; Vittecoq, D.; Salmon, D.; Amoura, Z.; Katlama, C.; Autran, B. Repertoire, diversity, and differentiation of specific CD8 T cells are associated with immune protection against human cytomegalovirus disease. J. Exp. Med. 2005, 201, 1999–2010. [Google Scholar] [CrossRef] [PubMed]
  37. Webster, G.; Bertoletti, A. Quantity and quality of virus-specific CD8 cell response: Relevance to the design of a therapeutic vaccine for chronic HBV infection. Mol. Immunol. 2001, 38, 467–473. [Google Scholar] [CrossRef]
  38. Webster, G.J.; Reignat, S.; Brown, D.; Ogg, G.S.; Jones, L.; Seneviratne, S.L.; Williams, R.; Dusheiko, G.; Bertoletti, A. Longitudinal analysis of CD8+ T cells specific for structural and nonstructural hepatitis B virus proteins in patients with chronic hepatitis B: Implications for immunotherapy. J. Virol. 2004, 78, 5707–5719. [Google Scholar] [CrossRef]
  39. Depla, E.; Van der Aa, A.; Livingston, B.D.; Crimi, C.; Allosery, K.; De Brabandere, V.; Krakover, J.; Murthy, S.; Huang, M.; Power, S.; et al. Rational Design of a Multiepitope Vaccine Encoding T-Lymphocyte Epitopes for Treatment of Chronic Hepatitis B Virus Infections. J. Virol. 2008, 82, 435–450. [Google Scholar] [CrossRef]
  40. Malmassari, S.; Lone, Y.C.; Zhang, M.; Transy, C.; Michel, M.L. In vivo hierarchy of immunodominant and subdominant HLA-A*0201-restricted T-cell epitopes of HBx antigen of hepatitis B virus. Microbes Infect. 2005, 7, 626–634. [Google Scholar] [CrossRef]
  41. Tsai, S.L.; Lee, T.H.; Chien, R.N.; Liao, S.K.; Lin, C.L.; Kuo, G.C.; Liaw, Y.F. A method to increase tetramer staining efficiency of CD8+ T cells with MHC–peptide complexes: Therapeutic applications in monitoring cytotoxic T lymphocyte activity during hepatitis B and C treatment. J. Immunol. Methods 2004, 285, 71–87. [Google Scholar] [CrossRef] [PubMed]
  42. Schnuriger, A.; Dominguez, S.; Guiguet, M.; Harfouch, S.; Samri, A.; Ouazene, Z.; Slama, L.; Simon, A.; Valantin, M.A.; Thibault, V.; et al. Acute hepatitis C in HIV-infected patients: Rare spontaneous clearance correlates with weak memory CD4 T-cell responses to hepatitis C virus. AIDS 2009, 23, 2079–2089. [Google Scholar] [CrossRef] [PubMed]
Cells 11 01015 g001 550
Figure 1. Anti-PD-1 therapy in PLWH with cancer does not significantly impact the biology of HIV. (A), HIV-DNA (cp/106cells) (upper panel) and HIV-DNA (log10 cp/106 cells) delta from baseline (lower panel), (B), HIV-RNA (cp/mL) (upper panel) and HIV-RNA (log10 cp/mL) delta from baseline (lower panel), (C), CD4 count (/mm3) (upper panel) and CD4 count (/mm3) delta from baseline (lower panel). Dotted lines denote decrease ≤ 0.5 log10 for CA HIV-DNA and increase ≥ 0.5 log10 for HIV-RNA. Red color denotes the RR patients represented with different shapes. EOT: end of treatment. Bonferroni corrected p-value: p ≤ 0.007, Wilcoxon-matched pairs signed Rank test. At cycle 1, n = 32; at cycle 2, n = 29; at cycle 3 or 4, n = 29; at cycle 9, n = 11; at cycle 15, n = 5; at cycle 27, n = 3; at cycle 51, n = 1; at end of treatment, n = 5. Experiments were monoplicates.
Figure 1. Anti-PD-1 therapy in PLWH with cancer does not significantly impact the biology of HIV. (A), HIV-DNA (cp/106cells) (upper panel) and HIV-DNA (log10 cp/106 cells) delta from baseline (lower panel), (B), HIV-RNA (cp/mL) (upper panel) and HIV-RNA (log10 cp/mL) delta from baseline (lower panel), (C), CD4 count (/mm3) (upper panel) and CD4 count (/mm3) delta from baseline (lower panel). Dotted lines denote decrease ≤ 0.5 log10 for CA HIV-DNA and increase ≥ 0.5 log10 for HIV-RNA. Red color denotes the RR patients represented with different shapes. EOT: end of treatment. Bonferroni corrected p-value: p ≤ 0.007, Wilcoxon-matched pairs signed Rank test. At cycle 1, n = 32; at cycle 2, n = 29; at cycle 3 or 4, n = 29; at cycle 9, n = 11; at cycle 15, n = 5; at cycle 27, n = 3; at cycle 51, n = 1; at end of treatment, n = 5. Experiments were monoplicates.
Cells 11 01015 g001
Cells 11 01015 g002 550
Figure 2. Anti-PD-1 treatment in PLWH with cancer results in T cell immunological activation. (A), Highest fold change at C2 or 3 from baseline in HLA-DR + cells among CD4+ and CD8 + T cells. (B), Highest fold change at C2 or 3 from baseline in Ki67+ cells among CD4+ and CD8+ T cells. (C), Highest fold change at C2 or 3 from baseline in CD38+ cells among CD8+ T cells. (D), Highest fold change at C2 or 3 from baseline in CD38 + HLA-DR + cells among CD8 + T cells. Red spots denote the RR patients and dotted lines denote fold change of 1. Wilcoxon- signed Rank test. FC: fold change. Experiments were monoplicates.
Figure 2. Anti-PD-1 treatment in PLWH with cancer results in T cell immunological activation. (A), Highest fold change at C2 or 3 from baseline in HLA-DR + cells among CD4+ and CD8 + T cells. (B), Highest fold change at C2 or 3 from baseline in Ki67+ cells among CD4+ and CD8+ T cells. (C), Highest fold change at C2 or 3 from baseline in CD38+ cells among CD8+ T cells. (D), Highest fold change at C2 or 3 from baseline in CD38 + HLA-DR + cells among CD8 + T cells. Red spots denote the RR patients and dotted lines denote fold change of 1. Wilcoxon- signed Rank test. FC: fold change. Experiments were monoplicates.
Cells 11 01015 g002
Cells 11 01015 g003 550
Figure 3. Anti-PD-1 therapy in PLWH with cancer induces dramatic decrease of PD-1 detection and slight and transient CTLA-4 expression increase. (A), Changes overtime in PD-1 expression on CD4 + (left) and CD8 + (right) T cells. (B), Changes overtime in CTLA-4 expression on CD4 + (left) and CD8 + (right) T cells. Bonferroni corrected p-value: p ≤ 0.007, Wilcoxon-matched pairs signed Rank test. Highest fold change at cycle 2 or 3 of (C), CTLA-4 + CD4 + (left) and CD8 + (right) T cells, and of (D), TIM-3 + CD4 + T (left) and CD8 + (right) among the patients without reservoir reduction (NoRR) and with reservoir reduction (RR). Mann–Whitney test. In each graph, red spots denote the patients RR. EOT: end of treatment. FC: fold change. At cycle 1, n = 32; at cycle 2, n =29; at cycle 3 or 4, n= 29; at cycle 9, n = 11; at cycle 15, n =5; at cycle 27, n = 3; at cycle 51, n = 1; at end of treatment, n = 5. Experiments were monoplicates.
Figure 3. Anti-PD-1 therapy in PLWH with cancer induces dramatic decrease of PD-1 detection and slight and transient CTLA-4 expression increase. (A), Changes overtime in PD-1 expression on CD4 + (left) and CD8 + (right) T cells. (B), Changes overtime in CTLA-4 expression on CD4 + (left) and CD8 + (right) T cells. Bonferroni corrected p-value: p ≤ 0.007, Wilcoxon-matched pairs signed Rank test. Highest fold change at cycle 2 or 3 of (C), CTLA-4 + CD4 + (left) and CD8 + (right) T cells, and of (D), TIM-3 + CD4 + T (left) and CD8 + (right) among the patients without reservoir reduction (NoRR) and with reservoir reduction (RR). Mann–Whitney test. In each graph, red spots denote the patients RR. EOT: end of treatment. FC: fold change. At cycle 1, n = 32; at cycle 2, n =29; at cycle 3 or 4, n= 29; at cycle 9, n = 11; at cycle 15, n =5; at cycle 27, n = 3; at cycle 51, n = 1; at end of treatment, n = 5. Experiments were monoplicates.
Cells 11 01015 g003
Cells 11 01015 g004 550
Figure 4. Anti-PD-1 therapy in PLWH with cancer does not significantly increase the in vivo frequency of HIV-specific CD8 + T cells. Thawed PBMC were stimulated during 6 h with HIV peptides (15 mers overlapping peptides covering RT, Nef and Gag) and then stained with intra-cellular anti-cytokine antibodies (anti-IFN-ɣ, anti- IL-2 and anti-TNF-α). HIV-specific CD8 + T cells were defined as IFN- ɣ producing CD8 + T cells after HIV stimulation. (A), Example dot plots of IFN-ɣ expression among CD8 + T cells after HIV stimulation at baseline and at cycle 2. (B), frequency of HIV-specific CD8+ T cells under ICB. (C), HIV-specific CD8 + T cells delta from baseline under ICB. Red colors denote the RR patients. EOT: end of treatment. At cycle 1, n = 32; at cycle 2, n =29; at cycle 3 or 4, n= 29; at cycle 9, n = 11; at cycle 15, n =5; at cycle 27, n = 3; at cycle 51, n = 1; at end of treatment, n = 5. Experiments were monoplicates.
Figure 4. Anti-PD-1 therapy in PLWH with cancer does not significantly increase the in vivo frequency of HIV-specific CD8 + T cells. Thawed PBMC were stimulated during 6 h with HIV peptides (15 mers overlapping peptides covering RT, Nef and Gag) and then stained with intra-cellular anti-cytokine antibodies (anti-IFN-ɣ, anti- IL-2 and anti-TNF-α). HIV-specific CD8 + T cells were defined as IFN- ɣ producing CD8 + T cells after HIV stimulation. (A), Example dot plots of IFN-ɣ expression among CD8 + T cells after HIV stimulation at baseline and at cycle 2. (B), frequency of HIV-specific CD8+ T cells under ICB. (C), HIV-specific CD8 + T cells delta from baseline under ICB. Red colors denote the RR patients. EOT: end of treatment. At cycle 1, n = 32; at cycle 2, n =29; at cycle 3 or 4, n= 29; at cycle 9, n = 11; at cycle 15, n =5; at cycle 27, n = 3; at cycle 51, n = 1; at end of treatment, n = 5. Experiments were monoplicates.
Cells 11 01015 g004
Cells 11 01015 g005 550
Figure 5. In PLWH with cancer, high levels of ICP expression are detected on HIV-specific T cells, without upregulation on anti-PD-1 therapy. Thawed PBMC were stimulated during 6 h with viral peptides and then stained with intra-cellular anti-cytokine antibodies (anti-IFN-ɣ, anti-IL-2 and anti-TNF-α). HIV-specific CD8 + T cells were defined as IFN- ɣ producing CD8+ T cells after HIV stimulation. (A), PD-1 expression among specific (black spots) and non-specific (blue square) CD8 + T cells at baseline depending on viral stimulations. Mann Whitney test. (B), CTLA-4, TIM-3 and LAG-3 expression among HIV-specific and non-specific CD8 + T cells after HIV stimulation (n = 18). Mann Whitney test. (C), expression of CTLA-4 (left), TIM-3 (middle) and LAG-3 (right) on HIV specific CD8 + T cells overtime. Red spots denote the patients with reservoir reduction. EOT: end of treatment. At cycle 1, n = 32; at cycle 2, n =29; at cycle 3 or 4, n= 29; at cycle 9, n = 11; at cycle 15, n =5; at cycle 27, n = 3; at cycle 51, n = 1; at end of treatment, n = 5. Experiments were monoplicates.
Figure 5. In PLWH with cancer, high levels of ICP expression are detected on HIV-specific T cells, without upregulation on anti-PD-1 therapy. Thawed PBMC were stimulated during 6 h with viral peptides and then stained with intra-cellular anti-cytokine antibodies (anti-IFN-ɣ, anti-IL-2 and anti-TNF-α). HIV-specific CD8 + T cells were defined as IFN- ɣ producing CD8+ T cells after HIV stimulation. (A), PD-1 expression among specific (black spots) and non-specific (blue square) CD8 + T cells at baseline depending on viral stimulations. Mann Whitney test. (B), CTLA-4, TIM-3 and LAG-3 expression among HIV-specific and non-specific CD8 + T cells after HIV stimulation (n = 18). Mann Whitney test. (C), expression of CTLA-4 (left), TIM-3 (middle) and LAG-3 (right) on HIV specific CD8 + T cells overtime. Red spots denote the patients with reservoir reduction. EOT: end of treatment. At cycle 1, n = 32; at cycle 2, n =29; at cycle 3 or 4, n= 29; at cycle 9, n = 11; at cycle 15, n =5; at cycle 27, n = 3; at cycle 51, n = 1; at end of treatment, n = 5. Experiments were monoplicates.
Cells 11 01015 g005
Cells 11 01015 g006 550
Figure 6. Anti-PD-1 therapy in PLWH with cancer does not significantly increase the frequency of virus-specific CD8 + cells. Thawed PBMC were stimulated during 6 h with viral peptides and then stained with intra-cellular anti-cytokine antibodies (anti-IFN-ɣ, anti-IL-2 and anti-TNF-α). Virus-specific CD8 + T cells were defined as IFN- ɣ producing CD8+ T cells after viral stimulation. Frequencies of (A), HHV-8, (B), HBV, (C), HCV, (D), EBV, and (E), CMV-specific-CD8+ T cells overtime. Delta from baseline of (F), HHV-8, (G), HBV, (H), HCV, (I), EBV, and (J), CMV-specific CD8 + T cells under ICB. Red colors denote the patients with reservoir reduction. EOT: end of treatment. Experiments were monoplicates.
Figure 6. Anti-PD-1 therapy in PLWH with cancer does not significantly increase the frequency of virus-specific CD8 + cells. Thawed PBMC were stimulated during 6 h with viral peptides and then stained with intra-cellular anti-cytokine antibodies (anti-IFN-ɣ, anti-IL-2 and anti-TNF-α). Virus-specific CD8 + T cells were defined as IFN- ɣ producing CD8+ T cells after viral stimulation. Frequencies of (A), HHV-8, (B), HBV, (C), HCV, (D), EBV, and (E), CMV-specific-CD8+ T cells overtime. Delta from baseline of (F), HHV-8, (G), HBV, (H), HCV, (I), EBV, and (J), CMV-specific CD8 + T cells under ICB. Red colors denote the patients with reservoir reduction. EOT: end of treatment. Experiments were monoplicates.
Cells 11 01015 g006
Cells 11 01015 g007 550
Figure 7. Immunovirological- profiles of the six PLWH patients treated with anti-PD-1 with a CA HIV-DNA decrease ≥ 0.5 log10. In each graph, red inverted triangles represent HIV-DNA (cp/106 cells), purple triangles represent HIV-RNA (cp/mL) and green squares represent HIV-specific CD8 + cells (% of total CD8 + T cells). Dotted lines represent CA HIV-DNA = 40 cp/106 cells, which is the detection threshold. EOT: end of treatment. The frequency of HIV-specific CD8 + T cells was not available for patient #31. Experiments were monoplicates.
Figure 7. Immunovirological- profiles of the six PLWH patients treated with anti-PD-1 with a CA HIV-DNA decrease ≥ 0.5 log10. In each graph, red inverted triangles represent HIV-DNA (cp/106 cells), purple triangles represent HIV-RNA (cp/mL) and green squares represent HIV-specific CD8 + cells (% of total CD8 + T cells). Dotted lines represent CA HIV-DNA = 40 cp/106 cells, which is the detection threshold. EOT: end of treatment. The frequency of HIV-specific CD8 + T cells was not available for patient #31. Experiments were monoplicates.
Cells 11 01015 g007
Table
Table 1. Clinical and biological characteristics at baseline.
Table 1. Clinical and biological characteristics at baseline.
PtCohortAgeSexType of CancersPv
Line		ICB TypeARTCD4 Count
(/mm3)		CD4/CD8 RatioHIV VL
(cp/mL)		HIV-DNA(cp/10 cells)HLA Typing
1CHIVA-271MNSCLC2NivoRilpivirine, Dolutegravir1830.528<40A*01:03
B*37:49
2CHIVA-268MNSCLC2NivoAbacavir, NevirapineNANA<1227A*03:11
B*07:27
3CHIVA-256MNSCLC1NivoEmtricitabine tenofovir disoproxil, Maraviroc8310.4193818A*24:29
B*38:44
4CHIVA-259MNSCLC1NivoEmtricitabine, Rilpivirine, Tenofovir alafenamide5960.9<1<40A*02:29
B*49:58
5CHIVA-265MNSCLC2NivoTenofovir, Emtricitabine, Bictegravir 4510.9 *352904A*02:29
B*07:15
6CHIVA-255FNSCLC1NivoDolutegravir, Abacavir, Lamivudine4990.61381287A*29:30
B*37:44
7CHIVA-268MNSCLC1NivoAbacavir, Efavirenz2410.7<20187A*02:11
B*39:40
8CHIVA-253FNSCLC1NivoEfavirenz, Emtricitabine, Tenofovir2910.4<2066A*02:66
B*49:52
9CHIVA-258MNSCLC1NivoDolutegravir, Abacavir, Lamivudine2490.2<20851A*01:03
B*08:51
10CHIVA-259MNSCLC1NivoLamivudine, Dolutegravir, Abacavir5830.4<1231A*02:03
B*18 27
11Onco
VIHAC		62MMelanoma0NivoElvitegravir, Emtricitabine, Tenofovir4550.8<1166A*02:26
B* 07:08
12Onco
VIHAC		69MNSCLC 0PembroEmtricitabine, Rilpivirine, Tenofovir2730.4<1<40A*02:24
B*35:57
13Onco
VIHAC		75MNSCLC1NivoLamivudine, Dolutegravir217142218A*25:31
B*40:51
14Onco
VIHAC		60FNSCLC0PembroDarunavir, Norvir, Raltegravir8882.1211749A*31:68
B*07:07
15Onco
VIHAC		63MNSCLC1NivoDolutegravir, Abacavir, Lamivudine2381.8<1<40A*03:23
B*44:53
16Onco
VIHAC		53MHL3NivoDolutegravir, Lamivudine3730.3<20173A*33:68 B*14:44
17Onco
VIHAC		53MNSCLC2NivoAbacavir, Lamivudine4050.6<1620A*02:29
B*40:49
18Onco
VIHAC		64MBladder2PembroAbacavir, Lamivudine, Nivérapine4491.1<1213A*02:31
B*07:40
19Onco
VIHAC		62MOropharynx2NivoDarunavir, Ritonavir1620.5<2080A*02:11
B*15:40
20Onco
VIHAC		58MKaposi Sarcoma4NivoDolutegravir, Abacavir, Lamivudine7282.147409A*33:68
B*14:44
21Onco
VIHAC		62MAnal2NivoDolutegravir, Lamivudine2091.2<20231A*02:24
B*35:44
22Onco
VIHAC		52MHead and neck1NivoDarunavir, Norvir, Raltegravir3690.8<20191A*30:33
B*07:15
23Onco
VIHAC		71FHead and neck1NivoDolutegravir3331.1<1<40A*02:24
B*44:50
24Onco
VIHAC		63MEye2CemiBictegravir, Emtricitabne, Tenofovir450.229<40A*02:33
B*14:53
25Onco
VIHAC		70MMelanoma0PembroEmtricitabine, Tenofovir, Névirapine4340.5<1181A*01:03
B*07:51
26Onco
VIHAC		56MNSCLC2PembroEmtricitabine, Tenofovir, Darunavir, Ritonavir4240.8<20166A*11:11
B*15:27
27Onco
VIHAC		62MBladder1PembroEfavirenz, Emtricitabine, Tenofovir9151,4<2099A*24:24
B*44:44
28Onco
VIHAC		58MNSCLC0PembroDarunavir, Doletugravir, Ritonavir, Tenofovir1920,4<173A*02:32
B*44:51
29Onco
VIHAC		62MNSCLC1NivoRaltegravir, Emtricitabine, Tenofovir5340.3<199NA
30Onco
VIHAC		60MBladder2PembroBictegravir, Emtricitabne, Tenofovir9690.6<1363NA
31Onco
VIHAC		59MNSCLC1NivoDarunavir, Ritonavir6991<1251NA
32Onco
VIHAC		60MNSCLC0PembroBictegravir, Emtricitabne, Tenofovir1690.4<159NA
allOncoVIHAC
69%		61M 88%NSCLC 63%
Bladder
9%		1Nivo 69%
Pembro
28%		 369120184
Abbreviations: Pt, patient; Pv, previous; ICB, immune checkpoint blockade; ART, antiretroviral therapy; M: male; F, female; NSCLC, non-small cell lung cancer; HL: Hodgkin lymphoma, Nivo, nivolumab; Pembro, pembrolizumab; Cemi, cemiplimab; NA, Not Available, %, percentage. In the line named “all”, absolute values are medians.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Baron, M.; Soulié, C.; Lavolé, A.; Assoumou, L.; Abbar, B.; Fouquet, B.; Rousseau, A.; Veyri, M.; Samri, A.; Makinson, A.; et al. Impact of Anti PD-1 Immunotherapy on HIV Reservoir and Anti-Viral Immune Responses in People Living with HIV and Cancer. Cells 2022, 11, 1015. https://doi.org/10.3390/cells11061015

AMA Style

Baron M, Soulié C, Lavolé A, Assoumou L, Abbar B, Fouquet B, Rousseau A, Veyri M, Samri A, Makinson A, et al. Impact of Anti PD-1 Immunotherapy on HIV Reservoir and Anti-Viral Immune Responses in People Living with HIV and Cancer. Cells. 2022; 11(6):1015. https://doi.org/10.3390/cells11061015

Chicago/Turabian Style

Baron, Marine, Cathia Soulié, Armelle Lavolé, Lambert Assoumou, Baptiste Abbar, Baptiste Fouquet, Alice Rousseau, Marianne Veyri, Assia Samri, Alain Makinson, and et al. 2022. "Impact of Anti PD-1 Immunotherapy on HIV Reservoir and Anti-Viral Immune Responses in People Living with HIV and Cancer" Cells 11, no. 6: 1015. https://doi.org/10.3390/cells11061015

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop