Next Article in Journal
Changes in Cystoscopic Findings after Intravesical Hyaluronic Acid Instillation Therapy in Patients with Interstitial Cystitis
Next Article in Special Issue
Clusterin Plasma Concentrations Are Decreased in Sepsis and Inversely Correlated with Established Markers of Inflammation
Previous Article in Journal
Dentigerous Cysts with Diverse Radiological Presentation Highlighting Diagnostic Challenges
Previous Article in Special Issue
Non-Coding RNA Networks as Potential Novel Biomarker and Therapeutic Target for Sepsis and Sepsis-Related Multi-Organ Failure
 
 
Correction published on 26 April 2023, see Diagnostics 2023, 13(9), 1555.
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

The Expression Levels and Concentrations of PD-1 and PD-L1 Proteins in Septic Patients: A Systematic Review

by
Mutiara Indah Sari
1,* and
Syafruddin Ilyas
2
1
Department of Biochemistry, Faculty of Medicine, Universitas Sumatera Utara, Medan 20155, Indonesia
2
Department of Biology, Faculty of Mathematics and Natural Sciences, Universitas Sumatera Utara, Medan 20155, Indonesia
*
Author to whom correspondence should be addressed.
Diagnostics 2022, 12(8), 2004; https://doi.org/10.3390/diagnostics12082004
Submission received: 24 June 2022 / Revised: 12 August 2022 / Accepted: 15 August 2022 / Published: 19 August 2022 / Corrected: 26 April 2023
(This article belongs to the Special Issue Biomarkers of Sepsis)

Abstract

:
Sepsis is a series of life-threatening organ dysfunction caused by an impaired host response to infection. A large number of molecular studies of sepsis have revealed complex interactions between infectious agents and hosts that result in heterogeneous manifestations of sepsis. Sepsis can cause immunosuppression and increase the expression of checkpoint inhibitor molecules, including programmed death protein (PD-1) and programmed death ligand 1 (PD-L1), and thus PD-1 and PD-L1 are thought to be useful as diagnostic and prognostic tools for sepsis. PD-1 is an inhibitor of both adaptive and innate immune responses, and is expressed on activated T lymphocytes, natural killer (NK) cells, B lymphocytes, macrophages, dendritic cells (DCs), and monocytes, whereas PD-L1 is expressed on macrophages, some activated T and B cells, and mesenchymal stem cells as well as various non-hematopoietic cells. This systematic review aims to assess the PD-1 and PD-L1 protein expression levels and concentrations in septic and other infectious patients.

1. Introduction

Sepsis is a series of life-threatening organ dysfunction caused by an impaired host response to infection. One study states the overall incidence rate of sepsis is 288 per 100,000 person/year, whereas for severe sepsis it is 148 per 100,000 person/year [1,2]. Based on the data collected from high-income countries, the incidence of sepsis is estimated to be 31.5 million cases annually worldwide. Of these cases, 19.4 million are severe sepsis, and 5.3 million resulted in death each year [3].
The understanding of the molecular pathobiology and immunology of sepsis is constantly growing. It was previously assumed that the hemodynamic manifestations of sepsis were primarily associated with excessive immune responses to specific pathogens [3]. However, a large number of molecular studies of sepsis have revealed a much more complicated process that results in the heterogeneous manifestations of sepsis [4].
In sepsis, the first step of the host response to pathogens is the activation of innate immune cells, which mainly consist of neutrophils, macrophages, monocytes, and natural killer (NK) cells [5]. This occurs through the binding of pathogen-associated molecular patterns (PAMPs), such as bacterial endotoxin and fungal β-glucan to specific pattern recognition receptors, on these cells [4]. Another source of such interactions is damage-associated molecular patterns (DAMPs) which may be intracellular materials or molecules released from dead or damaged host cells, such as mitochondrial ATP and DNA [4,6]. They bind to specific receptors on monocytes and macrophages such as toll-like receptors (TLRs), C-type lectin receptors, NOD-like receptors (nucleotide-binding oligomerization domain) and RIG-1-like (retinoic acid-inducible gene 1) [4,6]. This results in the activation of intracellular signal transduction pathways leading to the transcription and release of proinflammatory cytokines such as TNFα, IL-1, and IL-6 [4].
A prolonged immunosuppression state often follows and replaces the early proinflammatory state of sepsis. The number of T cells (helper and cytotoxic) decreases as a result of apoptosis, and there is a decreased response to inflammatory cytokines [7]. Postmortem studies of deceased septic patients from the ICU showed a global depletion of CD4+ and CD8+ T cells, most of which were found mainly in lymphoid organs such as the spleen [8]. Studies have also shown decreased production of important cytokines such as IL-6 and TNF in response to endotoxins [9]. In septic patients, neutrophils are found to express fewer chemokine receptors, and there is decreased chemotaxis in response to IL-8 [10].
Sepsis-induced immunosuppression increases the expression of checkpoint inhibitor molecules, such as programmed death protein (PD-1), programmed death ligand 1 (PD-L1), B and T lymphocyte attenuator (BTLA), T cell membrane protein-3 (TIM-3), LAG3, etc. [11]. PD-1 is an inhibitor of both adaptive and innate immune responses and is expressed on activated T lymphocytes, NK cells, and B lymphocytes, macrophages, dendritic cells (DC), and monocytes, as well as in hematopoietic and non-hematopoietic cells. PD-1 plays opposite roles. On one side, PD-1 negatively regulates lymphocyte activation and function, reducing the harmful immune responses and maintaining immune tolerance. On the other hand, PD-1 interferes with the protective immune response and causes the development of malignant cells [12,13,14,15,16,17].
PD-1 ability to suppress T cell activation depends on the phosphorylation of immunoreceptor tyrosine-based switch motif (ITSM), which leads to phosphorylation of downstream effector molecules that play a negative regulatory role, and inhibits T cell proliferation and cytokine production [18]. The postmortem studies of septic patients revealed severe dysfunction in the spleen cells and the lungs also exhibit immunosuppressive effects, which may be due to self-programmed immune cell death induced by immunosuppression [19]. Inhibitory receptors expression on the surface of T cells is upregulated by chronic antigen stimulation, which is an important immunosuppressive mechanism in sepsis as well [20].
T cell receptor signaling and cytokines, such as IL-2, IL-7, and interferon type I (IFN), can induce PD-1 expression in T cells. PD-1 expression is especially increased in activated T cells and can be regulated within 24 h depending on the strength or concentration of the stimulus. The inhibitory effect of PD-1 is mediated by the PD-L1 and PD-L2 ligands [21].
PD-L1 is expressed on macrophages, some activated T and B cells, and mesenchymal stem cells as well as various non-hematopoietic cells, such as hepatocytes, vascular endothelial cells, epithelial cells, myocytes, pancreatic islet cells, and astrocytes [4,21]. Several studies have shown that increased expression of PD-1 and PD-L1 by T cells, monocytes, and neutrophils, and immunosuppression due to sepsis is one of the main risk factors for death in septic patients. Therefore, immunosuppression in sepsis provides a new understanding as well as a promising new target for treatment and prediction of its prognosis [13].
The immunoregulatory role played by PD-1 and PD-L1 during the inflammatory process in sepsis activates the PD-1 and PD-L1 pathway mechanisms to evade the body’s defenses (immunosuppression). Increased expression of PD-1 and PD-L1 molecules, and decreased CD28 molecules and CD86 induced immunosuppression in mice with sepsis. The expression of PD-1 on T lymphocytes, and CD14+ CD16+ monocytes showed a significant increase in septic patients along with the PD-L1 on CD14+ CD16+ monocytes [22]. Therefore, PD-1 may play an important role in the regulation of T lymphocytes and monocytes in septic patients, especially CD16+ monocytes associated with inflammation. Studies have also shown that PD-1 levels are found in patients and experimental animals with impaired immunity such as autoimmune diseases, chronic infections, and cancer [23,24,25].
In this review, we conduct a systematic review of studies of the expression levels of PD-1 and PD-L1 and the concentrations of sPD-1 and sPD-L1 proteins in patients with sepsis and other systemic infectious diseases. We hypothesize that published studies in their entirety would report the benefits of examining PD-1 and PD-L1 expression for both the diagnosis and prognosis of sepsis.

2. Methods

The study was systematically reviewed following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines [26].
A systematic online search was carried out from 15 April 2022 to 26 May 2022 in 4 search engines, namely PubMed, Springer Link, Wiley Online Library, and Proquest. The search was carried out using the keywords ((Sepsis OR Septicemia OR Pyemia OR Pyaemia OR Pyohemia OR Blood Poisoning OR Bloodstream Infection) AND (Healthy volunteer OR Healthy participant OR Healthy subject OR Human volunteer OR Normal volunteer OR Control Group) AND (PD 1 OR PD-1 OR PD1 OR Programmed Death 1 OR Programmed Cell Death Protein 1 OR Programmed Cell Death 1 Protein)) AND ((Sepsis OR Septicemia OR Pyemia OR Pyaemia OR Pyohemia OR Blood Poisoning OR Bloodstream Infection) AND (Healthy volunteer OR Healthy participant OR Healthy subject OR Human volunteer OR Normal volunteer OR Control Group) AND (PD-L1 OR PD L1 OR PDL1 OR Programmed Death Ligand 1 OR Programmed Cell Death 1 Ligand 1)). The data taken are the name of the first author, year of publication, research design, study output, and conclusion. After excluding the articles that did not match the title, abstract, and duplication, all articles that met the inclusion criteria were assessed for full text.
Title and abstract screening, data extraction, and risk assessment of bias were performed. Samples were taken based on PRISMA guidelines with the inclusion criteria of this study, namely research with septic patients or other infectious patients who develop sepsis and there is a healthy control group, with the expression level or protein level of PD-1 or PD-L1, and journals in English. The exclusion criteria are review articles, comments, case reports, dissertations or master’s theses, research conducted on animals, research older than 10 years, and research conducted on children.
The relevance of the study was determined using the inclusion criteria presented in Table 1. The study had to meet the following relevant criteria: population, intervention, comparison, and desired outcome. All screening stages use predefined questions—formulated following the PICO—to select the publications that cover the scope of the review.

3. Results

The screening results of the search are presented in the PRISMA diagram as shown in Figure 1. By entering keywords and conducting a systematic search, we found 6099 studies across the search engines. In title and abstract screening, 6084 studies were excluded with the reasons: 5735 studies were irrelevant to this study, 337 studies were in the form of reviews, 11 studies were animal studies, and 1 study was a duplicate. After full text screening, 2 studies in a language other than English were excluded. The final result of the search is 13 studies that met the inclusion criteria in this systematic review, with 4 studies focused on PD-1, 2 studies focused on PD-L1, and 7 studies focused on PD-1 and PD-L1. The deadline for the search was 26 May 2022. The included studies are summarized in Table 2 and Table 3.
Of the 11 publications about PD-1, 9 found higher levels of PD-1 in septic patients, while 7 out of 9 publications about PD-L1 found higher levels of PD-L1 in septic patients. One of the two studies focuses on the PD-1 and PD-L1 in acute pancreatitis patients with and without infectious complications (IC), which includes infection, bacteremia, pneumonia, infected ascites, or urosepsis during admission and/or during the 90-day follow-up. This study finds significantly higher PD-1 and PD-L1 levels in acute pancreatitis patients with IC compared to non-IC on day 1 and day 3 (D1 p < 0.05; D3 p < 0.05). Meanwhile, the other study finds lower PD-1 levels in septic patients and no significant difference in the PD-L1 levels.
There are 7 publications studying the PD-1 and PD-L1 between survivors and non-survivors of sepsis, with 3 studies focused on PD-1, 1 study focused on PD-L1, and 3 studies focused on PD-1 and PD-L1. One study finds no significant difference in the sPD-1 levels between non-survivors and survivors of sepsis, but there is a significant increase in the sPD-L1 levels in the non-survivors group compared to the survivors group.

4. Discussion

As a member of the B7-CD28 superfamily, PD-1 plays an inhibitory role [39,40]. PD1 and PD-L1 can inhibit T and B cell functions such as cytokine production and cytotoxic activity and are thus important in immune regulation [41,42,43]. In addition, PD-1 also causes T cell “fatigue” so that T cells are dysfunctional, vulnerable to apoptosis, and unable to participate in an effective immune response, resulting in chronic viral infection [44]. Several studies have reported that the expression of PD-1 in lymphocytes and PD-L1 in monocytes causes immunosuppression during sepsis [34], leading to significantly increased levels in patients with sepsis [15,44,45].
PD-1 exists in two forms: a membrane-bound form and a soluble form (sPD-1). Like cytokines, sPD-1 circulates in the bloodstream to carry out its functions in the immune response and plays a major role in maintaining the balance of the PD-1/PD-L1 signaling pathway. Previous studies have shown that sPD-1 enhances T cell response through inhibition of the PD-1/PD-L1 signaling pathway and that excess sPD-1 can lead to immune imbalances in which auto-reactive T cells cannot function effectively; this can lead to pathological immune impairment [46]. Patients and experimental animals with autoimmune diseases, chronic viral infections, and cancer are found to have higher sPD-1 levels; however, correlations between sPD-1 levels and severe sepsis or septic shock are rarely reported [24,25,47].
This systematic review study strengthens the results of previous studies and existing theories, namely the increased PD-1 levels in septic patients compared to patients with non-infectious inflammation and healthy controls [27,28,29,31,32,33]. It was found that septic patients, especially those with severe sepsis and septic shock, have significantly higher levels of PD-1 expression in CD4+ or CD8+ T cells, PD-L1 expression in monocytes, sPD-1, and sPD-L1 levels [27,29,31,32,35,36,37], but there was no significant difference between the SIRS (Systemic Inflammatory Response Syndrome) group and healthy controls [34]. However, different results were obtained by Chang et al., 2014, which stated that increased PD-1 expression occurs only in CD8 T cells compared with non-septic patients, which is due to lymphocytes in septic patients having decreased IFN-γ and IL-2 and increased expression of PD-1 CD8 T cells [14]. This shows that both forms of PD-1 (membrane-bound and soluble form/sPD-1) are involved in the immune regulation of sepsis [29]. A secondary increase in sPD-1/sPD-L1 can occur when there is an increase in membrane-bound PD-1/PD-L1. When immune regulation is not controlled, the excessive sPD-1 can function as an antibody or a natural signaling inhibitor to block the PD-1/PD-L1 pathway, leading to impaired T cells activation and proliferation. sPD-1 can competitively bind to the PD-1 (PD-L1 and PD-L2) ligand, blocking the interaction on the surface of the cell membrane [24,48]. Therefore, a significantly increased level of sPD-1 may indicate more severe immunity impairment in patients with sepsis. This is supported by the results of research by Zhao et al., 2017 which stated that the more severe the sepsis is, the higher the levels of sPD-1/sPD-L1 will be. Increased sPD-1/sPD-L1 levels may indicate immune dysfunction in patients with severe sepsis or septic shock [34]. CD4+ PD-1+ expression levels and high heart rate are potential risk factors for sepsis [27].
Arens et al., 2016 found the opposite for PD-1 in septic patients in the long term. PD-1 levels were lower in septic patients compared to controls, while there were no significant differences in the PD-L1 levels between the two groups. However, BTLA receptor expression shows an inclination towards upregulation. As both are members of important negative regulators of T cell function, there is no clear conclusion. This may also be due to the small sample size of this study [30].
Among the septic patient groups, non-survivors showed higher levels of sPD-1 and sPD-L1 in peripheral blood, expression of PD-1 on CD4+ T cells and CD8+ T cells, and PD-L1 expression in monocytes than the survivors [31,32,35]. sPD-1 and sPD-L1 levels were positively correlated with the severity of sepsis. As the disease severity decreased in the survivor group, sPD-1 levels decreased in the first week. sPD-1 levels were an independent predictive factor for 28-day mortality on both day 1 and day 7. The prognostic evaluation of patients can be improved through monitoring sPD-1 levels during the first week of ICU stay [29].
This is supported by Huang et al., 2009 and Brahmamdam et al., 2010. A mouse model of sepsis shows a higher PD-1 expression level on the surface of peripheral blood macrophages and monocytes [47], and PD-1 antagonists significantly improved the survival of mice with severe sepsis or septic shock [49]. PD-1 overexpression in circulating T cells in septic patients is associated with decreased T cell proliferative capacity, increased secondary nosocomial infections, and death [47].
Different results were obtained by Liu et al., 2016. Non-survivors had significantly increased sPD-L1 levels compared to survivors (p < 0.05), but no statistical difference was found in serum sPD-1 levels between non-survivors and survivors (p > 0.05). Therefore, serum sPD-L1 may play a bigger role in sepsis compared to sPD-1 [35].
The percentage of CD4+ lymphocytes expressing PD-1 and CD14+ monocytes expressing PD-L1 on day 1 and day 3 was higher in acute pancreatitis patients compared to healthy controls and higher in the IC group of acute pancreatitis patients compared to non-IC [36]. This is supported by the theory that infectious factors play a role in the release of sPD-L1, which means that sPD-L1 is upregulated during infection and inflammation. For example, monocyte-derived DCs produced more sPD-L1 after in vitro experiments of LPS and TNF-α stimulation. Increased sPD-L1 was associated with increased levels of inflammation (CRP, PCT, WBC) and decreased immunological parameters (CD3+, CD4+ and CD8+). However, sPD-L1 levels decreased on the third day and decreased significantly on the seventh day compared to the first day of sepsis diagnosis. Compared to sepsis, septic patients had higher sPD-L1 levels compared to healthy controls, acute pancreatitis patients, and acute appendicitis, indicating that sPD-L1 is important for both identifying sepsis and assessing the severity of infection [33].
Liver cirrhosis (LC) patients show overexpression of PD-1 and PD-L1, which contribute to sepsis-associated immunosuppression and play an important role in bacterial infection [38]. Several other studies have also found increased levels of PD-1/PD-L1 and sPD-1/sPD-L1 in patients with liver diseases such as alcohol-related liver disease (ALD) [50] and chronic hepatitis B virus (CHBV) [46]. In this study, monocytic PD-L1 is excessive in LC patients, particularly those with severe complications of sepsis. Increased monocytic PD-L1 expression may be associated with immunosuppression and contribute to increased mortality in LC+ SS patients, though further investigation is needed for this hypothesis [38].
Jia et al., 2016 showed that both T2DM (type 2 diabetes mellitus) patients and patients with severe sepsis had decreased immunity. Compared to healthy controls, PD-1 levels in peripheral blood T lymphocytes increased in both T2DM and severe sepsis patients. Patients with severe sepsis also showed higher PD-1 levels than T2DM patients. Patients with severe sepsis and T2DM had similar T cell PD-1+ CD4+ and T cell PD-1+ CD8+ expression and mortality rates in patients with only severe sepsis. This study also showed that T2DM increased neither the 28-day mortality in patients with severe sepsis nor the expression of PD-1 on T cells. Non-survivors also had higher PD-1 levels on T cells which may be associated with more severe immunosuppression. The SS group and the SS+ T2DM group did not show a significant difference in the expression of PD-1 in T cells and instead showed a slight increase compared to healthy controls, likely because the increase in PD-1 in T2DM was less pronounced compared to severe sepsis. Septic patients, with or without T2DM, still had increased expression levels and concentrations of PD-1 [28].

5. Conclusions

Septic patients, especially those with severe sepsis and septic shock, had significantly higher PD-1 expression levels on CD4+ or CD8+ T cells, PD-L1 expression on monocytes, sPD-1, and sPD-L1 levels compared to patients with non-septic infections, non-infectious inflammation, and a healthy control group. Meanwhile, non-survivors showed higher levels of sPD-1 and sPD-L1 in peripheral blood, expression of PD-1 on CD4+ T cells and CD8+ T cells, and PD-L1 expression in monocytes than survivors. Blocking PD-1 by using inhibitory antibodies has been shown to restore T cell function, increase the antiviral response in T cells, decrease viral load in sepsis, and decrease apoptosis.

Author Contributions

Conceptualization, M.I.S.; formal analysis, M.I.S. and S.I.; data curation, M.I.S. and S.I.; writing—original draft preparation, M.I.S.; supervision, M.I.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Esposito, S.; De Simone, G.; Boccia, G.; De Caro, F.; Pagliano, P. Sepsis and septic shock: New definitions, new diagnostic and therapeutic approaches. J. Glob. Antimicrob. Resist. 2017, 10, 204–212. [Google Scholar] [CrossRef] [PubMed]
  2. Singer, M.; Deutschman, C.S.; Seymour, C.W.; Shankar-Hari, M.; Annane, D.; Bauer, M.; Bellomo, R.; Bernard, G.R.; Chiche, J.D.; Coopersmith, C.M.; et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016, 315, 801–810. [Google Scholar] [CrossRef] [PubMed]
  3. Purba, A.K.R.; Mariana, N.; Aliska, G.; Wijaya, S.H.; Wulandari, R.R.; Hadi, U.; Hamzah; Nugroho, C.W.; Postma, M.J. The burden and costs of sepsis and reimbursement of its treatment in a developing country: An observational study on focal infections in Indonesia. Int. J. Infect. Dis. 2020, 96, 211–218. [Google Scholar] [CrossRef] [PubMed]
  4. Gyawali, B.; Ramakrishna, K.; Dhamoon, A.S. Sepsis: The evolution in definition, pathophysiology, and management. SAGE Open Med. 2019, 7, 2050312119835043. [Google Scholar] [CrossRef]
  5. Vincent, J.L.; Moreno, R.; Takala, J.; De Mendonça, A.; Bruining, H.; Reinhart, C.K.; Suter, P.M.; Thijs, L.G. Working group on sepsis-related problems of the European society of intensive care medicine: The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. Intensive Care Med. 1996, 22, 707–710. [Google Scholar] [CrossRef]
  6. Relja, B.; Land, W.G. Damage-associated molecular patterns in trauma. Eur. J. Trauma Emerg. Surg. 2020, 46, 751–775. [Google Scholar] [CrossRef]
  7. Eckle, I.; Seitz, R.; Egbring, R.; Kolb, G.; Havemann, K. Protein C degradation in vitro by neutrophil elastase. Biol. Chem. Hoppe Seyler 1991, 372, 1007–1013. [Google Scholar] [CrossRef]
  8. Biemond, B.J.; Levi, M.; Cate, H.T.; van der Poll, T.; Büller, H.R.; Hack, C.E.; Cate, J.W.T. Plasminogen activator and plasminogen activator inhibitor I—Release during experimental endotoxaemia in chimpanzees: Effect of interventions in the cytokine and coagulation cascades. Clin. Sci. 1995, 88, 587–594. [Google Scholar] [CrossRef]
  9. Hotchkiss, R.S.; Tinsley, K.W.; Swanson, P.E.; Schmieg REJr Hui, J.J.; Chang, K.C.; Osborne, D.F.; Freeman, B.D.; Cobb, J.P.; Buchman, T.G.; Karl, I.E. Sepsis-induced apoptosis causes progressive profound depletion of B and CD4+ T lymphocytes in humans. J. Immunol. 2001, 166, 6952–6963. [Google Scholar] [CrossRef]
  10. Heagy, W.; Hansen, C.; Nieman, K.; Cohen, M.; Richardson, C.; Rodriguez, J.L.; West, M.A. Impaired ex vivo lipopolysaccharide-stimulated whole blood tumor necrosis factor production may identify “septic” intensive care unit patients. Shock 2000, 14, 271–276. [Google Scholar] [CrossRef]
  11. Patera, A.C.; Drewry, A.M.; Chang, K.; Beiter, E.R.; Osborne, D.; Hotchkiss, R.S. Frontline Science: Defects in immune function in patients with sepsis are associated with PD-1 or PD-L1 expression and can be restored by antibodies targeting PD-1 or PD-L1. J. Leukoc. Biol. 2016, 100, 1239–1254. [Google Scholar] [CrossRef] [PubMed]
  12. Han, Y.; Liu, D.; Li, L. PD-1/PD-L1 pathway: Current researches in cancer. Am. J. Cancer Res. 2020, 10, 727–742. [Google Scholar] [PubMed]
  13. Wilson, J.K.; Zhao, Y.; Singer, M.; Spencer, J.; Shankar-Hari, M. Lymphocyte subset expression and serum concentrations of PD-1/PD-L1 in sepsis-pilot study. Crit. Care 2018, 22, 1–7. [Google Scholar] [CrossRef] [PubMed]
  14. Turnis, M.E.; Andrews, L.P.; Vignali, D.A. Inhibitory receptors as targets for cancer immunotherapy. Eur. J. Immunol. 2015, 45, 1892–1905. [Google Scholar] [CrossRef]
  15. Chang, K.; Svabek, C.; Vazquez-Guillamet, C.; Sato, B.; Rasche, D.; Wilson, S.; Robbins, P.; Ulbrandt, N.; Suzich, J.; Green, J.; et al. Targeting the programmed cell death 1: Programmed cell death ligand 1 pathway reverses T cell exhaustion in patients with sepsis. Crit. Care 2014, 18, R3. [Google Scholar] [CrossRef]
  16. Guillerey, C.; Huntington, N.D.; Smyth, M.J. Targeting natural killer cells in cancer immunotherapy. Nat. Immunol. 2016, 17, 1025–1036. [Google Scholar] [CrossRef]
  17. Keir, M.E.; Francisco, L.M.; Sharpe, A.H. PD-1 and its ligands in T-cell immunity. Curr. Opin. Immunol. 2007, 19, 309–314. [Google Scholar] [CrossRef]
  18. Venet, F.; Monneret, G. Advances in the understanding and treatment of sepsis-induced immunosuppression. Nat. Rev. Nephrol. 2018, 14, 121–137. [Google Scholar] [CrossRef]
  19. Holub, M.; Dzupova, O.; Ruzkova, M.; Stranikova, A.; Bartakova, E.; Maca, J.; Benes, J.; Herwald, H.; Beran, O. Selected Biomarkers Correlate with the Origin and Severity of Sepsis. Mediat. Inflamm. 2018, 2018, 7028267. [Google Scholar] [CrossRef]
  20. Okazaki, T.; Maeda, A.; Nishimura, H.; Kurosaki, T.; Honjo, T. PD-1 immunoreceptor inhibits B cell receptor-mediated signaling by recruiting src homology 2-domain-containing tyrosine phosphatase 2 to phosphotyrosine. Proc. Natl. Acad. Sci. USA 2001, 98, 13866–13871. [Google Scholar] [CrossRef]
  21. Jubel, J.M.; Barbati, Z.R.; Burger, C.; Wirtz, D.C.; Schildberg, F.A. The role of PD-1 in acute and chronic infection. Front. Immunol. 2020, 11, 487. [Google Scholar] [CrossRef] [PubMed]
  22. Xia, Q.; Wei, L.; Zhang, Y.; Sheng, J.; Wu, W.; Zhang, Y. Immune checkpoint receptors Tim-3 and PD-1 regulate monocyte and T lymphocyte function in septic patients. Mediat. Inflamm. 2018, 2018, 1632902. [Google Scholar] [CrossRef] [PubMed]
  23. Greisen, S.R.; Rasmussen, T.K.; Stengaard-Pedersen, K.; Hetland, M.L.; Hørslev Petersen, K.; Hvid, M.; Deleuran, B. Increased soluble programmed death-1 (sPD-1) is associated with disease activity and radiographic progression in early rheumatoid arthritis. Scand. J. Rheumatol. 2014, 43, 101–108. [Google Scholar] [CrossRef] [PubMed]
  24. Cheng, H.Y.; Kang, P.J.; Chuang, Y.H.; Wang, Y.H.; Jan, M.C.; Wu, C.F.; Lin, C.L.; Liu, C.J.; Liaw, Y.F.; Lin, S.M.; et al. Circulating programmed death-1 as a marker for sustained high hepatitis B viral load and risk of hepatocellular carcinoma. PLoS ONE 2014, 9, e95870. [Google Scholar] [CrossRef]
  25. Xiao, H.; Huang, B.; Yuan, Y.; Li, D.; Han, L.F.; Liu, Y.; Gong, W.; Wu, F.H.; Zhang, G.M.; Feng, Z.H. Soluble PD-l facilitates 4-1BBL-triggered antitumor immunity against murine H22 hepatocarcinoma in vivo. Clin. Cancer Res. 2007, 13, 1823–1830. [Google Scholar] [CrossRef]
  26. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Prisma Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef]
  27. Gao, D.N.; Yang, Z.X.; Qi, Q.H. Roles of PD-1, Tim-3 and CTLA-4 in immunoregulation in regulatory T cells among patients with sepsis. Int. J. Clin. Exp. Med. 2015, 8, 18998–19005. [Google Scholar]
  28. Jia, Y.; Zhao, Y.; Li, C.; Shao, R. The Expression of Programmed Death-1 on CD4+ and CD8+ T Lymphocytes in Patients with Type 2 Diabetes and Severe Sepsis. PLoS ONE 2016, 11, e0159383. [Google Scholar] [CrossRef]
  29. Zhao, Y.; Jia, Y.; Li, C.; Shao, R.; Fang, Y. Predictive Value of Soluble Programmed Death-1 for Severe Sepsis and Septic Shock During the First Week in an Intensive Care Unit. Shock 2019, 51, 289–297. [Google Scholar] [CrossRef]
  30. Arens, C.; Bajwa, S.A.; Koch, C.; Siegler, B.H.; Schneck, E.; Hecker, A.; Weiterer, S.; Lichtenstern, C.; Weigand, M.A.; Uhle, F. Sepsis-induced long-term immune paralysis-results of a descriptive, explorative study. Crit. Care 2016, 20, 93. [Google Scholar] [CrossRef]
  31. Pan, T.; Liu, Z.; Yin, J.; Zhou, T.; Liu, J.; Qu, H. Notch Signaling Pathway Was Involved in Regulating Programmed Cell Death 1 Expression during Sepsis-Induced Immunosuppression. Mediat. Inflamm. 2015, 2015, 539841. [Google Scholar] [CrossRef] [PubMed]
  32. Shao, R.; Fang, Y.; Yu, H.; Zhao, L.; Jiang, Z.; Li, C.S. Monocyte programmed death ligand-1 expression after 3-4 days of sepsis is associated with risk stratification and mortality in septic patients: A prospective cohort study. Crit. Care 2016, 20, 124. [Google Scholar] [CrossRef] [PubMed]
  33. Pan, T.; Zhou, T.; Li, L.; Liu, Z.; Chen, Y.; Mao, E.; Li, M.; Qu, H.; Liu, J. Monocyte programmed death ligand-1 expression is an early marker for predicting infectious complications in acute pancreatitis. Crit. Care 2017, 21, 186. [Google Scholar] [CrossRef] [PubMed]
  34. Zhao, Y.; Jia, Y.; Li, C.; Fang, Y.; Shao, R. The risk stratification and prognostic evaluation of soluble programmed death-1 on patients with sepsis in emergency department. Am. J. Emerg. Med. 2018, 36, 43–48. [Google Scholar] [CrossRef] [PubMed]
  35. Liu, M.; Zhang, X.; Chen, H.; Wang, G.; Zhang, J.; Dong, P.; Liu, Y.; An, S.; Wang, L. Serum sPD-L1, Upregulated in Sepsis, May Reflect Disease Severity and Clinical Outcomes in Septic Patients. Scand. J. Immunol. 2017, 85, 66–72. [Google Scholar] [CrossRef]
  36. Liu, Q.; An, L.; Qi, Z.; Zhao, Y.; Li, C. Increased Expression of Programmed Cell Death-1 in Regulatory T Cells of Patients with Severe Sepsis and Septic Shock: An Observational Clinical Study. Scand. J. Immunol. 2017, 86, 408–417. [Google Scholar] [CrossRef]
  37. Sun, S.; Chen, Y.; Liu, Z.; Tian, R.; Liu, J.; Chen, E.; Mao, E.; Pan, T.; Qu, H. Serum-soluble PD-L1 may be a potential diagnostic biomarker in sepsis. Scand. J. Immunol. 2021, 94, e13049. [Google Scholar] [CrossRef]
  38. Lu, Y.; Wang, G.; Li, C. Expression of peripheral monocytic programmed death ligand-1 in severe sepsis combined with HBV-related cirrhosis. A pilot observational study. Cent. Eur. J. Immunol. 2021, 46, 217–224. [Google Scholar] [CrossRef]
  39. Brown, K.E.; Freeman, G.J.; Wherry, E.J.; Sharpe, A.H. Role of PD-1 in regulating acute infections. Curr. Opin. Immunol. 2010, 22, 397–401. [Google Scholar] [CrossRef]
  40. Sharpe, A.H.; Wherry, E.J.; Ahmed, R.; Freeman, G.J. The function of programmed cell death 1 and its ligands in regulating autoimmunity and infection. Nat. Immunol. 2007, 8, 239–245. [Google Scholar] [CrossRef]
  41. Freeman, G.J.; Long, A.J.; Iwai, Y.; Bourque, K.; Chernova, T.; Nishimura, H.; Fitz, L.J.; Malenkovich, N.; Okazaki, T.; Byrne, M.C.; et al. Engagement of the PD-1 immunoinhibitory receptor by a novel B7 family member leads to negative regulation of lymphocyte activation. J. Exp. Med. 2000, 192, 1027–1034. [Google Scholar] [CrossRef] [PubMed]
  42. Okazaki, T.; Honjo, T. PD-1 and PD-1 ligands: From discovery to clinical application. Int. Immunol. 2007, 19, 813–824. [Google Scholar] [CrossRef] [PubMed]
  43. Gianchecchi, E.; Delfino, D.V.; Fierabracci, A. Recent insights into the role of the PD-1/PD-L1 pathway in immunological tolerance and autoimmunity. Autoimmun. Rev. 2013, 12, 1091–1100. [Google Scholar] [CrossRef]
  44. Guignant, C.; Lepape, A.; Huang, X.; Kherouf, H.; Denis, L.; Poitevin, F.; Malcus, C.; Chéron, A.; Allaouchiche, B.; Gueyffier, F.; et al. Programmed death-1 levels correlate with increased mortality, nosocomial infection and immune dysfunctions in septic shock patients. Crit. Care 2011, 15, R99. [Google Scholar] [CrossRef] [PubMed]
  45. Zhang, Y.; Li, J.; Lou, J.; Zhou, Y.; Bo, L.; Zhu, J.; Zhu, K.; Wan, X.; Cai, Z.; Deng, X. Upregulation of programmed death-1 on T cells and programmed death ligand-1 on monocytes in septic shock patients. Crit. Care 2011, 15, R70. [Google Scholar] [CrossRef] [PubMed]
  46. Dai, S.; Jia, R.; Zhang, X.; Fang, Q.; Huang, L. The PD-1/PD-Ls pathway and autoimmune diseases. Cell. Immunol. 2014, 290, 72–79. [Google Scholar] [CrossRef]
  47. Huang, X.; Venet, F.; Wang, Y.L.; Lepape, A.; Yuan, Z.; Chen, Y.; Swan, R.; Kherouf, H.; Monneret, G.; Chung, C.S.; et al. PD-1 expression by macrophages plays a pathologic role in altering microbial clearance and the innate inflammatory response to sepsis. Proc. Natl. Acad. Sci. USA 2009, 106, 6303–6308. [Google Scholar] [CrossRef]
  48. He, Y.F.; Zhang, G.M.; Wang, X.H.; Zhang, H.; Yuan, Y.; Li, D.; Feng, Z.H. Blocking programmed death-1 ligand-PD-1 interactions by local gene therapy results in enhancement of antitumor effect of secondary lymphoid tissue chemokine. J. Immunol. 2004, 173, 4919–4928. [Google Scholar] [CrossRef]
  49. Brahmamdam, P.; Inoue, S.; Unsinger, J.; Chang, K.C.; McDunn, J.E.; Hotchkiss, R.S. Delayed administration of anti-PD-1 antibody reverses immune dysfunction and improves survival during sepsis. J. Leukoc. Biol. 2010, 88, 233–240. [Google Scholar] [CrossRef]
  50. Kasztelan-Szczerbinska, B.; Adamczyk, K.; Surdacka, A.; Rolinski, J.; Michalak, A.; Bojarska-Junak, A.; Szczerbinski, M.; Cichoz-Lach, H. Gender-related disparities in the frequencies of PD-1 and PD-L1 positive peripheral blood T and B lymphocytes in patients with alcohol-related liver disease: A single center pilot study. Peer J. 2021, 9, e10518. [Google Scholar] [CrossRef]
Figure 1. Flowchart for inclusion and exclusion.
Figure 1. Flowchart for inclusion and exclusion.
Diagnostics 12 02004 g001
Table 1. List of questions used for inclusion and exclusion of studies over the title and abstract, as well as the full text.
Table 1. List of questions used for inclusion and exclusion of studies over the title and abstract, as well as the full text.
Screening StageQuestionsResults
Title and abstract screening
  • Is this study focused on sepsis or other systemic infections on adults?
  • Does this study show the expression levels of PD-1 or PD-L1 or the concentrations of sPD-1 or sPD-L1?
  • Is this study published within the last 10 years?
  • Is the study design case control, cohort, cross sectional, randomized controlled trial (RCT), or meta-analysis?
Study included if all questions are answered with ‘yes’
Full text screening
  • Does this study have a control group as a comparison?
  • Is this study in English?
  • Is there the full access to this study?
  • Does this study have clear diagnostic criteria for sepsis?
  • Are there exclusion criteria for the septic patients in this study (such as immunodeficiency)?
Study included if all questions are answered with ‘yes’
Table 2. Study characteristics of expression level and concentration of PD-1 protein in septic patients.
Table 2. Study characteristics of expression level and concentration of PD-1 protein in septic patients.
ReferenceStudy DesignnPatientsOutcome
Gao et al., 2015 [27]Cross sectional367182 septic patients (54 mild sepsis, 46 severe sepsis, 82 septic shock), 185 healthy controls
  • Significantly higher expression levels of CD4+ CTLA-4+, CD4+ PD-1+ and CD8+ PD-1+ on all sepsis subgroups than the control group (p < 0.05)
  • Expression level of CD4+ PD-1+ is proven to be risk factors with statistical significances by logistic regression analysis (p = 0.003)
Jia et al., 2016 [28]Prospective and observational study29580 T2DM patients without sepsis, 88 severe septic (SS) patients without T2DM (29 non-survivors), 77 SS patients + T2DM (24 non-survivors), 50 healthy controls
  • Significantly increased percentage of PD-1+ CD4+ T cells and PD-1+ CD8+ T cells in the T2DM, SS and SS+ T2DM groups compared to the healthy control group and highest levels in the SS+ T2DM and SS group, but no significant difference between the SS+ T2DM group and the SS group
  • Significantly higher percentage of PD-1+ CD4+ T cells and PD-1+ CD8+ T cells, mean fluorescence intensity (MFI) PD-1+ CD4+ T cells and PD-1+ CD8+ T cells in non-survivors than survivors among SS patients with or without T2DM (p < 0.05), but in the survivor or non-survivor group, no significant difference between those with T2DM and those without T2DM (p > 0.05)
Zhao et al., 2019 [29]Cohort study15772 severe septic patients, 40 septic shock patients, 45 healthy controls; 112 septic patients were separated into 73 survivors and 39 non-survivors
  • Significantly higher levels of sPD-1, MFI value of PD-1 on CD4+ and CD8+ T cells in the patients with severe sepsis or septic shock on day 1 of ICU admission than healthy controls (p < 0.001)
  • Significantly higher sPD-1 in non-survivors than survivors (p < 0.05)
  • Significantly higher percentages and MFI values of PD-1 on CD4+and CD8+ T cells in non-survivors than survivors (p < 0.05)
  • No significant differences in the expression of PD-1 on CD4+ and CD8+ T cells and the expression of PD-L1 on monocytes between survivors and non-survivors between day 1 and day 7
Arens et al., 2016 [30]Cross sectional168 septic patients, 8 healthy controls
  • In CD4+ cells from septic patients, PD-1 receptor density appears to be downregulated
  • No substantial changes in CD8+ cells PD-1 receptor density in septic patients
Chang et al., 2014 [15]Cross sectional5843 septic patients, 15 non-septic patients
  • Increased expression of CD8 but not CD4 PD-1 in septic patients compared to non-septic patients
  • Increased CD8 T cell expression of PD-1 over time in the ICU as PD-L1 decreased
Pan et al., 2015 [31]Observational/clinical study8656 septic shock patients, 30 healthy controls
  • Significantly greater expressions of PD-1 in the CD4+ T cells in septic shock patients compared to healthy volunteers
  • Much higher expressions of PD-1 expressions on monocytes in septic shock patients compared to healthy volunteers
Shao et al., 2016 [32]Prospective cohort study16459 septic patients, 76 septic shock patients (49 survivors, 27 non-survivors), 29 healthy controls
  • Significantly increased percentage of PD-1/CD4+ T cells and PD-1/CD8+ T cells in patients with sepsis and septic shock (p < 0.05) compared to healthy controls, and was significantly higher in non-survivors than survivors of septic shock (p < 0.05)
Pan et al., 2017 [33]Prospective study9563 acute pancreatitis (25 with IC, 38 non-IC), 32 healthy controls
  • Significantly higher percentage of CD4+ lymphocytes expressing PD-1 on day 1 and day 3 in the group of acute pancreatitis patients with IC compared to non-IC (D1 p < 0.05; D3 p < 0.05)
  • The percentage of CD14+ monocytes expressing PD-L1 was more closely related to IC in acute pancreatitis than with PD-1 expression in CD4+ lymphocytes
Zhao et al., 2017 [34]Prospective cohort study655595 emergency department (ED) patients (130 SIRS, 276 sepsis, 121 severe sepsis, 68 septic shock; 465 septic patients were separated into 349 survivors and 116 non-survivors), 60 healthy controls
  • No significant difference in sPD-1 levels between the SIRS group and the healthy control group (p = 0.462)
  • Higher sPD-1 levels in the sepsis group compared to SIRS and healthy control group (p < 0.05)
  • Significant difference in pairwise comparison of sPD-1 levels in the sepsis, severe sepsis, and septic shock groups
  • As the severity of sepsis increases, sPD-1 levels gradually increase.
  • Higher sPD-1 levels in non-survivor group than the survivor group
  • sPD-1 showed the highest specificity (91.8%) for predicting 28-day mortality of septic patients
  • Lower probability of survival at 28 days in septic patients with sPD-1 levels higher than 2.38 ng/mL than septic patients with lower sPD-1 levels (p < 0.001)
Liu et al., 2016 [35]Observation study12091 septic patients (53 survivors and 38 non-survivors), 29 healthy controls
  • Significantly increased sPD-1 levels in septic patients compared to healthy controls (p = 0.000)
  • No statistical difference in serum sPD-1 levels between non-survivors and survivors (p > 0.05)
Liu et al., 2017 [36]Observation study13978 severe septic patients, 40 septic shock patients, 21 healthy controls; 118 septic patients were separated into 60 survivors and 58 non-survivors
  • Significantly increased PD-1 expression in regulatory T cells (Tregs) in septic patients compared to healthy controls, but no significant difference between the severe sepsis group and the septic shock group
  • Upregulation of PD-1 expression on CD4+ T cells in septic patients compared to healthy controls, but no significant difference between the severe sepsis and septic shock groups.
  • Significantly higher PD-1 expression in Treg and CD4+ T cells in non-survivors than survivors (p < 0.001 and p = 0.001, respectively)
Table 3. Study characteristics of expression level and concentration of PD-L1 protein in septic patients.
Table 3. Study characteristics of expression level and concentration of PD-L1 protein in septic patients.
ReferenceStudy DesignnPatientsOutcome
Sun et al., 2021 [37]Cross sectional15664 septic patients, 29 appendicitis patients, 33 acute pancreatitis patients, 30 healthy controls
  • Increased sPD-L1 levels on day 1 in septic patients compared to healthy controls, acute pancreatitis, and acute appendicitis patients
  • Significant positive correlation between sPD-L1 with inflammatory markers such as CRP (p = 0.001), PCT (p = 0.001), WBC (p = 0.012)
  • Significant negative correlation between sPD-L1 with immunological functional parameters such as CD3+ counts (p < 0.001), CD4+ counts (p < 0.001) and CD8+ counts (p < 0.001)
  • Decreased sPD-L1 levels on the third day and a significant decrease on the seventh day compared to the first day of sepsis diagnosis (p < 0.01)
  • No correlation between sPD-L1 and lymphocytes, levels of IgM, IgG, and IgA
Zhao et al., 2019 [29]Cohort study15772 severe septic patients, 40 septic shock patients, 45 healthy patients
  • Increased PD-L1 expression in patients with severe sepsis or septic shock.
  • Significantly higher levels of sPD-L1, MFI value of PD-L1 on monocytes in the patients with severe sepsis or septic shock on day 1 of ICU admission than healthy controls (p < 0.001)
  • Significantly different percentages of PD-L1 on monocytes between the severe sepsis and septic shock groups (p < 0.05)
  • Significantly higher percentages and MFI of PD-L1+ on monocytes in non-survivors than survivors (p < 0.05)
  • Positive correlation between sPD-L1 levels with the severity of sepsis
Arens et al., 2016 [30]Cross sectional168 septic patients, 8 healthy controls
  • No substantial difference for TLR4 and PD-L1 between the two groups
Chang et al., 2014 [15]Cross sectional5843 septic patients, 15 non-septic patients
  • Over two-fold increase in the percentage of PD-L1-expressing monocytes in septic versus non-septic patients
  • Monocytic PD-L1 from septic patients remained elevated but did not change over time
  • Increased CD8 T cell expression of PD-1 over time in the ICU as PD-L1 decreased
Pan et al., 2015 [31]Observational/clinical study8656 septic shock patients, 30 healthy controls
  • Significantly greater expressions of PD-L1 in the CD4+ T cells in septic shock patients compared to healthy volunteers
  • Much higher expressions of PD-L1 expressions on monocytes in septic shock patients compared to healthy volunteers
Shao et al., 2016 [32]Prospective cohort study 164 59 septic patients, 76 septic shock patients (49 survivors, 27 non-survivors), 29 healthy controls
  • Significantly increased percentage of monocytes expressing PD-L1 in patients with sepsis and septic shock (p < 0.05) compared to healthy controls, and was significantly higher in non-survivors than survivors of septic shock (p < 0.05)
Lu et al., 2021 [38]Observational pilot study11730 liver cirrhosis (LC) patients, 70 liver cirrhosis + severe sepsis (LC + SS) patients (59 with SS-induced acute on chronic liver failure (ACLF); 37 survivors, 22 non-survivors), 17 healthy controls (HC)
  • Higher percentage of monocytic PD-L1 in the LC + SS group compared to the LC group (p = 0.046) and HC group (p < 0.001) and lower in the HC group compared to the LC group (p = 0.018) [HC < LC < LC + SS]
  • Higher mean fluorescence intensity (MFI) of monocytic PD-L1 in the LC+ SS group compared to the LC (p = 0.046) and HC (p = 0.009) groups.
  • Significantly higher monocytic PD-L1 percentage (p = 0.004) and MFI (p = 0.008) in the deceased group than the living group
Pan et al., 2017 [33]Prospective study6463 acute pancreatitis (25 with IC, 38 non-IC), 32 healthy controls
  • Significantly higher percentage of CD14+ monocytes expressing PD-L1 on day 1 and day 3 in the group of acute pancreatitis patients with IC compared to non-IC (D1 p < 0.05; D3 p < 0.05)
  • The percentage of CD14+ monocytes expressing PD-L1 was more closely related to IC in acute pancreatitis than with PD-1 expression in CD4+ lymphocytes
Liu et al., 2016 [35]Observation study12091 septic patients (53 survivors and 38 non-survivors), 29 healthy controls.
  • Significantly increased serum sPD-L1 levels in septic patients compared to healthy controls (p = 0.000)
  • Significantly increased serum sPD-L1 levels in non-survivors compared with survivors (p < 0.05)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Sari, M.I.; Ilyas, S. The Expression Levels and Concentrations of PD-1 and PD-L1 Proteins in Septic Patients: A Systematic Review. Diagnostics 2022, 12, 2004. https://doi.org/10.3390/diagnostics12082004

AMA Style

Sari MI, Ilyas S. The Expression Levels and Concentrations of PD-1 and PD-L1 Proteins in Septic Patients: A Systematic Review. Diagnostics. 2022; 12(8):2004. https://doi.org/10.3390/diagnostics12082004

Chicago/Turabian Style

Sari, Mutiara Indah, and Syafruddin Ilyas. 2022. "The Expression Levels and Concentrations of PD-1 and PD-L1 Proteins in Septic Patients: A Systematic Review" Diagnostics 12, no. 8: 2004. https://doi.org/10.3390/diagnostics12082004

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop