Next Article in Journal
Healing of Extraction Sites after Alveolar Ridge Preservation Using Advanced Platelet-Rich Fibrin: A Retrospective Study
Next Article in Special Issue
In Vitro Bioassay for Damage-Associated Molecular Patterns Arising from Injured Oral Cells
Previous Article in Journal
Advancements in Finite Element Modeling for Cardiac Device Leads and 3D Heart Models
Previous Article in Special Issue
Biomimetic Remineralization of Artificial Caries Lesions with a Calcium Coacervate, Its Components and Self-Assembling Peptide P11-4 In Vitro
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

Changes of the Alveolar Bone Ridge Using Bone Mineral Grafts and Collagen Membranes after Tooth Extraction: A Systematic Review and Meta-Analysis

by
Nansi López-Valverde
1,*,
Bruno Macedo de Sousa
2 and
José Antonio Blanco Rueda
1
1
Department of Surgery, Biomedical Research Institute of Salamanca (IBSAL), University of Salamanca, 37008 Salamanca, Spain
2
Institute for Occlusion and Orofacial Pain, Faculty of Medicine, University of Coimbra, 3004-531 Coimbra, Portugal
*
Author to whom correspondence should be addressed.
Bioengineering 2024, 11(6), 565; https://doi.org/10.3390/bioengineering11060565
Submission received: 1 May 2024 / Revised: 21 May 2024 / Accepted: 29 May 2024 / Published: 3 June 2024
(This article belongs to the Special Issue Tissue Engineering for Regenerative Dentistry)

Abstract

:
Background: Alveolar preservation techniques for esthetic or functional purposes, or both, are a frequently used alternative for the treatment of post-extraction sockets, the aim of which is the regeneration of the lesion and the preservation of the alveolar bone crest. Methods: Studies published in PubMed (Medline), Web of Science, Embase, and Cochrane Library databases up to January 2024 were consulted. Inclusion criteria were established as intervention studies, according to the PICOs strategy: adult subjects undergoing dental extractions (participants), with alveoli treated with bone mineral grafts and collagen membranes (intervention), compared to spontaneous healing (comparison), and observing the response to treatment in clinical and radiological measures of the alveolar bone crest (outcomes). Results: We obtained 561 results and selected 12 studies. Risk of bias was assessed using the Cochrane Risk of Bias Tool, and methodological quality was assessed using the Joanna Briggs Institute. Due to the high heterogeneity of the studies (I2 > 75%), a random-effects meta-analysis was used. Despite the trend, no statistical significance (p > 0.05) was found in the experimental groups. Conclusions: The use of bone mineral grafts in combination with resorbable collagen barriers provides greater preservation of the alveolar ridge, although more clinical studies are needed.

1. Introduction

Dental extraction causes resorption of the alveolar ridge, starting this process immediately after extraction and causing a decrease in the vertical and horizontal dimensions of the alveolar ridge during the first 24 months [1]. To preserve the original dimensions of the ridge after extraction, either for esthetic or functional purposes or both, multiple techniques of grafting and bone substitutes in the empty socket, stabilized by resorbable or non-resorbable membranes, or simply by securing the blood clot with these membranes, have been proposed [2,3] (Figure 1).
A systematic review conducted by Avila-Ortiz et al. on 22 RCTs [4] evaluated the effect of different modalities of alveolar bone ridge preservation (ABRP) compared to spontaneous healing; however, there is currently no consensus on material choice, case selection, or appropriate clinical technique. Collagen bioresorbable membranes have been proposed as an ideal material because of their excellent tissue compatibility and low dehiscence rates, in addition to avoiding second extraction surgeries, as is the case with non-resorbable membranes [5]. Furthermore, collagen is considered a material of low immunogenicity; however, its biomedical potential is limited by its poor thermal stability and low enzymatic and mechanical resistance, so different crosslinking techniques have been proposed to minimize its degradation and improve its mechanical stability [6,7]. Based on these assumptions, a recent study by Radenković et al. concluded that the biocompatibility and stability of suitably crosslinked collagen membranes would be equal in functionality to non-resorbable membranes [8].
Graft materials need to be biocompatible and osteoconductive and provide an ideal environment for the recruitment of osteoblastic cells while being able to fill the bone defect and maintain the appropriate crestal level [9]. Although the ideal material is autologous bone, allografts, xenografts, and alloplastic materials capable of preserving the alveolar ridge have been developed and proposed in recent years. However, although these types of materials have been reported to decrease the levels of crestal resorption compared to non-grafted areas, there is no conclusive evidence to indicate the convenience of using one or another material, leaving this decision to the discretion of clinicians [10].
Alveolar healing exclusively by stabilizing the blood clot with collagen membranes has been associated with better preservation of the alveolar process compared to spontaneous healing since it has been shown that platelets can bind directly to collagen through Integrin a2b1 and Glycoprotein VI, favoring clot formation [11]. This blood clot alone induces bone maturation, differentiation, organization, and ultimately bone healing, and some studies have suggested that it may prove to be the best natural scaffold to obtain ideal bone quality and that future research on guided bone regeneration should consider the properties of the clot “per se” [12].
Nevertheless, clinicians should be aware of the possible intraoperative and postoperative complications they face when performing this type of invasive technique. The most frequent is the exposure of biomaterials, both membranes and graft materials, which, together with possible inflammation and infection, minimize the clinical results of regenerative therapy [13].
Reviews with meta-analyses focusing on the effect of collagen membranes on PRBA comparing spontaneous alveolar healing, healing by blood clot stabilization with collagen membranes, and healing with bone substitutes and collagen membranes are very scarce and, generally, focus on comparing spontaneous alveolar healing with healing with bone substitutes. Some only resort to the evaluation of these materials in what they call the “esthetic sector”. Moreover, the clinical scenarios are reduced to a single comparison: spontaneous healing and healing with bone substitutes and membranes. Our meta-analysis evaluated and compared the effect of collagen membranes on PRBA and compared spontaneous healing with healing by stabilization of the blood clot, alone or with the incorporation of bone substitutes, stabilized, in both situations, with collagen membranes.

2. Materials and Methods

2.1. Development of the Protocol and Registration

This systematic review and meta-analysis are presented in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Statement and Clinical Practice Guidelines [14,15].
The protocol of our meta-analysis was registered under the INPLASY number INPLASY202420094; DOI number 10.37766/inplasy2024.2.0094.

2.2. Focused Interest Question

“Are collagen membranes and bone mineral grafts effective in adults, in preserving the alveolar bone crest, following tooth extraction, compared to natural alveolar healing, in terms of clinical and radiological outcomes?”
The research question was posed according to the PICOs format and included intervention studies in adult patients undergoing dental extractions (P) comparing treatment with bone mineral grafts and collagen membranes (I) with spontaneous healing of the alveolus or with other different graft materials (C) to observe the effects on clinical and radiographic parameters (O), with only randomized clinical studies (s) being considered (Table 1).

2.3. Data Sources and Literature Search Method

Two reviewers (NL-V, BMS) independently searched four electronic databases (MEDLINE/PubMed, Embase, Cochrane Central, Web of Science) in the last thirteen years, up to January 2024. using the terms Medical Subject Headings (MeSH): Oral Sugery Procedures* OR Surgery, Oral* AND Alveolar Bone Loss*/prevention & control AND Alveolar Process/surgery AND Alveolar Ridge Augmentation* AND Biocompatible Materials AND Bone Transplantation AND Bone Regeneration AND Guided Tissue Regeneration AND Membranes OR Collagen Membranes AND Humans*. In addition, they performed a manual search and consultations in the gray literature, as well as consultations on the bibliographic references of the studies included in the review. All this was in order to obtain as much information as possible and to avoid bibliographic bias.

2.4. Inclusion and Exclusion Criteria

The original research studies were selected according to the following inclusion criteria: (i) randomized clinical trials (single or double blind) that included in the study more than 10 adult subjects (≥18 years); (ii) with alveolar bone preservation needs; (iii) that provided data on clinical and radiological measurements on width, height, and volume of the alveolar bone crest; (iv) with statistical methods that included mean numerical values and standard deviation; (v) published in English. Studies that did not follow all the above criteria, with a lack of relevant data, preclinical studies or in vitro studies, case series or clinical cases, literature reviews, and irrelevant studies (letters to the Editor, congress abstracts, etc.) were excluded.

2.5. Assessment of the Quality of the Reports

The included studies were methodologically evaluated using the tool developed by the Joanna Briggs Institute for RCTs (JBI MAStARI) that analyzes the methods used to synthesize the different types of evidence. The checklist consists of thirteen items whose answers are “yes”, “no”, “unclear” or “not applicable”. The answer “yes” was valued with one point. To include a study, it had to obtain a minimum score of seven [16].

2.6. Risk of Bias in Selected Studies

Two assessors (NL-V and BMS) independently assessed the risk of bias in the included studies using the Cochrane Risk of Bias Tool (RoB2) [17], which considers 7 domains of bias: 2 domains of selection bias (random sequence generation and Allocation concealment; 1 implementation bias (Blinding of participants and staff); 1 detection bias (blinding of outcome assessment); and 1 attrition bias (incomplete outcome data). Studies were assessed with “high,” “low,” and “borderline” risk of bias; “borderline” bias applied to that lacking information on a given bias. Discrepancies between the two reviewers were resolved by discussion and, in cases of doubt, by the opinion of a third reviewer (JABR). Cohen’s kappa index (κ) [18] was used to assess inter-reviewer agreement.

2.7. Meta-Analysis

Data were analyzed using Review Manager software (RevMan Software. Version 5.4.1, The Cochrane Collaboration, Copenhagen, Denmark; 2020). Due to the heterogeneity of the results, a random-effects meta-analysis was performed for studies that assessed BCW and for those that assessed BCH. CBV was reported by only one study. All were based on mean difference (MD) and standard deviation (SD) for estimating continuous data and for assessing categorical data, with 95% confidence intervals (CI). Heterogeneity was considered low with I2 = 0–30%; moderate, with I2 = 40–50%; substantial with I2 = 60–75% and high with I2 ≥ 75%. The threshold for statistical significance was set at p < 0.05. No meta-analysis of adverse effects was performed due to the scarcity of reports on the subject.

3. Results

The electronic search found 561 results, which, after eliminating duplicates, constituted 65 unique citations. Twenty-two full-text publications were evaluated, and 11 were excluded based on established criteria, resulting in 12 articles for evaluation [19,20,21,22,23,24,25,26,27,28,29,30] (Figure 2).

3.1. Study Characteristics

The included studies involved 390 adult subjects (>18 years), and 374 postextraction sockets were assessed in the different studies. Only one of the studies [23] evaluated CBV by panoramic radiograph and computed tomography, comparing alveolar filling using a bovine deproteinized bone graft coated with an absorbable collagen membrane with spontaneous healing. The number of patients ranged from 23 to 44. The study by Perelman-Karmon et al. [21] included the smallest number of subjects, and the study by Cook et al. [22] had the largest sample size. The study by Stumbras et al. [29] had the longest-term follow-up. Five studies used the molar area [23,25,26,27,28], and the others used the anterior areas of both the maxilla and mandible. The most commonly used graft material was bovine bone, together with resorbable collagen barrier membranes. Three studies used double barriers [25,26,27]. Four studies [19,20,24,25] used DBBM as a control; one, xenograft sponge [22], and six studies [23,26,27,28,29,30] used spontaneous healing as a control group. The description of the included studies is summarized in Table 2.

3.2. Methodological Rigor of the Studies

The methodological quality of all included studies ranged from acceptable (8 points) to very high (>10 points), as determined by the JBI-MAStARI critical appraisal checklist for RCTs. The studies that achieved the highest methodological quality were those of Jung et al., Lim et al., Cha et al. and Stumbras et al. with 12 points [26,27,28,29] and Gabay et al. [30] with 13 points (Table 3).

3.3. Risk of Bias within Studies

The risk of bias assessment of included studies using the Cochrane Risk of Bias Tool (RoB2) found no studies scoring positively in all domains. Random sequence generation (selection bias), incomplete outcome data (attrition bias), and selective reporting (reporting bias) were the domains with the highest compliance. Blinding of participants and staff and blinding in outcome assessment (performance and dropout biases, respectively) were unclear or noncompliant in the included studies. Nevertheless, seven studies [24,25,26,27,28,30] met most of the domains (Figure 3).

3.4. Meta-Analysis

Meta-analyses were performed for pooled studies evaluating BCH and BCW (Figure 4A,B, respectively) and for those comparing bone mineral graft and collagen membranes with spontaneous healing of the postextraction alveolus (Figure 5C,D, respectively). Only the study by Pang et al. [23] compared BCW between sockets filled with bone substitutes and collagen membranes, and those that healed by spontaneous healing (Figure 5E). Heterogeneity was high in all studies (I2 > 75%). All studies analyzed bone mineral grafting together with collagen membranes and compared it with spontaneous healing of the alveolus. The study by Stumbras et al. [29] was not included in the meta-analysis due to a lack of data supply. The results of the meta-analyses showed a trend toward dimensional change in the bone crest in both height and width, with bone mineral grafts and collagen membranes, although without statistical significance. p = 0.21 and p = 0.22 for the BCH and BCW evaluations, respectively, in the pooled study analysis; p = 0.17 and p = 0.13 for BCH and BCW assessments, respectively, in the analysis of studies comparing bone mineral grafting and collagen membranes with spontaneous alveolar healing. Only one study assessed BCW, and, despite finding a trend toward statistical significance, it was not measurable in the meta-analysis. The studies with the greatest statistical weight were those of Cha et al. and Gabay et al. [28,30]. The Forest Plot diagrams are shown in Figure 4 and Figure 5.

3.5. Publication Bias

The abscissa axis (X) represents the magnitude of the effect, and the ordinate axis (Y) represents the risk effect. The middle line represents the central estimator. The symmetry of most studies demonstrates a low publication bias, although the paucity of studies limits this consideration (Figure 6).

4. Discussion

Since the systematic review and meta-analysis by Avila-Ortiz et al. in 2018 [4], there have been few meta-analyses evaluating ABRP using bone substitutes and resorbable membranes. Therefore, the present review and meta-analysis aim to evaluate clinically and radiographically the efficacy of bone mineral grafts and collagen membranes in ABRP after tooth extraction compared to spontaneous healing. However, we must point out that there is a lack of well-designed RCTs in the scientific literature that adequately evaluate the different regenerative techniques, and most of the studies included in our meta-analysis were parallel-arm studies, and only one was a multicenter study [25]. On the other hand, our bibliographic search was limited to the last 13 years since previous studies could be obsolete in terms of techniques and biomaterials, given the constant updating in both aspects [31].
The main methodological weaknesses observed were related to the blinding of participants and personnel, although this performance bias is difficult to overcome in RCTs of surgical interventions. Interestingly, only one of the included studies [30] was enrolled in the ClinicalTrials.gov clinical trials registry, so this study was the best evaluated in terms of methodological quality (Table 3, Figure 3).
Nevertheless, qualitative analyses showed that bone grafts together with resorbable collagen membranes provided better results than collagen barriers alone, as stabilizers of the blood clot, or spontaneous healing of the alveolus; only the study by Gabay et al. [30] did not show statistical significance in the reduction in vertical and horizontal dimensions, due, according to the authors, to a larger than expected standard deviation and minor differences between the groups. Similarly, crestal width, although only measured by one study [23] and not comparable, showed a trend toward statistical significance (Figure 5E).
These data would be of clinical relevance when preserving alveolar volume, either for esthetic or functional purposes, or both, and would be consistent with other investigations [32,33].
It has been proven by histomorphometry that sockets filled with bone mineral grafts and stabilized by resorbable membranes show, after three months, lower amounts of bone tissue compared to sockets in which the clot is stabilized or those that heal spontaneously [29]; however, the neoformed bone increases with the passage of time [30,34]. Despite these observations, the literature does not clarify this bone immaturity and its clinical repercussions, especially when used to place implanted devices, as the esthetic defect may be compensated by soft tissue growth [35,36].
Our results in the different meta-analyses showed no significant differences established as ≤0.05; however, all showed a trend towards dimensional change in the ridge in height and width, suggesting the efficacy of mineral bone grafts and collagen membranes in alveolar preservation.
These results are far from the null hypothesis and the scientific evidence available in the scientific literature to date, so more well-designed RCTs to determine the effectiveness of the usual surgical technique of filling and sealing the alveolus in ABRP are recommended and justifiable.

Previous Literature

Numerous preclinical and clinical studies have addressed alveolar preservation to mitigate the process of bone resorption through clot stabilization procedures and placement of bone graft material within the extraction socket with or without the application of barrier membranes, and both clinical trials and animal studies appear to be trending towards the fact that socket sealing may increase alveolar wall preservation and reduce soft tissue loss [34,37,38]. You et al. [39] conducted a retrospective study on 58 subjects to demonstrate the efficacy of atherocollagen (collagen with low immunogenicity and longer degradation time) compared to deproteinized bone mineral substitutes in alveolar ridge preservation and found no difference between the groups. Chappuis et al. [40] were pioneers in investigating three-dimensional alterations in facial esthetic zones after tooth extraction in a prospective study on 39 subjects, concluding that thin-walled phenotypes showed pronounced vertical bone resorption compared to thick-walled phenotypes. Barone et al. [34] in a large sample of 58 patients reported that alveolar filling with collagenized porcine bone and a resorbable membrane was able to maintain the width and height of the grafted areas in better condition than the spontaneously healing control areas. Subsequently, a study by Karaka et al. [41] in a small sample of patients evaluated and compared the dimensional changes in spontaneously healed postextraction sockets with those that stabilized the clot using free gingival grafts, finding no differences between the groups in terms of width but did find differences in crestal height. An extensive review by Jambhekar et al. [42] showed that, after 12 weeks of waiting, the use of grafts results in less loss of alveolar dimensions compared with ungrafted sockets. In this regard, two reviews have been performed by the Cochrane Oral Health Group, one in 2015 and another in 2021; the latter included 524 extraction sites in 426 adult participants, concluding that, although ABRP techniques can minimize changes in residual alveolar ridge height and width, after 6 months after extraction, the evidence is uncertain and that there are no significant clinical differences in terms of different grafts and occlusal barriers, recommending well-designed RCTs [43]. All this is in agreement with the results of our meta-analysis.
As for preclinical studies, a very recent one by Han et al. [44] in a canine experimental model, in which they performed volumetric analysis with computerized microtomography and histological analysis, revealed incomplete resorption, after 24 months, of bone substitutes, which would raise the need for a debate on whether or not to use such biomaterials.
Other meta-analyses, published prior to ours, assessed different modalities of alveolar ridge preservation after extraction (alveolar filling with biomaterials, collagen sponge, or autologous blood-derived products) in molar and nonmolar teeth, compared with spontaneous healing. Avila-Ortiz et al. in 2014 conducted a systematic review and meta-analysis to demonstrate the effect of alveolar filling in nonmolar teeth compared with spontaneous healing [45]. In 2020, Avila-Ortiz et al. published another review and meta-analysis that assessed alveolar ridge preservation in two clinical scenarios: (a) extraction sites with intact bone walls and (b) sites with defects such as dehiscence or fenestrations, with biomaterial grafting and barrier application, compared to spontaneous healing of the alveolus [4]. The first meta-analysis showed that alveolar fillings could significantly hinder alveolar bone remodeling after extraction due to the influence of unknown local and systemic factors, according to the authors. In the second, larger review, the analysis of pooled studies revealed that the beneficial effects of this treatment appear to be most evident only in the prevention of horizontal bone resorption. Another review with meta-analysis by Bassir et al. [46], concurring with us, compared the efficacy of alveolar ridge preservation procedures with tooth extraction without intervention and found that, although alveolar ridge preservation procedures are effective in minimizing postextraction hard tissue dimensional loss, the use of alloplastic materials produces fewer desirable results. Apaza-Bedoya et al. [47] published a recent review with meta-analysis that aimed to identify the clinical, radiographic, and histologic outcomes of alveolar ridge preservation, using bone xenografts and absorbable barriers compared to spontaneous healing in the “esthetic zone”, highlighting the efficacy of techniques and materials in reducing post-extraction crestal bone changes in this area.
Our meta-analysis evaluated and compared three different clinical scenarios: (a) spontaneous healing, (b) clot stabilization using resorbable barriers, and (c) alveolar filling with bone substitutes and resorbable sealing barriers, both in esthetic and molar areas. (Figure 1). In this regard, it would be appropriate to highlight the relevance of the blood clot even as a scaffold in bone regeneration [13,48], highlighting that some studies have proposed spontaneous healing, without using filling materials or covering the alveolar entry, stabilizing exclusively the buccal contour [49]. Aravena et al. [50] in a randomized clinical trial of split mouth in a sample of 16 subjects, found the same clinical behavior, in dimensional and volumetric terms, between spontaneous healing by blood clot and alveolar filling using autologous platelet-rich fibrin. Already in the last century, bone regeneration of a defect exclusively by clot stabilization was highlighted [51,52] and, recently, Milillo et al. [13] pointed out that the clot, on its own, stabilized with an occlusive barrier, would be able to generate a higher quality bone, although they propose the association of the clot with a filler as the main support of the clot-scaffold.
Associated with the discrepancies in research are the costs of the treatments, which are proportional to the quantity and variety of materials used. Barootchi et al. [53] recently demonstrated the economic cost-effectiveness of the different therapeutic modalities according to the results obtained, concluding that greater expenditure does not imply greater efficacy in alveolar preservation, observing only a relative association between the expenditure derived from the use of barrier membranes and alveolar preservation.
Finally, we would like to highlight a series of situations that limit this meta-analysis, which are described below: the different commercial materials used in the different studies, each with its own corresponding manufacturing characteristics; the different follow-up periods and the lack of long-term reports; and the different surgical scenarios considered, whether for esthetic or surgical-implantological purposes. All this generates clinical and methodological heterogeneity, biasing the final results.

5. Conclusions

Based on the results obtained in this systematic review with meta-analysis, we draw the following conclusions for clinical practice:
  • The results of subgroup analyses demonstrated that ABRP using bone mineral grafts in combination with resorbable collagen barriers manifests a tendency for greater alveolar ridge preservation, both in height and width, than spontaneous healing.
  • The CBV, although assessed in only one study, showed the highest tendency to statistical significance, although it could not be evaluated in the meta-analysis.
  • Clinical practice can be focused in a certain direction, according to the characteristics and specific situation of each clinical case.
More well-designed clinical studies with long-term follow-up are warranted, recommended, and necessary to clarify our results.

Author Contributions

Conceptualization, N.L.-V.; methodology, N.L.-V.; software, B.M.d.S.; formal analysis, N.L.-V. and B.M.d.S.; data curation, N.L.-V. and B.M.d.S.; writing—original draft preparation, N.L.-V.; writing—review and editing, B.M.d.S.; supervision, J.A.B.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data can be made available on request by the authors.

Acknowledgments

This article has been partially funded by the research group “Avances en Salud Bucodental” of the University of Salamanca, directed by Javier Montero.

Conflicts of Interest

The authors declare no conflicts of interests.

References

  1. Tan, W.L.; Wong, T.L.; Wong, M.C.; Lang, N.P. A systematic review of post-extractional alveolar hard and soft tissue dimensional changes in humans. Clin. Oral Implants Res. 2012, 23 (Suppl. S5), 1–21. [Google Scholar] [CrossRef] [PubMed]
  2. Vittorini Orgeas, G.; Clementini, M.; De Risi, V.; de Sanctis, M. Surgical techniques for alveolar socket preservation: A systematic review. Int. J. Oral Maxillofac. Implants 2013, 28, 1049–1061. [Google Scholar] [CrossRef] [PubMed]
  3. Rodrigues, M.T.V.; Guillen, G.A.; Macêdo, F.G.C.; Goulart, D.R.; Nóia, C.F. Comparative Effects of Different Materials on Alveolar Preservation. J. Oral Maxillofac. Surg. 2023, 81, 213–223. [Google Scholar] [CrossRef] [PubMed]
  4. Avila-Ortiz, G.; Chambrone, L.; Vignoletti, F. Effect of alveolar ridge preservation interventions following tooth extraction: A systematic review and meta-analysis. J. Clin. Periodontol. 2019, 46 (Suppl. S21), 195–223. [Google Scholar] [CrossRef] [PubMed]
  5. Ayari, H. The use of periodontal membranes in the field of periodontology: Spotlight on collagen membranes. J. Appl. Biomed. 2022, 20, 154–162. [Google Scholar] [CrossRef] [PubMed]
  6. Lynn, A.K.; Yannas, I.V.; Bonfield, W. Antigenicity and immunogenicity of collagen. J. Biomed. Mater. Res. B Appl. Biomater. 2004, 71, 343–354. [Google Scholar] [CrossRef] [PubMed]
  7. Gu, L.; Shan, T.; Ma, Y.X.; Tay, F.R.; Niu, L. Novel Biomedical Applications of Crosslinked Collagen. Trends Biotechnol. 2019, 37, 464–491. [Google Scholar] [CrossRef] [PubMed]
  8. Radenković, M.; Alkildani, S.; Stoewe, I.; Bielenstein, J.; Sundag, B.; Bellmann, O.; Jung, O.; Najman, S.; Stojanović, S.; Barbeck, M. Comparative In Vivo Analysis of the Integration Behavior and Immune Response of Collagen-Based Dental Barrier Membranes for Guided Bone Regeneration (GBR). Membranes 2021, 11, 712. [Google Scholar] [CrossRef] [PubMed]
  9. Wood, R.A.; Mealey, B.L. Histologic comparison of healing after tooth extraction with ridge preservation using mineralized versus demineralized freeze-dried bone allograft. J. Periodontol. 2012, 83, 329–336. [Google Scholar] [CrossRef]
  10. Quisiguiña Salem, C.; Ruiz Delgado, E.; Crespo Reinoso, P.A.; Robalino, J.J. Alveolar ridge preservation: A review of concepts and controversies. Natl. J. Maxillofac. Surg. 2023, 14, 167–176. [Google Scholar] [CrossRef]
  11. Temmerman, A.; Vandessel, J.; Castro, A.; Jacobs, R.; Teughels, W.; Pinto, N.; Quirynen, M. The use of leucocyte and platelet-rich fibrin in socket management and ridge preservation: A split-mouth, randomized, controlled clinical trial. J. Clin. Periodontol. 2016, 43, 990–999. [Google Scholar] [CrossRef]
  12. Milillo, L.; Cinone, F.; Lo Presti, F.; Lauritano, D.; Petruzzi, M. The Role of Blood Clot in Guided Bone Regeneration: Biological Considerations and Clinical Applications with Titanium Foil. Materials 2021, 14, 6642. [Google Scholar] [CrossRef] [PubMed]
  13. Sanz-Sánchez, I.; Sanz-Martín, I.; Ortiz-Vigón, A.; Molina, A.; Sanz, M. Complications in bone-grafting procedures: Classification and management. Periodontol 2000 2022, 88, 86–102. [Google Scholar] [CrossRef]
  14. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
  15. Graham, R.; Mancher, M.; Miller Wolman, D.; Greenfield, S.; Steinberg, E. Washington (DC) US, Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. In Clinical Practice Guidelines We Can Trust; National Academies Press: Washington, DC, USA, 2011. [Google Scholar]
  16. Barker, T.H.; Stone, J.C.; Sears, K.; Klugar, M.; Tufanaru, C.; Leonardi-Bee, J.; Aromataris, E.; Munn, Z. The revised JBI critical appraisal tool for the assessment of risk of bias for randomized controlled trials. JBI Evid. Synth. 2023, 21, 494–506. [Google Scholar] [CrossRef] [PubMed]
  17. Minozzi, S.; Cinquini, M.; Gianola, S.; Gonzalez-Lorenzo, M.; Banzi, R. The revised Cochrane risk of bias tool for randomized trials (RoB 2) showed low interrater reliability and challenges in its application. J. Clin. Epidemiol. 2020, 126, 37–44. [Google Scholar] [CrossRef]
  18. Casagrande, A.; Fabris, F.; Girometti, R. Beyond kappa: An informational index for diagnostic agreement in dichotomous and multivalue ordered-categorical ratings. Med. Biol. Eng. Comput. 2020, 58, 3089–3099. [Google Scholar] [CrossRef]
  19. Mardas, N.; Chadha, V.; Donos, N. Alveolar ridge preservation with guided bone regeneration and a synthetic bone substitute or a bovine-derived xenograft: A randomized, controlled clinical trial. Clin. Oral Implants Res. 2010, 21, 688–698. [Google Scholar] [CrossRef] [PubMed]
  20. Mardas, N.; D’Aiuto, F.; Mezzomo, L.; Arzoumanidi, M.; Donos, N. Radiographic alveolar bone changes following ridge preservation with two different biomaterials. Clin. Oral Implants Res. 2011, 22, 416–423. [Google Scholar] [CrossRef]
  21. Perelman-Karmon, M.; Kozlovsky, A.; Liloy, R.; Artzi, Z. Socket site preservation using bovine bone mineral with and without a bioresorbable collagen membrane. Int. J. Periodontics Restor. Dent. 2012, 32, 459–465. [Google Scholar]
  22. Cook, D.C.; Mealey, B.L. Histologic comparison of healing following tooth extraction with ridge preservation using two different xenograft protocols. J. Periodontol. 2013, 84, 585–594. [Google Scholar] [CrossRef] [PubMed]
  23. Pang, C.; Ding, Y.; Zhou, H.; Qin, R.; Hou, R.; Zhang, G.; Hu, K. Alveolar ridge preservation with deproteinized bovine bone graft and collagen membrane and delayed implants. J. Craniofac. Surg. 2014, 25, 1698–1702. [Google Scholar] [CrossRef] [PubMed]
  24. Scheyer, E.T.; Heard, R.; Janakievski, J.; Mandelaris, G.; Nevins, M.L.; Pickering, S.R.; Richardson, C.R.; Pope, B.; Toback, G.; Velásquez, D.; et al. A randomized, controlled, multicentre clinical trial of post-extraction alveolar ridge preservation. J. Clin. Periodontol. 2016, 3, 1188–1199. [Google Scholar] [CrossRef] [PubMed]
  25. Nart, J.; Barallat, L.; Jimenez, D.; Mestres, J.; Gómez, A.; Carrasco, M.A.; Violant, D.; Ruíz-Magaz, V. Radiographic and histological evaluation of deproteinized bovine bone mineral vs. deproteinized bovine bone mineral with 10% collagen in ridge preservation. A randomized controlled clinical trial. Clin. Oral Implants Res. 2017, 28, 840–848. [Google Scholar] [CrossRef]
  26. Jung, R.E.; Sapata, V.M.; Hämmerle, C.H.F.; Wu, H.; Hu, X.L.; Lin, Y. Combined use of xenogeneic bone substitute material covered with a native bilayer collagen membrane for alveolar ridge preservation: A randomized controlled clinical trial. Clin. Oral Implants Res. 2018, 29, 522–529. [Google Scholar] [CrossRef] [PubMed]
  27. Lim, H.C.; Shin, H.S.; Cho, I.W.; Koo, K.T.; Park, J.C. Ridge preservation in molar extraction sites with an open-healing approach: A randomized controlled clinical trial. J. Clin. Periodontol. 2019, 46, 1144–1154. [Google Scholar] [CrossRef]
  28. Cha, J.K.; Song, Y.W.; Park, S.H.; Jung, R.E.; Jung, U.W.; Thoma, D.S. Alveolar ridge preservation in the posterior maxilla reduces vertical dimensional change: A randomized controlled clinical trial. Clin. Oral Implants Res. 2019, 30, 515–523. [Google Scholar] [CrossRef]
  29. Stumbras, A.; Januzis, G.; Gervickas, A.; Kubilius, R.; Juodzbalys, G. Randomized and Controlled Clinical Trial of Bone Healing After Alveolar Ridge Preservation Using Xenografts and Allografts versus Plasma Rich in Growth Factors. J. Oral Implantol. 2020, 46, 515–525. [Google Scholar] [CrossRef]
  30. Gabay, E.; Katorza, A.; Zigdon-Giladi, H.; Horwitz, J.; Machtei, E.E. Histological and dimensional changes of the alveolar ridge following tooth extraction when using collagen matrix and collagen-embedded xenogenic bone substitute: A randomized clinical trial. Clin. Implant Dent. Relat. Res. 2022, 24, 382–390. [Google Scholar] [CrossRef]
  31. Vergara-Merino, L.; Verdejo, C.; Carrasco, C.; Vargas-Peirano, M. Living systematic review: New inputs and challenges. Medwave 2020, 20, e8092. [Google Scholar] [CrossRef]
  32. Cardaropoli, D.; De Luca, N.; Tamagnone, L.; Leonardi, R. Bone and Soft Tissue Modifications in Immediate Implants Versus Delayed Implants Inserted Following Alveolar Ridge Preservation: A Randomized Controlled Clinical Trial. Part I: Esthetic Outcomes. Int. J. Periodontics Restor. Dent. 2022, 42, 195–202. [Google Scholar] [CrossRef] [PubMed]
  33. Walker, C.J.; Prihoda, T.J.; Mealey, B.L.; Lasho, D.J.; Noujeim, M.; Huynh-Ba, G. Evaluation of Healing at Molar Extraction Sites with and without Ridge Preservation: A Randomized Controlled Clinical Trial. J. Periodontol. 2017, 88, 241–249. [Google Scholar] [CrossRef] [PubMed]
  34. Borg, T.D.; Mealey, B.L. Histologic healing following tooth extraction with ridge preservation using mineralized versus combined mineralized-demineralized freeze-dried bone allograft: A randomized controlled clinical trial. J. Periodontol. 2015, 86, 348–355. [Google Scholar] [CrossRef]
  35. Maiorana, C.; Poli, P.P.; Deflorian, M.; Testori, T.; Mandell, F.; Nagursky, H.; Vinci, R. Alveolar socket preservation with demineralised bovine bone mineral and a collagen matrix. J. Periodontal Implant Sci. 2017, 47, 194–210. [Google Scholar] [CrossRef]
  36. Sun, D.J.; Lim, H.C.; Lee, D.W. Alveolar ridge preservation using an open membrane approach for sockets with bone deficiency: A randomized controlled clinical trial. Clin. Implant Dent. Relat. Res. 2019, 21, 175–182. [Google Scholar] [CrossRef] [PubMed]
  37. Kim, S.Y.; Kim, Y.K.; Kim, H.S.; Yun, P.Y.; Kim, S.G.; Choi, Y.H. Extraction socket sealing using palatal gingival grafts and resorbable collagen membranes. Maxillofac. Plast. Reconstr. Surg. 2017, 39, 39. [Google Scholar] [CrossRef] [PubMed]
  38. Guarnieri, R.; Di Nardo, D.; Di Giorgio, G.; Miccoli, G.; Testarell, L. Effectiveness of Xenograft and Porcine-Derived Resorbable Membrane in Augmentation of Posterior Extraction Sockets with a Severe Wall Defect. A Radiographic/Tomographic Evaluation. J. Oral Maxillofac. Res. 2019, 10, e3. [Google Scholar] [CrossRef]
  39. You, S.; Yu, F.; Fan, Q.; Xia, T.; Liang, L.; Yan, Q.; Zeng, H.; Shi, B. Radiographic comparison of atelocollagen versus deproteinized bovine bone minerals covered with a collagen membrane in alveolar ridge preservation: A retrospective study. BMC Oral Health 2023, 23, 901. [Google Scholar] [CrossRef]
  40. Chappuis, V.; Engel, O.; Reyes, M.; Shahim, K.; Nolte, L.P.; Buser, D. Ridge alterations post-extraction in the esthetic zone: A 3D analysis with CBCT. J. Dent. Res. 2013, 92, 195S–201S. [Google Scholar] [CrossRef]
  41. Karaca, Ç.; Er, N.; Gülşahı, A.; Köseoğlu, O.T. Alveolar ridge preservation with a free gingival graft in the anterior maxilla: Volumetric evaluation in a randomized clinical trial. Int. J. Oral Maxillofac. Surg. 2015, 44, 774–780. [Google Scholar] [CrossRef]
  42. Jambhekar, S.; Kernen, F.; Bidra, A.S. Clinical and histologic outcomes of socket grafting after flapless tooth extraction: A systematic review of randomized controlled clinical trials. J. Prosthet. Dent. 2015, 113, 371–382. [Google Scholar] [CrossRef] [PubMed]
  43. Atieh, M.A.; Alsabeeha, N.H.; Payne, A.G.; Ali, S.; Faggion, C.M.J.; Esposito, M. Interventions for replacing missing teeth: Alveolar ridge preservation techniques for dental implant site development. Cochrane Database Syst. Rev. 2021, 4, CD010176. [Google Scholar] [CrossRef] [PubMed]
  44. Han, H.S.; Lee, J.T.; Oh, S.; Cho, Y.D.; Kim, S. Effectiveness of a collagen matrix seal and xenograft in alveolar ridge preservation: An experimental study in dogs. Sci. Rep. 2024, 14, 163. [Google Scholar] [CrossRef] [PubMed]
  45. Avila-Ortiz, G.; Elangovan, S.; Kramer, K.W.; Blanchette, D.; Dawson, D.V. Effect of alveolar ridge preservation after tooth extraction: A systematic review and meta-analysis. J. Dent. Res. 2014, 93, 950–958. [Google Scholar] [CrossRef] [PubMed]
  46. Bassir, S.H.; Alhareky, M.; Wangsrimongkol, B.; Jia, Y.; Karimbux, N. Systematic Review and Meta-Analysis of Hard Tissue Outcomes of Alveolar Ridge Preservation. Int. J. Oral Maxillofac. Implants 2018, 33, 979–994. [Google Scholar] [CrossRef]
  47. Apaza-Bedoya, K.; Magrin, G.L.; Romandini, M.; Blanco-Carrión, J.; Benfatti, C.A.M. Efficacy of alveolar ridge preservation with xenografts and resorbable socket sealing materials in the esthetic region: A systematic review with meta-analyses. Clin. Implant Dent. Relat. Res. 2024, 26, 4–14. [Google Scholar] [CrossRef] [PubMed]
  48. Liu, Y.; Lin, D.; Li, B.; Hong, H.; Jiang, C.; Yuan, Y.; Wang, J.; Hu, R.; Li, B.; Liu, C. BMP-2/CPC scaffold with dexamethasone-loaded blood clot embedment accelerates clinical bone regeneration. Am. J. Transl. Res. 2022, 14, 2874–2893. [Google Scholar]
  49. Mattiola, A.; Bosshardt, D.; Schmidlin, P.R. The Rigid-Shield Technique: A New Contour and Clot Stabilizing Method for Ridge Preservation. Dent. J. 2018, 6, 21. [Google Scholar] [CrossRef] [PubMed]
  50. Aravena, P.C.; Sandoval, S.P.; Pizarro, F.E.; Simpson, M.I.; Castro-Adams, N.; Serandour, G.; Rosas, C. Leukocyte and Platelet-Rich Fibrin Have Same Effect as Blood Clot in the 3-Dimensional Alveolar Ridge Preservation. A Split-Mouth Randomized Clinical Trial. J. Oral Maxillofac. Surg. 2021, 79, 575–584. [Google Scholar] [CrossRef]
  51. Murray, G.; Holden, R.; Roschlau, W. Experimental and clinical study of new growth of bone in a cavity. Am. J. Surg. 1957, 93, 385–387. [Google Scholar] [CrossRef]
  52. Melcher, A.H.; Dreyer, C.J. Protection of the Blood Clot in Healing Circumscribed Bone Defects. J. Bone Jt. Surg. Br. Vol. 1962, 44, 424–430. [Google Scholar] [CrossRef]
  53. Barootchi, S.; Tavelli, L.; Majzoub, J.; Stefanini, M.; Wang, H.L.; Avila-Ortiz, G. Alveolar ridge preservation: Complications and cost-effectiveness. Periodontology 2000 2023, 92, 235–262. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Diagram of ABRP: (A), Spontaneous healing; (B), Clot stabilization by membrane; (C), Bone substitute and membrane.
Figure 1. Diagram of ABRP: (A), Spontaneous healing; (B), Clot stabilization by membrane; (C), Bone substitute and membrane.
Bioengineering 11 00565 g001
Figure 2. Flowchart.
Figure 2. Flowchart.
Bioengineering 11 00565 g002
Figure 3. Risk of bias of meta-analysis [19,20,21,22,23,24,25,26,27,28,30].
Figure 3. Risk of bias of meta-analysis [19,20,21,22,23,24,25,26,27,28,30].
Bioengineering 11 00565 g003
Figure 4. (A,B) Forest Plot of BCH and BCW (pooled studies) [20,22,23,24,25,26].
Figure 4. (A,B) Forest Plot of BCH and BCW (pooled studies) [20,22,23,24,25,26].
Bioengineering 11 00565 g004
Figure 5. (A,B) Forest Plot of BCH and BCW from studies comparing bone mineral graft and collagen membranes with spontaneous alveolar healing [23,28,30], (C) Forest plot of BCW [23].
Figure 5. (A,B) Forest Plot of BCH and BCW from studies comparing bone mineral graft and collagen membranes with spontaneous alveolar healing [23,28,30], (C) Forest plot of BCW [23].
Bioengineering 11 00565 g005
Figure 6. Funnel plots.
Figure 6. Funnel plots.
Bioengineering 11 00565 g006
Table 1. PICOs format.
Table 1. PICOs format.
PopulationAdult Subjects Undergoing Dental Extractions
InterventionBone mineral grafts and Collagen membranes
ComparisonsSpontaneous healing or other grafting materials
OutcomesTo observe the effects of treatment on the clinical and radiological measurements. (Δ BCW; Δ BCH; Δ CBV)
Study designRCTs
BCW, Bone Crest Width; BCH, Bone Crest Height; CBV, Crestal Bone Volume; RCTs, Randomized Controlled Trials.
Table 2. Description and characteristics of the studies included.
Table 2. Description and characteristics of the studies included.
Study,
Year
Type of StudySubjects Experimental AreaGrafted MaterialBarrier ProductsComparisonFollow-UpDropoutsOutcomes
Mardas et al., 2010
[19]
RCCT30Incisor, canine and premolar area (Maxilla and mandible)BBMCollagenDBBM8 months4The use of bone mineral grafts and collagen barrier in the alveolus preserved crestal height, but no differences in buccal and palatal bone plate width were observed between the groups.
Mardas et al., 2011
[20]
RCCT30 Incisor, canine and premolar area (Maxilla and mandible)BBMCollagenDBBM4 and 8 months3Preservation of the alveolar ridge with bovine xenograft or synthetic bone substitute and collagen barrier also preserves radiographic bone levels.
Perelman-Karmon et al., 2012
[21]
RCT23 Incisors and premolar area (Maxilla and mandible)BBMCollagenBBM without resorbable membrane coverage.9 months0The biomaterial grafted in fresh extraction sockets, together with resorbable collagen membranes, maintained its volume and favored the formation of new bone for future implant placement.
Cook et al., 2013
[22]
RCCT44Non-molar areaBBMNon-cross-linked and cross-linked porcine collagen types I and IIIXenograft sponge (70% cross-linked bovine collagen type I) 21 weeks6The use of xenograft material along with a resorbable collagen membrane for ridge preservation in non-molar extraction sites produced significantly more vital bone.
Pang et al., 2014
[23]
RCT30Molar areaBBMCollagenSpontaneous healing 3 and 6 months0Bovine bone grafting together with absorbable collagen membrane were effective in preserving the alveolar ridge; however, the neoformed bone in the experimental group was of poor quality.
Scheyer et al., 2016
[24]
RCMCT40MolarBBMCollagenDBBM with native collagen 6 months3The horizontal measurements of the extraction socket were significantly higher for those treated with BBM y bilayer collagen membrane. Vertical bone changes were not significant between the two treatment modalities. At 6 months, 37 of the 40 sites assessed had sufficient ridge dimension for implant placement. sufficient for implant placement
Nart et al., 2017
[25]
RCCT26Non-molar areaBBMBilayer collagenDBBM with 10% collagen5 months5A significant reduction in height and width was observed at the end of healing, but no statistically significant differences were observed between the BBM and DBBM-Collagen groups. Neoformed bone was similar in the experimental and control groups.
Jung et al., 2018
[26]
RCCT24Posterior areas (upper and lower jaw)BBMBilayer collagen Spontaneous healing3 and 6 months 6At 3 and 6 months crestal height and width were significantly higher in the test group vs. control group.
Lim et al., 2019
[27]
RCCT33Molar areasBBMBilayer collagen and native bilayer collagenSpontaneous healing4 months 4Despite the small sample size (29 sites), preservation of the horizontal alveolar ridge by open healing was advantageous; however, bone neoformation was better in the group treated with double collagen membrane, even though there was no statistical synification.
Cha et al., 2019
[28]
RCCT40Posterior upper molar areaBBM CollagenSpontaneous healing1,3 and 6 months1Preservation of the alveolar ridge in the posterior maxilla using BBM grafting was more effective and resulted in less need for sinus augmentation procedures at 6 months, compared to spontaneous healing of the alveolus.
Stumbras et al., 2020
[29]
RCT40Anterior maxilla (Incisor, canine area)BBMNative collagenSpontaneous healing12 months8Significantly greater new bone formation at sites grafted with BBM and collagen membrane, compared to spontaneous healing.
Gabay et al., 2022
[30]
RCT30Premolar, canine or incisor areaBBMCollagenSpontaneous healing6 months2Preservation of the alveolar ridge by DBBM resulted in a tendency to reduce the vertical and horizontal dimension.
RCCT, Randomized Controlled Clinical Trial; RCT, Randomized Clinical Trial; RCMCT, Randomized Controlled Multicenter Clinical Trial; BBM, Bovine Bone Mineral; DBBM, Deproteinized Bovine Bone Mineral.
Table 3. Results of the methodological assessment according to JBI MAStARI.
Table 3. Results of the methodological assessment according to JBI MAStARI.
Study and YearQ1Q2Q3Q4Q5Q6Q7Q8Q9Q10Q11Q12Q13Total Score
Mardas et al. [19]110?1?111111110
Mardas et al. [20]110?1?111111110
Perelman-Karmon et al. [21]??1??1111111?8
Cook et al. [22]110???111111?8
Pang et al. [23]1?1???11111119
Scheyer et al. [24]111?1?111111111
Nart et al. [25]111?1?111111111
Jung et al. [26]11111?111111112
Lim et al. [27]11111?111111112
Cha et al. [28]11111?111111112
Stumbras et al. [29]11111?111111112
Gabay et al. [30]111111111111113
Q1. Was true randomization used for assigning participants to treatment groups?; Q2. Was allocation to treatment groups concealed?; Q3. Were treatment groups similar at the baseline?; Q4. Were participants blind to treatment assignment?; Q5. Were those delivering treatment blind to treatment assignment?; Q6. Were outcomes assessors blind to treatment assignment?; Q7. Were treatment groups treated identically other than the intervention of interest?; Q8. Was follow-up complete and if not, were differences between groups in terms of their follow-up adequately described and analyzed?; Q9. Were participants analyzed in the groups to which they were randomized?; Q10. Were outcomes measured in the same way for treatment groups?; Q11. Were outcomes measured in a reliable way?; Q12. Was appropriate statistical analysis used?; Q13. Was the trial design appropriate, and any deviation from the standard RCT design accounted for in the conduct and analysis of the trial?
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

López-Valverde, N.; Macedo de Sousa, B.; Blanco Rueda, J.A. Changes of the Alveolar Bone Ridge Using Bone Mineral Grafts and Collagen Membranes after Tooth Extraction: A Systematic Review and Meta-Analysis. Bioengineering 2024, 11, 565. https://doi.org/10.3390/bioengineering11060565

AMA Style

López-Valverde N, Macedo de Sousa B, Blanco Rueda JA. Changes of the Alveolar Bone Ridge Using Bone Mineral Grafts and Collagen Membranes after Tooth Extraction: A Systematic Review and Meta-Analysis. Bioengineering. 2024; 11(6):565. https://doi.org/10.3390/bioengineering11060565

Chicago/Turabian Style

López-Valverde, Nansi, Bruno Macedo de Sousa, and José Antonio Blanco Rueda. 2024. "Changes of the Alveolar Bone Ridge Using Bone Mineral Grafts and Collagen Membranes after Tooth Extraction: A Systematic Review and Meta-Analysis" Bioengineering 11, no. 6: 565. https://doi.org/10.3390/bioengineering11060565

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop