Next Article in Journal
An Improved Method for Ship Target Detection Based on YOLOv4
Previous Article in Journal
A Review of the Recent Advances in Piezoelectric Materials, Energy Harvester Structures, and Their Applications in Analytical Chemistry
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Microbiological and Physical Changes Produced by Different Air–Powders on Contaminated Titanium Implant Surfaces: An In Vitro Pilot Study

by
Samy Francis
1,
Vito Carlo Alberto Caponio
2,
Francesca Spirito
2,
Vittoria Perrotti
3,* and
Alessandro Quaranta
4,5
1
Independent Researcher, Perth, WA 6000, Australia
2
Department of Clinical and Experimental Medicine, University of Foggia, 71122 Foggia, Italy
3
Department of Medical, Oral and Biotechnological Sciences, G. D’Annunzio University of Chieti-Pescara, 66100 Chieti, Italy
4
School of Dentistry, University of Sydney, Sydney, NSW 2010, Australia
5
Smile Specialists Suite, Newcastle, NSW 2300, Australia
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(3), 1301; https://doi.org/10.3390/app13031301
Submission received: 14 December 2022 / Revised: 10 January 2023 / Accepted: 13 January 2023 / Published: 18 January 2023

Abstract

:
Air–powder abrasive treatment has shown interesting results for dental implant treatments; however, which powder is most effective is still an open question. This in vitro pilot study aims to compare the ability of six different powders (sodium bicarbonate powder—65 µm and 40 µm; glycine powder; erythritol powder—with and without cetylpyridinium chloride and calcium carbonate) to remove biofilm from contaminated titanium discs and to evaluate the physical effects of such treatments on surface topography. Seventy-four titanium discs with two different surface roughness were treated after S. sanguinis contamination. Cleaning ability and surface changes were assessed by scanning electron and confocal laser scanning microscopy as well as profilometry. All treated surfaces showed minimal bacterial residues compared to untreated ones, regardless of the treatment provided (remaining biofilm range 11.4% to 28.4% on machined discs—range 10.7% to 18.3% on moderately rough surface discs). No relevant changes on the microscopic ultrastructure of the disc surfaces were noted. The different treatments reduced biofilm up to 89.3 and 88.6% on moderately rough and machined discs, respectively, and they all showed statistically significant superiority over calcium carbonate powder. None of the tested treatments rendered the disc surfaces biofilm-free. Therefore, combined mechanical and chemical decontamination methods are still recommended to achieve maximum biofilm removal for peri-implantitis treatment.

1. Introduction

Dental implants are nonbiologic devices surgically placed into jawbones to replace missing teeth. It is estimated that over 2 million dental implants are installed annually, and this number is expected to rise further over the next few years [1]. The peri-implant environment is a unique ecosystem [2], where the implant surface contacts the bone tissue by a phenomenon of incorporation called osseointegration. An osseointegrated implant is a sign of healthy and long-standing implant [3]. Unfortunately, this delicate ecosystem is continuously affected by different threats (e.g., biological, mechanical), leading to the loss of osseointegration. Peri-implantitis is the clinical sign of perturbations in the osseointegration; it is characterized by inflammation of the peri-implant connective tissue and by the progressive bone resorption, eventually associated with suppuration and bleeding until implant loss [4]. Bacterial plaque hosted in the oral cavity is still considered as the main risk factor for its onset [5]. Indeed, once bacteria biofilm accumulates on the implant surface, the inflammation leads to tissue destruction, and a greater implant surface is exposed, which becomes accessible to more pathogens [6], worsening the disease. Current evidence supports that peri-implant infection, caused by bacteria from dental biofilm, is the main cause of implant loss [7,8]. Therefore, the primary aim of peri-implantitis treatment is to remove the microbial biofilm [9], a crucial step for the survival of the implant [10].
Nonsurgical approaches for the treatment of peri-implant disease have been proved to be reliable in reducing clinical signs of peri-implant inflammation (e.g., bleeding on probing—BoP), although with limited capability to achieve complete disease resolution [11]. Surgical treatment of peri-implantitis includes access flap, removal of granulation tissue, implant surface decontamination and either guided bone regeneration (GBR) or pocket elimination resection [12]. In both approaches, implant surface decontamination is essential for the successful treatment of peri-implantitis. This can be achieved either chemically, using topical application of antiseptics [13] or antibiotics, or mechanically, using either air polishing abrasives or other available mechanical devices for biofilm removal with or without the combination of adjunctive antimicrobial agents [14]. This process has to be supported also by domiciliary compliance of patients in performing a good quality oral hygiene [15].
Today, the clinical setting is lacking reliable protocols as which regimens are the most effective in implant debridement while safeguarding the implant structure [16,17]. In this sense, air–powder abrasive treatment has shown interesting results [18,19]. It is based on the release of abrasive powder into a stream of compressed air to remove biofilm from the implant surface, and it is successfully employed in the treatment of peri-implantitis. However, there is still poor evidence on its effectiveness and adverse effects [20]. A range of abrasive powders are available, including sodium bicarbonate, amino-acid glycine salt, aluminum trioxide and calcium carbonate [21]. Lately, the use of the larger-particle size of sodium bicarbonate powder (65 µm) was shown to be more effective in biofilm removal but too abrasive to both teeth and implants surfaces. This led to the introduction to the market of a smaller-sized, amino-acid based particle, glycine (25 µm), as a less aggressive treatment option [22,23,24,25]. More recently, a smaller-sized particle powder, erythritol (14 µm), and a mixture of glycine and tricalcium phosphate [26,27] were utilized in implant decontamination. Erythritol is a natural sugar alcohol obtained through the reduction of erythrose [28]. It was noticed to be promising in removing biofilm, both in in vivo and in vitro studies [29,30]. While overall air abrasive powders showed superior decontamination power in comparison to other clinical modalities [19,31], studies comparing different air–powders in implant decontamination are very scarce and heterogeneous, with no superior material yet being established in clinical practice.
In this scenario, this in vitro pilot study aims to assess the ability of six different air–powder decontamination treatments to remove biofilm from contaminated titanium surfaces and to evaluate the effects of such treatments on titanium disc roughness and surface topography. We hypothesized that the different powders could show distinct biofilm cleaning efficacy and that they can alter discs’ surfaces diversely.

2. Materials and Methods

2.1. Study Design

Two types of pure titanium discs (9.0 mm diameter; 2 mm thickness, Southern Implants®, Irene, Gauteng, South Africa) with different surface roughness (Sa), machined surface rutile titanium (M), Sa: 0.04 µm, and moderately rough surface (MRS), Sa 1.3–1.5 µm, were tested in parallel to assess the ability of different powders to remove biofilm and to evaluate surface topography changes after treatment.
To our best knowledge, this is the first study comparing the ability of six different powders in removing biofilm; therefore, it was designed as a pilot study to generate data to run a power analysis for the sample size calculation for future studies. A total of 74 discs were used (37 M and 37 MRS): 68 were marked in the middle using a diamond bur, so that one half of the discs received treatment and the other half acted as a control. Six discs were left intact: two of these did not receive any treatment and acted as a control (contaminated untreated), and the remaining four discs were immersed in fresh Brain Heart Infusion (BHI) to assess the epifluorescence ability of BHI without bacteria. The experimental design is represented in Figure 1.

2.2. Assessment of Ability of Different Powders to Remove Biofilm

A sterile tube containing 10 mL of BHI was inoculated with a single colony of Streptococcus Sanguinis (S. sanguinis) and grown overnight for 24 h at 37 °C. Absorbance (OD600) of the culture was assessed and the culture transferred aseptically to a 15 mL Falcon tube and centrifuged at 3260 g for 15 min. The supernatant was gently poured off, and the bacteria were resuspended in an appropriate volume of fresh BHI medium containing 0.5% sucrose to yield an absorbance (OD 600) of 0.4.
A total of 56 scored contaminated and 2 control-contaminated untreated discs were transferred aseptically to culture plates, and 1 mL of bacterial culture was added to each disc. Discs were incubated at 37 °C for 72 h, with media being changed at 24 and 48 h. The remaining 4 unscored discs were incubated in 1 ml of fresh BHI + 0.5% sucrose for 72 h, without changing the medium. The other 12 scored uncontaminated discs (6 M and 6 MRS) were used without immersion in BHI to assess the fluorescence of different powders.
After 72 h, each of the 68 scored discs (28 M and 28 MRS incubated in bacterial culture and 12 discs not incubated in any culture) received treatment using different powders. Half of each disc was covered with parafilm, while the other half was left exposed to treatment.
An Original AirFlow® handpiece (EMS, Nyon, Switzerland) was kept at 5 mm distance from the disc surface. The angle of the handpiece was maintained at 90° [32,33,34], and treatment was standardized using a putty index (Figure 2A). Since the handpiece was stabilized in a perpendicular position in relation to the disk and the treatment was delivered through a stream of high pressured water and powder, most of the powder hit the disk perpendicularly, with some particle powders hitting the disk at a more acute angle which did not exceed 45°. Distilled water was used during treatment with different powders. Settings of the unit were kept to the maximum for both water and powder. Treatment was provided for 20 s in a back-and-forth movement, as follows:
(1)
Group 1 (4 M contaminated and 4 MRS contaminated, 1 M not contaminated, 1 MRS not contaminated): treated with air–powder mechanical instrumentation delivering sodium bicarbonate powder (Air flow powder Classic®, 65 µm, EMS, Nyon, Switzerland);
(2)
Group 2 (4 M contaminated and 4 MRS contaminated, 1 M not contaminated, 1 MRS not contaminated): treated with air–powder mechanical instrumentation delivering sodium bicarbonate powder (Air Flow powder Classic Comfort®, 40 µm, EMS, Nyon, Switzerland);
(3)
Group 3 (4 M contaminated and 4 MRS contaminated, 1 M not contaminated, 1 MRS not contaminated): treated with air–powder mechanical instrumentation delivering glycine powder (Air-Flow Powder Perio®, 25 µm, EMS, Nyon, Switzerland);
(4)
Group 4 (4 M contaminated and 4 MRS contaminated,1 M not contaminated, 1 MRS not contaminated): treated with air–powder mechanical instrumentation delivering erythritol powder (Air-Flow Powder Plus®, 14 µm, EMS, Nyon, Switzerland);
(5)
Group 5 (4 M contaminated and 4 MRS contaminated, 1 M not contaminated, 1 MRS not contaminated): Air Flow Plus® with 0.05% Cetylpyridinium chloride (CPC);
(6)
Group 6 (4 M contaminated and 4 MRS contaminated, 1 M not contaminated, 1 MRS not contaminated): treated with air–powder mechanical instrumentation delivering calcium carbonate (Prophy Pearls Classic®, 60–70 µm, Kavo, Biberach, Germany);
(7)
Group 7 (4 M contaminated and 4 MRS contaminated): treated with air–powder mechanical instrumentation delivering distilled water.
While half of each disc was still covered with the parafilm, discs were rinsed for 20 s using an air–water spray ejecting distilled water with air at 50 psi to remove potential powder deposits. The distance from the tip of the triplex syringe to the disc surface was 2 cm. This distance was standardized using a putty index (Figure 2B).
All the discs—including the untreated ones—were then rinsed in 3× 1 mL PBS and fixed for 24 h using 2.5% glutaraldehyde in PBS (74 discs in total). Fixed discs were rinsed in 3× 1 mL PBS and then were stored in another 1 ml of PBS.

2.3. Scanning Electron Microscopy (SEM) and Confocal Laser Scanning Microscopy (CLSM)

Prior to inoculation with bacterial biofilm, 10 titanium discs (5 M and 5 MRS) were evaluated by SEM (Zeiss 1555 VP-FESEM, Carl Zeiss MicroImaging GmbH, Gottingen, Germany) and CLSM (Eclipse Ti-E; Nikon, Tokyo, Japan) to provide a baseline.
In addition, the assessment of pristine discs, contaminated untreated discs and residual biofilm measurements after application of each treatment was conducted. Images were recorded using Nikon® Ti-E inverted motorized microscope with Nikon A1Si® spectral detector confocal system in 2 modes: reflection mode with emission wavelength of 575 nm and autofluorescence with emission wavelength of 525 nm. Reflection mode showed the surface topography of the disc only, while autofluorescence mode showed fluorescence of residual biofilm on each disc surface, allowing the quantitative assessment of residual biofilm volume after treatment. A pinhole of 127 nm and objective of 4× magnification were used during imaging. After imaging was completed, processing of the images was performed. A rectangle of 11.5 mm² was superimposed on each half of each disc, and fluorescence of each rectangle was measured and recorded by NIS-C Elements® software (version 5.2.1).
Following CLSM assessment, discs were dehydrated with graded ethanol solutions (50%, 70%, 80%, 90% and 100% ethanol solutions for 15 min each) and then dried in a critical point dryer. Discs were coated with platinum in preparation for SEM for qualitative analysis of surface changes. SEM assessment for surface alterations was performed at 1000× and 5000× magnifications using SE2 (WD: 20 mm, 3 KV) detector of Zeiss® 1555 VP-FESEM for both halves of each disc.

2.4. Evaluation of Surface Topography Changes after Treatment with Different Powders

Seven M and seven MRS discs were used to evaluate surface roughness changes after different treatments using a Zygo NewViewTM 6300® profilometer (New View 6300, Zygo Corporation, Middlefield, CT, USA) with 10× objective. The arithmetic mean of surface roughness (Ra) and root mean square of surface roughness (RMS) were determined for all the 14 discs included in this section of the experiment before any surface treatment was undertaken. Then treatment was provided for 20 s with the same experimental design mentioned above about the assessment of ability of different powders to remove biofilm (Section 2.2).
Ra and RMS of treated discs were assessed after treatment under the above-mentioned parameters.

2.5. Statistical Analysis

Statistical analyses for assessment of biofilm removal ability of different powders were made using one-way ANOVA, followed by Tukey test for multiple comparison. A p-Value of <0.05 was considered as statistically significant. All analyses were made using statistical software GraphPad Prism 7, GraphPad Software, La Jolla, CA, USA.

3. Results

3.1. CLSM

Both pristine M and MRS rendered minimal autofluorescence signal, same as samples immersed in BHI, without receiving any further treatment. Contaminated untreated MRS discs rendered more fluorescence compared to their machined counterpart. All treated surfaces showed significant reduction in fluorescence signal compared to untreated surfaces, regardless of the treatment provided.
Regarding M samples, overall reduced mean fluorescence reported exceeded 70%. The percentages of the disc surfaces free from the biofilm were for group 6, 71.6% (SD ± 5.4%); for group 2, 78.8% (SD ± 1.0%); for group 1, 80.5% (SD ± 0.9%); for group 5, 83.4% (SD ± 3.5%); for group 7, 86.8% (SD ± 1.8%); for group 4, 88.6% (SD ± 3.9%); and for group 3, 87.2% (SD ± 1.8%). Group 6 was found to be significantly less effective than distilled water and all the other tested powders. Group 3 was significantly more efficacious than groups 2 (p-Value 0.0395) and 6 (p-Value < 0.0001). In addition, group 4 was noticed to be significantly more active than groups 2 (p-Value 0.0121) and 6 (p-Value < 0.0001). Mean percentages of remaining bacteria are represented in Figure 3 and Figure 4.
As for the MRS sample, overall reduced mean fluorescence reported exceeded 80%. Respectively, the remaining biofilm decreased by 81.7% (SD ± 7.8%) for group 5; by 83% (SD ± 7.3%) for group 4, by 84.8% (SD ± 3.4%) for group 1; by 86% (SD ± 0.9%) for distilled water; by 86.1% (SD ± 5.3%) for group 3; by 88.2% (SD ± 1.9%) for group 2; and by 89.3% (SD ± 1.3%) for group 6 without statistically significant differences among any of the tested treatment modalities. Mean percentages of remaining biofilm are represented in Figure 5 and Figure 6.

3.2. SEM

Although machined discs looked smooth to naked eyes, they showed some minor surface irregularities under scanning electron microscope. Pristine uncontaminated discs, both M and MRS, showed a clear surface with no bacterial or powder residues. MRS discs showed a greater bacterial accumulation compared to M discs. The intact contaminated untreated discs, both M and MRS, showed more uniform bacterial accumulation compared to the contaminated part of the scored discs. All treated surfaces showed minimal to no bacterial residues compared to untreated surfaces, regardless of the treatment provided. No signs of disc microstructure alterations were observed in all the treated M and MRS samples (Figure 7—example for erythritol treatment).

3.3. Profilometry

Ra (arithmetic average of the absolute values of the profile height deviations from the mean line, recorded within the evaluation length) and RMS (the root mean square average of the profile height deviations from the mean line, recorded within the evaluation length) values of M surfaces were higher, such as reflecting higher surface roughness values, than their equivalent counterparts in MRS. Minor differences were noted in both Ra and RMS values before and after treatment. This difference is summarized in Table 1.

4. Discussion

The present study was designed to assess the ability of different powders to remove biofilm from contaminated titanium surfaces. To our best knowledge, this is the first study comparing the ability of six different powders in removing biofilm; therefore, it was designed as a pilot study to generate data to run a power analysis for the sample size calculation for future studies.
Results from this study highlighted differences on biofilm removal capacity of the various employed powders. Surface characteristics, such as roughness in M and MRS discs, affected the biofilm removal capacity, which was achieved by different kinds of powders in all the cases, although with different percentages (e.g., 88.6 ± 3.9% in M group 4 and 89.3 ± 1.3% in MRS group 6). Treatments were followed by no relevant changes to the microscopic ultrastructure of the disc surfaces. In this study, MRS and M contaminated treated samples showed significantly reduced residual biofilm after all provided treatment modalities, compared to untreated portions of contaminated discs. No statistically significant difference was noted in the biofilm removal ability of the tested modalities in MRS samples, while in M samples, group 3 (p-Value 0.0395) and group 4 (p-Value 0.0121) were significantly more effective than group 2 in biofilm removal. However, it should be noted that all provided treatments on M surfaces reduced the fluorescence signal to a level equivalent to or lower than the level of signal obtained from pristine discs.
A striking note to us was that distilled water was as effective in removing biofilm as all the other tested powders on both M and MRS surfaces. This finding could have been influenced by the fact that we tested a single micro-organism biofilm. Another influencing element could be the use of flat surface titanium discs, rather than screw-shaped implants. Both these factors could have influenced the ease of biofilm removal. Since this was a pilot study, we would recommend in future to repeat these treatments using a true multimicrobial biofilm on screw-shaped implants for more accurate and definitive outcomes.
Today, in clinical practice, implant surface decontamination is mandatory for a lasting successful implant treatment. Air–powder devices have been successfully employed to this purpose; however, no consensus exists on which powder should be used to maximize plaque removal while conserving the implant surface microstructure [19,31]. S. sanguinis was incubated for 3 days [35,36]. S. sanguinis displays an important role in the formation of bacterial plaque and is generally accepted as a suitable model for adherence studies [22,24,25,37]. In previous studies, sodium bicarbonate resulted effective in S. sanguinis biofilm removal [22,24,25]. However, despite the encouraging evidence coming from these previous studies, the efficacy of sodium bicarbonate was not further investigated in comparison with other powders or by assessing the structural changes of the titanium surface.
Concerning the ultrastructural changes, we ought to look at the changes in surface roughness values before and after treatment, rather than the absolute Ra and RMS values. It was noted that none of the tested treatments was able to cause changes in surface roughness values after 20s of treatment neither in M nor in MRS discs. These findings agree with other studies in the literature, which showed that different air–powders caused no titanium surface changes [31,38,39,40]. However, other studies showed surface roughness changes after different air–powder treatments [41,42,43,44]. The disparity between our findings and these studies findings could be attributed to either different tested titanium surfaces or different experiment parameters such as machine pressure, length of treatment time, angulation and distance of the handpiece from the tested surface or different settings of the machine used to measure surface roughness [19]. Moreover, most of previously published studies only performed a qualitative evaluation of such changes without a quantitative synthesis [19,38,45,46,47,48,49,50,51].
Considering the insignificant differences regarding biofilm removal among the different powders utilized, we would suggest the use of small-sized particle powders in everyday clinical practice. In fact, it has been shown that they are safe, effective and less aggressive on soft tissues [52,53]. Although in our controlled experiment no surface topography changes were induced using the larger particle air–powders, it has been shown in other studies that repeated use of large particle air-polishing powders caused changes to the titanium surface [26,27,47,54,55]. It is reasonable to consider small-particle size powders—such as erythritol—less aggressive on implant and tooth surfaces compared to large-particle size powders such as carbonates, as supported also by the results of a similar study [17].
As highlighted earlier, although there was significant improvement with all the tested modalities, none of the tested treatments rendered the tested surfaces biofilm-free. To date, this is one of the major shortcomings of peri-implant mechanical therapy, as none of the available mechanical treatments are able to completely remove the biofilm. This is of paramount importance during peri-implantitis treatment, as biofilm removal creates a biologically acceptable implant surface, an environment conducive to healing and regeneration. To compensate for this discrepancy, combined mechanical and chemical decontamination methods are recommended to achieve maximum biofilm removal during peri-implantitis treatment, matched with a collaborative self-care oral hygiene performed by patients.
One of the limitations of this study was the use of titanium discs instead of dental implants. Although dental implants are more complex in macro- and microtopography, the use of titanium discs—with identical surface topography to commercially available implants—has simplified the experiment, allowing for standardization. Another confounding factor was the use of S. sanguinis as a single micro-organism biofilm as a representative of the multifactorial polymicrobial biofilm. Although S. sanguinis is one of the early colonizers of the oral biofilm that changes the environment to allow for the colonization and growth of secondary colonizers, resulting in maturation of the biofilm [56], the use of single-organism biofilm might have had an effect on the biofilm removal ability of tested treatment modalities compared to a true polymicrobial biofilm. In addition, one of the constraints of this model of study is due to covering of half of the contaminated samples with parafilm. Although it protected the untreated part of the contaminated samples from being exposed to different powders and consequently removing biofilm during treatment, the mere action of covering the disc caused biofilm removal when it attached to the parafilm. Regarding profilometry, the use of limited number of samples is considered another limitation.
This pilot study paves the way for more complex investigations, overcoming the discussed limitations of this model, highlighting the importance of introducing to this field of study a standardized model of implant shapes and surfaces, multimicrobial-originated plaque and the need for both qualitative and quantitative assessment of microbial removal ability and ultrastructural changes of the titanium surface.

5. Conclusions

In conclusion, within the limitations of this pilot study, our outcomes suggest that biofilm grown on MRS surfaces can be removed to a similar extent by treatments tested to date. Different tested treatments were able to reduce biofilm up to 89.3% on MRS surfaces contaminated with S. sanguinis. Different tested treatments on M surfaces contaminated with S. sanguinis were able to reduce biofilm up to 88.6% and showed statistically significant superiority of all tested treatments over calcium carbonate powder. Within the tested parameters, treatment with different air–powders for 20 s did not alter the surface topography of both M and MRS surfaces.

Author Contributions

Conceptualization, S.F. and A.Q.; methodology, S.F., V.C.A.C. and V.P.; software, S.F.; validation S.F. and F.S.; formal analysis, S.F.; investigation, S.F., V.C.A.C., F.S., V.P. and A.Q.; resources, A.Q.; data curation, S.F.; writing—original draft preparation, S.F., V.C.A.C., F.S. and V.P.; writing—review and editing, V.P. and A.Q.; visualization, A.Q.; supervision, V.P. and A.Q.; project administration, A.Q.; funding acquisition, A.Q. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data can be requested to the corresponding authors.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Klinge, B.; Hultin, M.; Berglundh, T. Peri-implantitis. Dent. Clin. N. Am. 2005, 49, 661–676. [Google Scholar] [CrossRef] [PubMed]
  2. Barbagallo, G.; Santagati, M.; Guni, A.; Torrisi, P.; Spitale, A.; Stefani, S.; Ferlito, S.; Nibali, L. Microbiome differences in periodontal, peri-implant, and healthy sites: A cross-sectional pilot study. Clin. Oral Investig. 2022, 26, 2771–2781. [Google Scholar] [CrossRef] [PubMed]
  3. Hao, C.-P.; Cao, N.-J.; Zhu, Y.-H.; Wang, W. The osseointegration and stability of dental implants with different surface treatments in animal models: A network meta-analysis. Sci. Rep. 2021, 11, 13849. [Google Scholar] [CrossRef]
  4. Heitz-Mayfield, L.J. Peri-implant diseases: Diagnosis and risk indicators. J. Clin. Periodontol. 2008, 35, 292–304. [Google Scholar] [CrossRef]
  5. Zhang, Y.; Li, Y.; Yang, Y.; Wang, Y.; Cao, X.; Jin, Y.; Xu, Y.; Li, S.C.; Zhou, Q. Periodontal and Peri-Implant Microbiome Dysbiosis Is Associated with Alterations in the Microbial Community Structure and Local Stability. Front. Microbiol. 2021, 12, 785191. [Google Scholar] [CrossRef]
  6. Araujo, M.G.; Lindhe, J. Peri-implant health. J. Periodontol. 2018, 89 (Suppl. 1), S249–S256. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Buser, D.; Mericske-Stern, R.; Bernard, J.P.; Behneke, A.; Behneke, N.; Hirt, H.P.; Belser, U.C.; Lang, N.P. Long-term evaluation of non-submerged ITI implants. Part 1: 8-year life table analysis of a prospective multi-center study with 2359 implants. Clin. Oral Implants Res. 1997, 8, 161–172. [Google Scholar] [CrossRef] [PubMed]
  8. Ellegaard, B.; Baelum, V.; Karring, T. Implant therapy in periodontally compromised patients. Clin. Oral Implants Res. 1997, 8, 180–188. [Google Scholar] [CrossRef]
  9. Mombelli, A.; Lang, N.P. Microbial aspects of implant dentistry. Periodontol. 2000 1994, 4, 74–80. [Google Scholar] [CrossRef] [PubMed]
  10. Howe, M.S.; Keys, W.; Richards, D. Long-term (10-year) dental implant survival: A systematic review and sensitivity meta-analysis. J. Dent. 2019, 84, 9–21. [Google Scholar] [CrossRef]
  11. Roccuzzo, A.; De Ry, S.P.; Sculean, A.; Roccuzzo, M.; Salvi, G.E. Current Approaches for the Non-surgical Management of Peri-implant Diseases. Curr. Oral Health Rep. 2020, 7, 274–282. [Google Scholar] [CrossRef]
  12. Mahato, N.; Wu, X.; Wang, L. Management of peri-implantitis: A systematic review, 2010–2015. Springerplus 2016, 5, 105. [Google Scholar] [CrossRef] [Green Version]
  13. Tetè, G.; Cattoni, F.; Polizzi, E. Anti-discoloration system: A new chlorhexidine mouthwash. J. Biol. Regul. Homeost. Agents 2021, 35, 113–118. [Google Scholar] [CrossRef]
  14. Figuero, E.; Graziani, F.; Sanz, I.; Herrera, D.; Sanz, M. Management of peri-implant mucositis and peri-implantitis. Periodontol. 2000 2014, 66, 255–273. [Google Scholar] [CrossRef] [PubMed]
  15. Cattoni, F.; Tetè, G.; D’orto, B.; Bergamaschi, A.; Polizzi, E.; Gastaldi, G. Comparison of hygiene levels in metal-ceramic and stratified zirconia in prosthetic rehabilitation on teeth and implants: A retrospective clinical study of a three-year follow-up. J. Biol. Regul. Homeost. Agents 2021, 35, 41–49. [Google Scholar] [CrossRef] [PubMed]
  16. Grusovin, M.G.; Coulthard, P.; Worthington, H.V.; George, P.; Esposito, M. Interventions for replacing missing teeth: Maintaining and recovering soft tissue health around dental implants. Cochrane Database Syst. Rev. 2010, 2010, CD003069. [Google Scholar] [CrossRef]
  17. Matsubara, V.H.; Leong, B.W.; Leong, M.J.L.; Lawrence, Z.; Becker, T.; Quaranta, A. Cleaning potential of different air abrasive powders and their impact on implant surface roughness. Clin. Implant Dent. Relat. Res. 2020, 22, 96–104. [Google Scholar] [CrossRef] [PubMed]
  18. Francis, S.; Iaculli, F.; Perrotti, V.; Piattelli, A.; Quaranta, A. Titanium Surface Decontamination: A Systematic Review of In Vitro Comparative Studies. Int. J. Oral Maxillofac. Implants 2022, 37, 76–84. [Google Scholar] [CrossRef]
  19. Moharrami, M.; Perrotti, V.; Iaculli, F.; Love, R.M.; Quaranta, A. Effects of air abrasive decontamination on titanium surfaces: A systematic review of in vitro studies. Clin. Impl. Dent. Relat. Res. 2019, 21, 398–421. [Google Scholar] [CrossRef] [PubMed]
  20. Esposito, M.; Grusovin, M.G.; Worthington, H.V. Interventions for replacing missing teeth: Treatment of peri-implantitis. Cochrane Database Syst. Rev. 2012, 1, CD004970. [Google Scholar] [CrossRef]
  21. Petersilka, G.J. Subgingival air-polishing in the treatment of periodontal biofilm infections. Periodontol. 2000 2011, 55, 124–142. [Google Scholar] [CrossRef]
  22. Duarte, P.M.; Reis, A.F.; de Freitas, P.M.; Ota-Tsuzuki, C. Bacterial adhesion on smooth and rough titanium surfaces after treatment with different instruments. J. Periodontol. 2009, 80, 1824–1832. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Flemmig, T.F.; Arushanov, D.; Daubert, D.; Rothen, M.; Mueller, G.; Leroux, B.G. Randomized controlled trial assessing efficacy and safety of glycine powder air polishing in moderate-to-deep periodontal pockets. J. Periodontol. 2012, 83, 444–452. [Google Scholar] [CrossRef]
  24. Nemer Vieira, L.F.; Lopes de Chaves e Mello Dias, E.C.; Cardoso, E.S.; Machado, S.J.; Pereira da Silva, C.; Vidigal, G.M., Jr. Effectiveness of implant surface decontamination using a high-pressure sodium bicarbonate protocol: An in vitro study. Implant Dent. 2012, 21, 390–393. [Google Scholar] [CrossRef]
  25. Pereira da Silva, C.H.; Vidigal, G.M., Jr.; de Uzeda, M.; de Almeida Soares, G. Influence of titanium surface roughness on attachment of Streptococcus sanguis: An in vitro study. Implant Dent. 2005, 14, 88–93. [Google Scholar] [CrossRef]
  26. John, G.; Becker, J.; Schwarz, F. Effectivity of air-abrasive powder based on glycine and tricalcium phosphate in removal of initial biofilm on titanium and zirconium oxide surfaces in an ex vivo model. Clin. Oral Investig. 2016, 20, 711–719. [Google Scholar] [CrossRef] [PubMed]
  27. Tastepe, C.S.; Lin, X.; Werner, A.; Donnet, M.; Wismeijer, D.; Liu, Y. Cleaning effect of osteoconductive powder abrasive treatment on explanted human implants and biofilm-coated titanium discs. Clin. Exp. Dent. Res. 2018, 4, 25–34. [Google Scholar] [CrossRef]
  28. Moon, H.J.; Jeya, M.; Kim, I.W.; Lee, J.K. Biotechnological production of erythritol and its applications. Appl. Microbiol. Biotechnol. 2010, 86, 1017–1025. [Google Scholar] [CrossRef]
  29. Hägi, T.T.; Hofmänner, P.; Salvi, G.E.; Ramseier, C.A.; Sculean, A. Clinical outcomes following subgingival application of a novel erythritol powder by means of air polishing in supportive periodontal therapy: A randomized, controlled clinical study. Quintessence Int. 2013, 44, 753–761. [Google Scholar] [CrossRef]
  30. Hashino, E.; Kuboniwa, M.; Alghamdi, S.A.; Yamaguchi, M.; Yamamoto, R.; Cho, H.; Amano, A. Erythritol alters microstructure and metabolomic profiles of biofilm composed of Streptococcus gordonii and Porphyromonas gingivalis. Mol. Oral Microbiol. 2013, 28, 435–451. [Google Scholar] [CrossRef] [PubMed]
  31. Sahrmann, P.; Ronay, V.; Hofer, D.; Attin, T.; Jung, R.E.; Schmidlin, P.R. In vitro cleaning potential of three different implant debridement methods. Clin. Oral Implants Res. 2015, 26, 314–319. [Google Scholar] [CrossRef]
  32. Bühler, J.; Schmidli, F.; Weiger, R.; Walter, C. Analysis of the effects of air polishing powders containing sodium bicarbonate and glycine on human teeth. Clin. Oral Investig. 2015, 19, 877–885. [Google Scholar] [CrossRef] [PubMed]
  33. Tuchscheerer, V.; Eickholz, P.; Dannewitz, B.; Ratka, C.; Zuhr, O.; Petsos, H. In vitro surgical and non-surgical air-polishing efficacy for implant surface decontamination in three different defect configurations. Clin. Oral Investig. 2021, 25, 1743–1754. [Google Scholar] [CrossRef] [PubMed]
  34. Sawai, M.A.; Bhardwaj, A.; Jafri, Z.; Sultan, N.; Daing, A. Tooth polishing: The current status. J. Indian Soc. Periodontol. 2015, 19, 375–380. [Google Scholar] [CrossRef] [PubMed]
  35. Nakazato, G.; Tsuchiya, H.; Sato, M.; Yamauchi, M. In vivo plaque formation on implant materials. Int. J. Oral Maxillofac. Implants 1989, 4, 321–326. [Google Scholar] [PubMed]
  36. Wolinsky, L.E.; de Camargo, P.M.; Erard, J.C.; Newman, M.G. A study of in vitro attachment of Streptococcus sanguis and Actinomyces viscosus to saliva-treated titanium. Int. J. Oral Maxillofac. Implants 1989, 4, 27–31. [Google Scholar]
  37. Quintero, D.G.; Taylor, R.B.; Miller, M.B.; Merchant, K.R.; Pasieta, S.A. Air-Abrasive Disinfection of Implant Surfaces in a Simulated Model of Periimplantitis. Implant Dent. 2017, 26, 423–428. [Google Scholar] [CrossRef]
  38. Menini, M.; Piccardo, P.; Baldi, D.; Dellepiane, E.; Pera, P. Morphological and chemical characteristics of different titanium surfaces treated by bicarbonate and glycine powder air abrasive systems. Implant Dent. 2015, 24, 47–56. [Google Scholar] [CrossRef] [Green Version]
  39. Ronay, V.; Merlini, A.; Attin, T.; Schmidlin, P.R.; Sahrmann, P. In vitro cleaning potential of three implant debridement methods. Simulation of the non-surgical approach. Clin. Oral Implants Res. 2017, 28, 151–155. [Google Scholar] [CrossRef] [Green Version]
  40. Toma, S.; Lasserre, J.; Brecx, M.C.; Nyssen-Behets, C. In vitro evaluation of peri-implantitis treatment modalities on Saos-2osteoblasts. Clin. Oral Implants Res. 2016, 27, 1085–1092. [Google Scholar] [CrossRef]
  41. Kister, F.; Specht, O.; Warkentin, M.; Geis-Gerstorfer, J.; Rupp, F. Peri-implantitis cleaning instrumentation influences the integrity of photoactive nanocoatings. Dent. Mater. 2017, 33, e69–e78. [Google Scholar] [CrossRef] [PubMed]
  42. Kreisler, M.; Kohnen, W.; Christoffers, A.B.; Götz, H.; Jansen, B.; Duschner, H.; d’Hoedt, B. In vitro evaluation of the biocompatibility of contaminated implant surfaces treated with an Er: YAG laser and an air powder system. Clin. Oral Implants Res. 2005, 16, 36–43. [Google Scholar] [CrossRef] [PubMed]
  43. Cafiero, C.; Aglietta, M.; Iorio-Siciliano, V.; Salvi, G.E.; Blasi, A.; Matarasso, S. Implant surface roughness alterations induced by different prophylactic procedures: An in vitro study. Clin. Oral Implants Res. 2017, 28, e16–e20. [Google Scholar] [CrossRef] [PubMed]
  44. Ramaglia, L.; di Lauro, A.E.; Morgese, F.; Squillace, A. Profilometric and standard error of the mean analysis of rough implant surfaces treated with different instrumentations. Implant Dent. 2006, 15, 77–82. [Google Scholar] [CrossRef]
  45. Barnes, C.M.; Fleming, L.S.; Mueninghoff, L.A. SEM evaluation of the in-vitro effects of an air-abrasive system on various implant surfaces. Int. J. Oral Maxillofac. Implants 1991, 6, 463–469. [Google Scholar]
  46. Chairay, J.P.; Boulekbache, H.; Jean, A.; Soyer, A.; Bouchard, P. Scanning electron microscopic evaluation of the effects of an air-abrasive system on dental implants: A comparative in vitro study between machined and plasma-sprayed titanium surfaces. J. Periodontol. 1997, 68, 1215–1222. [Google Scholar] [CrossRef]
  47. Koka, S.; Han, J.; Razzoog, M.E.; Bloem, T.J. The effects of two air-powder abrasive prophylaxis systems on the surface of machined titanium: A pilot study. Implant Dent. 1992, 1, 259–265. [Google Scholar] [CrossRef]
  48. Parham, P.L., Jr.; Cobb, C.M.; French, A.A.; Love, J.W.; Drisko, C.L.; Killoy, W.J. Effects of an air-powder abrasive system on plasma-sprayed titanium implant surfaces: An in vitro evaluation. J. Oral Implantol. 1989, 15, 78–86. [Google Scholar]
  49. Razzoog, M.E.; Koka, S. In vitro analysis of the effects of two air-abrasive prophylaxis systems and inlet air pressure on the surface of titanium abutment cylinders. J Prosthodont. 1994, 3, 103–107. [Google Scholar] [CrossRef] [Green Version]
  50. Schwarz, F.; Ferrari, D.; Popovski, K.; Hartig, B.; Becker, J. Influence of different air-abrasive powders on cell viability at biologically contaminated titanium dental implants surfaces. J. Biomed. Mater. Res. B Appl. Biomater. 2009, 88, 83–91. [Google Scholar] [CrossRef]
  51. Tastepe, C.S.; Liu, Y.; Visscher, C.M.; Wismeijer, D. Cleaning and modification of intraorally contaminated titanium discs with calcium phosphate powder abrasive treatment. Clin. Oral Implants Res. 2013, 24, 1238–1246. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  52. Müller, N.; Moëne, R.; Cancela, J.A.; Mombelli, A. Subgingival air-polishing with erythritol during periodontal maintenance: Randomized clinical trial of twelve months. J. Clin. Periodontol. 2014, 41, 883–889. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. Petersilka, G.; Faggion, C.M., Jr.; Stratmann, U.; Gerss, J.; Ehmke, B.; Haeberlein, I.; Flemmig, T.F. Effect of glycine powder air-polishing on the gingiva. J. Clin. Periodontol. 2008, 35, 324–332. [Google Scholar] [CrossRef] [PubMed]
  54. Bennani, V.; Hwang, L.; Tawse-Smith, A.; Dias, G.J.; Cannon, R.D. Effect of Air-Polishing on Titanium Surfaces, Biofilm Removal, and Biocompatibility: A Pilot Study. Biomed. Res. Int. 2015, 2015, 491047. [Google Scholar] [CrossRef] [Green Version]
  55. Homiak, A.W.; Cook, P.A.; DeBoer, J. Effect of hygiene instrumentation on titanium abutments: A scanning electron microscopy study. J. Prosthet. Dent. 1992, 67, 364–369. [Google Scholar] [CrossRef]
  56. Socransky, S.S.; Haffajee, A.D.; Cugini, M.A.; Smith, C.; Kent, R.L., Jr. Microbial complexes in subgingival plaque. J. Clin. Periodontol. 1998, 25, 134–144. [Google Scholar] [CrossRef]
Figure 1. Assessment of ability of different powders to remove biofilm—experimental design. Specifically, a total of 34 machined (M) and 34 moderately rough surface (MRS) discs were examined. A total of 28 M and 28 MRS were contaminated, while 6 M and 6 MRS served as controls and were left uncontaminated. Thus, 4 M and 4 MRS discs were treated with sodium bicarbonate powder 65 µm (group 1), sodium bicarbonate powder 40 µm (group 2), glycine powder (group 3), erythritol powder (group 4), and the addition of cetylpyridinium chloride (group 5), calcium carbonate (group 6), and distilled water (group 7), respectively; the same treatments were applied to 1 not contaminated disc per group.
Figure 1. Assessment of ability of different powders to remove biofilm—experimental design. Specifically, a total of 34 machined (M) and 34 moderately rough surface (MRS) discs were examined. A total of 28 M and 28 MRS were contaminated, while 6 M and 6 MRS served as controls and were left uncontaminated. Thus, 4 M and 4 MRS discs were treated with sodium bicarbonate powder 65 µm (group 1), sodium bicarbonate powder 40 µm (group 2), glycine powder (group 3), erythritol powder (group 4), and the addition of cetylpyridinium chloride (group 5), calcium carbonate (group 6), and distilled water (group 7), respectively; the same treatments were applied to 1 not contaminated disc per group.
Applsci 13 01301 g001
Figure 2. (A) Handpiece at 90° angle and 5 mm distance from the disc receiving treatment. (B) Triplex Syringe at 2 cm from the disc receiving treatment.
Figure 2. (A) Handpiece at 90° angle and 5 mm distance from the disc receiving treatment. (B) Triplex Syringe at 2 cm from the disc receiving treatment.
Applsci 13 01301 g002
Figure 3. Percentage of remaining biofilm after decontamination of titanium discs with machined surface (M) by means of air abrasive unit delivering different powders.
Figure 3. Percentage of remaining biofilm after decontamination of titanium discs with machined surface (M) by means of air abrasive unit delivering different powders.
Applsci 13 01301 g003
Figure 4. Autofluorescence of bacterial biofilm on M discs before and after treatment with air abrasive unit delivering different powders: (A) pristine disc; (B) contaminated untreated disc; (C) group 7; (D) group 1; (E) group 2; (F) group 6; (G) group 3; (H) group 4; (I) group 5; (J) group 4 with boxes for fluorescence measurement; (K) diagram showing disc with fluorescence measurement rectangles. Magnification 4× and rectangle area of 11.5 mm2.
Figure 4. Autofluorescence of bacterial biofilm on M discs before and after treatment with air abrasive unit delivering different powders: (A) pristine disc; (B) contaminated untreated disc; (C) group 7; (D) group 1; (E) group 2; (F) group 6; (G) group 3; (H) group 4; (I) group 5; (J) group 4 with boxes for fluorescence measurement; (K) diagram showing disc with fluorescence measurement rectangles. Magnification 4× and rectangle area of 11.5 mm2.
Applsci 13 01301 g004aApplsci 13 01301 g004b
Figure 5. Percentage of remaining biofilm after decontamination of titanium discs with a moderately rough surface (MRS) by means of air abrasive unit delivering different powders.
Figure 5. Percentage of remaining biofilm after decontamination of titanium discs with a moderately rough surface (MRS) by means of air abrasive unit delivering different powders.
Applsci 13 01301 g005
Figure 6. Autofluorescence of bacterial biofilm on MRS discs before and after treatment with air abrasive unit delivering different powders: (A) pristine disc; (B) contaminated untreated disc; (C) group 7; (D) group 1; (E) group 2; (F) group 6; (G) group 3; (H) group 4; (I) group 5; (J) group 1 with rectangles for fluorescence measurement.
Figure 6. Autofluorescence of bacterial biofilm on MRS discs before and after treatment with air abrasive unit delivering different powders: (A) pristine disc; (B) contaminated untreated disc; (C) group 7; (D) group 1; (E) group 2; (F) group 6; (G) group 3; (H) group 4; (I) group 5; (J) group 1 with rectangles for fluorescence measurement.
Applsci 13 01301 g006
Figure 7. Samples of SEM imaging: (A) pristine M disc (1000×); (B) pristine M discs (5000×); (C) pristine MRS disc (1000×); (D) pristine MRS disc (5000×); (E) intact contaminated untreated M disc (1000×); (F) intact contaminated untreated M disc (5000×); (G) intact contaminated untreated MRS disc (1000×); (H) intact contaminated untreated MRS disc (5000×); (I) untreated half of contaminated M disc (1000×); (J) group 4 treated half of contaminated M disc (1000×); (K) untreated half of contaminated M disc (5000×); (L) group 4 treated half of contaminated M disc (5000×); (M) untreated half of contaminated MRS disc (1000×); (N) group 4 treated half of contaminated MRS disc (1000×); (O) untreated half of contaminated MRS disc (5000×); (P) group 4 treated half of contaminated MRS disc (5000×).
Figure 7. Samples of SEM imaging: (A) pristine M disc (1000×); (B) pristine M discs (5000×); (C) pristine MRS disc (1000×); (D) pristine MRS disc (5000×); (E) intact contaminated untreated M disc (1000×); (F) intact contaminated untreated M disc (5000×); (G) intact contaminated untreated MRS disc (1000×); (H) intact contaminated untreated MRS disc (5000×); (I) untreated half of contaminated M disc (1000×); (J) group 4 treated half of contaminated M disc (1000×); (K) untreated half of contaminated M disc (5000×); (L) group 4 treated half of contaminated M disc (5000×); (M) untreated half of contaminated MRS disc (1000×); (N) group 4 treated half of contaminated MRS disc (1000×); (O) untreated half of contaminated MRS disc (5000×); (P) group 4 treated half of contaminated MRS disc (5000×).
Applsci 13 01301 g007
Table 1. Evaluation of RA (roughness average) and RMS (root mean square) before and after treatment with different powders, expressed as means and standard deviations (S.D.) for machined surface and moderately rough surface discs.
Table 1. Evaluation of RA (roughness average) and RMS (root mean square) before and after treatment with different powders, expressed as means and standard deviations (S.D.) for machined surface and moderately rough surface discs.
TreatmentsMachined DiscsModerately Rough Surface
RMS Means ± S.D.Ra Means ± S.D.RMS Means ± S.D.Ra Means ± S.D.
Group 1—before treatment1.40 ± 0.311.10 ± 0.233.22 ± 0.212.45 ± 0.35
Group 1—after treatment1.60 ± 0.341.30 ± 0.223.01 ± 0.462.30 ± 0.45
Group 2—before treatment3.01 ± 0.442.47 ± 0.312.52 ± 0.401.95 ± 0.41
Group 2—after treatment3.14 ± 0.292.60 ± 0.352.55 ± 0.371.99 ± 0.22
Group 3—before treatment2.01 ± 0.301.61 ± 0.332.44 ± 0.471.89 ± 0.21
Group 3—after treatment2.04 ± 0.371.67 ± 0.322.42 ± 0.391.88 ± 0.38
Group 4—before treatment2.04 ± 0.291.57 ± 0.412.73 ± 0.182.13 ± 0.23
Group 4—after treatment2.14 ± 0.371.70 ± 0.32.77 ± 0.322.17 ± 0.25
Group 5—before treatment2.30 ± 0.441.86 ± 0.433.38 ± 0.252.57 ± 0.15
Group 5—after treatment2.49 ± 0.372.02 ± 0.393.40 ± 0.312.59 ± 0.21
Group 6—before treatment2.75 ± 0.212.22 ± 0.262.71 ± 0.282.10 ± 0.39
Group 6—after treatment2.60 ± 0.292.10 ± 0.332.48 ± 0.41)1.96 ± 0.15
Group 7—before treatment1.72 ± 0.421.07 ± 0.313.19 ± 0.372.42 ± 0.22
Group 7—after treatment1.60 ± 0.390.93 ± 0.372.96 ± 0.332.23 ± 0.29
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Francis, S.; Caponio, V.C.A.; Spirito, F.; Perrotti, V.; Quaranta, A. Microbiological and Physical Changes Produced by Different Air–Powders on Contaminated Titanium Implant Surfaces: An In Vitro Pilot Study. Appl. Sci. 2023, 13, 1301. https://doi.org/10.3390/app13031301

AMA Style

Francis S, Caponio VCA, Spirito F, Perrotti V, Quaranta A. Microbiological and Physical Changes Produced by Different Air–Powders on Contaminated Titanium Implant Surfaces: An In Vitro Pilot Study. Applied Sciences. 2023; 13(3):1301. https://doi.org/10.3390/app13031301

Chicago/Turabian Style

Francis, Samy, Vito Carlo Alberto Caponio, Francesca Spirito, Vittoria Perrotti, and Alessandro Quaranta. 2023. "Microbiological and Physical Changes Produced by Different Air–Powders on Contaminated Titanium Implant Surfaces: An In Vitro Pilot Study" Applied Sciences 13, no. 3: 1301. https://doi.org/10.3390/app13031301

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop