Validation of a Novel Diagnostic Test for Assessing the Risk of Peri-Implantitis through the Identification of the Microorganisms Present: A Pilot Clinical Study of Periopoc

Featured Application

This study aims to validate a new diagnostic test that predicts the risk of peri-implantitis, thereby enabling early intervention and improved implant survival rates.

Abstract

The aim of this parallel group study was to determine the clinical applicability of a newly developed bacterial test. We evaluated the ability of the test to detect five bacteria associated with peri-implantitis: Porphyromonas gingivalis, Tannerella forsythia, Aggregatibacter actinomycetemcomitans, Prevotella intermedia, and Treponema denticola. Sensitivity, specificity, and positive and negative predictive values were established. Furthermore, by analyzing the pre-test and post-test probabilities, likelihood ratios were established, and a Fagan nomogram was constructed. As the standard, the clinical criteria of peri-implantitis adopted in the latest classification of peri-implant diseases of 2018 were used. The sample consisted of 13 patients clinically diagnosed with peri-implantitis (various implant brands) with at least 1 year of loading, of whom 11 were included in the study. The healthy group comprised 10 patients who received implants (Ticare inhex hybrid) at the university dental clinic and were monitored and exhibited no signs or symptoms of peri-implantitis during 1 year of loading. The results indicated that this test has high sensitivity and low specificity; therefore, positive results will be of great importance for a confirmatory diagnosis of peri-implantitis. However, the test is not suitable as a screening tool.

1. Introduction

Peri-implant diseases (PIDs) pose a clear threat to the maintenance of dental implants. For this reason, early detection is of great importance.

In general, PIDs include peri-implant mucositis and peri-implantitis (PI). In Spain, the frequency of PIDs is approximately 50%, with the percentage of PI being 20% at the implant level and 24% at the patient level [1].

According to the 2018 World Workshop, the characteristic clinical signs that allow the diagnosis of peri-implantitis are a greater probing depth, bleeding on probing, suppuration, and radiographic evidence of bone loss after initial healing or a combination of these [2]. They are all late signs.

Given the irreversible destruction of tissue caused by PI, the development of new diagnostic tests that identify patients at risk of early PI is necessary.

A simple way to assess peri-implant health status could be by monitoring peri-implant crevicular fluid (PICF). Through PICF, the presence of subgingival pathogens, markers of tissue destruction or host responses through cytokines could be identified [3,4,5,6,7].

Because PI is considered a dysbiosis (similar to periodontitis), the bacterial component is considered the main triggering factor [8,9].

Some of these bacteria correspond to the Socransky red and orange complexes [10]. When healthy implants (HIs) were compared with diseased implants with PI, at least 19 bacteria were identified, including Porphyromonas gingivalis and Tannerella forsythia. Aggregatibacter actinomycetemcomitans, Eikenella corrodens, Fusobacterium nucleatum, Prevotella intermedia, Porphyromonas gingivalis, and Tannerella forsythia [2]. Other lesser-known species have also been identified, such as Desulfobulbus spp., Filifactor alocis, and TM7 spp [11,12]. On the other hand, we know that the presence of these bacteria alone need not indicate disease and that their presence in healthy conditions has been documented by Socransky and other relevant sources [10]. Furthermore, some PIs appear to be more resistant to treatment. This could be due both to local or systemic factors and to differences in the composition of the subgingival biofilm [13,14]. Therefore, having a clinical test to individualize treatment (a chairside test), especially for those patients who are refractory to treatment, is of great clinical interest [15].

Metagenomics analyses have enabled a better understanding of the species involved. Its application as a point-of-care (POC) remains slow and is often expensive, making its clinical application difficult. Therefore, the development of a simple “chairside test” for use as a “POC” is of great interest as a screening measure.

Periopoc^® is a new test based on the identification of the five bacteria most frequently associated with PIDs and periodontitis (PO). Its application requires a total of 20 min, and the results are qualitative, i.e., the presence or absence of target bacteria. In addition, controls are included to determine if the samples were processed properly in terms of sample amount and the incorporation of reagents.

The objective of this study was to validate this new diagnostic test, which claims to identify the five bacteria most frequently associated with the presence of PID and PO, against the clinical gold standard.

2. Materials and Methods

2.1. Patient Selection and Data Collection

This was a nonconsecutive sequential cross-sectional study. A total of 23 patients were examined, 10 of whom did not have PI (HI group) and 13 of whom had (PI group). After applying the eligibility criteria, 21 patients were included; the samples for 2 patients in the PI group were discarded because they were contaminated.

All samples were collected by the same researcher (NM) from patients referred to the Periodontics Department of the University of Murcia.

The research protocol and study design were approved by the Ethics Committee of the University of Murcia (ID: 2076/2018). The study was conducted following the guidelines established for observational studies (STROBE guide) available through the EQUATOR network [16].

Since the patients diagnosed with peri-implantitis corresponded to patients outside our center and we lacked data at the time of prosthesis installation, we adopted the case definition criterion without previous data. In summary, in the absence of previous radiographs, radiographic bone level ≥3 mm in combination with BOP and probing depths ≥6 mm is indicative of peri-implantitis.

2.2. Implants Evaluated

There was a wide variety of brands of implants in patients in the PI group who were treated at the University Dental Clinic (COU) after being referred by their dentists. All patients had their implants loaded for at least 1 year but not more than 18 months. Among the implants, 6 had internal hexagonal connections (core vent), 4 had external hexagonal connections (Branemark), and 1 had a conical connection (Morse cone).

In the HI group, the implants were all placed at the COU by the same professional, and the patient was followed up with at 1 week, 1 month, 3 months, and 6 months. All the implants in patients in the HI group Ticare Hybrid implants (Ticare, Mozo Grau, Valladolid, Spain). Patients in the HI group had no clinical signs of inflammation or radiographic bone loss. All patients in the HI group had a follow-up at 12 months after prosthesis placement.

2.3. Inclusion and Exclusion Criteria

The inclusion criteria were as follows:

• Patients referred to the Periodontics Department of the University of Murcia, Morales Meseguer University Hospital;
• Patients >18 years of age;
• Functional load of implants longer than 1 year and no longer than 18 months;
• Patients clinically and/or radiographically diagnosed with peri-implantitis (PI group) with the parameters detailed above or diagnosed as healthy (HI group);
• Patients who provided consent to take a sample.
• The exclusion criteria were as follows:
• Patients whose suppuration or bleeding on probing did not allow taking an adequate sample;
• Patients with uncontrolled systemic diseases;
• Patients who had taken antibiotics or bisphosphonates in the last 3 months;
• Patients who have received peri-implant treatment in the last 3 months;
• Polypharmacy patients (3 or more medications);
• Patients with disabling mental illnesses.

2.4. Data Collection

All patients were informed of the purpose of this study. Patient sex, age, medical history, smoking and alcohol consumption habits, and periodontitis diagnosis were recorded. Clinical indices were also collected, as were the plaque index, the depth of the pocket, and the gingival index, which served to place patients in either the HI group or PI group.

To obtain a sample of the population to be studied, 13 patients who were clinically and radiographically diagnosed with PI were selected, 11 of whom were included in the study; 2 were excluded due to the impossibility of obtaining a sample without saliva, blood or pus contamination.

Control samples were obtained from 10 patients without PI with characteristics similar to those with peri-implantitis and followed up regularly in the periodontics department.

The samples were all obtained by the same researcher. Each sample was labeled to indicate whether the patient was in the HI group or the PI group.

2.5. PerioPoc^® Procedure

Sampling was carried out beginning in January 2022, with a data collection period of 3 months (Figure 1, Figure 2 and Figure 3).

Briefly, before obtaining the sample, the site was isolated with cotton rolls and air dried. Two sterile paper tips were gently inserted into the deepest probing sites, focusing on 4 different sites, i.e., mesial, distal, buccal, and lingual, and held for 30 s.

Samples contaminated with blood, saliva or pus were discarded immediately. The samples considered suitable were placed in a sterile Dappen Dish, 6 drops of solution A were added, and the dish was shaken for 30 s. The samples were incubated at 95 °C for 6 min, and subsequently, different reagents were applied following the manufacturer’s protocol (Figure 4).

2.6. Statistical Analysis

The study variables were the existence or absence of PI and the Periopoc^® results.

The data were anonymized and saved on the computer at the Periodontics Department in a Microsoft Office Excel file, which was later exported to SPSS (IBM Corp. Released 2015. IBM SPSS Statistics for Windows, Version 23.0. Armonk, Westchester County, NY, USA: IBM Corp) for the comparison of means.

Tukey analysis was performed, as was Student’s test for independent samples to compare age and plaque and gingival indices.

To demonstrate the validity of this qualitative test, disease prevalence, test validity (sensitivity and specificity), and test safety (negative predictive value and positive predictive value) were calculated. Likelihood ratios were calculated, and a Fagan nomogram was constructed. An online calculator was used to determine the sensitivity, specificity, predictive values, likelihood ratios, and Fagan nomogram (http://araw.mede.uic.edu/cgi-bin/testcalc.pl) URL (accessed on 25 June 2023).

2.7. Ethical Considerations

This study was conducted in accordance with the ethical principles of the Declaration of Helsinki [17] and its subsequent updates.

This study was approved by an ethics committee, as it was an investigation that included patients (authorization number: 2076/2018).

3. Results

Twenty-one patients were included in this study, and two patients in the PI group were discarded due to sample contamination. Figure 4 shows the distribution of the patients (Figure 5).

The mean age of the patients was 58.52 years (confidence interval (CI) = 54.05–62.99); the mean age of male patients was 58.71 years (CI = 49.92–67.50); and that of female patients was 58.42 years (CI = 52.45–64.39).

The PI group comprised 11 patients from the Murcia region. The mean age of the patients was 58.45 years (CI = 53.63–63.27); 7 were females, and 5 were males. The mean age of the males was 60.75 years (CI = 46.37–75.12), and for the females it was 57.14 years (CI 51.33–62.95).

Finally, the HI group comprised 10 patients (3 males and 7 females) with a mean age of 58.60 years. The mean age of the males was 56.00 years (CI = 27.78–84.21), and the mean age of the females was 59.71 years (CI = 47.02–72.40) (Figure 6).

The clinical characteristics of the control group and peri-implant group are reported in

Table 1. Clinical characteristics of the HI group.

Age	Sex	Probing Depth	Plaque Index	Gingival Index
48	H	2 mm	2	1
51	M	1 mm	1	1
77	M	1 mm	2	1
77	M	1 mm	2	1
65	M	1 mm	2	1
53	M	1 mm	1	1
41	M	1 mm	1	1
51	H	1 mm	2	1
69	H	1 mm	2	1
54	H	1 mm	1	1

and

Table 2. Clinical characteristics of the PI group.

Age	Sex	Probing Depth	Plaque Index	Gingival Index
62	H	5 mm	3	3
73	H	5 mm	3	3
55	H	5 mm	3	3
61	M	4 mm	2	2
63	M	5 mm	2	2
55	M	6 mm	3	3
51	M	8 mm	2	2
60	M	9 mm	3	3
53	H	5 mm	2	2
47	M	6 mm	2	2
63	M	8 mm	3	3

. As would be expected, plaque and bleeding scores were clearly higher in the IP group than in the control group (Figure 7 and Figure 8).

The mean probing depth was significantly deeper in the PI group than in the HI group (p < 0.001).

To determine the diagnostic value of this test, sensitivity and specificity were assessed, as they are traditional parameters that demonstrate the validity of qualitative tests.

The validity test results indicated greater specificity than sensitivity; that is, the test better identifies patients who have the disease.

Sensitivity (S) was calculated as the quotient of the number of true positives (patients who have peri-implantitis and the test has been positive) among the total number of patients with PI.

The probability that the test identified a patient with PI was 0.09.

Specificity (E) was calculated as the quotient of the true negatives (patients without PI and a negative test) among the total number of patients in the HI group, with a maximum value of one.

Specificity was used to determine the probability that the test results are negative for those who do not have PI.

Prevalence was considered the proportion of patients with PI; in this study, the prevalence was 0.52. All results are summarized in

Table 3. Summary of the test results.

Parameter	Value	%
Sensitivity	0.09	9.1%
Specificity	1	100%
Prevalence	0.52	52%
Positive Post-test	1	100%
Negative Post-test	0.5	50%
Positive LR+	Infinity
Negative LR	0.91

.

To construct the Fagan nomogram [18], probability ratios (likelihood ratio) were calculated and plotted on graphs. A Fagan nomogram is a tool used to interpret the results of a medical or diagnostic test and evaluate the probability that a person has a disease or health condition (Figure 9).

4. Discussion

The diagnosis of PDs and PI is based primarily on clinical measurements such as probing depth, insertion loss, and bleeding on probing together with radiographic examinations. These procedures identify damage that has already occurred but do not provide any information about disease activity or future evolution [19].

Currently, more than 700 species of bacteria have been identified as potentially responsible for PDs and PI [15]. These bacteria form a complex network called biofilm that enhances their pathogenicity [10,20].

The presence of these bacteria is relevant because they are the main etiological factor and serve as early indicators of the need to start treatment, thus avoiding unnecessary overtreatment [21,22,23]. However, a positive microbiological test result does not necessarily imply the presence of disease or the need to use an antibiotic [24]. Furthermore, with the growing tendency of resistant bacteria, the need for a diagnostic test is imperative to reduce the uncertainty of starting treatment [25].

The choice of implants with one to one and a half years of loading was made due to the aggressive and rapidly progressive nature of peri-implantitis. For this reason, its early diagnosis and early treatment could significantly change the long-term outcome, without taking into account the other variables that could have influenced the presence of early-onset and highly aggressive peri-implantitis.

Time, technical difficulties, and cost must be considered for the routine application of tests. Despite the many potential markers and tests that exist, only a few have verified clinical usefulness [26]. Unfortunately, there is a long way to go regarding the development and validation of easy and reliable tests for the identification of microbial or host markers [4,5,7].

Diagnostic tests aim to provide quantitative information about the presence or absence of a particular condition or disease. To assess the diagnostic capability of a test, performance is quantified by providing calculating sensitivity, specificity, positive and negative predictive values, and likelihood ratios. These measures help determine how well a test can correctly identify true positive and true negative cases.

Diagnostic tests are often evaluated by comparing their results with a reference standard, i.e., the gold standard for diagnosis. Currently, to assess the presence of PI, the gold standard is still clinical and radiographic findings.

To our knowledge, only one previous study has evaluated the diagnostic value of PerioPoc (Arweiler et al. [15]). The other relevant sources are technical notes and reports from the manufacturer.

Arweiler et al. [15] provided sensitivity (85.15%) and specificity (100%) values. Herein, those data were used to calculate the predictive values and likelihood ratios and, thus, construct a Fagan nomogram. The data provided by Arweiler et al. [15] pertain to 497 patients, of whom 397 had PI and a true positive test result on 338 occasions and a false-negative test result on 59 occasions. There were a total of 100 healthy individuals, with 100 true-negative test results and 0 false-positive test results. In our study, the sample consisted of 21 patients, 10 of whom were healthy and 11 of whom had PI.

Next, the data obtained herein are compared with the data reported by Arweiler et al. [15].

The prevalence of the disease indicates the proportion of healthy and non-healthy sick individuals that make up the study sample. In the study by Arweiler et al. [15], their sample was composed mainly of patients with PDs (79%), while in our study, the sample had an almost equal proportion of patients with PI (52%) and patients without PI. This proportion has little effect on sensitivity and specificity because both are intrinsic to the test. However, prevalence is of great importance in determining the predictive values, which are dependent on the sample size and prevalence.

Sensitivity (S) is the probability that a patient with PI will have a positive test result. It is calculated as the quotient of true positives among the total number of patients. For PI, a sensitive test could negate the need for continuous radiographic examinations. High test sensitivity would allow clinicians to treat patients without the need to resort to other diagnostic tests.

In our study, the sensitivity was 9.1%; in the study by Arweiler et al. [15], the sensitivity was 85.14%. This is a very large difference. This difference may have occurred because the patients in the study by Arweiler et al. [15] had PD and those in our study had PI. Another possible cause of the difference could be related to the enrichment of Aggregatibacter actinomycetemcomitans in the samples in the study by Arweiler et al. [15] because these bacteria are found in a low proportion in PD.

Specificity (E) is the ability of a test to identify healthy individuals. It is a measure that does not depend on sample size. It is calculated as the quotient of true negatives (healthy individuals with a negative result) among healthy individuals.

Specificity was high in both studies. Therefore, 100% of patients who test positive will have bacteria involved in PI. Our results are consistent with those reported by Arweiler et al. [15]. No patient had a false-positive result in either of the two studies.

Predictive values or post-test probabilities represent the probability that a patient has the disease once the test result is known.

Positive predictive value (PPV) is defined as the probability that an individual with a positive test has the disease and corresponds to patients with positive tests among all positive tests. PPV depends on prevalence, specificity, and, to a lesser extent, sensitivity.

In the study by Arweiler et al. [15], the PPV was 1 or 100%, and in our study, the PPV was also 100%. This value indicates that 100% of positive tests identify the presence of target bacteria in patients with PD or PI, indicating the need to eradicate these germs.

Negative predictive value (NPV) is defined as the probability that an individual with a negative test does not have the disease, i.e., the individual is truly healthy. In the study by Arweiler et al. [15], the NPV was 63%, and in our study, the NPV was 50%. Although the values are close, the probabilities are different. In the study by Arweiler et al. [15], a negative test indicated that 37% of the patients who had a negative test result actually had the disease. In our study, a negative test indicated that 50% of the individuals who had a negative test result actually had the disease, meaning that half of the patients with PI may not be diagnosed. The reason why this value is so high may be due to threshold levels for the detection of germs being too high. Arweiler et al. [15] reported levels between species that ranged from 10⁴ and 10² depending on the species. Another difference may be due to the use of enriched suspensions of bacteria because sometimes the amount of bacteria present was very low, and therefore, the authors enriched the samples with cultures saturated with bacteria, specifically A. actinomycetemcomitans. Finally, another possible cause is the different profiles of the patients; in the study by Arweiler et al. [15], the patients had PDs, and in our study, the patients had PI. The bacterial load required to develop PI is much lower than that required to develop PDs.

The probability of a result for an individual with a disease and an individual without a disease is often compared using likelihood ratios. The advantage of this index compared to PPV and NPV is that unlike these, likelihood ratios do not depend on the proportion of patients in the sample but only on test sensitivity and specificity, hence, its usefulness when comparing diagnostic tests.

In the study by Arweiler et al. [15] study and in our study, the LR+ value was infinite, indicating that a positive result for PD or PI is conclusive.

However, in the study by Arweiler et al. [15], the LR− was 0.15 (considered good), and in our study, the LR− was 0.91 (considered bad), indicating that in our study, there were limitations in detecting the presence of disease (

Table 4. Utility of a test based on likelihood ratios.

LR+	LR−	Utility
10	<0.1	Highly relevant
5–10	0.1–0.2	Good
2–5	0.5–0.2	Regular
<2	>0.5	Bad

).

Using LRs, a Fagan nomogram, which is a graphic representation of the level of reduction in diagnostic uncertainty, can be constructed.

In the study by Arweiler et al. [15], the subsequent probability was 37%, and in our study, the subsequent probability was 50%. These data indicate that the reduction in uncertainty when considering a healthy individual with a negative test does not exclude the presence of the disease. Again, the differences between the results for the two studies can be explained by the populations studied, i.e., patients with PDs in the study by Arweiler et al. [15] and patients with PI in our study. In the study by Arweiler et al. [15], the probability that a patient with a negative test presented a PD was 37%. In our study, the probability that a patient with PI had a negative test was 50%.

These values are useful in determining the diagnostic power of the test [19]. Recommended values are provided in the following table.

A negative probability ratio (LR−) of 0.91, which is less than 0.5, indicated that the test has limited utility in ruling out the presence of PI.

Therefore, Periopoc^® can serve as a confirmatory test for patients with clinically and radiographically confirmed PI, but it is not valid for screening individuals for the presence of PI.

Bayes theorem and pretest probability can be converted into a linear graph or Fagan nomogram [18]. Haga clic o pulse aquí para escribir texto.

A Fagan nomogram has three columns: pretest probability, that is, the probability of having the disease before performing the test; likelihood ratio; and post-test probability. A straight line is drawn from the pretest probability value to the likelihood ratio, and the point at which the line crosses into the third column represents the probability of having the disease based on the test result [20].

In this study, a line was drawn from the 52% pretest probability (prevalence of PI in our sample) toward infinity (positive likelihood ratio), and another line was drawn from the pretest probability to the negative likelihood ratio of 0.91, indicating that the post-test probability was 50%.

Based on the Fagan nomogram, for 100% of patients who are sick and tested, only 50% of those who test positive will be truly ill with PI and proceed to treatment. The status of the other 50% will remain uncertain.

Therefore, Periopoc^® is an effective tool when yielding positive test results because the result correctly identifies patients with PI who need to be treated. However, a large number of patients with PI with negative test results will require more tests for the diagnosis of this pathology.

Finally, this new qualitative test is an inefficient alternative to quickly diagnose a pathology that requires other tests for a certain diagnosis.

The results of this study indicate that Periopoc^® may be insufficient for the identification of the five main pathogens of the peri-implant sulcus. The test had low sensitivity but high specificity. Therefore, this test is effective as a test to confirm a diagnosis of PI; however, a negative result does not rule out the presence of PI.

Study Limitations

One limitation of this study is a possible breakdown of the cold chain of reagents, making it difficult to detect inaccurate or even false results. Regarding the transport of the reagents, they are very sensitive to small changes in temperature.

Another limitation is that all bacterial species could not be identified in peri-implant tissue; that is, other bacteria different from the five studied may be closely related to the appearance of PI, a possibility that is indicated by the results of metagenomics studies.

It is also possible that the detection threshold of Periopoc^® may be too high for the actual bacterial load that appears in peri-implant grooves, i.e., the amount of bacteria present is not sufficient to activate the reagents.

Another limitation is the small sample size, with a total of 21 patients. Despite the power achieved, it was not possible to establish a statistically significant result and demonstrate the validity of Periopoc^®. Therefore, it is necessary to increase the sample size in future relevant studies.

The significance of the implant prosthetic abutment connection in peri-implantitis development lies in its impact on the stability, hygiene, and susceptibility of the implant site to bacterial colonization. However, our study did not focus on the inclusion of the type of connection on the presence of peri-implantitis, but on the presence or absence of peri-implantitis itself and the test result tested.

5. Conclusions

• PI is a disease that increases the risk of implant survival. Periopoc is not a quick alternative for diagnosis;
• The reagents used in the test deteriorate with changes in temperature; therefore, the tests cannot be used in dental clinics without a guarantee that the cold chain was preserved;
• This test is very unspecific and very sensitive; therefore, positive results will be of great importance for confirming a diagnosis of PI, but negative results do not reduce the level of uncertainty of a patient having PI;
• Our results show a high specificity of 100% is an excellent diagnostic confirmation test, although it has a low sensitivity of 52% which does not allow us to safely rule out a negative result. This assessment is confirmed with the predictive values (pretest) and likelihood ratios (post-test) as well as with the Fagan normogram which expresses a positive value at infinity and a negative value of 50%.