Methodological Quality of Consensus Guidelines in Implant Dentistry (Faggion et al. 2017)

Clinicians rely heavily on consensus guidelines when they develop treatment protocols for their patients. The idea is that the most experienced clinicians and academics meet and systematically develop what they feel is best practice in their particular field. Health care providers and regulators then rely on this distillation of knowledge, experience and evidence in their decision making. A systematic review of guideline quality in peer-reviewed medical literature was conducted in 1999  and found only a 43% adherence to reporting standards (Shaneyfelt et al. 1999). In 2003, a generic tool was created by the AGREE (Appraisal of Guidelines, REsearch and Evaluation) collaboration to create “common standards to improve the quality process and reporting of guideline development” (Cluzeau et al. 2003). With use the protocol evolved into AGREE II (Brouwers et al. 2010) which is made up of 23 items in 6 domains:

  1. Scope and Purpose
  2. Stakeholder Involvement
  3. Rigour of Development
  4. Clarity of Presentation
  5. Applicability
  6. Editorial Independence

The research question reference consensus guidelines in implant dentistry was: “Do consensus guidelines published in high ranked implant journals meet the requirements proposed in the AGREE II instrument? A secondary objective was to evaluate whether the inclusion of systematic reviews conducted to support the consensus guidelines improved their methodological quality.

Methods

Two authors (KA,MA) independently searched and evaluated consensus guidelines from the 6 highest impact implant dentistry journals (assigned by Journal Citation Reports) and Medline database via PubMed. The search was limited to May 2009 – February 2016 in line with the publication of AGREE II. Reasons for papers to be excluded was recorded.

Four authors (KA,TA,LM,MA), following assessor training then independently applied the AGREE II tools to the consensus guidelines as described in the user’s manual. Domain scores were presented as median percentages of the maximum possible with their respective interquartile ranges. Domain scores were divided into consensus guidelines, and consensus guidelines with systematic reviews.

Results

  • From an initial 258 publications 27 consensus guidelines fulfilled the inclusion criteria of which 19 was included for comparison
  • The journals were: Clinical Oral Implants Research (COIR), Clinical Implant Dentistry and Related Research (CIDRR), European Journal of Oral Implants (EJOI), The International Journal of Oral and Maxillofacial Implants (JOMI), Journal of Oral Implantology, and Implant Dentistry.
  • 26 guidelines were developed after meetings in Europe.
  • The European Association of Osseointigration (EAO) developed the most guidelines (n=9)
  • The number of authors ranged from 2-27 (median, 9)

Primary Outcomes

AGREE II Domains Consensus guidelines only.

Median is % of maximum score

Consensus guidelines plus systematic reviews.

 

Domain 4 (Clarity of presentation) Median, 75; (IQR 15.30) Median, 84.70;(IQR, 9.80).
Domain 1 (Scope and Purpose) Median, 69.40; (IQR, 36.20). Median, 79.20; (IQR, 73)
Domain 2 (Stakeholder Involvement) Median, 41.70; (IQR, 17.70) Median, 76.40; (IQR,18.10)
Domain 6 (Editorial Independence) Median, 41.7 ;(IQR, 83.30) Median, 56.30 (IQR, 34.40)
Domain 3 (Rigour of Development) Median, 30.70; (IQR, 26.50) Median, 50 (IQR,44.40)
Domain 5 (Applicability) Median, 26;(IQR, 12.50) Median, 26; (IQR, 20.80)

Conclusion (the author concluded)

“Methodological improvement of consensus guidelines published in major implant dentistry journals is needed. The findings of the present study may help researchers to better develop consensus guidelines in implant dentistry, which will improve the quality and trust of information needed to make proper clinical decisions”.

Comments

For the clinician in practice placing or restoring on dental implants this is a very important paper and goes a long way in explaining the disjoint between what is presented on the international conference circuit and what we see in day-day practice.

To help with interpreting the authors data I have modified it slightly to present the data as median with 95% confidence intervals and place the meta-analysis it in a forest plot.

The first chart show the AGREE II scores relating to the consensus guidelines that did not include the use of systematic reviews. The highest scoring domains were Domain 4. (Clarity of Presentation) and Domain 1. (Scope and Purpose) which score in the 70’s. There is a gap and the next four Domains in rank order are; 6. (Editorial Independence), 2. (Stakeholder Involvement), 3. (Rigour of Development) and lastly 6. (Applicability). These narrative guidelines therefore only fulfil 50% of the AGREE II criteria.

Agree II narr

The second chart show a meta-analysis comparing the narrative guideline scores compared with the consensus guidelines + systematic reviews, and there is roughly a 9% improvement bringing the score up to 60% compliance with AGREE II.

Forest plot

What becomes obvious however is that 18% improvement came out of Domain 3. (Rigour of Development) but 34% from Domain 6. (Editorial Independence). Going back then to the meta-analysis of consensus guidelines + systematic reviews the improvement has been weak and polarises the results.

Agree II SR

From a clinicians point of view what options do we have available to us to improve consensus guidelines?

 

  • Domains 1 (Scope and Purpose) and 4 (Clarity of Presentation) are good, we know what questions to ask and how to present them.
  • Domain 6 (Editorial Independence) is improving but there still needs to be more clarity over the editorial independence from funding bodies and conflicts of interest from the funding bodies and authors.
  • Domain 3. (Rigour of Development) More effort needs to be spent in this area as is is the foundation on which the guidelines are built, based on validated systematic methods for assessing and analysing the relevant primary research. Analysis also needs to include health benefits, side effects and complications. The guidelines should also be externally validated prior to publication.
  • Domain 2 (Stakeholder Involvement) at present is dominated by University professors and private practitioners with strong academic connections. This area would benefit from the views of clinicians working outside of specialist practice, health care providers, funders and patient groups.
  • Domain 6. (Applicability) is the weakest domain, this may be in part due to the lack of input at the stakeholder level . For clinicians in practice, and their patients cost implications are a major barrier to applying guideline recommendations.

 Links

 Primary paper

Faggion, C.M.J. et al., 2017. Methodological Quality of Consensus Guidelines in Implant Dentistry. PloS one, 12(1), p.e0170262. Available at: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=prem&NEWS=N&AN=28107405.

 Other references

Brouwers, M.C. et al., 2010. AGREE II: Advancing guideline development, reporting and evaluation in health care. Journal of Clinical Epidemiology, 63(12), pp.1308–1311.

Cluzeau, F. et al., 2003. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Quality & safety in health care, 12(1), pp.18–23. Available at: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1743672&tool=pmcentrez&rendertype=abstract.

Shaneyfelt, T.M., Mayo-Smith, M.F. & Rothwangl, J., 1999. Are guidelines following guidelines? JAMA: The Journal of the American Medical Association, 281(20), pp.1900–1905.

Attrition and implant research

Appraisal of “Clinical and Radiographic Outcomes of Dental Implant Therapy”*  by Mark-Steven Howe

*(J Derks et al. 2015; Jan Derks et al. 2015; Derks, J. Schaller, D. Håkansson, J. Wennström, J.L. Tomasi, C. and Berglundh 2015)

implant astra

Introduction

The initial study protocol(Jan Derks et al. 2015) was created to evaluate patient-reported outcomes following implant-supported restorative therapy in a randomly selected patient sample taken from the Swedish Social Insurance Agency. Having selected the data set the research evolved into a further study on peri-implantitis and implant survival. This was registered with ClinicalTrials.gov (NCT01825772) in 2013.

The research was broken into two papers using the same data set:

  1. The primary research question was to report on the prevalence, extent, and severity of peri-implantitis in a large and randomly selected patient sample identified from the data register of the Swedish Social Insurance Agency.(Derks, J. Schaller, D. Håkansson, J. Wennström, J.L. Tomasi, C. and Berglundh 2015)
  2. The secondary research question was to report on the loss of dental implants assessed in a large and randomly selected patient sample. (J Derks et al. 2015)

Methods:

A sample of 4716 patients aged between 45 and 75 who had received implants in 2003 were randomly selected from the data registered with the Swedish Social Insurance Agency. From the 3827 respondents, patient files were collected on 2765 for analysis. From this group 900 randomly selected patient were invited to a clinical examination. Nine years following implant placement 596 of the 900 were examined by two periodontists, the clinical examination also included radiographs.

The patients were divided into two groups by age (45-54 and 65-74). Two examiners extracted from the files basic patient data that included information on diabetes, cardiovascular diseases, periodontitis at the time of implant therapy, smoking and recall frequency. Clinical evaluation included data on private or public dental clinical setting, general practitioner or registered specialist for the Swedish National Board of Health and Welfare. Additional data was collected on whether the operators were specialists or general practitioners. Implants were categorized according to brand, length (

A multiple logistic regression model was used to analyse the data.

Results

 Peri-implantitis:

  “In 98 (23.0%) of the 427 patients with baseline radiographs, no signs of peri-implant disease were detected. In addition,137 (32.0%) patients exhibited only peri-implant mucositis, while 192 (45.0%) presented with peri-implantitis. Moderate/severe peri-implantitis was observed in 62 (14.5%) patients.”

Peri-implantitis effecting implants by brand

Implant brand Odds Ratio 95% Confidence interval
Straumann Dental Implant System 1
Brånemark System Replace Select 3.77 1.60 to 8.87 0.002
Astra Tech 3.55 1.29 to 9.77 0.014
Remaining implants 5.56 1.70 to 18.24 0.005

Implant Survival:

Early Loss Late Loss Total Loss @ 8.9 years
Patients affected 4.4% 4.2% 7.6%
Implants Lost 1.4% 2.0% 3.0%
Implant brand Odds Ratio Odds Ratio
Straumann Dental Implant System 1 1
Brånemark System Replace Select 1.94 6.13
Astra Tech 2.10 5.23
Remaining implants 7.79 58.15

 Authors conclusions

Patients with periodontitis and with ≥4 implants, as well as implants of certain brands and prosthetic therapy delivered by general practitioners, exhibited higher odds ratios for moderate/severe peri-implantitis. Similarly, higher odds ratios were identified for implants installed in the mandible and with crown restoration margins positioned ≤1.5 mm from the crestal bone at baseline. It is suggested that peri-implantitis is a common condition and that several patient and implant-related factors influence the risk for moderate/severe peri-implantitis

In terms of implant loss, the present study reported on outcomes in implant dentistry assessed in a large and randomly selected patient sample representing effectiveness of the treatment procedures. Almost 8% of patients had lost ≥1 implants, and several patient and implant-related factors influencing early and late occurring loss were detected.

Comments

This was a well-conducted long-term retrospective cross-sectional study of a large randomly selected group of Swedish patients treated by both specialists and general practitioners. It is interesting to note it was registered on ClinicalTrials.gov when it is not technically a clinical trial. The registered protocol inclusion criteria was for 55 – 85 year olds but appears to have changed later to 45-54 and 65-74.

The only major problem with the results was the effect of attrition bias (Schulz & Grimes 2002). After initial randomisation 41% of the data was not available/excluded and from this reduced data set a further 900 samples were randomly selected of which 34% of the data was not collected. Since this data had been randomised this loss of data will most likely bias the results and affect the internal validity of the study, as it is unlikely to be all due to random causes. For example, patients who have had poorer outcomes or are less motivated may exclude themselves from the consent process or clinical examination phase. Though considerably more time consuming a clearer picture of the prevalence of peri-implantitis and survival rates could have been achieved if an ‘intention-to-treat principal’ had been followed. That is taking the whole randomised populations data into account when producing the results either via sensitivity analysis or imputation of the missing data.By taking out the data relating to patients who died, are sick or moved house (assume increased failures among the non-attenders) one can impute the total loss figure to be somewhere between 2.o% and 11.o%. Research has been carried out on analysing attrition bias and drop-out rate (Heneghan et al. 2007) to identify different group characteristics from those who responded to the authors questionnaires and clinical examination.

A couple of additional observations where the tight criteria for moderate/severe peri-implantitis (Bleeding on probing/suppuration and bone loss greater than 2mm) considering the 9-year time frame and pooling the moderate and severe results together. This may have been done to increase sample size. Since the majority of the implants were 10mm long and the mean bone loss on the implants with peri-implantitis was 29%, >3mm of bone loss would have given better resolution for clinically significant peri-implantitis. It is also a pity the results were not collected at 10-years to fall into place with other long-term survival/success studies.

In conclusion, the results of this well conducted study should be interpreted with caution due to the high attrition rates and therefore, even though the differences in odds ratios for infection/implant survival rates between the various brands are most likely accurate the results may be over-optimistic when compared to some systematic reviews (Srinivasan et al. 2016)

References

Derks, J. Schaller, D. Håkansson, J. Wennström, J.L. Tomasi, C. and Berglundh, J., 2015. Effectiveness of Implant Therapy Analyzed in a Swedish Population:Prevalence of Peri-implantitis. Journal of Dental Research, 94(3), pp.44–51.

Derks, J. et al., 2015. Effectiveness of Implant Therapy Analyzed in a Swedish Population : Early and Late Implant Loss. Journal of Dental Research, 94(3), pp.44–51.

Derks, J. et al., 2015. Patient-reported outcomes of dental implant therapy in a large randomly selected sample. Clinical Oral Implants Research, 26(5), pp.586–591.

Heneghan, C. et al., 2007. Assessing differential attrition in clinical trials : self-monitoring of oral anticoagulation and type II diabetes. , 12.

Schulz, K.F. & Grimes, D.A., 2002. Epidemiology series Sample size slippages in randomised trials : exclusions and the lost and wayward. , 359, pp.781–785.

Srinivasan, M. et al., 2016. Dental implants in the elderly population: a systematic review and meta-analysis. Clinical oral implants research. Available at: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=medp&NEWS=N&AN=27273468.