Currently, teachers experience a general problem, a surge of published systematic reviews where slightly modified search criteria have led to slightly different bunch of papers with slightly different results of meta-analyses. Systematic reviews have once been welcomed as valuable tool to either end a story once and forever (if evidence for or against a certain treatment or association was overwhelming), or call for more conclusive randomized controlled trials (RCT) after still open questions had been identified. If, after any new RCT, editors of our professional journals would accept considering a new systematic review for publication, which basically ruminates already published RCT summaries but adds just another study without changing main conclusions, it will in fact become difficult to keep pace with what some call “emerging evidence”.
One main reason why evidence based medicine has to be taught to undergraduate students is to provide future health care workers with proper tools and train specific skills to conduct brief systematic reviews of identified randomized controlled trials themselves. Here on this blog, I had posted a couple of quick examples, see here, here and here.
In particular the latter of the above examples has dealt with the question whether the large multi-center trial by Engebretson et al. (2013), which had reported no effects of periodontal therapy on HbA1c levels in diabetics, would nullify the conclusion (that nonsurgical periodontal treatment may reduce HbA1c levels by about 0.4%) of previous meta-analyses of smaller and mainly single-center RCTs with similar settings.
Engebretson et al. (2013) had listed possible shortcomings of their study. However, that oral hygiene of study participants had not improved was considered by most of our professional leaders scandalous. Further issues for unprecedented criticism included “nearly normal” HbA1c levels at the outset and extreme obesity of participants. Engebretson’s unwelcome results had been reviled by an armada of 21 editors of our key journals, presidents of our main scientific societies, and further periodontal experts. Criticism had culminated in a very strange recommendation.
“Given the inconlusive nature of these data, we recommend that the existing body of evidence in which meta-analyses consistently conclude that successful periodontal therapy appears to improve glycemic control, should instruct us until results from future studies are reported. We urge all interested parties to refrain from using these study results as a basis for future scientific texts, new research projects, guidelines, policies, and advice regarding the incorporation of necessary periodontal treatment in diabetes management.” (My emphasis.)
In other words, forget about Engebretson et al. and continue quoting more favorable results from existing meta-analyses of RCTs on the effect of periodontal therapy on diabetes control. A quick analysis revealed that it won’t nullify a mean HbA1c reduction in diabetics by nonsurgical periodontal therapy, but that considerable heterogeneity was introduced by including Engebretson’s study which may in fact lower the grade of evidence. I had entered meta-data of Engebretson and Kocher 2013 in an amazing tool for meta-analysis and had added findings by Engebretson et al. (2013). That might have been premature, see below.
In an announcement for his talk about periodontal treatment effects on type 2 diabetes at Europerio 8 in London later this year, exasperated Professor Thomas Kocher of Greifswald University in Germany promises to “dissect” the large multicenter trial by Engebretson et al. (2013) who could not find an effect on glycated hemoglobin in type 2 diabtes mellitus. The study had been published in late 2013 in JAMA, not in New England (Journal of Medicine). The large multicenter trial had long been attacked for not yielding the desired results (“a publication which we were really waiting for”).
Kocher was asked to talk in London about “why all the other small studies showed an effect” and he wants to find out “the issues why we [?] couldn’t see anything in the Engebretson study”. Well, it was actually Wenche Borgnakke who had got 20 other “reviewers” aboard who had already dissected the study by Engebretson et al. and has called for censorship.
As noted by Engebretson and Kocher 2013 in one of the numerous previous systematic reviews of RCTs on the effect of nonsurgical periodontal therapy and reported in Table 1 of their article, problems with the design of these small-scale, mainly single-center studies, which included some trials with adjunctive antibiotics, were plentiful. Problems with low and high baseline HbA1c levels and with questionable periodontal outcomes had been reported as well. Engebretson and Kocher (2013) report possible publication bias which means nothing else that studies without an effect on HbA1c might have gone unpublished. Based on this particular and numerous other systematic reviews, the evidence that nonsurgical periodontal therapy in fact has a relevant beneficial effect on HbA1c levels in type 2 diabetics may actually be regarded moderate. The study by Engebretson et al. adds heterogeneity to any meta-analysis which may downgrade this evidence to low. That is what our thought leaders alerts. That’s why censorship.
As reported before, the large muticenter intervention trial by Engebretson et al. (2013) who reported no effect of nonsurgical treatment of periodontitis on HbA1c levels in patients with type 2 diabetes mellitus has harshly been critized by our thought leaders. Last month, JAMA published a number of letters to the editor. One letter by Chapple, Borgnakke and Genco identified important problems in Engebretson’s paper including “problems with the study design, execution, data interpretation and reporting.”
“First, the periodontal therapy provided failed to clinically manage the periodontal infection and associated inflammatory burden. Residual plaque levels of 72% and bleeding scores of 42% are far below the consenus for expected outcomes [reference to van der Weijden et al., J Clin Periodontol 2002; 29(suppl 3): 55-71, 90-91]. Therefore, no conclusions can be drawn about the effect of clinicaly effective periodontal therapy on HbA1c in patients with type 2 diabetes.
Second, control of diabetes at baseline was predominantly good (mean HbA1c levels, 7.8%), with less than 60% of patients having HbA1c levels greater than 8.0% (HbA1c level <9.0% was an inclusion criterion). With the mean HbA1c value close to the therapeutic target, we would not expect an intervention to improve HbA1c substantially.
We are concerned about the reliance on statistical significance to justify a conclusion of no effect when the clinical therapy failed to deliver the expected standard of care.”