The large multi-center intervention trial by Engebretson et al. (2013), who had reported lack of any effect of non-surgical periodontal therapy on HbA1c levels in type 2 diabetes mellitus patients after 3 and 6 months, has been harshly criticized because of very moderate clinical improvements as regards pocket depth reduction (from mean 3.26 mm at baseline) of 0.4 mm (95% CI 0.4; 0.5 mm) and reduction of bleeding on probing (from mean 62%) of 19% (95% CI 15.7; 22.4). What was even more concerning was that, 6 months after seemingly intense treatment (at least 160 minutes of scaling and root planing followed by oral hygiene instruction and, for two weeks, twice daily mouthwash with 0.12% clorhexidine digluconate; then, at both 3- and 6-month follow-up examinations, further oral hygiene instructions and scaling/root planing for another hour), bleeding on probing was still seen at an average of 40% sites while, on average, 70% tooth surfaces were still covered by plaque (from 86% at baseline).
These are undeniable problems of the study. Claims that periodontal treatment was insufficient and, as a consequence, periodontal infection still present in most patients after periodontal therapy, may in fact be justified. It is the sheer size of the attack which is so appalling. Each and every editor of our professional journals and numerous further pundits, altogether 21, had joined, well, the public execution of the study’s principle investigator. Because of unwelcome results of a study with the potential of ending a story, or illusion, once and forever. And, absolutely inappropriate attempts of intimidation of scientists when writing,
“Given the inconlusive nature of these data, we recommend that the existing body of evidence in which meta-analyses consistently conclude that successful periodontal therapy appears to improve glycemic control, should instruct us until results from future studies are reported. We urge all interested parties to refrain from using this study results as a basis for future scientific texts, new research projects, guidelines, policies, and advice regarding the incorporation of necessary periodontal treatment in diabetes management.” (Emphasis added.)
So, censorship. This is absolutely unscientific. Meta-analyses are always preliminary and must incorporate new results on a continuous basis.
I read yesterday with grief that one of our profession’s greatest scientists, Dr. Anne D. Haffajee, has deceased in August, just two years after we lost her collaborator for so many decades, another giant of periodontal research, Dr. Sigmund S. Socransky, both at Forsyth in Boston. I have to admit that I had never met Dr. Haffajee personally. In the early 1980s we had a brief exchange of letters about an anaerobic gas-flushed syringe for obtaining a sample of the incredible oxygen-sensitive subgingival microflora in deep periodontal pockets. The rather complicated device had been invented at Forsyth (as so many other things), and Anne suggested that we should just manufacture our own syringe. It was not for sale.
Currently, and apparently, data of a large study, which has been collected in the U.S. and Sweden between 1999 and 2004, is being published in two series of papers. The purpose is to report clinical and microbiological observations made in a randomized controlled trial in which seven different treatments (periodontal surgery, systemic amoxicillin and metronidazole, topical tetracycline fibers, and all combinations) were compared to standard periodontal therapy, i.e., scaling and root planing alone, in hundreds of patients with chronic periodontitis. A criticized traditional approach to data analysis where huge numbers of site-specific data were aggregated at the patient level (and highly valuable site-specific data is lost) seems to be published in our major journal, Journal of Clinical Periodontology, see, for example, the most recent paper by Socransky et al. (2013). Site-specific data analyses, at least seemingly taking into account the hierarchical structure and non-independence of observations made in a given subject are dealt with in papers which appear in the open access Journal of Oral Microbiology. I had critically commented on the previous paper by Mdala et al. (2012) [pdf] on multilevel modeling on clinical parameters after eight different treatment modalities before on this blog, see here. There I wrote,
“Unfortunately, the random parts in the models (whose design has to be considered insufficient) are not explained or even shown […] although one would like to see variance partition at the subject, tooth, and site levels, and whether common rules of thumb in decision making mainly based on site-specific clinical features hold. Frequently reported problems with extrabinomial variation, in particular underdispersion, in multilevel logistic regression models of periodontal data […] are not discussed. The main purpose of applying rather sophisticated models apparently was estimation based on just fixed effects rather than previous simple calculation of averages.”
The new paper by Mdala et al. (2013) [pdf] on bacterial counts after complex treatment of chronic periodontitis, which erroneously keeps “multilevel analysis” in the title, has abandoned the idea of multilevel modeling of bacterial data. Instead, authors turn to the more traditional approach of marginal models such as Generalized Estimating Equations (GEE). GEE, which correctly takes into account the non-independence of observations made in a given subject, may indeed be preferred when interest is mainly on the effect of explanatory variables on the response, and correlation structure is rather considered a nuisance. On the other hand, the main advantage of multilevel modeling as exercised in the previous paper by Mdala et al. (2012), with its unbiased dealing with the hierarchical structure of the data is, besides obtaining correct estimates of fixed effects, an analysis of the random part of the model, i.e., variances and covariances. It may indeed provide new and deep insights into phenomena and mechanisms operating at the level of interest, the periodontal site.
In lectures and seminars for graduates and undergraduate students, or many invited talks I have long pointed with regrets to the way how painstakingly collected site-specific periodontal data are frequently treated when aggregating them at the subject level. The main reason for intentionally (or unintentionally?) losing instantly much if not most information by applying this approach is usually mentioned: statistical pundits have pointed to the fact that standard statistical tests require independent observations. On a number of occasions I have used for clarification just an abstract of a published paper by Forsyth Institute’s periodontal group which explains the methods applied,
“Subgingival plaque samples were taken from the mesial aspect of each tooth in 635 subjects with chronic periodontitis and 189 periodontally healthy subjects. The samples were individually analyzed for their content of 40 bacterial species using checkerboard DNA-DNA hybridization (total samples = 21,832). Mean counts, % DNA probe counts and percentage of sites colonized at >105 were determined for each species in each subject and then averaged in each clinical group. Significance of difference between groups was determined using Mann-Whitney test. Association between combinations of species and periodontal status was examined by stepwise logistic regression analysis.” (Emphasis added.)
So, tens of thousands of bacterial samples were analyzed for forty bacterial species using highly sophisticated and expensive molecular biological methods (and most probably clinical signs of periodontal disease were assessed as well) which had been collected in more than 800 subjects; and then all this wealth of information is just dumped when mean counts and percentages of sites more heavily colonized were calculated in each of the subjects. One would have saved enormous an amount of money and time if one pooled sample per subject had been analyzed, and I still would argue that this biological pooling the material would be more reasonable than considering statistically pooled data. And then, data were analyzed by non-parametric Mann-Whitney test which uses ranks of observations rather than observations themselves. Well, as this approach has been adopted by the group in dozens if not hundreds of scientific papers since the late-1990s and hardly ever criticized, a more interesting approach using multilevel modeling had been applied in the summer when very old clinical data of one of Forsyth’s therapy studies had been re-analyzed and -published in a new open-access microbiological journal (implying that the respective microbiological data are upcoming). The paper by Mdala et al. (2012) [pdf], apparently using sites as statistical units in a time series model, would of course be most welcome provided the analysis was flawless and the given information complete. But since conclusions regarding adjunct antibiotic treatment of chronic periodontitis are grave and may imply even recommendations, a close look at what has been done is mandatory. I had contacted meanwhile the corresponding author of the paper and had been provided with some additional material which is strangely not accessible on the journal’s web page (although mentioned as supplementary material in the text). But my main concerns have not been addressed so far. In the abstract, the authors write (and CAL means here clinical attachment level, SURG periodontal surgery, PD pocket depth; AMOX and MET refer to respective systemic drugs amoxicillin and metronidazole, while TET is a no longer available tetracycline fiber for topical use in periodontal pockets),
“Surgically treated patients experienced most CAL loss. Adjunctive therapy including SURG was most effective in reducing PD. Combining SURG with AMOX, MET, and TET gave significant clinical benefits. Past and current smoking habits were significant predictors of deeper PD. Only current smoking was a significant predictor of CAL loss. Bleeding, accumulation of plaque, gingival redness, and suppuration were significant predictors of further CAL loss and deeper PD.”
The pretty old study (most data had been collected more than ten years ago, but this information can only be retrieved from another paper by the group describing separate aspects) has had a complicated design, where eight treatment modalities had randomly be assigned to patients. Scaling and root planing was the standard treatment provided to which the other, increasingly complex, treatments were compared. When the authors claim in the abstract (and I am afraid that many clinicians skip reading the whole, likely incomprehensible, paper) that surgically treated patients experienced most loss of clinical attachment (without giving any numbers), one may already be alerted. Buried on the last page of the manuscript (Table 5), the average difference of change in attachment level two years after periodontal surgery (as compared to what was achieved with scaling and root planing alone) was in fact minus 0.40 mm which in fact compares unfavorably to plus 0.67 mm (as compared to scaling and root planing alone) in the group which received systemic amoxicillin and metronidazole in addition to scaling and root planing. (Note that all subjects received subgingival scaling. Notably, the paper does not favor, as others of the group in fact do, only systemic antibiotics). All other treatment modalities yielded (and it is clear that all estimates were derived from a hierarchical, four-level model) suboptimal (as compared to the group receiving amoxicillin and metronidazole) but superior results than scaling and root planing alone, just between plus 0.67 and minus 0.40 mm. So, all mean results were within 1.07 mm difference to those achieved with scaling and root planing. This might actually demand an exercise on the difference between statistical significance and clinical relevance in another posting. And, of course, on overwhelming evidence of certain periodontal conditions being indications or contraindications for periodontal surgery. Timely randomized controlled clinical studies are most welcome as they provide, if conducted properly, a high level of evidence. But there is progress over time. That periodontal surgery is mainly indicated if periodontitis has resulted in certain anatomical defects, ie. deep infrabony lesions and furcation involvement which are largely inaccessible to scaling and root planing, but not indicated otherwise is meanwhile commonplace. Since these lesions are less frequently found than more accessible periodontal lesions and since periodontal surgery is by definition much more invasive (minimally invasive surgical techniques had apparently not be applied), I would not be too surprised when the net effect in a group of patients treated with (mostly not indicated) periodontal surgery would be slight loss of attachment (as compared to scaling and root planing which was mostly indicated). Continue reading