In a recent analysis of thousands of randomized controlled trials (RCT) in eight journals a simple method was offered which might enable skeptical scientist identification of data fabrication. Editor of the Anaesthesia journal John B. Carlisle of Torbay Hospital, UK, looked at baseline differences of means in more than 5000 randomized controlled trials, mainly in the field of Anesthesiology, but also more than 500 published in JAMA and more than 900 published in the New England Journal of Medicine . His study went online earlier this week. Analyzed articles were published between 2000 and 2015. In brief, if randomization was successful, baseline differences should be small. Giving p-values for baseline differences (in order to indicate successful randomization) is actually discouraged since they are not really interpretable, but Carlisle calculated them anyway. If the null hypothesis is true, p-values have a uniform distribution. So p-values between 0 and 1 would be equally likely.
The quite provocative question “Are dentists dreadful [statisticians]?” had been asked in 2000 in an introductory comment to a series of educational papers in Community Dental Health which were authored/coauthored by biostatistician Mark Gilthorpe, now Leeds University. His main objective then was to make dentists aware of multilevel modeling, something which had then been applied by dental statisticians for a dozen or so years earlier. At that time, MLwiN had been released by the Centre of Multilevel Modeling, now located at Bristol University; a special software which facilitates the overall easy application of a wide range of multilevel models of data with a hierarchical data structure often found in dental and, in particular, periodontal research. While on the one hand seemingly promoting more sophisticated modeling, Gilthorpe and some of his co-workers tiredlessly exposed, in the following years, in fact dreadful statistics in published papers in Dentistry and warned not further educated clinicians not to do their own analyses but rather rely on expert statistical help. However, Gilthorpe did not realize a frequent unwillingness of common biostatisticians to make themselves familiar with more sophisticated methods such as multilevel modeling, which is otherwise rarely used in Medicine.