Are Neuroscientists Dreadful Statisticians?

The quite provocative question “Are dentists dreadful [statisticians]?” had been asked in 2000 in an introductory comment to a series of educational papers in Community Dental Health which were authored/coauthored by biostatistician Mark Gilthorpe, now Leeds University. His main objective then was to make dentists aware of multilevel modeling, something which had then been applied by dental statisticians for a dozen or so years earlier. At that time, MLwiN had been released by the Centre of Multilevel Modeling, now located at Bristol University; a special software which facilitates the overall easy application of a wide range of multilevel models of data with a hierarchical data structure often found in dental and, in particular, periodontal research. While on the one hand seemingly promoting more sophisticated modeling, Gilthorpe and some of his co-workers tiredlessly exposed, in the following years, in fact dreadful statistics in published papers in Dentistry and warned not further educated clinicians not to do their own analyses but rather rely on expert statistical help. However, Gilthorpe did not realize a frequent unwillingness of common biostatisticians to make themselves familiar with more sophisticated methods such as multilevel modeling, which is otherwise rarely used in Medicine.

In a survey conducted by Aarts et al. (2014) from the University of Amsterdam, The Netherlands, which was published online last week in Nature Neuroscience,  the authors recognized that 53% of 314 reviewed papers over an 18-month period from five renoowned journals included experimental designs in which multiple observations were collected from a single research object. They correcly state that “These so-called ‘nested designs’ yield data that canno be considered to be independent, and so violate the independency assumption of conventional statistical methods such as the t test.”

As Scientific American reports,

“’We didn’t see any of the studies use the correct multi-level analysis,’ says Sophie van der Sluis, the lead researcher. Seven percent of the studies did take steps to account for clustering, but these methods were much less sensitive than multi-level analysis in detecting actual biological effects.  The researchers note that some of the studies surveyed probably report false-positive results, although they couldn’t extract enough information to quantify precisely how many.  Failure to statistically correct for the clustering  in the data can increase the probability of false-positive findings to as high as 80 percent—a risk of no more than 5 percent is normally deemed acceptable.”

So, one may extend Dr. Gilthorpe’s provocative question 14 years ago to “Are neuroscientists dreadful [statisticians]?” Well, maybe.

As for us, multilevel modeling has widely been applied in dental and, in particular periodontal, data analyses in the last decade and some of my own papers may be found here. There is still a long way to convince dental researchers that site-specific observations should preferably be analyzed as such and not aggregated at a higher level. It is important to facilitate, not to restrict. Editors of our dental journals are more or less aware of the common peculiarity of multiple obersavtions made in an oral cavity and should be prepared to identify reviewers with a benevolent expert approach to avoid grave mistakes on one hand but, on the other, encourage application of mutilevel modeling.

If time allows I will post, in the next coming weeks, on this blog chapters of a special manual for MLwiN which uses my own published data in order to guide young scientists in Dentistry through the useful program.

30 March 2014 @ 11:46 am.

Last modified March 30, 2014.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s