As the saying goes, "There are three kinds of lies: lies, damned lies, and statistics." We know that's true because statisticians themselves just said so.
A stunning report published in the Annals of Internal Medicine concludes that researchers often ask statisticians to make "inappropriate requests." And by "inappropriate," the authors aren't referring to accidental requests for incorrect statistical analyses; instead, they're referring to requests for unscrupulous data manipulation or even fraud.
The authors surveyed 522 consulting biostatisticians and received sufficient responses from 390.
Then, they constructed a table (shown below) that ranks requests by level of inappropriateness. For instance, at the very top is "falsify the statistical significance to support a desired result," which is outright fraud. At the bottom is "do not show plot because it did not show as strong an effect as you had hoped," which is only slightly naughty.
On the right, the authors report how often the biostatisticians estimated that they received such a request over the past five years. The results are jaw-dropping.
The absolute worst offense (i.e., being asked to fake statistical significance) occurred to 3% of the survey respondents. Another 7% reported being asked to change data, and a whopping 24% -- nearly 1 in 4 -- said they were asked to remove or alter data. Unequivocally, that is a request to commit scientific fraud.
Of the less serious offenses, 55% of biostatisticians said that they received requests to underreport non-significant results.
Liar, Liar
It's quite remarkable that a scientist would have the audacity to ask another professional to fudge data. While there is simply no excuse for the egregious offenses (e.g., falsifying statistical significance), some of the other lesser offenses may not reflect maleficence but ignorance. Scientists often aren't very good at statistics, and they may make inappropriate requests simply because they don't know any better. The study didn't tease that out.
Still, this study should serve as a reminder that the ongoing reproducibility crisis may have, at least in part, a more sinister explanation.
Source: Min Qi Wang, Alice F. Yan, Ralph V. Katz. "Researcher Requests for Inappropriate Analysis and Reporting: A U.S. Survey of Consulting Biostatisticians." Ann Intern Med 169(8): 554-558. Published: 16-Oct-2018. DOI: 10.7326/M18-1230
A stunning report published in the Annals of Internal Medicine concludes that researchers often ask statisticians to make "inappropriate requests." And by "inappropriate," the authors aren't referring to accidental requests for incorrect statistical analyses; instead, they're referring to requests for unscrupulous data manipulation or even fraud.
The authors surveyed 522 consulting biostatisticians and received sufficient responses from 390.
Then, they constructed a table (shown below) that ranks requests by level of inappropriateness. For instance, at the very top is "falsify the statistical significance to support a desired result," which is outright fraud. At the bottom is "do not show plot because it did not show as strong an effect as you had hoped," which is only slightly naughty.
On the right, the authors report how often the biostatisticians estimated that they received such a request over the past five years. The results are jaw-dropping.
The absolute worst offense (i.e., being asked to fake statistical significance) occurred to 3% of the survey respondents. Another 7% reported being asked to change data, and a whopping 24% -- nearly 1 in 4 -- said they were asked to remove or alter data. Unequivocally, that is a request to commit scientific fraud.
Of the less serious offenses, 55% of biostatisticians said that they received requests to underreport non-significant results.
Liar, Liar
It's quite remarkable that a scientist would have the audacity to ask another professional to fudge data. While there is simply no excuse for the egregious offenses (e.g., falsifying statistical significance), some of the other lesser offenses may not reflect maleficence but ignorance. Scientists often aren't very good at statistics, and they may make inappropriate requests simply because they don't know any better. The study didn't tease that out.
Still, this study should serve as a reminder that the ongoing reproducibility crisis may have, at least in part, a more sinister explanation.
Source: Min Qi Wang, Alice F. Yan, Ralph V. Katz. "Researcher Requests for Inappropriate Analysis and Reporting: A U.S. Survey of Consulting Biostatisticians." Ann Intern Med 169(8): 554-558. Published: 16-Oct-2018. DOI: 10.7326/M18-1230
No comments:
Post a Comment