Michael D. Shaw
Science matters to any health-oriented website, because the largest component of government and privately financed scientific research is—or at least purports to be—involved with health. Figures from a few years ago peg annual US spending (private and public sectors) on research and development at nearly $500 billion. As such, any time a service provider (scientist in this case) and lots of money are present, politics is sure to also be in the mix.
While scientists have struggled to get funding since the Middle Ages, the explosion of technology in the past 50 years, along with the proliferation of large “research universities” has changed science from a search for truth to a search for money. But, it gets worse. Consider that you have in place an enterprise worth half a trillion dollars that is almost completely devoid of meaningful outcomes evaluation. In fact, in many cases, the most positive outcome—and an endpoint in itself—is that a paper is accepted and published in a scientific journal.
Back in 2005, John Ioannidis, a professor of medicine at Stanford, published a paper entitled “Why Most Published Research Findings Are False.” Subsequent efforts to reproduce previously published results of hundreds of experiments, including a widely publicized paper entitled “Estimating the Reproducibility of Psychological Science,” cast huge doubts on a large number of studies. No doubt, the implications go far beyond the field of psychology. Care to guess how much money was spent on these failed research efforts?
Ironically, most attacks on junk science are directed toward either the experimental design; the transparent data dredging in search of some result; or the use of sketchy statistics to show a “link” between some cause and supposed effect. Hardly ever will you see arguments challenging other researchers in a particular field to duplicate such results—which has always been the gold standard for scientific truth.
In one memorable case, I was able to personally confront (via telephone) the lead researcher on an especially ridiculous and politically motivated “study” that attempted to link levels of phthalates (eeee-ville chemicals) in pregnant women to certain neonatal intelligence performance tests. This work was published in a respected journal, and received wide press coverage. Here’s a short list of what was wrong…
1. Even if you were willing to accept the absurd premise, the results were wildly inconsistent, and in some cases showed better performance in babies from mothers with higher phthalate levels. As such, no meaningful trend was demonstrated at all. Moreover, there was precious little difference between “high” and “low” phthalate levels.
2. The researcher admitted to classic “dry-labbing.” No original work was done whatsoever. She was able to obtain urine test data on a few dozen women, along with neonatal records—which would only have been possible since this research was done at a large teaching hospital. Is it “science” to compare two columns of data which already exist, and then get crummy results?
3. The demographics of the women involved, and a bit of reading between the lines in the published paper, implied that there were plenty of confounding factors—including illegal drug use during pregnancy by many of the participants. I mean, if you’re looking at neonatal cognitive function, why bother with such drug use? Far better to jump on the chemophobia bandwagon and assault phthalates.
4. There was nothing at all remarkable about the phthalate levels in these women. Indeed, virtually all American women showed levels in the range measured. Thus, there was nothing special about this cohort, other than the fact that free and easy data was readily available! Never mind the tone-deaf anti-scientific bias, whereby looking at how their use of illegal drugs during pregnancy would have been a much more interesting study, but was ignored.
I challenged the lead author on these and other points, and she had no argument against them. But, then she played her trump card: “It got published, didn’t it?”
And so it did. She got a grant to tout a preconceived politically-motivated chemophobic finding, and did it in a most slothful manner. Admittedly, she was enterprising in determining the political bent of the granting agency. Sadly, this is but one terrible study among thousands.
.
Wnen science gets political--Part 2
Science matters to any health-oriented website, because the largest component of government and privately financed scientific research is—or at least purports to be—involved with health. Figures from a few years ago peg annual US spending (private and public sectors) on research and development at nearly $500 billion. As such, any time a service provider (scientist in this case) and lots of money are present, politics is sure to also be in the mix.
While scientists have struggled to get funding since the Middle Ages, the explosion of technology in the past 50 years, along with the proliferation of large “research universities” has changed science from a search for truth to a search for money. But, it gets worse. Consider that you have in place an enterprise worth half a trillion dollars that is almost completely devoid of meaningful outcomes evaluation. In fact, in many cases, the most positive outcome—and an endpoint in itself—is that a paper is accepted and published in a scientific journal.
Back in 2005, John Ioannidis, a professor of medicine at Stanford, published a paper entitled “Why Most Published Research Findings Are False.” Subsequent efforts to reproduce previously published results of hundreds of experiments, including a widely publicized paper entitled “Estimating the Reproducibility of Psychological Science,” cast huge doubts on a large number of studies. No doubt, the implications go far beyond the field of psychology. Care to guess how much money was spent on these failed research efforts?
Ironically, most attacks on junk science are directed toward either the experimental design; the transparent data dredging in search of some result; or the use of sketchy statistics to show a “link” between some cause and supposed effect. Hardly ever will you see arguments challenging other researchers in a particular field to duplicate such results—which has always been the gold standard for scientific truth.
In one memorable case, I was able to personally confront (via telephone) the lead researcher on an especially ridiculous and politically motivated “study” that attempted to link levels of phthalates (eeee-ville chemicals) in pregnant women to certain neonatal intelligence performance tests. This work was published in a respected journal, and received wide press coverage. Here’s a short list of what was wrong…
1. Even if you were willing to accept the absurd premise, the results were wildly inconsistent, and in some cases showed better performance in babies from mothers with higher phthalate levels. As such, no meaningful trend was demonstrated at all. Moreover, there was precious little difference between “high” and “low” phthalate levels.
2. The researcher admitted to classic “dry-labbing.” No original work was done whatsoever. She was able to obtain urine test data on a few dozen women, along with neonatal records—which would only have been possible since this research was done at a large teaching hospital. Is it “science” to compare two columns of data which already exist, and then get crummy results?
3. The demographics of the women involved, and a bit of reading between the lines in the published paper, implied that there were plenty of confounding factors—including illegal drug use during pregnancy by many of the participants. I mean, if you’re looking at neonatal cognitive function, why bother with such drug use? Far better to jump on the chemophobia bandwagon and assault phthalates.
4. There was nothing at all remarkable about the phthalate levels in these women. Indeed, virtually all American women showed levels in the range measured. Thus, there was nothing special about this cohort, other than the fact that free and easy data was readily available! Never mind the tone-deaf anti-scientific bias, whereby looking at how their use of illegal drugs during pregnancy would have been a much more interesting study, but was ignored.
And so it did. She got a grant to tout a preconceived politically-motivated chemophobic finding, and did it in a most slothful manner. Admittedly, she was enterprising in determining the political bent of the granting agency. Sadly, this is but one terrible study among thousands.
.
Wnen science gets political--Part 2
By Michael D. Shaw
Last week’s article began a discussion of the pernicious influence of money, and therefore politics on science. Some commentators will point to the launch of the Sputnik 1 satellite, by the Soviets on October 4, 1957, as the wake-up call to America to greatly expand its technology programs. But, there’s more to it than that.
The end of World War II represents a significant turning point in our history. People sacrificed, and demanded that their post-war lives improve. If we could develop an atomic bomb, we should be able to apply the benefits of technology to our everyday lives. For that, we would need far more college-educated individuals. Indeed, the notion was promoted that virtually all in the emerging baby boomer cohort should go to college. If not a four-year college, then certainly at least a two-year institution.
Colleges and universities grew (and are still growing), with many new campuses being built, within existing state systems—not to mention extensive construction projects on legacy facilities. Of course, such growth requires plenty of money. One way to enhance revenue is for the administration to grab its vig on all research contracts secured by its faculty.
But, to obtain research grants, intrepid faculty members must first develop a reputation—scientific street cred, if you will. This is best done by publishing research papers, in so-called “high-impact” journals. A journal’s impact factor is a measure of the frequency with which an average article in a journal has been cited in a particular year. By way of example, the New England Journal of Medicine’s impact factor is 59.558. Nature clocks in at 38.138; and Annual Review of Public Health at 10.240.
Getting stuff published will also put a faculty member on what’s presently left of the tenure track, allowing him to achieve the coveted title of full professor. Thus, we have the phenomenon of “publish or perish.”
Journal impact factor data is widely disseminated, and obviously favors journals of general interest, over specialized publications. Still, no matter how obscure a researcher’s field might be, he would surely prefer to have his paper in NEJM, rather than Obstetrics & Gynecology (5.175).
So, publishing is imperative, and the best way to get a study published is to either have some sort of “sexy” finding, or to reinforce the current orthodoxy. Tens of billions of dollars have been spent on climate research, with only a tiny fraction of publications going against the grain. Even so, the dreaded increase in atmospheric carbon dioxide has not produced the widely predicted rise in temperatures. Search for truth, anyone?
For politically-motivated matters specifically related to public health, one need look no further than the government’s asinine promotion of the high carb/low fat diet. Countless papers have been published supposedly “proving” this contention, using all manner of statistical chicanery…only that diet is demonstrably not healthy for at least half the population, which is prone to metabolic syndrome.
For those keeping score at home, the high carb/low fat meme has changed a bit. The face-saving narrative now is that high carb versus low carb doesn’t make much of a difference. Which brings us to another time I was able to confront a researcher over the phone. And this guy, whose name I will not reveal, is an extremely well-known Harvard don.
The conclusion of his paper was that low carb was no better for weight reduction or health than the traditional higher carb diet touted by so many “experts.” But there was a serious problem with his research. He and his associates arbitrarily defined “low carb” as a 40 percent carb diet. Never mind that by this deluded reckoning a sizable portion of the US would already be on—or close to—a “low carb” diet. No rational person would think a diet is “low carb” if it were more than 20 percent carbohydrates.
Needless to say, this nonsense was published in a high-impact journal, the researcher got his grant, and Harvard got its vig. Search for truth, anyone?
Last week’s article began a discussion of the pernicious influence of money, and therefore politics on science. Some commentators will point to the launch of the Sputnik 1 satellite, by the Soviets on October 4, 1957, as the wake-up call to America to greatly expand its technology programs. But, there’s more to it than that.
The end of World War II represents a significant turning point in our history. People sacrificed, and demanded that their post-war lives improve. If we could develop an atomic bomb, we should be able to apply the benefits of technology to our everyday lives. For that, we would need far more college-educated individuals. Indeed, the notion was promoted that virtually all in the emerging baby boomer cohort should go to college. If not a four-year college, then certainly at least a two-year institution.
Colleges and universities grew (and are still growing), with many new campuses being built, within existing state systems—not to mention extensive construction projects on legacy facilities. Of course, such growth requires plenty of money. One way to enhance revenue is for the administration to grab its vig on all research contracts secured by its faculty.
But, to obtain research grants, intrepid faculty members must first develop a reputation—scientific street cred, if you will. This is best done by publishing research papers, in so-called “high-impact” journals. A journal’s impact factor is a measure of the frequency with which an average article in a journal has been cited in a particular year. By way of example, the New England Journal of Medicine’s impact factor is 59.558. Nature clocks in at 38.138; and Annual Review of Public Health at 10.240.
Getting stuff published will also put a faculty member on what’s presently left of the tenure track, allowing him to achieve the coveted title of full professor. Thus, we have the phenomenon of “publish or perish.”
Journal impact factor data is widely disseminated, and obviously favors journals of general interest, over specialized publications. Still, no matter how obscure a researcher’s field might be, he would surely prefer to have his paper in NEJM, rather than Obstetrics & Gynecology (5.175).
So, publishing is imperative, and the best way to get a study published is to either have some sort of “sexy” finding, or to reinforce the current orthodoxy. Tens of billions of dollars have been spent on climate research, with only a tiny fraction of publications going against the grain. Even so, the dreaded increase in atmospheric carbon dioxide has not produced the widely predicted rise in temperatures. Search for truth, anyone?
For politically-motivated matters specifically related to public health, one need look no further than the government’s asinine promotion of the high carb/low fat diet. Countless papers have been published supposedly “proving” this contention, using all manner of statistical chicanery…only that diet is demonstrably not healthy for at least half the population, which is prone to metabolic syndrome.
For those keeping score at home, the high carb/low fat meme has changed a bit. The face-saving narrative now is that high carb versus low carb doesn’t make much of a difference. Which brings us to another time I was able to confront a researcher over the phone. And this guy, whose name I will not reveal, is an extremely well-known Harvard don.
The conclusion of his paper was that low carb was no better for weight reduction or health than the traditional higher carb diet touted by so many “experts.” But there was a serious problem with his research. He and his associates arbitrarily defined “low carb” as a 40 percent carb diet. Never mind that by this deluded reckoning a sizable portion of the US would already be on—or close to—a “low carb” diet. No rational person would think a diet is “low carb” if it were more than 20 percent carbohydrates.
Needless to say, this nonsense was published in a high-impact journal, the researcher got his grant, and Harvard got its vig. Search for truth, anyone?
No comments:
Post a Comment