“Peer-review” status is often used to endorse scientific papers cherry-picked because they support a bias.
Many scientists are not impressed with the peer-review processes scientific journals use. Like democracy, this peer-review is better than all the available alternatives but it certainly doesn’t guarantee published scientific papers are problem-free.
Sure, peer-reviewed sources are better than others which have no quality control. But it is still a matter of “customer beware.” The intelligent users of scientific literature must do their own filtering – make their own critical judgements of the likely reliability of reported scientific findings.
Despite this people often use the “peer-reviewed” description to endorse published finding (especially if they confirm their own biases) without any critical assessment. This happens a lot in on-line debates of “controversial” issues.
Here I will go through the details of peer-review of a recently published paper which anti-fluoride activists are endorsing and promoting, but others are critcising. The paper is:
Malin, A. J., & Till, C. (2015). Exposure to fluoridated water and attention deficit hyperactivity disorder prevalence among children and adolescents in the United States: an ecological association. Environmental Health, 14.
I have discussed this paper in recent posts (see More poor-quality research promoted by anti-fluoride activists and ADHD linked to elevation not fluoridation). The journal, Environmental Health, has a transparent peer-review process which provides access to the names and reports of the reviewers. This reveals problems with the review process in this case. Below I discuss the responsibility of the authors, reviewers and the journal for the problems with this paper and its reported findings.
Authors’ responsibilities
The authors are clearly committed to a pet theory that fluoride is a neurotoxicant which could contribute to ADHD prevalence. Nothing wrong with that – we all feel committed to our hypotheses. We wouldn’t be human if we didn’t. But the best way to produce evidence for a hypothesis is to test it in a way that could prove it wrong.
In this case the authors found a correlation between ADHD prevalence in US states and the amount of community water fluoridation in each state. Trouble is, one can find just as good a correlation, or even a better correlation, with many other criteria for which state prevalence statistics are available. I listed a few in ADHD linked to elevation not fluoridation. Some of these factors are also correlated with community water fluoridation suggesting the correlation reported by Malin and Till (2015) may be deceptive.
A proper test of the fluoridation hypothesis would include considering the effect of including such confounders together with fluoridation in their statistical analysis. Malin and Till (2015) did include one other criteria – the median household income for 1992 – but did not include any others. I find this surprising because they acknowledged ADHD results from interaction of genetic and environmental factors. While fluoridation is not usually considered a relevant factor things like smoking and premature births are and there is conflicting evidence about the role of economic factors like poverty and income.
In my article ADHD linked to elevation not fluoridation I showed ADHD prevalence is better explained by a few of these factors without any input from water fluoridation.
I can’t help feeling the limited consideration of confounding factors results from a desire to protect the fluoridation hypothesis and therefore not test it properly.
Reviewers responsibilities
Again, such a desire is only human. But reviewers should have picked this up during their own considerations.
Interestingly, only one of the two reviewers raised possibility other confounders – specifically lead levels. This is of course valid as lead is a recognised neurotoxicant – but why did none of the reviewers question why other factors like smoking, premature births and social or regional factors were not considered?
I believe that is because both reviewers had research interests directed at chemical toxicity and not ADHD or similar mental characteristics. A matter of someone with a hammer only seeing nails.
The reviewers and their research interests are:
Marc Weisskopf whose reviews are available here and here.
“Some examples of my current work are exploring how exposure to, e.g., lead, manganese, and air pollution affect cognitive function and psychiatric symptoms; how exposure to Agent Orange and other herbicides used in Vietnam relate to the development of PD; and how formaldehyde and lead exposure relate to the development of ALS.”
Anna Choi whose review is here.
“Dr Choi’s research focuses on the effects of environmental exposures on health outcomes. She has been studying the birth cohorts in the Faroe Islands where exposures to environmental chemicals including mercury, PCBs, and PFCs are increased due to traditional marine diets. In addition, she also studies the effects of the contaminants on cardiovascular function and type 2 diabetes among the Faroese septuagenarians. She is also actively involved in the research on the impact of nutrients as possible negative confounders that may have caused an underestimation of methylmercury toxicity. Dr Choi’s other research interests include studying the adverse effects of fluoride exposure in children.”
Why were reviewers with a wider research experience not chosen? This journal allows authors to propose suitable reviewers themselves. Or the reviewers may have been chose by the associate editor handling this paper – Prof David Bellinger. His research focuses on the neurotoxicity of metabolic and chemical insults in children. So again it may just be the blinkered view of someone whose research background stressed the role of neurotoxicants rather than other factors likely to influence ADHD prevalence.
The journal’s responsibility
I noticed that one of the two chief editors (who have final say over acceptance of submitted papers) of this journal is Prof Philippe Grandjean. He himself has been actively promoting the idea that fluoride is a neurotoxicant purely on the evidence of the metareview of Choi et al (2012). Yes he is a coauthor of that review and Choi is one of the reviewers of the Malin and Till paper. The review of Choi et al (2012) related to areas of mainly China where fluoride concentrations are higher than used in community water fluoridation. Areas where endemic fluorosis is common.
I have to wonder if Grandjean’s well-known position on fluoride and community water fluoridation was a consideration in choosing this journal for publication.
Others have commented that the journal Environmental Health is considered low-quality based on its low impact factor. I do not know the area well enough to pass judgement myself. However, I notice that the journal charges authors for publishing their paper (£1290/$2020/€1645 for each article accepted for publication.) This sort of charge, associated with poor quality peer-review makes me suspicious. I have commented on these sort of journal before in my post Peer review, shonky journals and misrepresenting fluoride science.
Conclusions
This is one example of peer-review and paper acceptance which brings into question the idea of using publication and peer-review as endorsement of a study’s quality. I am sure this is not an isolated case. Even with the best of intentions journal editors and reviewers are limited by their own areas of expertise. Journal publication and peer-review is a far from perfect process – even if it is preferable to current alternatives.
Unfortunately activists will promote poor quality studies like this by blindly using the study’s peer-review status.
The intelligent reader should beware of such blind endorsements. Knowing the human foibles which exist in the research and publication processes such a reader will consider the contents of the paper and not rely on peer-review status. They will consider the evidence and conclusions critically. And if they don’t have enough background to make their own critical assessment they will consider the views of others with the required expertise and not blindly accepting what political activists tell them.
Footnote
Just came across this article referring to peer-review problems in journals published by BioMed Central – Major publisher retracts 43 scientific papers amid wider fake peer-review scandal.
BioMed Central publishes the Journal Environmental Health discussed in this post. I am not suggesting this paper was part of the peer-review racket discussed in the article. But the news item does highlight the point I am making that intelligent readers need to consider published scientific papers carefully and critically and not blindly rely on “peer-review” endorsement.
