Reproducibility Crisis, Part 3: Scientific Skepticism

Scientific Skepticism


A recent segment on the TV show Last Week Tonight addressed the media’s shortcomings in reporting scientific information. Citing headlines such as “Dogs Don’t Like Hugs & Can Raise Stress” and “Glass of Red Wine Equivalent to Hour of Gym Time”, Oliver lambasted mass-media outlets for their sensationalism and overall poor understanding of science. The segment addressed p-hacking, the post hoc generation of a hypothesis based on ‘statistical significance’ found in a data set, and scientific ‘experts’ who preach pseudoscience on the public stage.

The goal of the segment, seemingly, was to restore the viewer’s faith in science—to help encourage the epiphany that the science that we hear on the nightly news is merely a distorted representation of what science actually is. By explaining the various mechanisms by which science could be distorted, Oliver provided the non-specialist some basic tools for discerning the scientific nature of a headline they read on the television. In a sense, he introduced skepticism to a population that generally submits to scientific authority.

Negative Priming Effects

While it is generally valuable to combat the quacks and the ‘clickbaity’ headlines with skepticism, at times the skepticism can come to undermine the entire scientific enterprise altogether. In a lecture last year for the National Research Council of the National Academies of Science, the University of Pennsylvania Professor of Communications Kathleen Hall Jamieson warned of the negative priming effects of attacking science (or representations of science) on the public stage. Attacks on science or scientists could corrode the public’s trust in the scientific enterprise, similarly to how mentions of John McCain’s age before the 2008 general election were shown to correlate with drops in his polling numbers. Negative priming leads to the formation of unfavorable associations that affect future actions and opinions.

(note: several major priming experiments were among those found to be irreproducible in the Reproducibility Project’s recent analysis of psychological studies).

Healthy Skepticism Can Easily be Blurred with Radical Skepticism

The concept of scientific authority may seem oppressive. Absolute authority is responsible for stifling creative thought and silencing the voices of dissenters.  Scientific authority is not, in its purest sense, rooted in personality or dogma, but rather in experience. It would seem to make intuitive sense that those most qualified to talk about science are those who have spent the most time studying and progressing science. One is not expected to submit to scientific authority in any way, but just to acknowledge the prevailing opinions of a given scientific field.

The complete rejection of scientific authority is a major impetus for the anti-science movement. Unfortunately, segments such as Oliver’s that were meant to give good science the PR boost it desperately needs, can lead some down the path of absolute scientific skepticism. Healthy skepticism can easily be blurred with radical skepticism if one does not have the proper grounding in a technical field.

Skepticism is one of the best things science has brought to the modern world. It is important to realize, however, the dangers of reinforcing such skepticism. Suddenly, 99% of scientists agreeing is not a consensus but a conspiracy. Suddenly, the retraction of a paper by a major scientific body is not a matter of correctness, but rather censorship.

The Media’s Role

The blame cannot be placed solely on the media. Scientists, in attempts to secure funding and prestige, sensationalize their own results and provide overly hopeful projections for the applications of their research. Moreover, science has been facing a reproducibility crisis that not only questions the legitimacy of certain fields of research, but also the integrity of the research scientists.  In many cases, the media is merely compounding the sensationalism and distortion that could already be seen in the original scientific papers.

Positive Priming Effects from the Normal Scientific Progress

A 2015 study by a professor of journalism at the University of Gothenberg analyzed whether the media’s coverage of research misconduct in Sweden from 2002-2013 had an effect on the public’s trust in science. While the study was initiated as a way of understanding the decline of trust in science in Swedish society, the data surprisingly showed that media coverage of misconduct was correlated with greater trust in science. The author suggested that the surprising result is likely due to consumers of coverage of research misconduct also being the most likely to consume coverage of general scientific progress. The positive priming effects from the normal scientific progress were able to outweigh the negative priming of the misconduct, and in turn lead to a much more complex perception of the scientific enterprise.

The best way to combat the possibly damaging negative priming effects of the coverage of scientific scandals is to continue to cover scientific progress. The more that legitimately groundbreaking scientific breakthroughs are made apparent to the public, the more receptive they will be to integrating the substantive criticism of a scientist (or a scientific field) to a healthy form of skepticism, as opposed to one that completely undermines trust in the scientific enterprise.

Go back to Part 1 or Part 2 of the ‘Reproducibility Crisis’ Series

  • author's avatar

    By: Wade Miller

    Wade Miller is a senior chemistry major at the University of Pennsylvania. In addition to his interest in drug development and pharmacology, Wade has studied both the philosophy and history of science. Wade is also member of the Philadelphia branch of Neuwrite, a collaborative writing group for scientists and writers.

  • author's avatar