This year 2017 Edge Annual question was: What scientific term or concept ought to be more widely known? Like every year, there is delicious bunch of wildly interesting ideas discussed by the 207 contributors. Absolutely worth reading. However…

In the motivation to the question, on the occasion of Edge’s 20th anniversary, John Brockman, says:

Of all the scientific terms or concepts that ought to be more widely known to help to clarify and inspire science-minded thinking in the general culture, none are more important than “science” itself.

And what is science?

Science is nothing more nor less than the most reliable way of gaining knowledge about anything, whether it be the human spirit, the role of great figures in history, or the structure of DNA.

But how reliable is the most reliable way of gaining knowledge? In my humble opinion, this should have been one of the topics widely discussed this year. Surprisingly, it is not^(*). Unfortunately the proportion of published research likely to be false is higher than is desirable.

Maybe John Broockman should have invited John Ioannidis to have a broader perspective about this key question. Ioannidis has been fighting for better research practices and, in particular, improving the “reproducibility” of scientific research, for years. His paper “Why Most Published Research Findings Are False”⁽¹⁾ has been the most downloaded technical papers from PLoS Medicine.

The challenges to reproducible science are systemic and cultural, but that does not mean they cannot be met. This is the conclusion of a paper he has just published in Nature this month⁽²⁾: “A manifesto for reproducible science.”

A major challenge for scientists is to be open to new and important insights while simultaneously avoiding being misled by our tendency to see structure in randomness. The combination of:

• apophenia (the tendency to see patterns in random data),
• confirmation bias (the tendency to focus on evidence that is in line with our expectations or favoured explanation) and
• hindsight bias (the tendency to see an event as having been predictable only after it has occurred)

can easily lead us to false conclusions.

In a world where most of the relevant causal relationships have already been discovered, it is not difficult to see patterns where none exist. And with an ever-increasing capacity to experiment, any outrageously improbable thing is bound to happen.

The paper concludes with a quote by Richard Feynman: “The first principle is that you must not fool yourself – and you are the easiest person to fool.”

____________________

(*) There are a few scattered references here, here, and here, but methodology does not seem cool enough, after all.

(1) Ioannidis, J. (2005). Why Most Published Research Findings Are False PLoS Medicine, 2 (8) DOI: 10.1371/journal.pmed.0020124

(2) Munafò, Marcus R., Brian A. Nosek, Dorothy V. M. Bishop, Katherine S. Button, Christopher D. Chambers, Nathalie Percie du Sert, Uri Simonsohn, Eric-Jan Wagenmakers, Jennifer J. Ware, and John P. A. Ioannidis. 2017. ‘A Manifesto for Reproducible Science’. Nature Human Behaviour 1 (January): 21. doi:10.1038/s41562-016-0021.

Featured Image: Threats to reproducible science, op.cit.