Will more data help us understand the brain? This is a hot question today, with large scale efforts like US’s BRAIN Initiative, or the European Human Brain Project, betting on it.

There is a popular belief in neuroscience that we are primarily data limited, and that producing large, multimodal, and complex datasets will, with the help of advanced data analysis algorithms, lead to fundamental insights into the way the brain processes information. These datasets do not yet exist, and if they did we would have no way of evaluating whether or not the algorithmically-generated insights were sufficient or even correct.

To throw some light on this question, Eric Jonas and Konrad Paul Kording have devised a clever trick, applying reverse-engineering to a microprocessor as a model “organism”. They describe the effort in a paper—”Could a Neuroscientist Understand a Microprocessor?”—published in PLOS Computational Biology last week⁽¹⁾.

Validating our understanding of complex systems is incredibly difficult when we do not know the actual ground truth.

here we take a classical microprocessor as a model organism, and use our ability to perform arbitrary experiments on it to see if popular data analysis methods from neuroscience can elucidate the way it processes information. Microprocessors are among those artificial information processing systems that are both complex and that we understand at all levels, from the overall logical flow, via logical gates, to the dynamics of transistors.

The microprocessor they chose was no other than the mythical 6502, the 8-bit microprocessor behind Apple II and many other of the first popular “personal” computers and video game consoles at that time, such as Atari, Nintendo, or Commodore.

The conclusion? Inconclusive. Obviously the brain is not a microprocessor—though presently we usually talk as if it was—but the study is a cautionary tale about the promises of modern brain science:

We show that the approaches reveal interesting structure in the data but do not meaningfully describe the hierarchy of information processing in the microprocessor. This suggests current analytic approaches in neuroscience may fall short of producing meaningful understanding of neural systems, regardless of the amount of data. Additionally, we argue for scientists using complex non-linear dynamical systems with known ground truth, such as the microprocessor as a validation platform for time-series and structure discovery methods.

Neuro and data scientist beware! (Big) Data is not enough.

____________________

(1) Jonas E, Kording KP (2017) Could a Neuroscientist Understand a Microprocessor? PLoS Comput Biol 13(1): e1005268. doi:10.1371/journal.pcbi.1005268