If you are in a hurry, the short answer to this question is no, it does not.
Current enthusiasm about big data is based on an extreme inductivist optimism. Hykel Hosni and Angelo Vulpiani explain why in a readable note(1).
There are two equally radical, yet opposite methodologies. A reductionist one, based on deduction from first principles, and a naıve-inductivist one, based only on data. The availability of unprecedented amounts of data and increasingly sophisticated algorithmic techniques (machine learning) has led some luminaries and interested parties to claim that we can dispense with theory, modelling or even hypothesising. Chris Anderson famously announced “the end of theory” in Wired in 2008: the data deluge makes the scientific method obsolete.
All this enthusiasm is loosely rooted on two presuppositions:
- First, the idea that big data will lead to much better forecasts.
- Second, it will do so across the board, from scientific discovery to medical, financial, commercial and political applications.
Hosni and Vulpiani challenge both:
- More data may lead to worse predictions.
- A suitably specified context is crucial for forecasts to be scientifically meaningful.
They use the representative example of weather forecasting, the mother of all approaches to prediction, where the early attempts at arriving at a quantitative solution turned out to be unsuccessful precisely because they took into account too much data.
This is the takeaway for would-be big data practitioners: Big data constitutes a great opportunity for scientific and technological advance, with a potential for considerable socio-economic impact. The role of modelling cannot be discounted: not only larger data-sets, but also the lack of an appropriate level of description may make useful forecasting practically impossible.
in spite of a persistent emphasis on a fourth paradigm (beyond the traditional ones, i.e. experiment, theory and computation) based only on data, there is as yet no evidence data alone can bring about scientifically meaningful advance. To the contrary, (…) up to now it seems that the unique way to understand some non-trivial scientific or technological problem, is following the traditional approach based on a clever combination of data, theory (and/or computations), intuition and wise use of previous knowledge.
In other words, don’t throw out your science textbooks just yet.
(1) Hosni, Hykel, and Angelo Vulpiani. 2017. ‘Forecasting in the Light of Big Data’, May. doi:10.1007/s13347-017-0265-3.