In a remarkable column under the title ‘Fraud is good news’, in the Dutch Newspaper NRC Next of 27th June 2012, Arjen van Veelen tells us about the progress of psychology as a respectable science. Recently, several cases of fraud have been uncovered in the Netherlands, concerning social psychologists who invented or massaged their data into compliance with preconceived outcomes. Van Veelen thinks this is in fact good news: psychology has come off age since it works with data instead of vague theories. This means that the findings of psychology finally lend themselves to falsification; the affairs of data fraud are a case in point. Van Veelen reminds us that compared to mathematics psychology is still inferior, but by now it is ‘altogether better off than domains such as theoretical sociology, history of art or legal philosophy. The latter do not engage in experimentation, but rather deal in opinions-with-footnotes. They have no data. Even if they wanted to, they could not fraud their data. So, the Karl Popper-test should be adapted: only if fraud is possible, we are dealing with science. [my translation, MH].’ So much for Van Veelen. The reference to mathematics is somewhat ironic: maths does not deal with data and Van Veelen seems to focus on statistics which many mathematicians think is not really mathematics.
Since I have been appointed at the Institute of Computing and Information Sciences (iCIS) at the Faculty of Natural Sciences, Mathematics and Informatics (Radboud University Nijmegen) my puzzlement over the way scientists from all domains speak of data as the Holy Grail of scientific research has increased. Untenable guarantees of soundproof scientific evidence have been attributed to data and data mining. In the field of Informatics, however, it is assumed that a dataset that is used for scientific research must be aggregated, searched and made accessible in a way that allows for the testing of eventual findings. On the one hand care must be taken that neither the data nor their searcheability can be manipulated, while on the other hand scientific method requires that other researchers can do similar research on the same datasets. This way the findings of advanced pattern-recognition techniques can be verified and falsified. However, this is merely the beginning of good scientific practice.
To qualify as science such a practice must pay attention to (1) the manner in which reality is cut up into bits and bytes to become machine-readable and (2) the way the data points are correlated. Both processes involve suppositions and presumptions which require transparency, because they co-determine the outcome. We may assume that statisticians and empirical scientists are aware of the suppositions they have to make to reduce the flux of reality into manipulable digits, but we cannot assume that they are aware of the presumptions that are inherent in their disciplinary perspectives. Hermeneutics [according to some undoubtedly a pertinent example of vague theories] has demonstrated that our thinking, describing and perceiving necessarily depend on presumptions. These presumptions are the indispensible ‘bias’ that allows us to create order in the world in which we must act. Not a necessary evil, but a remarkably productive precondition for our engagement with reality. Cognitive psychologist Gigerenzer has written a number of relevant considerations which throw new light on the idea of our bounded rationality and the cognitive bias it entails. He explains that such a bias is not a missed opportunity or a restriction of our rationality, but – on the contrary – offers clear added value as compared to systems that keep calculating to optimize decision-making. The point is, however, that this type of ‘bias’ or intuitive presumptions are not neutral. They matter. They influence the outcome of knowledge acquisition, perception and action.
Lawyers and legal philosophers should not get carried away by the data fetishism of some of our journalists, marketeers, politicians, policy makers and scientists. From the perspective of law and the Rule of Law it is important to keep a sharp eye on the fact that even the outcome of solid empirical research can be re-searched or analysed from more than one perspective. And this can lead to different outcomes. In a court of law, but also in administrative decision-making processes based on information-driven computing systems lawyers should see to it that they get their finger behind the ‘bias’ that ‘runs’ the system. Not because ‘bias’ is bad, but because it co-determines the quality and the direction of the outcome. If we do not develop a hermeneutics of the algorithms that inform our cognitive outlook, fair decision-making becomes elusive. To the extent that public decision-making is based on knowledge mined from numerous databases that contain unprecedented amounts of data, citizens have a right to a measure of transparency. Even if the data has been anonymized the knowledge that is mined from it will influence decision-making. This requires a hermeneutic investigation of how the data as well as the search techniques have been ‘framed’ [as social psychologists would say]. I dare to venture that empirical scientists can – for this reason – learn from theoretical scientists, such a legal philosophers. Not in the least because their acquaintance with the Rule of Law requires legal philosophers to pay attention to procedural fairness and the kind of adversarial examinations that also inform Popper’s notion of falsification.
Deze post is ook beschikbaar in: Dutch