'The ‘rule of law’ implications of data-driven decision-making: a techno-regulatory perspective' by Emre Bayamlıoğlu and Ronald Leenes in (2018)
Law, Innovation and Technology comments
Techno-regulation is a prominent mechanism for regulating human behaviour. One type of techno-regulation concerns automated decision-making with legal effects. While automated decision-making (ADM) systems in the public domain have traditionally been based on conscious design of decisional norms, increasingly, Data Science methodologies are used to devise these norms. This data-driven approach causes frictions with the underlying principle of public-sector decision-making, namely adherence to the rule of law. In this paper we discuss three major challenges data-driven ADM poses to the Rule Law: law as a normative enterprise, law as a causative enterprise and law as a moral enterprise.
The authors argue
Since the industrialisation, we have witnessed an influx of novel artefacts, objects, and more recently automated systems that come to play a significant role in what we do, how we perceive and interpret the world, how we make our choices, and under what conditions.
We see the emergence of ‘algorithmic authority’ as the legitimate power of ‘code’ to direct human action and also to impact which information is considered true.
Issues surrounding (big) data analytics and automated decision-making (ADM), such as those touching on privacy and data protection, have been widely studied, but the enabling and restricting role of data-driven solutions as techno-regulatory orders have remained mostly unanalysed.
Although studies on techno-regulation frequently analyse and characterise technology for its normativity, research theorising the regulatory relevance of Big Data analytics as a normative order in itself is much sparser.
As the world of data has become the test bed for social sciences, economic innovation and state administration, the need for research explaining and framing the regulatory dimension of the data-driven practices is ever more critical.
This article contributes to this venture. It departs from the premise that data-driven ADM processes, governed by complex algorithms, are either embodiments of existing normative orders, or they themselves enact ad hoc regulatory orders with or without a legal basis. In terms of regulatory constraints and capacities, data-driven ADM systems go much beyond existing legal decision-making based on codified legal norms. Although both types of systems (data-driven versus code-driven as Mireille Hildebrandt calls them) regulate human behaviour, their assessment from a rule of law perspective is different. In fact, data-driven ADM systems undermine the rule of law and hence, developers, lawyers and subjects of decisions by these systems should pay attention.
The paper is organised as follows. First, in Section 2, we revisit techno-regulation as a mechanism to regulate human behaviour and describe how conscious implementation of norms is being augmented or replaced by norms derived from data analytics. Next, in Section 3, we discuss some shortcomings and effects of this turn towards data-driven ADM. Section 4 addresses the challenges that these shortcomings cause for the rule of law as the backbone of legal decision-making. Section 5 concludes the paper with some reflections and a call for action.
They conclude
The pervasive employment of data-driven systems is indicative of our current and future dependence on technologies incorporating, articulating and amplifying computational and calculative rationalities – linking ends to means in novel and humanly unintelligible ways.
Counting, calculating, accounting and eventually computing – a hectic obsession of modern humans – now has reached the point where we turn blind to almost anything that falls beyond or outside of our measuring capacity. The social complexity we live in dictates a paradigm where knowledge is limited without measurement. This current prevailing understanding of data analytics and technology is rooted in the political philosophy of modern societies which is predicated upon a distinction between politics and science, according to which, while the former is supposedly based on values, the latter seeks for “objective truth”.
The problem with the emerging data-driven epistemology is that the kind of knowing it suggests is not always what we aim for or desire if we want to maintain the rule of law, but simply what technology allows us. Or as David Berry put it: ‘subtractive methods of understanding reality (episteme) produce new knowledges and methods for the control of reality (techne)’.
Data-driven processes increasingly re-embody norms within a form of an instrumentalized rationality. Data-driven instrumental reason converts each dilemma, conflict or antagonism, however material and fundamental, into a mere paradox which could be counteracted by the application of logic – substituting interests with the requirements of the technique and the normativity of law with the performativity of the algorithm. Big data constrains the possibilities for political and moral choices by reducing governance to a technical process of adaptation, and law to a process of optimisation – rendering politics a mere question of “better-doing”.
If the rule of law is taken as a meta-principle which primarily presupposes an autonomous subject who could effectively reason against the norms and introduce a novel interpretation, the type of law that the data-driven paradigm implements, leaves no room for effective contestation – but only rationalised logical and probabilistic reasoning. This results in an all or nothing approach which hardly complies with the principles of proportionality, subject autonomy, expediency and certainty. At some point, the binary nature of Turing computation and its logical consistency eliminates any discretionary power as a capacity of the legal system to import extraneous knowledge to produce answers to the ‘hard cases’.
As the consequences of such formalisation of reason, our aims and values like justice, equality, happiness, solidarity and tolerance, which have been inherent in or sanctioned by reason since the Enlightenment, lose their intellectual ground. Although such values exist in the constitutions of the sovereign states, they lack any confirmation by reason or agency to link them to an objective reality.
