RobotDebt

'The New Digital Future for Welfare: Debts Without Legal Proofs or Moral Authority' by Terry Carney in (2018) 41(30 UNSW Law Journal 783 comments 

This article reviews Centrelink’s online compliance initiative (‘OCI’) to determine whether the Senate Community Affairs References Committee was right to recommend that Centrelink resume responsibility for obtaining all information necessary for calculating working age payment debts based on verifiable actual fortnightly earnings rather than on the basis of assumed averages, or whether responsibility has always remained with Centrelink when the person is unable to easily provide records. It argues that legal responsibility ultimately has always rested with Centrelink in such cases and outlines distributional justice and best practice reasons why the OCI system should be brought into compliance with the law.

Carney argues

A new digital future for administration and administrative review is much discussed, with Britain touted as a leader. Automation of decision-making through application of machine learning algorithms is one way efficiency and accuracy is pursued, including Australia’s online compliance intervention (‘OCI’) debt recovery system – colloquially known as ‘robo-debt’ – which is one part of the government’s Better Management of the Welfare System initiative projected to recover $2.1 billion of social security ‘overpayments’ over four years.

It is trite maths that statistical averages (whether means or medians) tell nothing about the variability or otherwise of the underlying numbers from which averages are calculated. Only if those underlying numbers do not vary at all is it possible to extrapolate from the average a figure for any one of the component periods to which the average relates. Otherwise the true underlying pattern may be as diverse as the experience of Australia’s highly variable drought/flood pattern in the face of knowledge of ‘average’ yearly rainfall figures. Yet precisely such a mathematical fault lies at the heart of the introduction from July 2016 of the OCI machine-learning method for raising and recovering social security overpayment debts. This extrapolates Australian Taxation Office (‘ATO’) data matching information about the total amount and period over which employment income was earned, and applies that average to each and every separate fortnightly rate calculation period for working-age payments.

ATO data-matching previously was very properly used to trigger further enquiries about a portion of the approximately 300 000 discrepancies (and possible debts) identified annually by the Department of Human Services (better known to the public as Centrelink). Based on risk management profiling, Centrelink formerly selected around seven per cent of discrepancies for manual review and enquiry, to obtain firm information about actual earnings in each payment fortnight (whether provided by the person or from invoking its compulsory powers to require employers to provide pay slip records, or banks to disclose statements). From July 2016 the OCI scheme targets and raises debts in every case where the person cannot disprove the possible overpayment (or its quantum), such as by producing or obtaining copies of pay slips. The Ombudsman identified the dramatic scale of the change, writing that ‘DHS estimates it will undertake approximately 783 000 interventions in 2016–17 compared to approximately 20 000 compliance interventions per year under the previous manual process’. While the full pipeline effect of the new system had yet to be fully felt due to the lag in review processing, debt cases lodged with the Administrative Appeals Tribunal (‘AAT’) increased by 28.5 per cent in the first full year of the OCI scheme in 2016–17, compared to the previous year (rising from 3387 to 4354).

Why is this of legal, policy and moral interest? It is of interest to the law because, as argued below, the so-called ‘practical onus’ to establish a debt and its size continues to remain with Centrelink; the failure of a person to ‘disprove’ the possibility of a debt is not a legal foundation for a debt.

This is not new. It was recognised in Centrelink’s preOCI guideline which, while (somewhat dubiously) accepting averaging as a ‘last resort’, correctly added ‘[t]he raising and recovery of debts must satisfy legislative requirements. Evidence is required to support the claim that a legally recoverable debt exists’.  And it is also of legal interest because the nature of the issue (the monetary, moral and practical implications of contending that a debt is owed) raises the bar for Centrelink in terms of its discharge of that practical onus. This is what I loosely term the ‘rule of law’ challenge (Part II below).

It is of wider policy interest because, in practice, when confronted with suggestions of having an overpayment, often from up to seven years ago, the least literate, least powerful, and most vulnerable alleged debtors will simply throw up their hands, assume Centrelink knows that there really is a debt, and seek to pay it off as quickly as possible. Alleged debtors do so even though the Ombudsman’s report demonstrated that most debts calculated this way were greatly inflated, and that some were false (zero debts), and they continue doing so because the otherwise worthy recommendations of the Ombudsman and the Parliamentary Community Affairs Committee fail to correct the fundamental legal error. It is of moral or ethical interest because Centrelink did not advise recipients of the need to keep pay records for longer than six months, and because it is difficult to see how the current system meets requirements of model-decision-making at primary level or ‘model litigant’ obligations for internal and AAT review. Finally, it is of interest because it is a test-bed for assessing the fitness for purpose of the administrative review system (especially its normative impact on good primary decision-making) and as a window into the digital future (Part III below).

The article briefly concludes in Part IV by arguing that the OCI system urgently be rendered compliant with the law, lest it undermine public confidence in the positive contribution machine learning can bring to better administration.

Carney concludes

Machine learning algorithms and other digital applications to improve the accuracy and efficiency of decision-making are unquestionably the way of the future, as the rapid expansion of such systems across a wide range of administrative settings in the USA testifies. Apart from the rule of law challenge in designing such systems, there is the challenge of rendering it consistent with principles of sound administration, such as the 27 principles laid down by the Administrative Review Council in its 2004 report, or Jerry Mashaw’s accuracy, efficiency and ‘dignity’ objectives. As demonstrated, machine learning initiatives contravene dignity and fairness principles if citizens are disadvantaged by presumed digital literacy (access to or ability to use computers), lack of understanding of the true nature of the issue (as in not knowing that fortnightly income outcomes are very different to application of an average), are overcome by (possibly misplaced) feelings of fear and guilt about a suggested moral wrong such as incurring a debt, or otherwise ‘cause a disproportionate impact on members of certain classes or groups’.

Addressing such concerns calls for creativity and ongoing debate on alternatives. Just as modes of achieving accountability alter when, say, delivery of welfare is shifted from government to private sector auspices (as with job placement services in Australia) or to charitable agencies (as in some instances in the United States of America), such changes are not intrinsically better or worse, but call for careful weighing up of attributes of these radically different ‘regimes’. Arguably so too when moving from more traditional human agency decision-making to greater (or complete) reliance on machine learning systems. For example reasonable minds still differ over whether legal paradigms of greater ‘transparency’ of system design and operations is the answer, with Desai and Kroll persuasively arguing instead for a ‘technological’ remedy of incorporating into regulatory and accountability frameworks the ‘trust but verify’ approach adopted by the sector when building and testing systems. By contrast, Coglianese and Lehr assess machine learning against traditional legal standards of non-delegation, due process (procedural fairness), non-discrimination and transparency;  worthy standards of course, but ones which the history of robo-debt demonstrates proved to be inadequate to redress its systemic deficiencies, at least within the current system of review and appeal. Once Australia’s OCI scheme is reformed to be compatible with the rule of law and more compatible with best practice principles of administration, attention should turn to these wider considerations for other machine learning digital initiatives.

However, there are also potential lessons for refinement of AAT practice. Lorne Sossin argues that selection of the best model of a tribunal, and its finer aspects of design, ideally should involve:

a holistic enterprise, involving the expertise of policy-makers and lawyers, administrators and IT professionals, organizational and behavioural specialists together with communication experts. All aspects of the tribunal experience should be considered together – that is, the statutory authority of the tribunal together with its physical and virtual presence, the budget and staffing of the tribunal together with its approach to proportionality or streaming of caseloads, the rule-making together with the strategies for accessibility, inclusion and accommodations.

For instance, the conversational, symbolic and other atmospherics of hearings can be critical to real engagement and accessibility. 

Refining administrative review to fit contemporary circumstances is not new, as Lorne Sossin observes. For its part, the SSCSD of the AAT, as successor to the Social Security Appeals Tribunal (‘SSAT’), experienced significant changes over the last decade as it balanced competing pressures of justice and efficiency. For instance the legislative requirement for quick decision-making is said to stifle the degree to which the AAT itself now actively seeks additional information from agencies or others. The raw numbers demonstrate the dramatic pressures on the SSCSD, with a 44 per cent increase in appeal numbers in the last five years coinciding with a roughly 33 per cent decrease in membership; and perhaps not entirely coincidentally, a seven percentage point decline in set aside decisions (from 27 per cent to 20 per cent) over the decade 2007–17.  So procedure matters.

The main implication I suggest is that the SSCSD should focus on ways in which its decision-making can boost the normative or educative impact of review in improving primary decision-making. Given that the illegality of OCI debt raising suggested here continued unchecked for 18 months as at the date of writing, despite AAT1 decisions invalidating it, and that those legal doubts remained unbroached publicly, it is clear that neither the normative nor the educative power of current review is optimal. Selective publication of AAT1 decisions, especially where, for whatever reason, the agency elects not to seek review AAT2 of an adverse AAT1 decision,  is one remedy. Another is being alert to unintended consequences – such as any premature but notionally ‘voluntary’ withdrawal of applications by under-informed applicants during any pre-hearing screening; or the absence of feedback to the agency when delivering oral decisions; or any undue exercise of powers to endorse (unpublished) settlement of such matters at AAT2 – all of which weaken educative feedback loops to decision-makers.

Given that many people in receipt of working age payments are vulnerable and thus may not participate in their hearing,  care is also needed in exercising dismissal and reinstatement powers. For instance it is wrong to apply the same test when dealing with a reinstatement application as when dismissing for failure to attend a scheduled hearing: the latter is mainly a question of whether the person was adequately notified and any excuse they may have for non-attendance, while reinstatement crucially also requires application of a presumption of reinstatement and some consideration of the merits of the matter. Channelling Juliet Lucy, there may also be other creative (and highly costeffective) possibilities. Examples include greater use of pre-hearing powers to require Centrelink provision of additional documents or information before AAT1 hearings, or clarification of analysis or reasoning, effectively ‘front-ending’ considerations otherwise only incorporated as part of AAT directions when deciding the application.

Ultimately, however, the aim must be to ensure that primary decision-making is of the highest quality, integrity and legality, minimising the need for what Juliet Lucy terms the AAT’s function of ‘facilitat[ing] “administrative second thoughts”’