AI Regulation

'Lessons from the FDA for AI' (AINow Institute) by Sarah Myers West and Amba Kak comments 

 When we initiated this project at the start of 2023, a growing chorus of voices was mobilizing in favor of stronger and, importantly, ex ante or premarket regulatory scrutiny for artificial intelligence. Industry leaders and regulators alike were calling for stronger standards to bring a sense of order and stability to the sector; the UK AI Summit even had major AI labs commit (in principle) to premarket testing of certain AI models; and there was a general, pervasive sense that some friction was necessary in the frenzied AI industry. The devil would be in the details: Would a licensing scheme, such as that lobbied for by Microsoft, bring stronger scrutiny to the sector, or would it tip the scales in favor of incumbents? How should responsibility be distributed among the many actors involved in development along the AI supply chain, from the base models to end products? Taking time and space to read and think deeply to arrive at viable answers seemed well worth doing. 

As we write this executive summary in July 2024, enacting premarket enforcement of any kind seems like a distant prospect: the conviction that “something should be done” fell prey to a lack of political will to move on actual proposals, while those that did gain traction contained worrying carve-outs. Recent case law brought down from the Supreme Court may create further barriers to agency-led regulation, Congress remains sharply divided on the path forward after losing months to industry-centered deliberation, and Silicon Valley’s venture-capital class is rallying around a deregulatory agenda for AI as central to the Republican presidential election agenda.  Why, amid these headwinds, read up on lessons from the Food and Drug Administration, one of the most regulated industries in the US? 

As the report that follows illustrates, the example of the FDA is most instructive not as a road map for how to approach AI, but as a set of lessons on attuning ex ante regulation to an evolving market and its products. The question at hand is not whether we need an “FDA for AI,” since that crude formulation will inevitably lead to unhelpfully vague answers. Rather, how the FDA transformed the pharmaceutical sector in the United States, from a domain of snake oil salesmen and quack doctors to a market that produces lifesaving drugs that are tested rigorously enough for people around the world to travel to the US just to obtain them, holds key insights for regulatory debates on AI. 

Amy Kapczynski, a member of the advisory council for this project, has written that we didn’t know much about how drugs worked until an agency existed to motivate companies to research them, producing the evidence necessary for entry into the market. One could think about the long-critiqued opacity and unreliability of AI systems along the same lines. 

Because market entry hinges on approval by a regulator, the structure of the pharmaceutical market also provides strong motivations for compliance with the law. Similar motivations are lacking in AI, where regulatory fines often amount to a budget line that deep-pocketed tech firms build into their financial planning year by year. 

Ex ante regulation also provides consistency and reliability for industry players and regulators alike, in contrast to the whack-a-mole approach that constitutes AI regulation at present. 

And while many conversations about artificial intelligence center on risk management, from the FDA example we can learn about the necessity of also validating the efficacy of these systems, enabling a meaningful evaluation of the trade-offs between risks and benefits rather than relying on breezy assertions that AI is inherently innovative. 

To be clear, this is not meant to portray the FDA as a shining beacon: if anything, it is a clear example of how regulatory hurdles hinder competition between firms, leading to bloat. Market incentives are often tipped against sufficient investment in providing affordable access to drugs for populations not seen as highly valued by corporate shareholders. The revolving door between regulators and industry has been and remains an endemic challenge, and real ethical concerns need to be raised about a funding model that relies on corporate fees to support testing and evaluation. 

We do want to emphasize, however, that strong arguments can be made that regulation that orders an otherwise unruly and unpredictable market, that provides not only incentives for beneficial corporate behavior that would not be induced by the market alone, but also disincentives for risky behavior by any one entity that would tarnish the market as a whole, can offer significant value to industry and to the public at large. 

As we launch this report amid more uncertainty around imminent regulatory possibilities, we’re left with new questions to wrestle with: What is the landscape of possibility for regulation post-Chevron? How will First Amendment challenges be navigated for premarket regulatory proposals? How should AI regulators interface with the existing jurisdiction and authorities of sectoral regulators? How should we think about appropriate benchmarks for sociotechnical evaluation, both of the risks associated with AI systems and their efficacy? What audiences are most relevant to increased generation of documentation and information about the AI market (for example, consumers, businesses, regulatory authorities, third-party auditors, journalists and civil society members), and how does this lead to greater accountability? How should regulators ensure the information is meaningful and relevant to that audience? How can regulatory intervention be structured to incentivize compliance? What penalties will be most meaningful to this sector? How do we draw boundaries around the “AI industry” as the focus of regulatory scrutiny? What counts as an “AI firm”? 

AI is already a regulated technology, and the companies developing and deploying AI are bound by existing law. As the frequent refrain of many US government officials goes, There is no AI exemption to the law on the books. Despite this, the current regulatory environment for AI leaves much to be desired: the penalties are paltry given the deep pockets of many tech firms; it’s almost entirely reliant on ex post accountability for harms surfaced by underresourced regulators and investigative journalists; and the haphazard nature of the regulatory approach means that it is difficult, if not impossible, to clearly conscribe boundaries around the “AI market.” 

Given these weaknesses, the idea of creating a novel regulatory agency for AI waxes and wanes alongside waves of attention to the sector, as we’ve outlined in Box 1. This report does not endorse, one way or another, whether or not we need a new regulatory agency for AI. Instead, it seeks to draw lessons from the analogical model most frequently referenced in relation to a stronger regulatory regime for AI: the Food and Drug Administration (FDA). 

Delving deeply into this model surfaced the following key insights: 

An “FDA for AI” is a blunt metaphor to build from. A more productive starting point would look at FDA-style regulatory interventions and how they may be targeted at different points in the AI supply chain: 

Discussions about an “FDA for AI” often operate in a broad analogical manner—a blunt instrument for a conversation deserving of greater nuance. Rather than simply porting over to AI the functions of a large agency whose regulatory toolbox includes many varied approaches, a supply chain approach to understanding AI development adds useful conceptual clarity to conversations about appropriate regulatory design. 

FDA-style interventions might be better suited for certain parts of the AI supply chain than others: 

The FDA’s approach translates most directly at the level of the application or eventual use case, where it is most tractable to validate the safety and effectiveness of an AI product. 

By contrast, attempting similar interventions at other stages of AI development, such as the base or “foundation model” layer, presents potentially intractable challenges like how to identify in advance the universe of possible harms using empirical evaluation. Here, other regulatory design approaches, such as financial regulation and its treatment of systemic risk, may offer more useful corollaries. 

At minimum, mandates for clear documentation of base models, including the data used to train them, will be necessary to enable evaluation at the application layer. 

It is important to clearly differentiate between the “‘users” of AI applications, which are the entities procuring AI systems, and the people or communities the system is used on—the “subjects” of AI’s use. Often there is a significant power differential between “users” and “subjects,” which regulatory interventions must also account for. 

The FDA model offers a powerful lesson in optimizing regulatory design for information production, rather than just product safety. This is urgently needed for AI given the lack of clarity on market participants and the structural opacity surrounding AI development and deployment. 

The FDA has catalyzed and organized an entire field of expertise that has enhanced our understanding of pharmaceuticals and creating and disseminating expertise across stakeholders far beyond understanding incidents in isolation. AI is markedly opaque in contrast: mapping the ecosystem of companies and actors involved in AI development (and thus subject to any accountability or safety interventions) is a challenging task absent regulatory intervention. 

This information-production function is particularly important for AI, a domain where the difficulty—nay, impossibility—of interpretability and explainability remain pressing challenges for the field, and where key players in the market are incentivized against transparency. Over time, the FDA’s interventions have expanded the public’s understanding of how drugs work by ensuring firms invest in research and documentation to comply with a mandate to do so—prior to the existence of the agency, much of the pharmaceutical industry was largely opaque, in ways that bear similarities to the AI market. 

Many specific aspects of information exchange in the FDA model offer lessons for thinking about AI regulation. For example, in the context of pharmaceuticals, there is a focus on multistakeholder communication that requires ongoing information exchange between staff, expert panels, patients, and drug developers. Drug developers are mandated to submit troves of internal documentation, which the FDA then reformats for the public. 

The FDA-managed database of adverse incidents, clinical trials, and guidance documentation also offers key insights for AI incident reporting (an active field of research). It may motivate shifts in the AI development process, encouraging beneficial infrastructures for increasing transparency of deployment and clearer documentation. 

The lack of consensus on what counts as efficacy (rather than safety) is a powerful entry point for regulating AI. There will always be potential harms from AI; the regulatory question thus must consider whether the benefits outweigh the harms. But to know that, we need clear evidence—which we currently lack—of the specific benefits offered by AI technologies. 

A lesson from the FDA is that safety and efficacy of products must be evaluated in parallel. In the context of AI, policymaking has tended to index heavily on safety and harm and not focus as intently on evaluating or challenging the fundamental premise of efficacy, or on presenting a concrete appraisal of risks and benefits. 

To serve the public interest, measures of efficacy should be considered carefully so that they are not primarily or solely indexed on profit or growth, but take into account benefits to society more generally. Regulatory approaches in AI should require developers of AI systems to explain how an AI system works, which societal problems it attempts to address, and what benefits it offers—not just to evaluate where it fails. 

Efficacy evaluation could present an existential challenge to some domains and applications of AI where we currently lack the necessary methods to validate the ostensible benefits of AI usage, given widespread failures in machine learning research to reproduce the findings published in papers. 

Premarket approval is potentially the most powerful stage of regulatory intervention: this is where alignment between regulatory power and companies’ incentives to comply reach their peak. 

Past the point of market entry, the FDA retains some ability to act in the public interest, through market surveillance and recalls—but we see a significant drop in the agency’s ability to act and its track record for doing so successfully. 

In both the context of the FDA and in AI, assuring downstream compliance after a product enters the market is a regulatory challenge. Post-market surveillance is a challenge for AI given the varied provenance of AI system components, but currently characterizes the bulk of ongoing AI regulatory enforcement. 

Looking to the FDA analogy, downstream accountability occurs through mechanisms such as recalling products after the fact, though its ability to enact these remedies is weakened once they are in commercial use. Applied to AI, this is made even more challenging given the difficulty in clearly identifying the chain of provenance for particular components of AI systems. 

In the context of the FDA, companies remain liable for harms caused to the public after drugs are made available for wide release, but establishing liability and then demonstrating causation in the AI context are significant barriers. Currently, the bulk of regulatory enforcement of existing law in AI occurs ex post, and is thus subject to these challenges. 

To have teeth, any regulatory intervention targeting the AI sector must go far beyond the current standard of penalties to meaningfully challenge some of the biggest companies in the world. 

The FDA model hinges on the FDA’s ability to prevent pharmaceutical companies from marketing drugs to physicians—without which they cannot sell their drugs on the market. Controlling this essential gate to market entry is what grants the FDA a big stick, critical to its effectiveness as a regulator; under present conditions, no corollary gates to market entry for AI companies exist. 

The power of FDA regulation also comes from other actors in the system, from physicians to insurance companies, who can themselves refuse to recommend or cover a product if they believe it not helpful. This has acted as an important second line of defense in pharmaceuticals, where the regulatory process has failed to be sufficiently rigorous; there are also corollaries in other industries such as banking and insurance. This deserves stronger development in the context of AI, where the dependencies and sites of friction remain comparatively immature. 

Greater transparency into what constitutes the market itself, and the process through which AI products are sold, will be important to AI governance. Currently, the contours of what constitutes the “AI market” are underspecified and opaque. 

FDA regulation for pharmaceuticals is triggered by the “marketing” of a drug as a critical gate to entry. In other industries, there are gates around the sale of certain products, which may be preferable over marketing given First Amendment concerns. Any attempt at sector-specific AI regulation will run into a thorny set of definitional questions: What constitutes the AI market, and how do products enter into commercial use? Moreover, conceptual clarity that the entity procuring the AI system is often not the same as the individual the system is used on is key, given that AI systems are frequently used by comparatively powerful entities on the less powerful, necessitating interventions that go beyond deceptive marketing and protect the interests of the public at large. 

The funding model for regulatory agencies matters tremendously to its effectiveness, and can inadvertently make the regulator beholden to industry motives. 

The FDA utilizes fees paid by industry players to fund its review process, which ensures adequate resourcing for reviews. However, under the present model, the FDA must submit its budgets regularly to companies paying fees, making them responsible to the companies it is reviewing for its accounting. This is a significant weakening of the agency’s power and risks creating leverage by industry. 

FDA-style documentation requirements for AI would already be a step-change from the current accountability vacuum in AI. Encouraging stronger monitoring and compliance activities within AI firms like recordkeeping and documentation practices would generate organizational reflexivity as well as provide legal hooks for ex post enforcement. 

Introducing FDA-style functions into the AI governance process could motivate restructuring of the development practices, and potentially the operating model, of AI developers. In and of itself, this would create greater internal transparency and accountability within AI firms that would convey societal benefits, and aid the work of enforcement agencies when they need to investigate AI companies.

Surveillance

'Managing and Monitoring Mobile Service Workers via Smartphone App: A case study on worker monitoring, algorithmic management and software for "field service management"' by Wolfie Christl comments 

Mobile service workers, whether they are technicians, homecare workers or cleaning personnel, are increasingly being managed and monitored through smartphone apps. Data that was previously unavailable to employers is now being recorded and evaluated. As these apps give workers instructions about which client to visit next, how to get there and which tasks to perform at the client site, they become algorithmic managers. In the background, powerful software systems for “field service management” help employers organize, coordinate and schedule client visits, work orders and tasks. They promise to optimize, streamline and automate task allocation and help dispatchers and managers supervise workers, monitor their location, assess work performance and identify undesired behavior. 

This case study explores how employers can use software and smartphone apps to manage and monitor mobile service workers. It focuses on the potential implications for employees in Europe and makes two contributions. First, it summarizes survey-based research on how employers actually use these technologies and how workers are affected. Second, it examines software that is available on the market. To illustrate wider practices, it investigates Microsoft software for “field service management”, which is part of the company’s comprehensive “Dynamics 365” system. The investigation aims to identify, examine and document data practices that affect workers, based on a detailed analysis of technical documentation and other publicly available sources. 

Microsoft’s “field service management” system provides extensive functionality for algorithmic management, performance control and behavioral monitoring:

Via mobile app, workers receive instructions about work orders, client destinations, travel routes and a list of predefined tasks to be performed at client sites. The required arrival times and expected durations of work orders and tasks serve as target times. Workers confirm via app when they travel to clients, complete tasks or take breaks. Tasks can include sub-tasks with step-by-step instructions. Consequently, the app structures, directs and micromanages work, aligning it with rigidly defined processes. As it constantly reminds workers of time constraints and deviations, it includes implicit mechanisms for performance and behavior control. 

Employers can utilize behavioral monitoring and performance control to supervise and pressure workers. Dispatchers can see their real-time location and travel routes on a map. They can monitor the current degree of completion of work orders and tasks. For completed work, they can see how much time a worker actually spent on it in relation to the target time. The system can remind both dispatchers and workers of cost limits associated with the time spent on work orders. To keep contractually agreed response times low, it can show a timer to dispatchers that counts the minutes and seconds that have elapsed since the creation of a work order. 

New work orders can result from client inquiries or machines sending error codes, or they are automatically generated on a recurring basis, for example, based on service agreements. Organizations can define standardized work orders that include specific service tasks, instructions and estimated durations. The specified durations serve as target times and are used to distribute and schedule actual work orders to workers. The system can automatically schedule and dispatch work orders, and, as such, automatically assign tasks to workers. It can generate schedules for all workers for entire days or calculate a new schedule every 30 minutes. To match work orders to workers, it considers workers’ availability, location, predicted travel times and skill profiles. Automated scheduling is based on customizable optimization goals. The “maximize productivity” goal leads to schedules with minimized travel and idle times. The system can optionally create schedules that require workers to travel to or from client sites outside their working hours. Besides fully automated scheduling, it can also semi-automatically recommend workers for particular work orders. 

Microsoft offers to predict how long it will take to complete particular work based on past data on work activities and “AI” models. It outlines possible reasons for deviations between predicted and previously specified durations by suggesting, for example, that a particular client, region, weekday, task or worker will likely increase the time required to carry out the work. As such, it may accuse workers of being slower than expected. This functionality can help dispatchers “enhance their team’s performance”, according to Microsoft. 

The system is designed to rate and rank workers according to a wide range of performance and behavior metrics over the previous year, including by the number of completed work orders, the time spent on them in relation to target times and the time spent travelling, on break and in an “idle” state, i.e. without an assignment. Managers can identify undesirable behavior by evaluating how often workers missed the target time or arrived late to a client. They can assess workers by how much revenue they generate and by how satisfied the customers are with their work, based on surveys sent to clients after work was completed. Group-level metrics, for example, on the average time spent completing certain types of work in relation to targets, can also create pressure to speed up work. Employers can use Microsoft’s “Power BI” system to create almost any type of report. 

While GPS location tracking is optional, many features rely on it. Microsoft recommends recording a worker’s location every “60 to 300 seconds”. Clients can be offered access to the current location of the scheduled worker and their estimated arrival time. The system systematically exposes personal data on worker behavior to employers, who can, for example, view records about workers’ exact whereabouts over time. It provides access to enriched location records that indicate, based on geofencing, at which time workers have entered or exited certain client sites or other areas. The system also provides records about remote assistance calls, including their start and end times, and summarizes how much time each worker spent on those calls. 

Despite concerns about the reliability of “generative AI”, Microsoft has rushed to put its CoPilot technology into many products, including field service management. CoPilot promises to summarize information, create draft work orders based on customer emails and draft email replies. While Microsoft advertises it as a means to accelerate work, dispatchers are told to review “AI-generated content” because it can be “incorrect”. The system can be integrated with Microsoft 365 and other Microsoft software. Dispatchers and workers can handle work orders directly from within Outlook and Teams, turning them into task management systems. 

Employers can customize Microsoft’s field service technology and use it in more or less problematic ways. Some intrusive pre-built reports are not available for German customers. Employers can add custom workflows to manage and monitor different types of mobile work. Microsoft’s Home Health system extends its field service technology with functionality specific to homecare. A brief investigation of field service technology offered by the major German vendor SAP shows that its system also offers intrusive performance monitoring functionality. 

Implications for workers. The review of survey-based research on the practical use of field service technology in Austria, Norway, the UK and the US shows that these systems can have significant negative implications for workers. Digital task documentation in the name of billing, quality management or workload balancing can quickly evolve into far-reaching algorithmic management via app. Task direction and monitoring via smartphone app generally leads to increased surveillance and digital control at work. Employers may intentionally misuse the data for purposes other than it was originally collected, including for making negative decisions about workers. Standardized processes, rigid performance targets and automated scheduling can accelerate and intensify work and undermine work discretion. Knowledge that once resided with workers now increasingly lies within technical systems

University Management

'Strategic Bureaucracy: The Convergence of Bureaucratic and Strategic Management Logics in the Organizational Restructuring of Universities' by Peter Woelert and Bjørn Stensaker in (2024) Minerva comments 

Over recent decades, the organizational dimensions of universities have taken a center stage in analyses of higher education policy reform and governance change (e.g., Bleiklie, Enders, and Lepori 2015; Fumasoli and Stensaker 2013; Seeber et al. 2015). Research from different parts of the world has documented a changing university where key organizational trends include greater centralization and formalization, more external and internal reporting and accountability pressures, and the growth of an increasingly professionalized and managerial administrative apparatus within universities (e.g., Christensen 2011; Croucher and Woelert 2022; Ramirez and Christensen 2013). 

Across the literature examining the changing organizational governance of universities, one can identify two related but differently accentuated narratives concerning the observed changes. The first narrative is broadly associated with analyses of public sector reform along New Public Management (NPM) lines and the associated policy and governance changes (Ferlie et al. 1996). Key elements in this narrative are, first, the state’s off-loading of responsibilities for organizational governance to universities and increases in universities’ institutional autonomy in operational matters, and second, increases in universities’ accountability to government authorities and other key stakeholders setting the broader policy goals and objectives (e.g., Capano 2011; Christensen 2011; Enders, de Boer, and Weyer 2013). This shift towards increased institutional autonomy and accountability entails new and expanded administrative responsibilities and demands that, so the narrative goes, compel universities to increasingly acquire the characteristics of formalized, centralized, and hierarchical organizations (Bleiklie, Enders, and Lepori 2015; Musselin 2006). In view of these apparent changes, universities thus can be said to have undergone an organizational process of bureaucratization. 

The second narrative is related to the first in that it also sees the environment as the core driver of change within universities. However, in contrast to linking organizational change in the university directly to public sector reform and ‘steering at a distance’, this narrative foregrounds the emergence of dynamic forms of institutional competition including those associated with markets or quasi-markets (see Jungblut and Vukasovic 2018) as a key driver of change. Intensifying institutional competition for domestic and international students and university ranking positions (Brankovic 2018; Espeland and Sauder 2007), the narrative then goes, has made it imperative for universities to become comprehensively managed organizations capable of strategic decision-making and swift internal restructuring to effectively identify and realize opportunities offered by their environment (see, e.g., Krücken and Meier 2006; Thoenig and Paradeise 2016). In short, according to this narrative, an increasingly competitive and uncertain environment has driven universities to transform into strategically managed organizations. 

Despite the ongoing centrality of these two narratives to accounts of university reform and change, the question of how specifically the two associated organizational logics – bureaucratic and strategic – interrelate in the restructuring of universities has received little attention. This is in parts because the strategic organizational logic, on a more general level, has been frequently yet simplistically painted as implying a radical departure from bureaucratic forms and processes (see on this point, e.g., Hoggett 2007; Wright, Sturdy and Wylie 2012). Applied to the domain of universities, such ‘post-bureaucratic’ notion of strategic management thus provides little scope to account for any common ground or convergence between the two logics in processes of organizational restructuring and change. 

This is an issue also since more recent empirical studies from around the world appear to present a mixed picture as to how universities are changing as organizations (see, e.g., Bleiklie, Enders, and Lepori 2017; Ramirez and Christensen 2013; Seeber et al. 2015). There is, for example, a range of evidence suggesting that universities have become more tightly integrated and managed as organizations (Bleiklie, Enders, and Lepori 2015, Seeber et al. 2015). Yet there are also signs of ongoing fragmentation in university organization due to the successive addition of new administrative layers that ultimately appear to have expanded the bureaucratic dimensions of university life (Maassen and Stensaker 2019, Ramirez and Christensen 2013; Woelert 2023). 

In this conceptual paper, we argue that bureaucratic and strategic logics, despite their different emphases and points of departure, converge and combine with respect to key dimensions of universities’ internal governance and organizing, ultimately giving rise to a hybrid form of organizational governance we refer to as ‘strategic bureaucracy’. We suggest that the manifestation of strategic bureaucracy within universities is inter alia characterized by a strong focus on strategic leadership and the associated management techniques alongside intensification of organizational features and dimensions traditionally associated with bureaucratic governance such as formalization and hierarchical authority. 

The key research questions guiding our discussion are: 1. What are the key characteristics of bureaucratic and strategic logics in a university setting? 2. How are the bureaucratic and strategic organizational logics articulating within universities? 3. What are some of the key organizational implications arising from this articulation between both logics? 

Our use of the notion of organizational logic throughout this paper is motivated by the ambition to conceptualize (a) distinctive forms or types of collective rationality that frame, legitimize, and guide organizational activities; and (b) the relationships between these forms. There are affinities to the institutional logics conception that has become widely popular in the social sciences over recent decades, and which assumes that typically there are several such forms, or logics, to be found and interacting within organizations, and which further posits that understanding of the articulation of such different forms is key to understanding organizational change also (Thornton, Ocasio, and Lounsbury 2012). In contrast to the institutional logics perspective and its ambition to integrate macro-, meso-, and micro-levels of analysis (see Thornton, Ocasio, and Lounsbury 2012), our analyses remain, however, more modestly focused on the organizational level and, in particular, do not attempt to integrate individual or micro-level dimensions or foundations.

'Turning universities into data-driven organisations: seven dimensions of change' by Janja Komljenovic, Sam Sellar and Kean Birch in (2024) Higher Education comments 

Universities are striving to become data-driven organisations, benefitting from data collection, analysis, and various data products, such as business intelligence, learning analytics, personalised recommendations, behavioural nudging, and automation. However, datafication of universities is not an easy process. We empirically explore the struggles and challenges of UK universities in making digital and personal data useful and valuable. We structure our analysis along seven dimensions: the aspirational dimension explores university datafication aims and the challenges of achieving them; the technological dimension explores struggles with digital infrastructure supporting datafication and data quality; the legal dimension includes data privacy, security, vendor management, and new legal complexities that datafication brings; the commercial dimension tackles proprietary data products developed using university data and relations between universities and EdTech companies; the organisational dimension discusses data governance and institutional management relevant to datafication; the ideological dimension explores ideas about data value and the paradoxes that emerge between these ideas and university practices; and the existential dimension considers how datafication changes the core functioning of universities as social institutions. 

Universities recognise the potential value of their digital data and strive to become data-driven organisations that collect, analyse, structure, manage, and use data and data products in their strategic and operational activities. As one of the participants in the focus groups we held during our research on the digitalisation of higher education (HE) in the UK noted: I think every university knows that the data they hold is the wealth of the institution, whether that’s data about how people are behaving or what they’ve actually produced. But that is, at the end of the day, that is the most valuable thing you have. (G6P3). 

This imaginary of the value of digital data is supported and encouraged by policymakers and sectorial agencies (Gulson et al., 2022). Jisc, a digital technology and data agency supporting HE in the UK, has recently launched the Data Maturity Framework, which universities can use to assess their ‘data capability’ and guide strategic change. The Higher Education Statistics Agency (HESA) led the Data Futures Project, which aimed at sector-level data collection and analysis to modernise HE data collection and make it more efficient. These initiatives are further driving the marketisation of HE in the UK (Williamson, 2018) and supporting commercial actors to economically benefit from university data (Komljenovic, 2020), including the recent emergence of educational data brokers (Arantes, 2023). 

Datafication refers to the ‘quantification of human life through digital information, very often for economic value’ (Mejias & Couldry, 2019, p.1), which involves representing social and natural worlds in machine readable digital formats (Williamson et al., 2020) with significant social consequences. In education, datafication consists of collecting and processing data at all levels, from individual to institutional, national and beyond, impacting education stakeholders’ discursive and material practices (Jarke & Breiter, 2019). 

We specifically focus on digital data collected by or registered in digital platforms and digital infrastructure. In many industries, data are valuable when aggregated into big data, allowing more sophisticated analyses, such as group analysis and comparison of individuals for targeted advertising (Birch et al., 2021; Pistor, 2020). In HE, policymakers and educational leaders are attempting to improve quality, efficiency, and impact via datafication at the sectoral and institutional levels (Eynon, 2013). Imaginaries of precision education promise to deliver personalisation akin to other sectors, such as medicine and agriculture (Kuch et al., 2020). 

This omnipresent and techno-deterministic belief in the value of data acts as a mythical belief in magic in that it evokes the ideas of seamless functionality with impressive end experience without attention to how it works or the means with which this was achieved, including struggles, efforts, risks, and costs (Elish & boyd, 2018). However, a paradox emerges as this belief in the value of data is not realised in HE, at least not to the extent that stakeholders would wish; yet it continues to drive investment, business models, actions, and strategies (Komljenovic et al., 2024, 2024b). Currently, data are both valuable and not valuable. Various actors, including EdTech companies and universities, experiment and look for ways to realise economic and social value from data. 

Universities are diverse along many dimensions, including size and resources, which are particularly important for datafication. These differences mean they organise data processes differently. Having thousands of students and staff, universities have to manage petabytes of data, which is a complex task technologically, financially, and legally. The costs of data storage alone have substantially increased, on top of other new costs related to establishing and maintaining the digital ecosystems required for datafication. Universities also deal with legacy software, problems integrating various systems and data flows, ensuring data security, facing cyberattacks, and more. Moreover, diverse actors formally and informally scrutinise universities concerning their data and digital practices (Komljenovic et al., 2024, 2024b). 

In this article, we focus on the UK as an illustrative case due to the high level of digitalisation and datafication of HE (Williamson, 2019). We aim to recognise UK universities’ needs and aims to become data-driven organisations and analyse the challenges they face as they pursue the datafication journey. We first examine datafication in HE and then elaborate on our methodological approach. We then turn to our analysis, structured around seven interrelated dimensions of change, followed by a brief conclusion calling for democratic and relational datafication in HE.

Laundries

The 93 page 'Impacts of money laundering and terrorism financing: Final report' (Australian Institute of Criminology, 2024) by Alicia Schmidt comments 

This report outlines a conceptual model of the social and economic impacts of money laundering and terrorism financing. Drawing on a comprehensive literature review and stakeholder interviews, it identifies possible economic, societal and sectoral impacts. Economic impacts are those that affect the economy at a macro level and include reductions in economic growth and foreign direct investment and the distortion of exchange and interest rates. Societal impacts include changes in crime levels—predicate offences which generate illicit proceeds that are then laundered, crimes financed using laundered funds and crimes attracted to areas where money laundering occurs—and the associated costs to the community. They also include the consequences of terrorism enabled by terrorism financing, including the costs of terrorist attacks and the impact on national reputation. Sectoral impacts include damage to the reputation of the financial sector and other regulated entities, the crowding out of legitimate competitors, artificial increases in prices (eg real estate prices), and lost tax revenue. Importantly, not all impacts are harmful; potential benefits of money laundering include the recovery of proceeds of crime from the enforcement of the anti-money laundering and counter-terrorism financing (AML/CTF) regime, the profitability of certain sectors that facilitate or enable money laundering, and the growth of the AML/CTF industry. Having identified these impacts, this report assesses their significance in the Australian context and sets out a path towards quantifying the impacts identified as both relevant and measurable.

Cth Attorney-General

'The Changing Role of the Attorney-General' by Matthew Groves in (2024) 52(2) Federal Law Review comments 

The office of Attorney-General is an ancient one that remains central to the legal system. The Attorney-General exercises many functions and powers important to the legal system but is also a politician and member of cabinet. This article explains the key functions of the Attorney-General and also the political position of that officer. The focus of the article is upon federal law. It also examines three federal Attorneys-General of modern times and considers how their actions have influenced or reflected the changing conceptions of that office. The article does not suggest that changing conceptions of the office are either good or bad, but instead that they are a reality. The article suggests that these changes provide a reason to reconsider some of the traditional privileges of the Attorney-General, such as the power to grant a fiat in judicial review claims and the Attorney’s privileged position in standing for judicial review of administrative action. 

Public law is replete with loose edges and inexact rules. We tend to associate most of that uncertainty with concepts and doctrines. Some of the cornerstones of public law remain elusive, such as the nature of judicial power, which is central to the allocation of power under the Australian Constitution. The High Court has acknowledged that the judicial power it exercises cannot be defined in a comprehensive manner. The same is true of jurisdictional error, which has assumed a central role in modern Australian administrative law but may never be precisely defined. There are also numerous loose edges around many of the offices vital to our public law framework. George Winterton noted that the key tenets of responsible government may be clear, but ‘the edges are fuzzy and ill-defined’.  The place and role of cabinet in our governance is an example. Cabinet is arguably now the single most important political institution in Australia, whether at the federal, state or territory level of government. Cabinet is the apex predator of our polity. It is cabinet that determines government business of the day, such as what legislation will be tabled before parliament. Cabinet is also the arbiter, or at least the decisive forum for approval, of key decisions in government in a range of decisions, such as judicial and other public appointments, or decisions about Australia’s entry into international treaties.  Legislative and other processes may underpin those decisions, but they will only be triggered after the imprimatur of cabinet is given. Yet, the Australian Constitution makes no mention of cabinet. That constitutional omission has not obscured our understanding of cabinet because the basic rules governing cabinet conduct are widely known. Cabinet is not unlike a rough street gang. It demands solidarity and complete confidence, settles agreements in-house and ruthlessly casts out any member who breaks those rules. 

This paper examines an office that sits within cabinet and is subject to expectations that may conflict with those governing cabinet. That office is the Attorney-General. The Attorney-General exerts a singular authority in our legal system and has long been described as the nation’s first law officer.  The same description is also given to the Attorneys-General of the states and territories. As with cabinet, the Attorney-General is an institution we have inherited from English political practice, placed at the centre of many aspects of government, but one which we have done little to define. Australian political practice has introduced one important distinction, which is that the Attorney-General is a member of cabinet. The political loyalty that follows from cabinet membership affects the Attorney-General but precisely how remains unclear. Many aspects of the role of the Attorney-General have been defined, and arguably changed, by those people who occupy it. This paper draws from two federal Attorneys-General of modern times, whose conception of their role has served to change it and also provoked considerable academic discussion. The paper uses those selected examples to consider the potential consequences for the role of the Attorney-General as the defender of the judiciary and the officer deemed to be able to represent the public interest in judicial review. The paper argues that the first role has changed and that the second role should. The paper also argues that, if the politicians who occupy the office of Attorney-General may change aspects of that role, it is equally legitimate for the courts to adjust those aspects of the office which are based in the common law. But it is useful to first sketch the role of the Attorney-General and some of the unique powers and privileges of that office.

Perceptions of GenAI Assessment

'Perceived Impact of Generative AI on Assessments: Comparing Educator and Student Perspectives in Australia, Cyprus, and the United States' by René F. Kizilcec, Elaine Huber, Elena C. Papanastasiou, Andrew Cram, Christos A. Makridis,Adele Smolansky, Sandris Zeivots and Corina Raduescu in (2024) Computers and Education: Artificial Intelligence comments 

The growing use of generative AI tools built on large language models (LLMs) calls the sustainability of traditional assessment practices into question. Tools like OpenAI's ChatGPT can generate eloquent essays on any topic and in any language, write code in various programming languages, and ace most standardized tests, all within seconds. We conducted an international survey of educators and students in higher education to understand and compare their perspectives on the impact of Generative AI across various assessment scenarios, building on an established framework for examining the quality of online assessments along six dimensions. Across three universities, 680 students and 87 educators, who moderately use generative AI, consider essay and coding assessments to be most impacted. Educators strongly prefer assessments that are adapted to assume the use of AI and encourage critical thinking, while students' reactions are mixed, in part due to concerns about a loss of creativity. The findings show the importance of engaging educators and students in assessment reform efforts to focus on the process of learning over its outputs, alongside higher-order thinking and authentic applications. 

In a remarkable convergence of research and real-world impact, the sudden emergence of ChatGPT has sent shockwaves through the global landscape of education. As students, educators, and university administrators grapple with the practical implications of generative AI, it becomes abundantly clear that we stand at the precipice of a new era. Since the release of GPT-3, a groundbreaking large language model (LLM) released by OpenAI, and its offspring, the user-friendly ChatGPT conversational interface, researchers are both excited and filled with trepidation over its boundless possibilities and transformative potential (Cotton et al., 2023; Farazouli et al., 2023; Nikolic et al., 2023). Generative AI, as defined by Weng (2023), refers to a technology that utilizes deep learning models to generate content that closely resembles human expression in response to complex and diverse prompts. These tools have the ability to produce conversational-style text that closely resembles human writing, as well as other visual and auditory media. They can be used to create systems that operate in ways that resemble human cognition and behavior (Siemens et al., 2022; Markel et al., 2023; Park et al., 2023). For example, ChatGPT and its derivatives is increasingly utilized for language translation, human-like conversation with chatbots, writing articles, stories, computer code, and other forms of written content ( Cotton et al., 2023). 

Generative AI tools promise many benefits in education, such as increasing student engagement in learning tasks, providing timely feedback, aiding research and collaboration, and improving accessibility (Kasneci et al., 2023). For example, AI technology can provide immediate feedback via automated grading (Mate and Weidenhofer, 2022) and facilitate the provision of meaningful feedback in large cohorts (Bernius et al., 2022). At the same time, AI raises serious concerns about the validity of widely used assessment practices, especially concerns about academic integrity and bypassing important learning processes (Swiecki et al., 2022). Because standard assessment practices focus on evaluating the final products like essays to measure learning, researchers have highlighted the potential for plagiarism as a key challenge with using ChatGPT for assessment in higher education (Cotton et al., 2023). Students can potentially use generative AI tools like ChatGPT to cheat on online assessments by submitting essays that are not their own work. The problem might be more prevalent in online assessments where students tend to feel more distant from their instructors (Papanastasiou and Solomonidou, 2023). 

Educators can face challenges distinguishing between students' own work and responses generated by AI tools, making it difficult to assess students' level of understanding and their ability to apply the material (Mao et al., 2024). Unless educators and academic institutions adapt to this new reality, generative AI can undermine academic integrity in online assessments and the purpose of higher education to educate students, which may reduce the signaling effects and inherent value in formal educational attainment (Cotton et al., 2023). To address this major problem, scholars have called for applying AI in classrooms in such a way that promotes self-regulated and more productive learning, rather than treating it as a replacement for human effort in the learning process (Hopfenbeck et al., 2023; Mao et al., 2024; Swiecki et al., 2022). 

AI has been framed as a transformative resource that educators and students can leverage in teaching and learning. Weng (2023) suggests ways to employ generative AI tools such as raising awareness of these tools, using them in class, in assessments, and engaging in discussions with students about their promises and challenges. They argue that this is more productive than either banning them or giving them a central role in the curriculum. Integrating generative AI with assessments can also transform assessment practices and experiences, for example, by immersing students in simulated learning environments where they can safely and repeatedly practice skills (Markel et al., 2023). This paradigm shift may require the development of new assessment approaches and policies that achieve a balance between the advantages of AI and the imperative to maintain academic integrity (Chan and Chen, 2023). 

Bearman et al. (2023) argued that educational assessment practice has not kept up with the digital transformation. Students and educators require better guidance on how to engage in meaningful interactions with AI systems for the purpose of assessment (Viberg et al., 2024). These interactions would directly assess students' learning process, critical thinking, and evaluative skills, not just their knowledge and comprehension. To this end, we expect to see revised guidelines and recommendations for educational assessment policies, incorporating input from stakeholders involved in assessment design to address the two major questions around AI integration in education: ‘what’ to assess (Sabzalieva and Valentini, 2023) and ‘how’ to assess it (Chan and Chen, 2023). 

There are many ongoing conversations around what types of assessments are needed given the capabilities of generative AI tools. Bearman and Luckin (2020) emphasize that machines lack the ability to define quality or establish standards, making it crucial to develop assessment designs that prioritize the distinctly human capacity for defining quality standards. This raises questions on assessment standards and a move towards more authentic, adaptive, and continuous assessment (Gašević et al., 2023). Adapting current assessment practices in response to the ubiquitous availability of generative AI tools is timely but also effortful. As AI continues to play a pivotal role in society, assessments need to be adapted to ensure that they assess students authentically and ethically. Assessment approaches that foster human expertise and judgment are primed to gain greater significance through digital technologies (Dann, 2014; Nieminen et al., 2023; Bearman and Luckin, 2020). 

This moment presents a rare opportunity for real innovation in current assessment practices, because most commonly used assessments were not conceived with access to powerful generative AI tools in mind. To meet the moment, we need to understand educators' and students' perspectives on the issue to achieve sustainable advances in assessment practices. We are especially interested in how much the perspectives of these two stakeholders—educators and students—are in alignment to provide a common ground. This may vary across contexts shaped by the local pace of technological adoption, institutional characteristics, cultural differences, and linguistic variation in technological efficacy (i.e., generative AI tools may work better in English than in other languages such as Greek). Within this context, we pose the following three research questions: (1) Which types of assessments do educators and students consider to be most impacted by generative AI? (2) How do educators and students think that students will be using generative AI in completing assessments? (3) And what are their preferences and attitudes toward adapting assessments to incorporate generative AI? Answering these questions by building on an established framework for examining assessment quality for university online assessments is essential since such knowledge is needed to guide efforts to reform future assessment practices.

Netherlands Pseudolaw

'Sovereign Citizen Groups in the Netherlands are Arming Themselves: Cause for Concern?' by Menso Hartgers (International Centre for Counter-Terrorism, 2024) comments 

On 9 July 2024, the first court hearing of a Dutch citizen who attempted to illegally acquire firearms and his arms dealer began. The suspect is part of the so-called Common Law Netherlands Earth (in Dutch Common Law Nederland Earth), a sovereign citizen group that rejects the democratic rule of law in the Netherlands, believing that they are not subject to government authority. Instead, they argue to be bound only by their interpretation of natural law and selective historical legal principles. The Public Prosecutor asserts that the group attempted to illegally acquire firearms to instigate a revolution. This case is indicative of a possible trend of sovereign citizens and anti-government extremists who not only espouse anti-democratic ideas but are also attempting to seize the means to realise their ideations. In a March 2024 report, ICCT already encountered online incitements for violence by Common Law Netherlands Earth, which advocated for “civil arrest” to “fight back” against government and law enforcement. This rhetoric is clearly a call to arms against the democratic rule of law. The ongoing trial evidences that the group’s adherents are heeding these calls, quite literally arming themselves. This short analysis aims to further shed light on the group, reflecting on developments of sovereign citizen movements abroad, and outlining possible challenges ahead. 

Anti-government extremism (AGE) encompasses movements, networks, and individuals that reject government legitimacy and undermine the democratic legal order, often rooted in conspiracy narratives, particularly surrounding the existence of an ‘evil elite’ that supposedly runs the world from behind the scenes. Sovereign citizens are a subset of anti-government extremists. Their beliefs in an evil elite are often centred around the notion that man-made, government mandated laws and statutes are ways to oppress ‘the people’ and strip them of their individual rights and sovereignty. They position themselves outside of this ‘oppressive’ legal regime by advocating for the establishment of their own courts and parallel societies instead. 

Common Law Netherlands Earth is part of an international genre of sovereign citizen collectives that is notably found in the UK and ‘British Heritage countries’ (i.e., former British colonies that have adopted and continue to use particular legislative or judiciary structures like courts inspired by British colonialism) such as the US, Canada, and Australia. The existence of these various ‘Common Law’ groups is predicated on a shared narrative of an aspired supersession of natural laws over existing legal structures and democratic norms. In the case of Common Law Netherlands Earth, it must further be noted that, based on limited open-source intelligence, the group may have changed its name or split off under a different nomenclature. 

ICCT’s early investigations into the Dutch anti-government and sovereign citizen extremist scene revealed that the group is (or was) comparatively well organised. Its official website, which is currently defunct, neatly listed several Telegram channels through which it could share sovereign citizen content and messages, while also linking to international affiliate organisations like the Australian People of the Commonwealth, the Republic of Kanata in Canada, and the People of Krystal City in Spain. Visitors of the website could also be directed to toolkits and documents, including procedural guidelines detailing the establishment of Common Law people’s tribunals. In the eyes of groups that adopt Common Law views, these people’s tribunals serve as Common Law alternatives to Dutch courts. 

Perhaps more worryingly are the ‘sheriffs’ training materials that the group distributes through its website and various Telegram channels. In Common Law speak, sheriffs, are those individuals empowered to execute the verdicts of people’s tribunals. According to the group’s doctrine, Common Law sheriffs are authorised to arrest, detain, and use force against transgressors. A training manual that was examined by ICCT mentions that “sheriffs and their deputies must be qualified and trained in the expert use of force, to arrest and detain suspects, convicted criminals, and enemies of the republic.” (NB: the Netherlands is a constitutional monarchy, not a republic. This reference could allude to a different type of State that sovereigns envision). Moreover, the document states that “all sheriffs must be armed and equipped with protective gear.” 

Against the backdrop of the group’s public advocacy for arming its members, the currently ongoing trial of a Common Law sovereign citizen—who the Public Prosecutor alleges is a ‘sheriff’—attempting to acquire firearms is alarming. It shows that sovereign groups are earnest in their attempts to arm themselves. Coupled with their democratic state rejectionist ideology, the likelihood of an escalation of violence against state and democratic institutions increases. Moreover, as ICCT’s report indicates, efforts are made to actively recruit members with a military or law enforcement background. This is corroborated by previous convictions of Dutch Common Law sovereign citizens on weapon charges. In September 2023, a veteran was sentenced to eighteen months jail (twelve of which are probationary) for attempting to purchase firearms. The Public Prosecutor alleged that the man had intended to disseminate these firearms to other veterans, although the court did not rule this sufficiently evidenced. Nevertheless, this case, along with other cases of active-duty and veteran sovereign citizens in neighbouring countries such as Germany raises concerns about the tactical experience, technical expertise, and operational capabilities of Common Law and other sovereign citizen groups to handle firearms.