AI Regulation

'Humans Outside the Loop' by Charlotte Tschider  comments 

Artificial Intelligence is not all artificial. After all, despite the need for high-powered machines that can create complex algorithms and routinely improve them, humans are instrumental in every step used to create it. Through data selection, decisional design, training, testing, and tuning to managing AI’s developments as it is used in the human world, humans exert agency and control over these choices and practices. AI is now ubiquitous: it is part of every sector and, for most people, their everyday lives. When AI development companies create unsafe products, however, we might be surprised to discover that very few legal options exist to actually remedy any wrongs. 

This paper introduces the myriad of choices humans make to create safe and effective AI products, then explores key issues in existing liability models. Significant issues in negligence and products liability negligence schemes, including contractual limitations on liability, separate organizations creating AI products from the actual harm, obscure the origin of issues, and reduce the likelihood of plaintiff recovery. Principally, AI offers a unique vantage point for analyzing the relative limits of tort law in these types of technologies, challenging long-held divisions and theoretical constructs, frustrating its goals. From the perspectives of both businesses licensing AI and AI users, this paper identifies key impediments to realizing tort goals and proposes an alternative regulatory scheme that reframes liability from the human in the loop to the humans outside the loop.

Privacy

'Privacy Theater in the Bankruptcy Courts' by Christopher Bradley in (2023) 74  Hastings Law Journal comments 

The sharply resource-constrained environment of bankruptcy breeds fierce competition for the value that remains in a bankrupt firm.'Managers are tasked with maximizing the overall value of the firm, while those with individual rights in the firm's assets or claims against the firm fight to protect their interests. Businesses may seek to restructure or escape contractual obligations and other burdensome relationships in order to maximize their assets' value-a process that can seem inequitable to those who dealt with the debtor prebankruptcy. The determinations made in bankruptcy proceedings often go beyond business and financial issues and implicate public policy. For example, in recent years, debtors have sought relief from liabilities to victims of asbestosis, sexual assault, or opioid addiction. 

The general bankruptcy goal of value maximization is often in tension with the interests of consumers. As a matter of policy, consumer interests may be worth protecting, but usually consumers lack individual incentive to involve themselves in the bankruptcy process. As a result, consumers can see gift card values wiped out; warranties, leases, and loans abandoned or dramatically restructured; or private data sold to the highest bidder. 

The efforts of bankrupt firms to monetize consumers' private data is a prime example of how unresolved policy conflicts can end up in the bankruptcy system. Although the intersection between privacy law and the big business of consumer data has become a major focus of policymakers, scholars, the business community, and consumer advocates, the legal regime governing the commercial use of data in bankruptcy proceedings remains contested and often unclear. To the stakeholders in a financially distressed firm, consumers' private data is a resource that can be monetized. While a healthy firm might hesitate to exploit private data out of concern over reputational risk or liability, these scruples matter less to financially distressed firms. 

The pressure to capitalize on data has been acute in quickly transforming industries such as retail and healthcare. Thousands of companies in these fields have experienced financial distress, closed offices and stores, and sought bankruptcy relief.' Meanwhile, the value of consumer data has continued to rise, tempting cash-strapped companies to sell this data regardless of any risk to their reputation or harm to consumers. 

Recognizing this issue, Congress passed legislation to protect consumer privacy in bankruptcy proceedings in the sweeping Bankruptcy Abuse Prevention and Consumer Protection Act of 2005. Congress did so in reaction to a high-profile case in which a failing dot-com e-retailer, Toysmart, sought to sell its consumer data-including data collected from and about children-to the highest bidder, despite the company's promises never to do so. Regulators opposed the sale, the national media covered the battle, and the sale was scuttled. In response to public outcry, Congress created the regime that Senator Patrick Leahy, the law's primary legislative sponsor, pronounced would prevent future breaches of such privacy promises. 

This law created a new institution and procedural rules to address the tension between bankruptcy value maximization and the protection of consumer data.  When certain conditions are met, the law provides for the appointment of a "consumer privacy ombudsman" (an "ombud") to investigate a sale of consumers' private information, and to advise the court on whether the sale should be allowed. The rationale underlying this law is that by forcing the retention of an outside expert to report to the court, the process will ensure consideration of the impact of the sale on the privacy interests of consumers. 

In a companion article [noted below], I summarize the contributions that ombuds have made to the law of privacy. There, I show that privacy law, as applied by ombuds, imposes some guardrails on companies' sale of consumer data, but that these restrictions leave businesses with considerable latitude to collect, use, and transact in data. In addition, privacy law in this area relies heavily on the assumption that companies will continue to uphold the privacy promises that they made to their consumers after the sale of data is complete. While the companion article deals with ombuds' impact on privacy law in general, this Article addresses the operation of the regime in actual bankruptcy proceedings. This Article provides an empirically grounded analysis of the regime that Congress established. Drawing primarily from a hand-collected dataset of every case from 2005 to 2020 where private data was offered for sale, an ombud was appointed, and a written report was submitted to a bankruptcy court, this Article presents the first comprehensive empirical study of who ombuds are, what they charge, and what they do. This Article also presents evidence of the lengths to which parties in dozens of cases have gone to avoid the appointment of ombuds. 

The Article shows that the consumer privacy ombudsman regime falls far short of its promise of protecting consumers from the misuse of their data by distressed entities. Most ombuds are capable and accomplished privacy experts, and undoubtedly their work sometimes helps consumers. In the RadioShack bankruptcy, for instance, the personal information of more than 117 million individuals was put up for sale. Together with regulators, the ombud crafted a regime that protected much of the most sensitive consumer information that RadioShack was seeking to monetize. But regulators and the media are rarely as active as they were in the RadioShack case, and absent such pressure, ombuds operate within a legal and institutional system that does not equip them to energetically protect consumers' private data. 

The law applies in a narrow subset of cases where consumers' private information is sold, and while the work of the ombuds has some impact in those cases, it is less than most would assume. Ombuds routinely follow a set framework derived from a prominent FTC settlement. Ombuds' analyses typically rely on neither technical knowledge nor legal expertise beyond that which most lawyers could attain with a few hours of research-particularly if, as this Article recommends, prior reports are collected and published so that future ombuds, future debtors in bankruptcy, and the public can better understand what ombuds do. 

Ultimately, the consumer privacy ombudsman regime is best understood as a form of "privacy theater," intended to reassure the public that their data is safe by giving an exaggerated sense of protection. As privacy law scholar Paul Schwartz articulates, "privacy theater . .. seeks to heighten a feeling of privacy protection without actually accomplishing anything substantive in this regard." A legal regime that functions as privacy theater is "largely ritualistic," used to "create a myth of oversight," and used to sustain that myth while obscuring the darker realities. The consumer privacy ombudsman regime mobilizes public trust in experts and courts to "create a myth of oversight" while adding little "substantive" protection. 

While there are good reasons to enhance consumer privacy protections when companies are financially stressed, the current system does not do the job. Instead, appointments are rarely made, and when they are, these appointments provide relatively minimal protection. This Article presents a series of proposals to reform the law and the institutions governing commerce in consumers' private data. It further suggests that ombuds' role could be played by U.S. Trustees' lawyers, or could be shaped into a more traditional form where the proponents of a sale present necessary proofs to the judge to decide on the sale's compliance with the law. 

In addition, the Bankruptcy Code could be changed to make its privacy protections more meaningful. Most obviously, the many gaps in the current law could be fixed so that all sales of private information comply with governing law, and that the burden of demonstrating compliance with the law lies more clearly with the proponents of the sale. In addition, lawmakers could shift ombuds from the relatively neutral role they have generally played and instruct them to serve as protectors of consumers' interests, investigate proposed sales more actively, and advocate more directly for consumers. 

Changes beyond the bankruptcy system should also be considered. Because financially distressed businesses present special risks regardless of whether they are in bankruptcy, and because there is no workable way to identify and target distressed businesses for additional privacy scrutiny, commercial privacy law reforms arguably should apply to all business transactions involving consumers' private information. Distress should be merely one factor for regulators or courts to consider as they weigh the propriety of a transfer of private data or the need for data protection. Some potential reforms involve requiring regulatory preclearance of transactions in data, or requiring advance notification of such transactions so that consumers, regulators, and others can act if necessary to protect consumer privacy. 

Part I summarizes the existing consumer privacy ombudsman regime and the broader context of the law of privacy the regime operates in. This Part discusses the public concern over the protection of data from misuse by distressed entities, which led to the creation of the ombudsman regime, and surveys the significant limitations of that regime. Part II explores how this regime is implemented in practice. It shows that many sales of private data take place without the appointment of an ombud, and that even when ombuds are appointed, they show keen awareness of the limits of their statutory responsibilities. Part III surveys possible reforms to the consumer privacy ombudsman regime. It begins with potential changes to the bankruptcy system, and then suggests more sweeping changes to transactions in private data by businesses outside of bankruptcy as well.

Bradley's 'Privacy for Sale: The Law of Transactions in Consumers’ Private Data' in (2023) 40(1) Yale Journal of Regulation states 

Lawmakers, regulators, consumer advocates, and the business community have focused increasing attention on the policy issues that arise at the intersection of privacy, technology, and commerce. Yet the law governing what businesses can do with consumer data remains unsettled and unclear. The United States has no dedicated and comprehensive privacy law, relying instead on a patchwork of general consumer protection laws and industry-specific regulations like HIPAA. The FTC has created what scholars have called a “common law of privacy” through its enforcement actions and published guidance, but how privacy law applies to business practices often remains uncertain. 

This Article uncovers a large new trove of privacy law, elaborating the jurisprudence of privacy with reports submitted to courts in which hundreds of millions of consumers’ private information has been put up for sale. A unique provision of bankruptcy law requires the appointment of a privacy expert when consumer information is put up for sale, to report on the sale’s legality. These expert reports constitute an unrecognized but substantial body of privacy law. The Article presents and analyzes reports submitted from 2005 to 2020—a hand-collected dataset gathered from 141 court dockets. The reports dramatically increase what is known about how the “common law of privacy” applies in practice to sales of consumer data in a legal forum, and what the future of privacy law may hold. 

The reports generally advocate a pro-transactional view and permit sales to proceed in spite of existing privacy promises so long as the purchasers’ use of consumer data will be roughly consistent with the sellers’. They rely on aspects of the traditional “notice and choice” regime that has guided privacy law, but they also include substantive consideration of the reasonable expectations that consumers may have formed or of the sensitivity of the information to be transferred. Thus, the reports reflect privacy law’s shift beyond strictly consent-based contractarian models and toward more substantive and context-based approaches. 

The reports also speak to the institutional context of regulation of commerce in consumer information. On the one hand, the reports impose significant limits on companies selling private data, which suggests that expert oversight and supervision mechanisms, such as the legal regime that generated these reports, can play an important role in privacy regulation on the ground. But the reports are, on the whole, timid and formulaic, hewing closely to existing precedent and showing little inclination to adapt or develop it even when novel circumstances might justify a change in course. This hesitancy indicates that privacy law’s continuing development requires leadership from federal and state policymakers.

Sabotage

'“Attacking” Big Data: Strategic Competition, the Race for AI, and the International Law of Cyber Sabotage' by Gary Corn and Eric Talbot Jensen comments 

This chapter begins with a discussion of the national security threat that China’s AI development efforts pose, and the importance of big data to those efforts. It then moves to a review of potential cyber-enabled operations, particularly as applied to data, that could impede or thwart China’s AI development. The chapter then proceeds to a review of the international law implications of cyber sabotage, beginning with a discussion of the jus ad bellum and followed by a review of other relevant aspects of the international law of state responsibility such as the rule of prohibited intervention, principles of state sovereignty, and the doctrines of countermeasures and necessity. 

Various cyber means allow for the sabotage of China’s AI development in ways that do not violate the prohibition on the use of force, and may not, depending on how the cyber means are employed, implicate the doctrine of prohibited intervention. Carefully crafted precision tools might be used to poison China’s progress in AI without crossing any thresholds barred by international law.

Tikanga

The NZ Law Commission Study Paper Tikanga and the Law comments

In 2001, Te Aka Matua o te Ture | Law Commission published its Study Paper Māori Custom and Values in New Zealand Law to examine the impact of tikanga on state law and consider ideas for future state law reform projects that might give effect to tikanga. The Study Paper has had an enduring influence on the consideration of tikanga in both legal and policy contexts and remains one of our most frequently cited publications. 

As we note in the Introduction, since 2001 there have been many developments in the ways that tikanga and state law intersect. Tikanga is increasingly being woven into statute and the common law while, at the same time, gaining wider recognition within state law as being an independent source of rights and obligations. Yet tikanga is not well understood outside of Māori communities. The breadth and depth of tikanga is often overlooked and misunderstood. This has potential implications both for the integrity of tikanga and the coherent development of state law. 

In October 2021 the Minister of Justice asked the Commission to review the role of tikanga concepts in state law. We identified two main goals. One was to provide an account of what tikanga is. The second was to address how tikanga and state law might best engage. 

In approaching the first goal, we have been acutely conscious of the immense significance of tikanga to Māori. We sought the assistance of pūkenga (experts) to guide us. Their directive was clear — any account of tikanga must occur from “the inside”, grounded in mātauranga (Māori knowledge). Early in the project, we also identified that mainstream consideration of the legal dimensions of tikanga was sparse. This paper attempts to fill this gap. 

In approaching the second goal, we outline the evolution of state law as it relates to tikanga. This then sets the scene for the final part of our paper, in which we provide guidance on how state actors might engage with tikanga in a way that maintains the integrity of both tikanga and state law.

The Commission comments 

... there have been many developments in the way that tikanga is addressed by state law. Tikanga concepts are prevalent in legislation. Tikanga “has been and will continue to be recognised” in the development of Aotearoa New Zealand’s common law in cases where it is relevant. Tikanga has been judicially described as the first law of Aotearoa New Zealand, a “free-standing legal framework”, and a “third source of law”. The largest cohort of courts, Te Kōti ā Rohe | District Court, has begun a process of incorporating tikanga into the fabric of its operation. 

In consequence, demands have become pressing for legal and policy practitioners to engage more authentically with tikanga. Tikanga is still not well understood outside of the Māori communities where it is practised and lived, raising many questions. What is tikanga? Where do we find it? To whom does tikanga apply? As state law and tikanga engage, how and where are proper boundaries set — and by whom? How will risks and challenges be managed, enabling these systems to interact well? Our work has been done in a context where engagement between tikanga and state law is actively occurring, heightening the urgency. 

Given these pressures, paths forward are needed to guide those approaching tikanga and assure all concerned that state law and tikanga are able to engage with one another with integrity. These two systems are already interacting. In this Study Paper, we have taken the view that there is little utility in the Commission readdressing whether or why state law and tikanga should engage. Instead, we have focused primarily on identifying ways in which they may properly do so, that are respectful of both systems’ parameters. The Study Paper’s purpose is to offer guiding frameworks that will enable the coherence and integrity of both tikanga and state law to be maintained. To legitimately address tikanga, an authentic understanding of it is also needed. The Study Paper aims to build understanding of tikanga that is both grounded in mātauranga (Māori knowledge) and connected with the law. 

As Tā Edward Taihakurei Durie says, tikanga “is the set of values, principles, understandings, practices, norms and mechanisms from which a person or community can determine the correct action in te ao Māori”. Tikanga has long been recognised as having “the character and authority of law” and continues to shape and regulate the lives of Māori as it always has. Some aspects of tikanga are already a part of state law as a result of being incorporated, as described above, by courts through the common law and by Parliament in legislation. 

At the same time, tikanga scholars acknowledge the “immense” ambit of tikanga, encompassing philosophical, ethical and social frameworks, processes and norms. As Dr Carwyn Jones for instance considers, tikanga has aspects of ritual and custom and spiritual and socio-political dimensions that go far beyond the legal domain, as well as being Māori legal knowledge and Māori legal tradition. Distinguished Professor Hirini Moko Mead describes the importance of precedent and procedure, saying that tikanga provides “procedures to be followed in conducting the affairs of a group or an individual” that are “established by precedents through time … validated by usually more than one generation”. He also says that: Tikanga are tools of thought and understanding. They are packages of ideas which help to organise behaviour and provide some predictability in how certain activities are carried out. They provide templates and frameworks to guide our actions and help steer us through some huge gatherings of people and some tense moments in our ceremonial life. They help us to differentiate between right and wrong with built-in ethical rules that must be observed. Sometimes tikanga help us survive. 

When addressing tikanga in this Study Paper, we accordingly have preferred to use the word “tikanga” over other choices such as “Māori law”. The label “law” is unduly narrow, even while tikanga without doubt has legal quality. Reference to “Māori” in the phrase “Māori law” also tends to mask the reality of localised and variable expressions of tikanga among iwi, hapū, marae, whānau and other hapori Māori (“hapori” meaning section of a kinship group, society or community). For similar reasons, we do not use the terms “Māori custom law” or “custom law” — although, as will be explained in Parts Two and Three, custom law is a phrase accurately used to describe one common law category of tikanga recognition. In this Study Paper, we simply call tikanga: tikanga.

Data Buying

'Government Purchases of Private Data' by Matthew Tokson in Wake Forest Law Review (Forthcoming) comments 

 In recent years, numerous government entities from the Department of Homeland Security to local police departments have begun to purchase location and other data from specialized brokers in order to track individuals’ activities over time. Much of this data is constitutionally protected. Yet, while some government actors have concluded that the Fourth Amendment regulates these purchases, most have determined that purchasing data is a valid way of bypassing the Constitution’s restrictions. 

This Article addresses the increasingly prominent issue of government purchases of private data, and examines broader issues of privacy protection in an era of commercial markets in personal information. The Article questions the widespread assumption that the Fourth Amendment can never apply to commercial purchases. Police officers can generally purchase an item available to the public without constitutional restriction. But a closer examination of data markets demonstrates that sensitive cellphone data is not publicly available or exposed. Rather, the vendors who sell such data do so either exclusively to law enforcement agencies or in large, anonymized chunks to other marketing companies. Because sensitive cellphone data remains functionally private, a government purchase of such data violates the Fourth Amendment. 

The Article then challenges the idea that consumers waive their rights in their cellphone data when they use apps or other services. The explanations customers see when an app asks for permission to access their data are often insufficient or misleading, and they typically say nothing about personal data being sold to other parties. Further, penalizing users for disclosing their data to service providers creates harmful incentives and is incompatible with meaningful Fourth Amendment protection in the digital age. 

The Article sits at the intersection of consumer privacy and Fourth Amendment law, as poorly regulated markets in personal data and flawed concepts of consumer consent now threaten to erode fundamental constitutional rights. The Article draws broader lessons about the inadequacy of consumer privacy law in the United States. It examines the potential for private surveillance to become government surveillance, via technical and legal interoperability. And it assesses a variety of possible solutions through which legal actors can prevent commercial markets in private data from undermining Fourth Amendment rights.

Generative AI

'Talkin’ ‘Bout AI Generation: Copyright and the Generative AI Supply Chain' by Katherine Lee, A. Feder Cooper and James Grimmelmann comments 

"Does generative AI infringe copyright?" is an urgent question. It is also a difficult question, for two reasons. First, “generative AI” is not just one product from one company. It is a catch-all name for a massive ecosystem of loosely related technologies, including conversational text chatbots like ChatGPT, image generators like Midjourney and DALL·E, coding assistants like GitHub Copilot, and systems that compose music and create videos. Generative-AI models have different technical architectures and are trained on different kinds and sources of data using different algorithms. Some take months and cost millions of dollars to train; others can be spun up in a weekend. These models are made accessible to users in very different ways. Some are offered through paid online services; others are distributed on an open-source model that lets anyone download and modify them. These systems behave differently and raise different legal issues. 

The second problem is that copyright law is notoriously complicated, and generative-AI systems manage to touch on a great many corners of it. They raise issues of authorship, similarity, direct and indirect liability, fair use, and licensing, among much else. These issues cannot be analyzed in isolation, because there are connections everywhere. Whether the output of a generativeAI system is fair use can depend on how its training datasets were assembled. Whether the creator of a generative-AI system is secondarily liable can depend on the prompts that its users supply. 

In this Article, we aim to bring order to the chaos. To do so, we introduce the "generative-AI supply chain": an interconnected set of stages that transform training data (millions of pictures of cats) into generations (a new, potentially never-seen-before picture of a cat that has never existed). Breaking down generative AI into these constituent stages reveals all of the places at which companies and users make choices that have copyright consequences. It enables us to trace the effects of upstream technical designs on downstream uses, and to assess who in these complicated sociotechnical systems bears responsibility for infringement when it happens. Because we engage so closely with the technology of generative AI, we are able to shed more light on the copyright questions. We do not give definitive answers as to who should and should not be held liable. Instead, we identify the key decisions that courts will need to make as they grapple with these issues, and point out the consequences that would likely flow from different liability regimes.

Wrongheaded

In Woodhead v Commissioner of Police [2023] QDC 143 Cash DCJ states

[6] ... Orally, the appellant addressed brief submissions about whether Mr Weller’s evidence of tailgating should be accepted. The rest of his submissions raised the all-too-common nonsense claims that the defendant was possessed of dual legal identities and that he had not consented to the application of the law.  ...

[8] Material filed by the appellant in the appeal, though substantial in nature, sheds no further light on how he says the trial miscarried. What I have assumed to be his written submissions concerning the appeal contain some breathtakingly wrongheaded assertions. Outside of the usual pseudo-law nonsense, the appellant cherry-picks from irrelevant and inapplicable material. For example, a report from the Australian Law Reform Commission. Another is the extraordinary assertion that:

On the 1st of July 2023, the National Anti-Corruption Act 2022 comes into force. Every Australian is now entitled to ask for and get a jury trial in every matter, civil or criminal, and it is corruption to refuse in either a state or federal Court. This is because in 1995 the “Kable principle” was argued in the High Court and became common law in 1996. The Kable principle is that no state can make a law contradicting section 79 Constitution and no state can make a law discriminating against a subject of the King, and in any criminal matter, the common law standard must apply.

[9] It is a curious submission to make, not least of which because section 79 of the Constitution is a provision concerned with the number of judges of a Court exercising federal jurisdiction. Perhaps it was intended the reference be to section 73 or 71 of the Constitution or section 79 of the Judiciary Act. In any event, any citizen capable of reading who looks either at the Constitution or the decision of the High Court in Kable would immediately understand they provide no support for these outlandish claims. 

[10] Further claims made by the appellant in his speech in Court sought to maintain the discredited fiction that he is somehow not the appellant or not the person who committed the traffic offences because he has dual legal personalities. This was an idea I rejected and still reject (see R v Sweet [2021] QDC 216). There were the further usual references to equity and trusts. It was impossible to comprehend much of the speech. The suggestion that statute law is without a source of authority is, of course, entirely misconceived. In the late 19th century and early 20th century, some of the people of Australia came together to form the Commonwealth. Notably, Aboriginal Australians were excluded from that process. By common consent, the participants in the process created a source of authority: the Constitution of the Commonwealth of Australia. By section 4, State Parliaments are authorised to make law and it is pursuant to this authority that Queensland has the power to make laws for the “peace, welfare and good government” of the state. The laws contravened by the appellant are part of the valid laws of this state (see, generally, Hubner v Erbacher [2004] QDC 345). 

[11] Not for the first time, I am moved to observe that it is sad to see a person such as the appellant, who is seemingly capable of industry and thought, diverting his time and effort in the fruitless pursuit of ideas promoted by charlatans, fraudsters, crackpots and racists. There exists in Queensland a recognised body of statute and common law. As a society, we have developed techniques and processes for interrogating and developing that law. Those who wish to stand outside that law and ignore long recognised processes must realise they bear the onus of rationally explaining why these almost universally accepted understandings are wrong. Until such time, the claims of such people will continue to be summarily dismissed.

Disinformation

'The efficacy of Facebook’s vaccine misinformation policies and architecture during the COVID-19 pandemic' by David A David A Broniatowski, Joseph R Simons, Jiayan Gu, Amedia M Jamison and Lorien C Abroms in (2023) 9(37) Science Advances comments 

 Online misinformation promotes distrust in science, undermines public health, and may drive civil unrest. During the coronavirus disease 2019 pandemic, Facebook—the world’s largest social media company—began to remove vaccine misinformation as a matter of policy. We evaluated the efficacy of these policies using a comparative interrupted time-series design. We found that Facebook removed some antivaccine content, but we did not observe decreases in overall engagement with antivaccine content. Provaccine content was also removed, and antivaccine content became more misinformative, more politically polarized, and more likely to be seen in users’ newsfeeds. We explain these findings as a consequence of Facebook’s system architecture, which provides substantial flexibility to motivated users who wish to disseminate misinformation through multiple channels. Facebook’s architecture may therefore afford antivaccine content producers several means to circumvent the intent of misinformation removal policies. 

Online misinformation undermines trust in scientific evidence (1) and medical recommendations (2). It has been linked to harmful offline behaviors including stalled public health efforts (3), civil unrest (4), and mass violence (5). The coronavirus disease 2019 (COVID-19) pandemic has spurred widespread concern that misinformation spread on social media may have lowered vaccine uptake rates (6, 7). Therefore, policymakers and public officials have put substantial pressure on social media platforms to curtail misinformation spread (8, 9). 

Years of “soft” remedies—such as warning labels by Twitter (10), YouTube (11), and Facebook (12) and attempts by these platforms to downrank objectionable content in search—have demonstrated some success (13); however, misinformation continues to spread widely online, leading many to question the efficacy of these interventions (14). Some have suggested that combining these soft remedies with “hard” remedies (15)—removing content and objectionable accounts (16–18)—could largely curtail misinformation spread (19). However, evidence for the short-term efficacy of hard remedies is mixed (20–24), and the long-term efficacy of these strategies has not been systematically examined. Hard remedies have also spurred accusations of censorship and threats of legal action (25, 26). There is therefore a critical need to understand whether this combination of remedies is effective—i.e., whether it reduces users’ exposure to misinformation—and if not, why not. 

Any evaluation of the efficacy of these remedies must be grounded in a scientific understanding of why misinformation spreads online. Prior work indicates that misinformation may spread widely on social media if it is framed in a manner that is more compelling than true information (27, 28). Users appear to prefer sharing true information when cued to think about accuracy (13); however, the social media environment may interfere with peoples’ ability to distinguish truth from falsehood (29, 30). Other studies have suggested that social media platforms’ algorithms facilitate the creation of “echo chambers” (31, 32), which increase exposure to content from like-minded individuals. Accordingly, prior interventions have focused on altering the social media environment to either reduce users’ exposure to misinformation or inform them when content is false. However, recent evidence suggests that people use online algorithms to actively seek out and engage with misinformation (33). Therefore, on the basis of prior theory (34, 35), we examine how a social media platform’s “system architecture”—its designed structure that shapes how information flows through the platform (34)—enables antivaccine content producers and users to flexibly (36) establish new paths to interdicted content, facilitating resistance to content removal efforts. 

We analyzed Facebook because it is the world’s largest social media platform. In December 2020, when COVID-19 vaccines first became available, Facebook had 2.80 billion monthly active users (37). As of April 2023, this number had grown to 2.99 billion monthly active users (25). We therefore conducted an evaluation of Facebook’s attempts to remove antivaccine misinformation from its public content throughout the COVID-19 pandemic. 

Of primary interest were the following three research questions: (i) Were Facebook’s policies associated with a substantial decrease in public antivaccine content and engagement with remaining antivaccine content? (ii) Did misinformation decrease when these policies were implemented? (iii) How might Facebook’s system architecture have enabled or undermined these policies?

...Our findings suggest that Facebook’s policies may have reduced the number of posts in antivaccine venues but did not induce a sustained reduction in engagement with antivaccine content. Misinformation proportions both on and off the platform appear to have increased. Furthermore, it appears that antivaccine page administrators especially focused on promoting content that outpaced updates to Facebook’s moderation policies: The largest increases appear to have been associated with topics falsely attributing severe vaccine adverse events and deaths to COVID-19 vaccines.