24 November 2024

Open

'The future(s) of open science' by Philip Mirowski in (2018) 48(2) Social Studies of Science comments 

Almost everyone is enthusiastic that ‘open science’ is the wave of the future. Yet when one looks seriously at the flaws in modern science that the movement proposes to remedy, the prospect for improvement in at least four areas are unimpressive. This suggests that the agenda is effectively to re-engineer science along the lines of platform capitalism, under the misleading banner of opening up science to the masses. We live in an era of trepidation over the future of science. It is all the more noteworthy, then, that science policy circles have embraced an open infatuation with ‘open science’. The whole thing kicked off in the later 2000s, with rumors concerning something called ‘Science 2.0’. In January 2012, the New York Times (Lin, 2012) then had the good sense to promote the rebranding of this imaginary as ‘open science’. The British Royal Society intervened close on its heels in 2012, with a public relations document entitled Science as an Open Enterprise (Royal Society, 2012). Subsequently, this was rapidly followed by popularizing books (Nielsen, 2012; Weinberger, 2012) and a plethora of government white papers, policy documents and articles (e.g. OECD, 2015; CNRS, 2016; Strasser and Edwards, 2015; Vuorikari and Punie, 2015; Weinberger, 2012). All sorts of institutes and think tanks (the Ronin Institute, Center for Open Science, openscienceASAP, UK Open Data Institute, PCORI, Laura and John Arnold Foundation) sprouted across the landscape, dedicated to propounding the virtues of open science for all and sundry. The NIH even teamed up with the Wellcome Trust and the Howard Hughes Medical Institute to offer a much ballyhooed ‘Open Science Prize’ consisting of six awards to various teams of the not-very-princely sum of $80K with which to launch (?) their prototypes. The concept was trundled out to the public in the format of a 2017 PBS television Series ‘The Crowd and the Cloud’, funded by the NSF. Congressional mandates stipulating ‘openness’ were hidden in the US ‘Crowdsourcing and Citizen Science Act’, itself folded into the 2016 ‘American Competitiveness and Innovation Act’. 

Back in Europe in 2013, the G8 Science Ministers formally endorsed a policy of encouraging open science. In May 2016 the EU Competitiveness Council issued a mission statement that all scientific articles should be ‘freely accessible’ by 2020 (Enserink, 2016). ‘The time for talking about Open Access is now past. With these agreements, we are going to achieve it in practice’, the Dutch state secretary for education, culture, and science, Sander Dekker, added in a statement. Lord knows, the last thing an EU bureaucrat has patience with is talking about something not at all well understood. This, in turn, led to a programmatic ‘Vision for Europe’ in 2016 of ‘Open Innovation, Open Science’. The taken-for-granted premise that modern science is in crying need of top-to-bottom restructuring and reform turns out to be one of the more telling aspects of this unseemly scrum, a melee to be in the vanguard of prying science ‘open’. But the language is deceptive: In what sense was science actually ever ‘closed’, and who precisely is so intent upon cracking it open now? Where did all the funding come from to turn this vague and ill-specified opinion into a movement? 

To even pose these questions in a sober and deliberate manner, while making direct reference to the actual history of science, constitutes a challenge to the prophets of openness, because it conflicts with their widespread tendency to treat the last three or more centuries of science as operating in essentially the same monolithic modality. The so-called ‘scientific method’, once it appeared, persisted relatively unchanged, or so goes the undergraduate version of Western Civ. To evade the admission that scientific research and dissemination might actually have been structured differently across diverse epochs and geographical eras, the prophets of openness instead rapidly pivot to a completely unsupported theory of technological determinism to motivate their quest. Change is inevitable, they preach, due to some obscure imperatives concerning the computer and the internet and social media. Once scientists acquiesce to the implacable imperatives of the information revolution, it is said, they will discover that science itself should necessarily become more ‘open’, and the whole society will naturally benefit. 

The layers of confusion surrounding open science rival a millefeuille, and can be just as sticky. The quickest way to cut through the confection is to acknowledge that science has been constituted by a sequence of historical regimes of epistemic and logistical organization, long before the current craze for ‘openness’; this proposition could be perhaps patterned after the arguments made in what has been called the literature on ‘historical epistemology’ (e.g. Daston, 1994; Hacking, 1992). Much of this literature tends to make its case in the format of what used to be called ‘stage theories’: descriptions of historical sequences of relatively internally coherent modes, hegemonies or regimes, structured according to certain key self-images and practices, and punctuated by periods of instability and transition. Indeed, I shall argue that the open science movement is an artifact of the current neoliberal regime of science, one that reconfigures both the institutions and the nature of knowledge so as to better conform to market imperatives. 

But before that, it is necessary to take note of the slippery connotations and motives behind the open science movement. For some, it denotes mere open access to existing scientific publications; for others, it portends a different format for future scientific publication; for yet others, it signifies the open provision of scientific data; for others, it is primarily about something like open peer review; and for still others, the clamor for openness purports to welcome the participation of non-scientists into the research process, under the rubric of citizen science. Of course, these are individually wildly disparate phenomena; but it is noteworthy that many of the proponents and cheerleaders glide rather effortlessly between these diverse conceptions, and that in itself provides a clue to the deep structure of the emergent world of open science. Each ‘reform’ might accidentally have been deemed the imperative of the ‘same’ technological development or, conversely, they might each exemplify a more profound shift in epistemology. Thus, rather than track each of the above sub-components individually, I will approach the problem of understanding open science from the broader perspective of asking: What sort of thing is it that open science proposes to fix about older science? 

Mody (2011) writes that if an ‘epochal break has any features worth studying, they should be visible, in some way, down at the microlevel of practice’ (p. 64). I agree with this precept. The way to make the case for a structural break in the nature of modern science is to link some broad abstract cultural ideas about knowledge to pronounced transformations of scientific practice at the microlevel. The primary manifestations of the new regime are the marriage of an ethos of what has been called ‘radically collaborative science’ with the emergent structures of ‘platform capitalism’, all blessed under the neoliberal catechism of the market as super information processor. The ultimate objective of this paper is to describe how this marriage works; but it turns out to be more informative to begin by surveying the infirmities of recent science that the open science advocates claim they can fix.

AI, Trade and the WTO

The WTO 'Trading with intelligence: How AI shapes and is shaped by international trade' report comments

The widespread and transformative impact that artificial intelligence (AI) is currently having on society is being felt in all areas, from work, production and trade to health, arts and leisure activities. New applications of AI are expected to create unprecedented new economic and societal opportunities and benefits. However, significant ethical and societal risks are also associated with the development and application of AI. These risks have implications for all these areas too, including trade. AI is a global issue, and as governments increasingly move to regulate AI, global cooperation is more important than ever. 

Against this backdrop, the present report examines the intersection of AI and international trade. It begins with a discussion of why AI is a trade issue, before delving into the ways in which AI may shape the future of international trade. It discusses key trade-related policy considerations raised by this technology and provides an overview of government initiatives taken both to promote and to regulate AI. The report also highlights the looming risk of regulatory fragmentation and its impact, in particular on trade opportunities for micro, small and medium- sized businesses. Finally, the report discusses the critical role of the WTO in facilitating AI-related trade, ensuring trustworthy AI and addressing emerging trade tensions. 

Why is AI a trade issue? 

AI is distinct from other digital technologies in several key ways, and it has the potential to affect international trade significantly. It is a general-purpose technology, capable of adapting to a wide range of domains and tasks with unprecedented flexibility and efficiency. It relies on large datasets to learn and improve its performance and accuracy. AI's functions and efficiency can evolve rapidly, leading to dynamic shifts in its capabilities and autonomy. Finally, its inherent complexity and opacity, as well as its potential failures and biases, raise significant concerns related to matters such as how to understand the reasons for and basis of AI decisions and recommendations, or regarding ethics and broader societal implications. AI can be leveraged to overcome trade costs associated with trade logistics, supply chain management and regulatory compliance. By enhancing trade logistics, overcoming language barriers, and minimizing search and match costs, AI can make trade more efficient. It can help to automate and streamline customs clearance processes and border controls, navigate complex trade regulations and compliance requirements, and predict risks. AI-based tools can be used in trade finance, and can significantly enhance supply chain visibility by providing real-time data analytics, predictive insights and automated decision-making processes. All of this could lower trade costs and, as a result, level the playing field for developing economies and small businesses, helping them to overcome trade barriers, enter global markets and participate in international trade. 

AI can transform patterns of trade in services, particularly digitally delivered services. It can enhance productivity, especially in services sectors that rely on manual processes, by enabling low-skilled workers to leverage best practices of more high-skilled workers more effectively. For example, generative AI can amplify the performance of business consultants by up to 40 per cent compared to those not using it. Greater productivity gain is also observed for lower-skilled workers (Dell’Acqua et al., 2023). Research also shows that access to generative AI increases the productivity of call centre workers by an average of 14 per cent, and by 34 per cent specifically for novice and low-skilled workers (Brynjolfsson et al., 2023). AI can also foster the development of innovative services and increase demand for them. However, while AI can enhance trade in digitally delivered services significantly, it has contributed to reducing the demand for certain traditional services. AI-enabled automation can also reduce the necessity to outsource certain services. 

AI can increase demand and trade in technology-related products. Because AI systems often rely on real-time data streams and seamless connectivity, the adoption of AI is spurring demand for complementary goods related to information and communications technology (ICT) infrastructure and information technology (IT) equipment. These include computer and telecommunications services, specialized development tools and software libraries. For example, the global market for AI chips was valued at US$ 61.5 billion in 2023 and it has been projected that it could reach US$ 621 billion by 2032 (S&S Insider, 2024). As many of these goods and services are often supplied by a small number of economies, international trade serves as a major channel to foster AI development worldwide. Further upstream in the value chain, trade in the extraction and processing of critical metals and minerals, as well as trade in energy, are also likely to gain in importance. In addition, AI has substantially heightened the demand for data, fundamentally reshaping the landscape of data usage and trade. 

By affecting productivity, and through shifts in production dynamics, AI may reshape economies' comparative advantages. AI is expected to enhance productivity across all economic sectors in both developed and developing economies, and to change the composition of inputs required for production, placing greater emphasis on capital investment, rather than on labour inputs. This shift in production dynamics could reshape trade patterns. Conversely, new sources of comparative advantage may emerge from factors like educated labour, digital connectivity and favourable regulations. Because AI is energy-intensive, economies with abundant renewable energy may also gain comparative advantages. However, although AI can potentially benefit all economies, the development and control of AI technology are likely to remain concentrated in large economies and companies with advanced AI capabilities, resulting in industrial concentration. The adoption of AI can drive productivity increases across various sectors and reduce trade costs, leading to global gains in trade and GDP. Simulations using the WTO global trade model show that, under an optimistic scenario of universal AI adoption and high productivity growth up until 2040, global real trade growth could increase by almost 14 percentage points. In contrast, a cautious scenario, with uneven AI adoption and low productivity growth, projects trade growth of just under 7 percentage points. The simulation further shows that, while high-income economies are expected to see the largest productivity gains, lower-income economies have better potential to reduce trade costs. 

The global trade and GDP impact of AI varies significantly across economies and sectors, depending on choices made concerning innovation and policies. While trade growth in high-income economies remains relatively stable across projected scenarios, low-income economies could experience much higher trade growth under the scenarios of universal AI adoption and high productivity growth (18.1 percentage points) compared to those of uneven AI adoption and low productivity growth (6.5 percentage points). The simulation results suggest that if developing economies improve their AI readiness by strengthening digital infrastructure, enhancing skills and boosting innovation and regulatory capacity, they will be in a better position to adopt AI effectively. 

These simulations show that digitally delivered services1 are expected to experience the highest trade growth. In an optimistic scenario of universal AI adoption, digitally delivered services are projected to see cumulative growth of nearly 18 percentage points relative to the baseline scenario, the largest increase across all sectors. The expected impact of AI on real trade growth also differs within sectors. Potentially digitally delivered services such as education, human healthcare, and recreational and financial services, as well as manufacturing sectors such as processed food, are projected to experience significant trade growth, largely driven by trade cost reductions. Meanwhile, sectors related to natural resource extraction and manufacturing sectors such as textiles are expected to see limited growth. 

The policies of AI and trade 

The discussion on how AI might reshape international trade raises important policy questions. The risk of a growing divide resulting from applications of AI is significant, as are data governance challenges and the need to ensure that AI is trustworthy and to clarify how it relates to intellectual property (IP) rights. The implementation of AI at the domestic, regional and international levels entails both benefits and risks, and a lack of coordination could cause increasing regulatory fragmentation with regard to AI. Addressing the risk of a growing AI divide is essential to leverage the opportunities offered by this technology. Currently, the capacity to develop AI technology is concentrated in a few large economies, and this is creating a significant divide between economies that are leading research and development (R&D) in AI – in particular China and the United States – and the rest of the world. This imbalance could be further exacerbated by the use of government subsidies to develop AI. The risk of industry concentration within a few large firms could also intensify the divide between firms. These features, combined with the opacity of AI algorithms and the possibility of tacit collusion among competitor firms to maintain higher prices, present challenges for competition authorities. 

The rise of AI is raising important data governance issues that will need to be addressed to prevent further digital trade barriers. Cross-border data flows are essential to AI, as vast amounts of data are needed to train AI models, as well as minimize possible biases. Thus, restrictions on data flows can slow AI innovation and development, increase costs for firms, and negatively impact trade in AI-enabled products. A recent study (OECD and WTO, 2024) found that if all economies fully restricted their data flows, this could result in a 5 per cent reduction in global GDP and a 10 per cent decrease in exports. However, the large datasets required by AI models raise significant privacy concerns. Therefore, a reasonable trade-off between accessing large amounts of data to train AI models and protecting individual privacy must be found. 

Ensuring that AI is trustworthy without hindering trade can be challenging. “AI trustworthiness” means that it meets expectations in terms of reliability, security, privacy, safety, accountability and quality in a verifiable way. However, given the behaviour and opaque nature of AI systems, as well as the potential dual-use of some AI products (i.e., for both civilian and military applications), striking a balance between ensuring that AI is trustworthy and enabling trade to flow as smoothly as possible may prove especially challenging. The evolutionary nature of AI makes regulation a perennial moving target. “Traditional" regulations and standards for goods, which normally focus on tangible, visible and static product requirements, may not be fully capable of addressing all of the different types of potential risks, including the ethical and societal questions that may result from the integration of AI into goods and services. Regulating to address questions of public morals, human dignity and other fundamental rights, such as discrimination or fairness, is not only challenging, but is also prone to causing regulatory fragmentation because the meaning and relative importance of such values may vary across societies. 

AI also poses new conceptual challenges for the traditional, “human-centric” approach to IP rights. Issues that deserve particular attention include the protection of AI algorithms and of copyrighted material for training AI, and the protection and ownership of AI generated outputs. These questions may call for a re-evaluation of existing IP legal frameworks. 

The immense potential of AI has prompted governments around the globe to take action to promote its development and use while mitigating its potential risks. At the domestic level, more and more jurisdictions are putting in place AI strategies and policies to enhance their AI capabilities. The number of economies having implemented AI strategies increased from three in 2017 to 75 in 2023. According to Stanford University's 2024 "AI Index", 25 AI-related regulatory measures were adopted in the United States in 2023, compared to just one in 2016, while the European Union has passed almost 130 AI-related regulatory measures since 2017. However, most domestic AI policy initiatives are being implemented by developed economies, which could further deepen the existing AI divide between developed and developing economies: while around 30 per cent of developing economies have put AI policy measures in place, only one least-developed country (LDC) – Uganda – has done so according to data from the Organisation for Economic Co-operation and Development (OECD) AI Policy Observatory. Also high on governments’ policy agendas are domestic initiatives to promote access to data through open data and data-sharing initiatives, with a view to fostering domestic innovation and competition, protecting privacy and controlling the flow of data across borders. What is emerging is a landscape of fragmented measures and heterogeneous domestic initiatives, which may lead to regulatory fragmentation. 

This fragmentation extends beyond AI-specific regulations to include sector-specific legislation, such as IP and data regulations, which also impact AI. In addition, the design of some border measures imposed on the hardware components and raw materials crucial to AI systems can affect competitors in other economies, leading to trade- distorting effects and further exacerbating fragmentation. The economic costs of regulatory fragmentation, in particular for small businesses, highlight the importance of mitigating regulatory heterogeneity; according to OECD and WTO (2024), the economic costs of the fragmentation of data flow regimes along geo-economic blocks amount to a loss of more than 1 per cent of real GDP. The increasing number of bilateral and regional cooperation initiatives on AI governance, many focusing on different priorities, add to the risk of creating a multitude of fragmented approaches. 

For example, while some bilateral cooperation initiatives focus primarily on aligning AI-related terminology and taxonomy, and on monitoring and measuring AI risks, others prioritize collaboration to promote alignment in general terms or focus primarily on AI safety and governance. Likewise, some regional initiatives prioritize human rights and ethics, while others focus on economic development and growth. 

Regional trade agreements (RTAs) and digital economy agreements are important vehicles to promote and regulate AI. AI-specific provisions have started to be incorporated into such agreements, but they mainly take the form of “soft” – i.e., non-binding – provisions focusing on the importance of collaboration to promote trusted, safe and responsible use of AI. Several AI-specific provisions explicitly refer to trade. Digital trade provisions included in RTAs, such as provisions on data flows, data localization, protection of personal information, access to government data, source code,2 competition in digital markets, and customs duties on electronic transmissions, are also important for AI development and use. The number of RTAs with digital trade provisions has been growing steadily since the early 2000s, and by the end of 2022, 116 RTAs – representing 33 per cent of all existing RTAs – had incorporated provisions related to digital trade (López-González et al., 2023). However, the depth of digital trade provisions included in RTAs varies significantly, reflecting diverging approaches. Few developing economies and LDCs have negotiated digital trade provisions. Disciplines on trade in services in RTAs are also an important channel through which governments' trade policies and trade obligations can affect the policy environment for AI, but the level of commitments undertaken differs significantly across economies. 

The last few years have witnessed a wave of international initiatives related to AI. While there are elements of complementarity among such initiatives and alignment on core principles, different initiatives prioritize different aspects of AI governance. A number of initiatives also contain various common elements that have important trade and WTO angles, such as the recognition of the role of regulations and standards, the need to avoid regulatory fragmentation, the importance of IP rights, the importance of privacy, personal data protection and data governance, and the importance of international cooperation, coordination and dialogue. Several of these initiatives also address the environmental impacts of AI. 

However, there is still no global alignment on AI terminology. Differing priorities, the overlap between initiatives, and lack of global agreement on key terminology could pose challenges at the implementation stage, limiting efforts to prevent fragmentation and to put in place a coherent global AI governance framework. Nevertheless, beyond initiatives to govern AI, an increasing number of international organizations, such as the International Telecommunication Union (ITU), the United Nations Educational, Scientific and Cultural Organization (UNESCO), the United Nations Industrial Development Organization (UNIDO) and the World Bank, are developing courses on AI and integrating AI in their technical assistance activities, some of which have a trade component. The WTO, as the only rules-based global body dealing with trade policy, can contribute to promoting the benefits of AI and limiting its potential risks. It can play an important role in limiting regulatory fragmentation, promoting the development of trustworthy AI and access to it, and facilitating trade in AI-related goods and services, thereby enabling the growth of AI and promoting innovation through IP. 

What role for the WTO? 

WTO rules and processes promote global convergence. The WTO is a forum that promotes transparency, non-discrimination, discussion, the exchange of good practices, regulatory harmonization, non-mandatory policy guidance, and global alignment through the negotiation of new binding trade rules on trade. Transparency provisions included in WTO agreements allow WTO members, as well as economic operators and consumers, to be kept abreast of latest regulatory developments. One example is the enhanced transparency provisions in the Technical Barriers to Trade (TBT) Agreement. By requiring early notification of regulatory measures and allowing opportunities to provide comments on these measures at a draft stage, the TBT Agreement can help to prevent obstacles to trade, as well as promote and accelerate global convergence. WTO members are increasingly notifying a wide range of regulations on digital technologies to the TBT Committee. For instance, more than 160 notifications have been made on regulations addressing cybersecurity and the Internet of Things (IoT)/robotics, both of which are relevant for AI. More recently, the TBT Committee has started receiving notifications of AI-specific regulations. Another example is the WTO Trade Policy Review Mechanism, which contributes to transparency in members’ trade policies. Finally, in terms of possible new substantive rules, various issues negotiated under the Joint Statement Initiative on E-commerce, which currently brings together 91 WTO members, may matter for AI. 

The WTO also provides a global forum for constructive dialogue, the exchange of good practices, and cooperation. This enables discussion among members of how best to design nuanced, flexible and adaptable regulatory solutions to address the goods, services and IP-related aspects of AI in a coordinated manner. In some areas, the WTO also promotes regulatory harmonization and coherence by encouraging the use of international standards, mutual recognition and equivalence, and through various "soft law" instruments, such as voluntary committee guidelines. 

The WTO is the cornerstone of global efforts to facilitate trade in services and goods that enable or are enabled by AI. Various aspects of the WTO rulebook can contribute to promoting the development of and access to AI. For example, the General Agreement on Trade in Services (GATS) plays an important role in shaping a policy environment that facilitates the development and uptake of AI. A majority of WTO members (out of 141 schedules of commitments, 84, or 60 per cent, contain commitments on computer services) have made specific commitments on market access and national treatment related to ICT services, which play a fundamental role in enabling and promoting AI. However, commitments in other sectors remain limited, and barriers to services trade remain high in overall terms. When it comes to goods, the Information Technology Agreement (ITA) aims to increase worldwide access to high-technology goods essential to AI by eliminating tariffs on the ICT products it covers. Meanwhile, the TBT Agreement can help to ensure that, when governments adopt AI standards and regulations, these are, to the extent possible, not trade-restrictive, and are optimal for attaining policy objectives. The Trade-Related Aspects of Intellectual Property Rights (TRIPS) Agreement aims to foster a balanced IP system that incentivizes innovation through the enforcement and protection of IP rights, while promoting dissemination of and access to technology, to the mutual benefit of both producers and users of technological knowledge. Various WTO agreements also include provisions to promote the transfer of technology, and this can play an important role in the development of AI. Finally, the WTO Agreement on Government Procurement (GPA) 2012 promotes access to internationally available new AI technologies. Various principles, provisions and guidelines in the WTO rulebook can support trade in AI systems and AI-enabled products by minimizing international negative spillovers. Examples include the non-discrimination principle and the Agreement on Trade-Related Investment Measures (TRIMS), which recognizes that certain investment measures can restrict and distort trade and states that members may not apply investment measures that discriminate against foreign products or lead to quantitative restrictions. When it comes to technical regulations, standards and certification procedures, the TBT Agreement provides that regulatory intervention shall not be discriminatory nor any more trade-restrictive than necessary to achieve the intended policy objectives, and that it should, when justified, be subject to periodic reviews. And the Agreement on Subsidies and Countervailing Measures (SCM) can play a crucial role in navigating the dual aspects of AI development, by promoting technological innovation while preventing negative spillovers in international trade from government financial support. 

The WTO can help to prevent and settle trade tensions and frictions. The practice of raising "specific trade concerns" (STCs) allows WTO committees to serve as a venue for defusing potential trade tensions with regulatory measures in a cooperative, pragmatic and non-litigious way. In the TBT Committee, for instance, members have already been using this practice to discuss and address concerns with regulations involving a wide range of digital technologies and issues, including IoT, autonomous vehicles, 5G in robotics, industrial automation, cybersecurity, and more recently AI. The WTO also serves as a global forum to settle trade-related disputes. While there has been no dispute on AI so far, the WTO Dispute Settlement System has dealt with resolving disputes related to various aspects of the digital economy. 

The WTO promotes inclusiveness through special and differential treatment and technical assistance for developing economies. WTO agreements recognize the constraints faced by developing economies and, for this reason, include various special and differential (S&D) treatment provisions to help them to implement WTO rules and participate more effectively in international trade. Technical assistance and capacity-building are key pillars of the WTO’s work and play a fundamental role in furthering understanding of the WTO rules and agreements, as well as of other topics relevant to trade. Multi-stakeholder programmes, such as Aid for Trade and the Enhanced Integrated Framework, could, however, be leveraged further to help developing economies seize the benefits of AI for trade. 

As a forum for negotiation, discussion and rule-making, the WTO provides a multilateral framework that can help address the trade-related aspects of AI governance. Nevertheless, AI may have implications for international trade rules. Although it is a new technology, AI is developing rapidly, and is certainly already advanced enough to be a subject of discussions at the WTO. Its cross-cutting nature requires a cross-cutting policymaking approach to promote policy coherence. 

While AI governance extends beyond trade, trade remains a crucial element within AI governance. The WTO can contribute significantly to developing a robust AI governance framework. This report is a first attempt to explore some key implications of AI for trade and trade rules. As AI continues to evolve, governments should continue to discuss the intersection of AI and trade and its possible implications for the WTO rulebook.

23 November 2024

Vilification

The New South Wales Law Reform Commission report on Serious racial and religious vilification addresses s 93Z of the Crimes Act 1900 (NSW). The Commission states 

1.3 On 14 February 2024, the NSW Attorney General asked us to expeditiously review and report on the effectiveness of s 93Z of the Crimes Act 1900 (NSW) (Crimes Act) in addressing serious racial and religious vilification in NSW. Throughout this review, we heard about the significant impact that hate-based conduct has on individuals, groups and our wider community, historically and at the present time. We acknowledge public interest in the operation of s 93Z has increased following the events in Israel and Gaza on and after 7 October 2023. However, after consulting widely, we have concluded that s 93Z should not be amended in response to the specific issues raised by the terms of reference. 

Based on the concerns raised with us, we recommend the NSW Government consider:

• commissioning a separate review of the effectiveness s 21A(2)(h) of the Crimes (Sentencing Procedure) Act 1999 (NSW) (Sentencing Procedure Act), which enables motivations of hatred and prejudice to be considered as aggravating factors on sentence, and 

• measures to improve the collection of data on hate crimes when offences other than s 93Z are charged for hate-related incidents. ...

It notes 

 concerns expressed by some community groups about the low number of prosecutions under s 93Z. In particular, some were dissatisfied at the criminal justice response to the experiences of individuals and groups when allegations of vilification and hate-based conduct have been reported. 

Data from the Bureau of Crime Statistics and Research (BOCSAR) shows that, as at July 2024, 7 people had charges under s 93Z finalised. Of these people: • 2 were found guilty of an offence under s 93Z, and • 5 had the charge(s) under s 93Z withdrawn. Both convictions were appealed before the District Court. Out of the 2 convictions: • 1 was quashed on 6 February 2024 after a successful appeal, and • 1 was upheld on appeal by the District Court on 7 June 2024 (that is, after this review commenced). There were 2 further convictions in 2020. However, they were annulled because the NSW Police Force commenced prosecutions without the consent of the Director of Public Prosecutions (DPP), which was required at the time. 

The requirement to obtain DPP consent before commencing a prosecution was removed from s 93Z in January 2024. This was intended to streamline the prosecution process. ... 

This report does not make recommendations about the ADA As we explain in chapter 2, s 93Z operates alongside the civil anti-vilification protections in the ADA. These cover other forms of vilification, that is, public acts that incite hatred, serious contempt or severe revulsion on the basis of: • race • transgender status • HIV/AIDS status • homosexuality, or • religious belief, affiliation or activity (or lack of such belief, affiliation or activity). ... 

1.29 A range of organisations argued that the civil and the criminal frameworks should be reviewed holistically, as part of our ongoing review of the ADA. Additionally, some suggested it was premature to consider reforms to s 93Z while the ADA was under review. For instance, the NSW Bar Association suggested that concerns about the operation of the criminal law may be addressed if the civil vilification regime was improved. 

1.30 We acknowledge the relationship between the vilification protections, and there are good arguments for considering them together in a holistic review. However, we are bound by our terms of reference which focus, in this instance, on the criminal law response to serious racial and religious vilification in s 93Z. 

1.31 Accordingly, this report does not consider several issues raised with us in submissions and consultations. These include whether: • the list of protected attributes in either s 93Z or the ADA should be expanded, including to recognise intersectional experiences of vilification21 • the terminology used to describe the attributes currently protected by s 93Z or the ADA should change22 • s 93Z and the ADA should be aligned in terms of the attributes protected and/or the way common elements are defined23 • the civil protection against religious vilification, introduced into the ADA in 2023, could be improved,24 and • the civil complaints mechanisms, and the framework for civil remedies, should be reformed. ... 

1.37 Summary of our key reasons 

Throughout this review, we heard about the significant and increasing effect that vilification has on our community. We outline these concerns in chapter 3. While we acknowledge these concerns, we do not recommend reform to s 93Z to address the issues raised by our terms of reference. 

Section 93Z needs to be understood as part of the broader legal system in which it operates. This includes other, general criminal offences and the civil vilification framework (outlined in chapter 2). Section 93Z has a protective purpose, in that it aims to protect identified groups from threats of or incitements to violence. It also has a symbolic purpose, signifying that the community does not condone this conduct.  There was widespread support for criminalising this conduct in a specific vilification offence, as s 93Z currently does. 

One of the factors that led to this review was the low number of prosecutions under s 93Z. However, the low number does not, of itself, make the case for reform. The fact that an appeal against a conviction under s 93Z has been dismissed demonstrates that the section is operable and has a role to play in appropriate circumstances. 

As we discuss in chapter 3, the low numbers of prosecutions may be due to a range of factors other than the elements of the offence. The factor most often raised with us is that police may prefer to charge general offences. In many cases, these offences are more familiar to police, are easier to prove and have higher maximum penalties. 

There is no clear community consensus, even among religious and multicultural groups, that s 93Z requires reform in response to the issues raised by our terms of reference. Indeed, many cautioned against such reforms.   

Expanded criminalisation comes with risks and is not always the best tool to achieve social policy aims. In particular, we are aware that extending the criminal law can have unintended consequences, especially for those groups already overrepresented in the criminal justice system. Specific concerns were expressed about the potential impact on Aboriginal people. 

There is also a need to be cautious of any reforms that might over-complicate the law and cause further uncertainty or litigation. 

In the following chapters, we detail the responses to the various options suggested in our Options Paper. While views differed in relation to various options, the weight of opinion was that none of these options should be pursued. 

The exception was the potential removal of recklessness as a mental element. Opinions in submissions divided more evenly on this issue. However, this change would not strengthen s 93Z or address the concerns that prompted our review. Finally, as we further explain in chapter 3, the law is only one part of a wider range of measures necessary to promote social cohesion in NSW. Non-legal measures may be more effective in achieving this aim. 

However, we agree that more could be done to improve the visibility and to track the effectiveness of the wider criminal justice response to hate crime. We recommend that the NSW Government consider commissioning a review of the effectiveness of s 21A(2)(h) of the Sentencing Procedure Act. 

We also recommend that the NSW Government consider measures to improve data collection in relation to the prosecution of general offences in response to hate crime.

22 November 2024

WHO Genome Guidelines

This month's WHO 'Guidance for human genome data collection, access, use and sharing' notes  

Collecting, accessing, using and sharing genomic data from humans is fraught with ethical, legal, social and cultural issues. Nevertheless, the potential benefits of genomics can only be realized if such data is collected, accessed, used and shared. Consequently, the Science Council report set the promotion of ethical, legal, equitable, and responsible sharing of human genome data as a specific goal. 

This complementary document seeks to achieve that goal by outlining globally applicable principles for collecting, accessing, using and sharing human genome data. These principles serve as a compass to guide policy-makers, researchers, clinicians, and all those involved in human genome data, how they should collect, access, use and share human genome data in ways that advances genomics for individual and population health, protects individual and collective rights and interests, and fosters public trust. Equally, they provide individuals, their families and communities from whom human genome data is accessed with an understanding of the principles upon which their data will be collected, accessed, used and shared. The principles described recognize the importance and value of human genome data. Its use is critically important if we are to realize the promise of genomics for all, but this must be stewarded in such a way that identifies and mitigates the ethical, legal, social and cultural issues that are likely to arise. These principles offer normative guidance and serve as a call to action, urging all of those involved in the use of human genome data to uphold and implement them. 

1. INTRODUCTION 

The science and practice of genomics hold great promise and potential to improve individual and population health for present and future generations. To realize this potential, there is a need to enable the collection, access, use and sharing of human genome data within and across differing health and research sectors. Achieving this requires proactively addressing the ethical, legal, societal and cultural issues. It also requires acknowledging that there are risks associated with both the use of and the non-use of human genome data (1). Such risks must be balanced and mitigated to protect the interests of individuals, families and communities, while at the same time promoting the health and well-being of present and future generations. Efforts to scale up collection, access to, use and sharing of human genome data must recognize the related mistrust that can exist among some individuals, families and communities. This is an ongoing challenge, partly due to continuing exploitative practices, as well as capacity and power imbalances between the different stakeholders (2-4). The diversity of datasets and the under-representation of many populations in existing datasets must also be addressed to help reduce existing inequities, facilitate equitable access to the potential benefits of genomics, and advance global equity in genomics (5). However, addressing diversity and representation must be done in a way that does not perpetuate harms and protects privacy and confidentiality, if scientific quality and global equity in genomics are to be achieved. Consideration must also be given to the trans-generational impact of genomics, as decisions that are made today on collection, access, use and sharing could affect future generations. 

The integration of genomics into health systems requires a data life cycle approach, with guidance enabling collection, access to, use and sharing of human genome data within and across health and research sectors locally, nationally and internationally. To support research and the integration of genomics into health systems – and following the WHO Science Council 2022 report Accelerating access to genomics for global health (6) that recommended the promotion of ethical, legal, equitable and responsible use and sharing of human genome data – WHO has developed this document, which sets out principles for human genome data collection, access, use and sharing. 

2. PURPOSE AND SCOPE 

This document sets out globally applicable and inter-connected principles on the collection, access, use and sharing of human genome data, to promote human health and well-being, including responsible medical advances and scientific research. This document is rooted in human rights law (7-8). It complements and builds upon current laws, policies, frameworks and other guiding documents in this space (including 9-19) and encourages their development where none exist. 

The principles for human genome data collection, access, use and sharing are intended to:

•Promote social and cultural inclusiveness, equity and justice. •Promote trustworthiness within the data lifecycle •Foster integrity and good stewardship •Promote communal and personal benefits •Promote the use of common principles in laws, policies, frameworks and guidelines, within and across countries and contexts. 

In addition, these principles aim to build and strengthen capacity and awareness of individuals, families and communities from whom genome data are collected, to enable them to have more control over their genome data. 

Implementing these principles requires a comprehensive approach throughout the entire data life cycle. They apply to all prospective and retrospective collections of human genome data, and are designed to complement and inspire legal and ethical regulations, frameworks and guidelines at both the national and community- specific level (e.g. research community). WHO recognizes that the implementation of some of these principles may differ for retrospective data (e.g. secondary use of data). 

This document applies only to human genome data. Pathogen genome data (20) and microbiome data do not fall within its remit. 

Human genome data are typically linked with other health information that is critically important to its interpretation. WHO strongly encourages making other health data available with human genome data, subject to approval and mitigating any associated risks that may arise. It may be reasonable to apply these same principles to health data collection, access, use and sharing. Human genome data are obtained from biological samples thus these principles equally apply to them. Biological samples are a finite resource and have cultural significance in many contexts. Collecting, accessing, using, and sharing biological samples therefore may require additional considerations beyond those identified in this document (11). 

This document sets out principles that are intended to set normative standards in collection, access, use and sharing of human genome data. Each principle is followed by recommendations that can be used to guide the application and implementation of these principles in practice. The application of these principles in practice depends on giving careful attention to the health and research context in which human genome data are collected, accessed, used and shared. This will include: specific considerations of the individuals, families and communities providing the data; the purpose of collection, access and use; and the capacity, resources, skills and expertise of those collecting, accessing, using and sharing the data. Equally, national legal and ethical frameworks, as well as social and cultural values, impact the application of these principles. Individual and collective values may vary, giving rise to tensions when implementing these principles. In such circumstances, implementing these principles may require additional and careful deliberation and review. 

These principles are intended to be used by those responsible for governing, overseeing and managing human genome data, as well as stakeholders in the data life cycle within health and research contexts, including individuals, families and communities from whom human genome data originate, and the private sector. 

3. PRINCIPLES FOR HUMAN GENOME COLLECTION, ACCESS, USE AND SHARING 

3.1. To affirm and value the rights of individuals and communities to make decisions 

A commitment to affirm and value the rights and interests of individuals with capacity to make informed decisions about their human genome data throughout the data life cycle. In addition, a commitment to affirm the best interests of, and support for, individuals who do not have the capacity to make decisions for themselves. The use of human genome data has implications beyond the individual, and the relevant views of family members and communities on collection, access to, use and sharing of these data should be taken into account throughout the data life cycle.  

Recommendations: • Human genome data collection, access, use and sharing should be aligned with the needs, preferences and values of individuals, families and communities throughout the data life cycle. Informed consent is a critical component for the ethical use of human genome data and includes the right of and clear mechanism for an individual to withdraw, but there can be justified limitations to this right (e.g. when the results of the use of human genome data have been publicly shared). Any such limitations must be subject to approvals being in place and communicated clearly in advance. Informed consent should be as specific and granular as possible in relation to the potential uses (including by for-profit entities and the potential to share the data to train artificial intelligence), benefits and harms possibly resulting from the use of human genome data, the infrastructure hosting the data (including location and access modalities), and this information must be tailored to respect social and cultural contexts. The most appropriate informed consent model (e.g. specific, broad, tiered or dynamic informed consent) depends upon the individual/local context. Informed consent should be supported by governance frameworks and processes, and individuals should be informed of such processes. In circumstances where it is not possible to identify a specific purpose for human genome data use, informed consent to broad categories of human genome data collection, access, use and sharing may be permissible, provided such collection, access, use and sharing is subject to appropriate safeguards. 

These safeguards include, at a minimum, • • • • • governance frameworks and processes on the re-use of the data that should be informed by community engagement and oversight by a body such as a data access committee. Such a committee should ideally be independent and have responsibility for reviewing access requests and monitoring compliance with conditions set out in the access approval. The broader the informed consent, the more safeguards are required. Individuals, families and communities should have access to clear, transparent, accessible, understandable and ongoing communication about their human genome data collection, access, use and sharing, for those who wish to receive that information. This ongoing communication should, where possible, continue throughout the data life cycle. Individuals and their representative communities should be engaged in the governance and decision-making process regarding collection, access to, use and sharing of human genome data, including the development of appropriate informed consent models and processes. Children, when sufficiently mature to understand what is involved in their participation, should be given the opportunity to affirm the informed consent previously given on their behalf or to withdraw their consent from that point onwards. The right of the child to an open future (i.e. the right to know and the right not to know) should be given due consideration, when collecting, accessing, using and sharing human genome data from children. Measures regarding the protection of marginalized groups and populations, including individuals who are not able to consent or in need of additional support, protection or assistance, should be carefully thought out and implemented. 

3.2. Social justice 

A commitment to uphold individual and collective values and enable collection, access to, use and sharing of human genome data in ways that: (i) promote the highest attainable standard of health, individual and collective well-being; (ii) address the needs of underserved and marginalized individuals, families and communities, and those experiencing greater health burdens; (iii) reduce socioeconomic inequalities and health inequities; (iv) promote global equity; and (v) avoid individual and group discrimination and stigmatization. A commitment to enable access to adequate resources, skills, training, capacity building and capacity- strengthening for researchers, all health care professionals, genomic data administrators, policy-makers, individuals, families, communities and other stakeholders involved in human genome data collection, access, use and sharing. Fulfilment of this commitment requires greater effort in some countries and contexts than others due to existing inequities. 

Recommendations: • The purposes to be served by human genome data collection, access, use and sharing should give due consideration to local health needs and burdens, taking account of the interconnectedness between the local, national and international health ecosystems, which are critical to ensuring the global impact of genomics and improving global equity. 

• Return of results to individuals should be considered in cases where: results are clinically relevant and could be validated; return is feasible within the local health setting; and the return of results is legally and ethically permissible. 
• An approved policy should be developed for the return of individual results and should be in line with the individuals’ informed consent and respect the privacy and confidentiality of the individual. 
• Policies and procedures to protect individuals, families and communities from stigmatization and discrimination that can result from the association between genome data, community membership and health status should be developed in advance and regularly updated. They should be developed in collaboration with communities through meaningful community engagement, particularly those who may be at higher risk of stigmatization and discrimination. 

3.3. Solidarity 

A commitment to stand in solidarity with others by ensuring equitable access to human genome data and fair distribution of its benefits and burdens. This includes data collection, access, use and sharing, within and across communities, and acknowledges the need to address differences in capacity and existing inequities between different individuals, families or communities, countries and regions. 

Recommendations: • • The rights and interests of individuals, families and communities providing human genome data for collection, access, use and sharing should continue to be protected, particularly as efforts to scale up diversity and representation are increased. Decisions on human genome data collection, access, use and sharing should include an assessment of both the potential risks and potential benefits, and commitments to • • facilitating access to any resulting benefits for individuals, families and communities. Commercial interests should not unfairly limit collection, access to, use and sharing of human genome data. Governance processes should be introduced to clearly identify responsibilities and duties for all those involved in the data life cycle, and to specify sanctions in case of non-compliance. These sanctions should be sufficiently serious to act as deterrents to help avoid harm to individuals, families and communities. 

3.4. Equitable access to and benefit from human genome data 

A commitment to achieving equitable collection access to, use and sharing of human genome data and its resulting benefits. This means actively addressing power imbalances and inequities among different stakeholders that may hinder these efforts. A commitment to increase diversity and representation in datasets and decision-makers overseeing collection, access to, use and sharing of human genome data, without contributing to further harm for current and future generations. A commitment to ensuring that individuals, families and communities whose human genome data are collected, accessed, used and shared fairly benefit from its use. 

Recommendations: • Increasing representation of datasets across diverse populations is critical, but inclusion alone is insufficient to achieving equity. It must be paired with the meaningful participation of individuals, families and communities affected by decisions regarding the collection, access, use and sharing on their human genome data. Their involvement in decision-making and the development of governance frameworks is necessary, as differing cultural perspectives • • on human genome data can influence these processes. The equitable sharing of potential risks and benefits across and within communities, including affordable access to resulting benefits, should be considered in advance of collection, access, use and sharing of human genome data, and where possible and needed, informed by community engagement. Capacity building and strengthening should be considered as part of any collection, access to, use and sharing of human genome data. 

3.5. Collaboration, cooperation and partnership A commitment to promote mutually beneficial local, national and international collaboration, cooperation and partnership, including public–private partnership, between those involved in all aspects of human genome data collection, access, use, and sharing, acknowledging that to achieve this will require a rebalancing of power and representation between individuals, families, communities, countries, regions, and other stakeholders. 

Recommendations: • • • • Decisions on governance processes for human genome collection, access, use and sharing should be made collaboratively between all relevant stakeholders. Decisions on collection, access to, use and sharing of human genome data should include discussions on potential risks and benefits to individuals, families and communities from which the human genome data is collected. Policies should clarify that human genome data should be collected, accessed, used and shared with consideration for protecting and confidentiality to improve human health and wellbeing, with ethical safeguards. Ensure the interoperability of platforms to • facilitate collection, access, use and sharing of human genome data between institutions both nationally and internationally, and in the public and private sectors. To promote collaborative decision-making and effective partnerships, efforts should focus on building and strengthening capacity and improving health literacy on genomics and human genome data among all stakeholders. This includes both those contributing their data and those involved in making decisions about its collection, access, use, and sharing. It may incorporate targeted educational initiatives to increase public awareness and understanding of human genome data and the importance of and implications of its collection, access, use and sharing. 

3.6. Stewardship of human genome data A commitment to encourage, enable and sustain ethical, legal, socially and culturally appropriate, and responsible, human genome data collection, access, use and sharing by committing to: (i) develop processes to enable equitable collection, access to, use and sharing of human genome data; (ii) follow the current ethical practices on human genome data; (iii) identify and minimize potential risks in human genome data collection, access, use and sharing; and (iv) respect applicable laws and guidance, including laws on privacy and data protection. 

Recommendations: • Suitable models should be identified that provide equitable access to human genome data. They should be implemented in ways that best protect individuals, families and communities across different contexts. Efforts should be made to mitigate the environmental impact of data processing, storage and use. The collection, access, use and sharing • • of human genome data should align with other current relevant guidance, such as the Findable, Accessible, Interoperable, Reusable (FAIR) principles (21), the Collective Benefit, Authority to Control, Responsibility, Ethics (CARE) principles (22) and the TRUST code (23). Sufficient attribution should be given for the source(s) of human genome data. Timely access to human genome data should be granted, but justified, reasonable and • • • proportionate time delays can be permitted. To ensure that data collection and subsequent access, use, and sharing is in line with cultural and social priorities and considerations, community and stakeholder engagement should be ongoing throughout the data life cycle. Resources required to sustain the use of human genome data (e.g. financing, infrastructure, and personnel) should be considered at the outset of human genome data collection and also reviewed through the data life cycle. Specific guidelines, policies and frameworks should be put in place to ensure that current ethical, legal, privacy, data protection, and security standards and practices are followed, recognising that they may be informed by standards and practices on health data • • generally. This may require the establishment of data governance structures and oversight mechanisms (e.g. data access committees). Such standards and practices may need to evolve over time to reflect advances in technology, the state of the art, and societal norms. Robust data security measures should be implemented to safeguard genetic information from unauthorized access, breaches or misuse. This might include encryption, access controls, regular security audits, and compliance with data protection regulations. Provide training and resources to all those involved in the data life cycle on ethical data handling, privacy protection, and responsible data stewardship practices for human genome data. 

3.7. Transparency 

A commitment to provide openly available and easily accessible information on policies and processes that describe human genome data collection, access, use and sharing, including how the data are to be protected. A commitment to transparency also includes making research findings readily accessible to individuals, families, communities and other stakeholders who shared genomic data. 

Recommendations: • Publicly available policies should describe the criteria for deciding on collection, access to, use and sharing of human genome data, the processes for decision-making, how human genome data is protected, and how such policies were developed. 

• These policies should describe how the right to privacy is protected and who is responsible for ensuring respect to this right throughout the data life cycle. 
 
• Systems and mechanisms should be put in place to enable communication with individuals, families and communities about the use of their human genome data, and related research results. This should include plain language summaries of key insights and education materials and should be openly available to all. Individuals, families and communities should be informed about how they can exercise their rights related to their human genome data. 

3.8. Accountability 

A commitment to establishing processes that enable and promote responsible collection, access, use and sharing of human genome data and that prevents human genome data misuse, accompanied by mechanisms that hold individuals, institutions and other stakeholders accountable for failure to adhere to such processes. 

Actions: • Establish mechanisms that assign roles and responsibilities to those involved throughout the collection, use and sharing of human genome data, including for cases related to negligence or data misuse. Responsible stakeholders should be identified prior to human genome data collection, access, use and sharing. 

 • Mechanisms, including regulations and policies, should be put in place to guard against the misuse of human genome data. This includes limiting collection, access to, use and sharing of human genome data with stakeholders who cannot adequately protect the data. Such policies should, at a minimum, support the right to privacy, prohibit collection, access to, use and sharing of human genome data that stigmatize or discriminate against the individual, their family or their community. They should also prohibit any attempt to re-identify the individual, and prohibit the unauthorized collection, access to, use and sharing of such data. 
 
 • Mechanisms should be put in place to ensure that stakeholders use human genome data in a secured and trustworthy manner, and that those responsible for human genome data misuses are held to account. Human genome data collection, access, use and sharing should be subject to checks on the purpose of data use. Data audit trails and systems for tracking and auditing data collection, access, use and sharing should also be implemented to monitor compliance with data sharing agreements, regulatory requirements, and ethical guidelines.

AI Magic

'The reanimation of pseudoscience in machine learning and its ethical repercussions' by Mel Andrews, Andrew Smart and Abeba Birhane in (2024) 5(9101027) Cell comments 

 Machine learning has a pseudoscience problem. An abundance of ethical issues arising from the use of machine learning (ML)-based technologies—by now, well documented—is inextricably entwined with the systematic epistemic misuse of these tools. We take a recent resurgence of deep learning-assisted physiognomic research as a case study in the relationship between ML-based pseudoscience and attendant social harms—the standard purview of “AI ethics.” In practice, the epistemic and ethical dimensions of ML misuse often arise from shared underlying reasons and are resolvable by the same pathways. Recent use of ML toward the ends of predicting protected attributes from photographs highlights the need for philosophical, historical, and domain-specific perspectives of particular sciences in the prevention and remediation of misused ML. 

The present perspective outlines how epistemically baseless and ethically pernicious paradigms are recycled back into the scientific literature via machine learning (ML) and explores connections between these two dimensions of failure. We hold up the renewed emergence of physiognomic methods, facilitated by ML, as a case study in the harmful repercussions of ML-laundered junk science. A summary and analysis of several such studies is delivered, with attention to the means by which unsound research lends itself to social harms. We explore some of the many factors contributing to poor practice in applied ML. In conclusion, we offer resources for research best practices to developers and practitioners. 

The fields of AI/machine learning (ML) ethics and responsible AI have documented an abundance of social harms enabled by the methods of ML, both actual and potential. Although the topic is comparatively more obscure, critics have also sought to draw attention to the epistemic failings of ML-based systems: failures of functionality and scientific legitimacy.  The connection between the ethicality and epistemic soundness of deployed ML, however, has received scant attention. 

We urge that if the field of AI ethics is to be efficacious in preventing and remediating the social harms flowing from deployed ML systems, it must first grapple with discrepancies between the presumed epistemic operation of these tools and their in-practice ability to achieve those aims. While such an observation is not novel (see Raji et al.), we build on prior work, both in offering an analysis of the issue from a philosophical vantage point and in venturing into the intricacies of in-practice epistemic and ethical misuses of ML systems. We argue that philosophical, historical, and scientific perspectives are necessary in confronting these issues and that ethical and epistemic issues cannot, and should not, be confronted independently. 

A recent surge of deep learning-based studies have claimed the ability to predict unobservable latent character traits, including homosexuality, political ideology, and criminality, from photographs of human faces or other records of outward appearance, including Alam et al., Chandraprabha et al., Hashemi and Hall, Kabir et al., Kachur et al., Kosinski et al.,  Mindoro et al.,  Parde et al.,  Peterson et al.,Mujeeb Rahman and Subashini,  Reece and Danforth,  Su et al.,  Tsuchiya et al.,  Verma et al.,  Vrskova et al.,  and Wang and Kosinski.  In response, government and industry actors have adapted such methods into technologies deployed on the public in the form of products such as Faception,  Hirevue,  and Turnitin.  The term of art for methods endeavoring to predict character traits from human morphology is “physiognomy.” Research in the physiognomic tradition goes back centuries, and while the methods largely fell out of favor with the downfall of the Third Reich, the prospects of ML have renewed scientific interest in the subject. Much like historical forays into this domain, this new wave of physiognomy, resurrected and yet not, apparently, sufficiently rebranded, has faced harsh criticism on both ethical and epistemic grounds. 

This critical response, however, has yet to explore how the confused inferential bases of these studies are responsible for their ethically problematic nature. There are several conclusions we wish to draw from the detailed study of these examples, which we believe extrapolate to the relation between ethical and epistemic issues in deployments of ML at large. (1) No inference is theory neutral. (2) Leaving a theory or hypothesis tacit means it is not held to account for, and its conclusions are not critically evaluated before the results of such work are deployed or acted upon. (3) If a study informs a policy, intervention, or technology that will materially impact human lives—in other words, if a study is at all informative—and it misrepresents the human reality within which it is being deployed, it should be expected that harms to humans will arise. Wrong theories generate wrong interventions. Wrong interventions cause harm. (4) ML models are developed and deployed to extract complex, high-dimensional statistical patterns from large datasets. These complex patterns are typically taken to represent unobservable latent features of the systems from which their training data were drawn. The norms and procedures established for correctly inferring unobservable latent variables from correlational measures differ by scientific field and must be indexed to subject matter. (5) Meta-narratives and cycles of hype surrounding ML, we argue, play a direct role in encouraging errant usage of the tools. When ML tools are proclaimed to deliver false inferences, the outcomes are rarely ethically innocuous. This is true in general but is all the more salient for ML tools deployed in socially sensitive arenas. In bringing to light the connection between pseudoscientific methods in applied ML and the ethical harms they perpetuate, we hope to encourage greater care in the design and usage of such systems. 

Physiognomy resurrected 

“Physiognomy” is “the facility to identify, from the form and constitution of external parts of the human body, chiefly the face, exclusive of all temporary signs of emotions, the constitution of the mind and the heart.” Georg Christoph Lichtenberg, 1778

Recent years have seen an abundance of papers promulgating physiognomic methods resting on ML models.  Work of this ilk is undertaken by academic research groups, private firms, and government agencies. A number of representative instances of each claim to have trained ML classifiers to predict personality, behavioral, or identity characteristics from image, text, voice, or other biometric data. Inferred labels have included race,  sexuality,  mental illness,  criminal propensity, autism, and neuroticism. These studies have predominantly relied on deep learning neural networks (DNNs), sometimes in tandem with more simplistic regression techniques. The practice of wielding the methods of ML toward the (putative) prediction of internal mental states, dispositions, or behavioral propensities based on outwardly visible morphology has been labeled “AI pseudoscience,” “digital phrenology,” “physiognomic AI,” “AI snake oil,” “bogus AI,” and “junk science.” These technologies, however, do not only exist in the abstract—a growing number of companies now market physiognomic capabilities, including the ability to detect academic dishonesty in students and future performance in prospective employees. Remarkably, a single tool marketed to defense contractors boasts of the ability to predict “pedophilia,” “terrorism,” and “bingo playing.” 

In this section, we review the details of several representative examples of physiognomic ML. These case studies are intended to be illustrative of the kinds of reasoning, epistemic foundations, and logic behind research and applications of automated inference from images portraying human likenesses. The studies presented here are intended to be representative of the genre and not a comprehensive overview. 

Inferring sexual orientation 

Utilizing DNNs,  Wang and Kosinski extract features from images of human faces, which they then regress in a supervised learning task against self-reported sexual orientation labels. The classifier achieved 81% and 71% accuracy scores on sexual orientation for male and female subjects, respectively. These findings represent a higher classification accuracy than experimentally determined human judgment. The researchers scraped their data from social media profiles, claiming that training their classifiers on “self-taken, easily accessible digital facial images increases the ecological validity of our results.”  Wang and Kosinski report that the “findings advance our understanding of the origins of sexual orientation.”  The authors of the study explain the ability of their models to discriminate sexual orientation with the claim that “the faces of gay men and lesbians tend to be gender atypical.”  The validation of this hypothesis depended on the training of an additional DNN for gender discrimination. This classifier assigned a likelihood to each face image of being female. The researchers then interpreted this likelihood as a measure of facial femininity, assessing the faces of homosexual-tagged individuals against an average femininity score for heterosexual individuals. The researchers claimed that their results revealed that “the faces of gay men were more feminine and the faces of lesbians were more masculine than those of their respective heterosexual counterparts.”  “The high accuracy of the classifier,” Wang and Kosinski report, “confirmed that much of the information about sexual orientation is retained in fixed facial features.”  The contention of the researchers is that high classification accuracy of sexual orientation from facial features, alongside the evidence they supply for the gender-atypicality of facial morphology, lends support for a particular theory of the genesis of same-sex attraction. The proposed hypothesis is the prenatal hormone theory (PHT) of homosexuality, which proposes that same-sex attraction is a developmental response to atypical testosterone exposure in fetal development. Wang and Kosinski’s results, they claim in their preprint, “provide strong support for the PHT, which argues that same-gender sexual orientation stems from the underexposure of male fetuses and overexposure of female fetuses to prenatal androgens responsible for the sexual differentiation of faces, preferences, and behavior.” 

Personality psychology 

Kachur et al. write that “morphological and social cues in a human face provide signals of human personality and behaviour.” Their stated hypothesis is that a “photograph contains cues about personality that can be extracted using machine learning.” The authors further claim to have “circumvented the reliability limitations of human raters by developing a neural network and training it on a large dataset labelled with self-reported Big Five traits.” Here, deep learning is invoked as a means to obtain objectivity beyond human judgment; however, the training dataset was self-labeled by human raters. The predictive accuracy is interpreted as prima facie evidence for their hypothesis that structural features of human faces contain information of human personality and behavior, and the authors state that their “study presents new evidence confirming that human personality is related to individual facial appearance.” 

In this study, participants self-reported personality characteristics by completing an online questionnaire and then uploaded several photographs, which the researchers then used to construct their training and test datasets. In this example, as in Wang and Kosinski, researchers used the accuracy of their ML model as confirmatory evidence of a joint causal basis for both facial morphology and self-reported personality. Kachur et al. report “several theoretical reasons to expect associations between facial images and personality” including that “genetic background contributes to both face and personality.” Kachur et al. described their results as being indicative of “a potential biological basis” to the discovered association between face images and self-reported personality characteristics. 

“Abnormality” classification 

A recent study constructed a “normal” and “abnormal” human facial expression dataset for the purpose of automatically detecting such abnormal traits as drug addiction, autism, and criminality from facial images. The authors argued that “facial expression reflects our mental activities and provides useful information on human behaviors.” Kabir et al. “developed a combined method of Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) to classify human abnormalities.” “This approach,” they contend “analyzes the human face and finds the abnormalities, such as Drug addiction, Autism, Criminalism [sic].” 

The researchers utilized images “gathered from the web using the web gathering technique,” although the details of this technique were not further elucidated. It is not made clear within the scope of the manuscript on what basis images were classified as “normal,” “drug addicted,” “autistic,” or “criminal.” The researchers reported a validation accuracy of 89.5% on the four categories. The provenance of the labels is left undisclosed in this study, as are the validation criteria. 

In a similar vein, Vrskova et al. claim to be able to diagnose “abnormal” human activities such as “begging,” “drunkenness,” “robbery,” and “terrorism” from video footage. 

Lie detection 

Automated deception detection has long been of interest to law enforcement, judicial systems, academic institutions, corporations, and governments. A recent study by Tsuchiya et al. utilized facial analysis and ML toward the putative automatic detection of deception for remote job-interview scenarios. The stated purpose of this research was to create an ML-based tool to detect when someone on video call might be lying. Participants in this study were asked to knowingly generate false descriptions of images while being recorded via video and biometric sensors. The researchers then used these data to train an ML model to predict deception-based facial or head movements, pulse rate, or eye movements. The researchers obtained a high accuracy rate using their classifier on the four participants used in the study. As in the other studies reviewed here, the predictive accuracy of the model was taken to substantiate the hypothesis that particular facial features or movements are evidence of unobservable character or behavioral traits—in this instance, deception. 

Criminality detection 

A study by Wu and Zhang purported to “empirically establish the validity of automated face-induced inference on criminality.” The authors trained four canonical ML models on a dataset of ID photographs of Chinese citizens to predict the label of criminality. Wu and Zhang stated that their models detect “criminality based solely on still face images, which is free of any biases of subjective judgments of human observers.”33 The convolutional neural network achieved an accuracy rate of 89.51% at picking out subjects who had been arrested for a crime. Hashemi and Hall claim to have also developed a deep learning-based criminality detector.

Amazon

'Amazon's Quiet Overhaul of the Trademark System' by Jeanne C Fromer and Mark P McKenna in (2025) 113 California Law Review comments

 Amazon's dominance as a platform is widely documented. But one aspect of that dominance has not received sufficient attention-the Amazon Brand Registry's sweeping influence on firm behavior, particularly in relation to the formal trademark system. Amazon's Brand Registry serves as a shadow trademark system that dramatically affects businesses' incentives to seek legal registration of their marks. The result has been a dramatic increase in the number of applications to register, which has swamped the U.S. Patent and Trademark Office and created delays for all applicants, even those that previously would have registered their marks. And the increased value of federal registration has drawn in bad actors who fraudulently register marks that are in use by others on the Amazon platform and use those registrations to extort the true owners. 

Amazon's policies also create incentives for businesses to adopt different kinds of marks. Specifically, businesses are more likely to claim descriptive or generic terms, advantageously in stylized form or with accompanying images, and to game the scope limitations that would ordinarily attend registration of those marks. And the same Amazon policies have given rise to the phenomenon of "nonsense marks" - strings of letters and numbers that are not recognizable as words or symbols. In the midst of these systemic changes, Amazon has consolidated its own branding practices, focusing on a few core brands and expanding its use of those marks across a wide range of products. In combination, Amazon's business model and Brand Registry have overhauled the American trademark system, and they have done so with very little public recognition of the consequences of Amazon's business approach. Amazon's impact raises profound questions for trademark law, and for law more generally. There have been powerful players before, and other situations in which private dispute resolution procedures have affected parties' behavior. But Amazon's effect on the legal system is unprecedented in scale and scope. What does (and should) it mean that one private party can so significantly affect a legal system? Do we want the trademark system to have to continually adapt to Amazon's rules? If not, how can the law disable Amazon from having such a profound impact? In this regard, we explore the ways in which Amazon's practices might both help and hurt competition, be harmful to the trademark system, and reshape how we think about trademark law at its foundation. 

Margo Bagley comments 

The Brand Registry Program rewards owners of federally registered trademarks with a cheap and efficient dispute resolution process, which both allows them to object to uses of their mark on the website and also gain a higher priority in search results than if their mark was unregistered. 

Fromer and McKenna identify at least ten impacts of Amazon’s program on the trademark system: Amazon 1) created a shadow trademark system that incentivized applicants to file for federal trademark registrations which 2) swamped, the PTO resulting in 3) significant delays for all applicants (plus a cluttered register) and opportunities for bad actors to 4) fraudulently register “in use” but unregistered marks and then 5) extort the legitimate owners (often small entities relying on traditional trademark priority rules) who would now risk losing the ability to sell goods on Amazon.  

In addition, the authors detail how Amazon’s policies led to a change not just in the magnitude of registrations but also 6) in the kinds of marks being registered, nullifying the effect of historical limitations on the registration of 7) descriptive and generic terms (bad actors can register them with the USPTO as stylized marks/use disclaimers, but Amazon’s Registry only matches text so those legal limitations are ignored) while also giving rise to a wide variety of 8) “nonsense marks” (if you’ve shopped on Amazon, you’ve seen them), strings of unpronounceable letters that have no meaning to consumers but qualify for Amazon’s registry which then favors them in search results. As the authors explain, “when search and purchase are not necessarily done by people who need to remember a brand name, businesses just need something to make the algorithm prefer them. Nonsense will do.” 

And if these externalities to Amazon’s Brand Registry Program were not bad enough, the company has simultaneously 9) elevated its own brands on its site so that one is more likely to see Amazon brands first in searches, even in searches specifically for other brands. This gives Amazon more power vs. third-party brands and 10) “decenters” branding by blunting the traditional source identification and search cost reduction benefits branding is designed to provide.