The authors comment
Overview
Between 2016 and 2019 the Metropolitan Police Service (MPS) conducted a total of 10 test deployments, trialling live facial recognition (LFR) technology during policing operations. This research was initiated in order to provide an independent academic report on this process. Researchers observed the final six test deployments, beginning in Stratford (Westfield) in June 2018. Researchers joined officers on location in the LFR control rooms, engaged with officers responding on the ground, and attended briefing and de-briefing sessions in addition to planning meetings. A number of legal and other documents prepared by the MPS were also reviewed.
This report is focused on issues arising in relation to the MPS’ LFR test deployments. It does not directly engage with broader issues regarding the legality, or use, of LFR technology by law enforcement agencies. As such, although certain elements of the analysis presented herein may be relevant to future debates, no conclusions are drawn in that regard.
This report centres on the overall governance of the LFR test deployments, the procedures and practices of LFR in operational settings (as observed over the course of the test deployments), and human rights compliance. A draft of this report was submitted to the MPS so that any factual errors could be noted, and to provide a right of reply. After review of the document, the MPS chose not to exercise this right. This report is independent, was externally funded as part of the ESRC Human Rights, Big Data & Technology Project, and the findings and opinions expressed are those of the authors alone.
Human rights law requires that any interference with individuals’ rights be in accordance with the law, pursue a legitimate aim, and be ‘necessary in a democratic society’.
As detailed in the report, it is highly possible that the LFR trial process adopted by the MPS would be held unlawful if challenged before the courts. In particular, this report concludes that the implicit legal authorisation claimed by the MPS for the use of LFR - coupled with the absence of publicly available, clear, online guidance - is likely inadequate when compared with the ‘in accordance with the law’ requirement established under human rights law. This demonstrates a need to reform how the trialling or incorporation of new technology and policing practices is approached by the MPS, and underlines the need for the effective incorporation of human rights considerations into all stages of the MPS’ decision making process, including with respect to if, and how, trials should be undertaken. It also highlights a need for meaningful engagement with, and debate on, these issues at a national level.
The report highlights a number of issues arising from the LFR test deployments, and raises some significant concerns. These concerns are most notable with respect to:
(1) The research process adopted by the MPS to trial LFR technology. The test deployments offered an opportunity to both examine technical accuracy and to understand the implications of LFR on police operations. However, MPS trial methodology focused primarily on the technical aspects of the trials. There is less clarity on how the test deployments were intended to satisfy the non-technical objectives, such as those relating to the utility of LFR as a policing tool. This report raises concerns that the process adopted by the MPS was inadequate with respect to addressing the non-technical objectives identified.
(2) The absence of an explicit legal basis for the use of LFR, and concerns that the implicit legal basis identified by the MPS is inadequate in relation to the ‘in accordance with the law’ requirement established by human rights law. This is compounded by the absence of online guidance capable of addressing the ‘foreseeability’ of how LFR technology was utilised. Without explicit legal authorisation in domestic law, it is highly possible that police deployment of LFR technology may be held unlawful if challenged before the courts;
(3) Human rights law requires that any rights interference be ‘necessary in a democratic society’. MPS analysis did not effectively address this requirement, and it is considered highly possible that the MPS’ test deployments of LFR technology would not be regarded as ‘necessary in a democratic society’ if challenged before the courts; and
(4) Operational factors relating to inconsistency in the adjudication process, including a presumption to intervene, problems with how the MPS engaged with individuals, and difficulties in obtaining the consent of those affected.
Research Methodology Adopted for the Report
Six test deployments were observed from beginning to end. Observations extended to attendance at pre-deployment police briefings for each test deployment and post- deployment debriefings which usually took place the following day. Observation mainly focused on the operational practices of the intelligence units in the control room: that is, the activities of officers monitoring LFR camera feeds, deliberating over computer-generated matches and, when deemed appropriate, issuing instructions to intercept matched persons. Research also engaged street-based intervention teams responsible for intercepting individuals matched to watchlists by the LFR system, plain clothes officers deployed at the test sites, and uniformed officers involved in LFR-related public facing activities. Researchers were also invited to several LFR planning meetings.
All documents provided by the MPS were examined. A variety of interview techniques were used to gain additional data. Observations involved detailed conversations with a wide range of MPS staff. These included operational officers, individuals holding tactical and strategic roles, and those engaged in the technical evaluation of LFR. Formal interviews were also conducted with a number of key external stakeholders including oversight bodies, technology evaluation specialists and civil society organisations.
The Nature of the Test Deployments Undertaken by the MPS, and Appropriateness of the Trial Method Technology adopted
Numerous voices, including the Surveillance Camera Commissioner and the Biometrics Commissioner, have stressed the importance of trialling emergent technologies. Several attempts have been made to offer principles to guide this process, yet their development has been piecemeal and no agreed national standards or established oversight mechanisms exist. It is within this context that the MPS trials of LFR took place.
The MPS’ trial methodology focused primarily on technical aspects, examining the performance of the technology in live settings. There did not appear to be a clearly defined research plan that set out how the test deployments were intended to satisfy the non-technical objectives, such as those relating to the utility of LFR as a policing tool. This necessarily complicated the trial process, and affected its effectiveness and overall utility.
It is unclear if alternative approaches to testing LFR were considered and discarded. It is uncertain whether the initial decision to trial LFR considered the use of simulated conditions, with volunteer-based watchlists (as adopted in Berlin) or live condition trials focused on technical performance but not policing responses (as in the United States).
The mixing of trials with operational deployments raises a number of issues including with respect to consent, public legitimacy and trust. A key concern is the lack of a clear distinction between research objectives regarding the trial of LFR technology, and the policing objectives associated with operational deployments. This holds particular meaning when considering differences between an individual’s consent to participate in research and their consent to the use of technology for police operations. For example, from the perspective of research ethics, someone avoiding the cameras is an indication that they are exercising their entitlement not to be part of a particular trial or are protecting their own right to privacy. From a policing perspective, this same behaviour may acquire a different meaning and serve as an indicator of suspicion. This resulted in a number of issues relating to how police officers engaged with individuals on the ground.
The absence of national leadership at government level – including a lack of clear lines of responsibility regarding whether trials should be conducted, and if so how – leaves police evaluation teams with the enormous task of not only undertaking scientific evaluation, but also compensating for a lack of national leadership by recreating and reinterpreting policy anew. While this tension may apply to other trials of police equipment, it is particularly acute in the case of LFR given its intrusive nature, and requires urgent attention for future testing of this technology (as well as other technological innovations). A key element in this regard must be considering and building in human rights compliance from the outset of any trial process, including with respect to if, and how, any trials should be undertaken.
The Legal Basis Underpinning the MPS’ Use of LFR
No explicit legal basis exists authorising the MPS’ use of LFR technology. The legal mandate documents prepared by the MPS reference a number of different sources of law, including the common law, the Human Rights Act 1998, the Freedom of Information Act 2000, the Protection of Freedoms Act 2012, the Data Protection Act 2018, and the Regulation of Investigatory Powers Act 2000. Of these, only the common law and the Protection of Freedoms Act 2012 could potentially establish an implicit legal basis for LFR. The other sources either relate to public access to information regarding police activity or regulate the use of LFR technology, without establishing explicit legal authorisation for LFR as such.
The difficulty with relying upon the common law or the Protection of Freedoms Act 2012 as sources of implicit legal authorisation vis-à-vis the use of LFR technology is the ambiguity that will inevitably arise. The ‘in accordance with the law’ test established under human rights law incorporates a number of different elements, relating both to the existence of a legal basis and the quality of that legal basis. Key in this regard is protection against arbitrary rights interferences, and foreseeability with respect to how the law will be applied. Existing case law reinforces the concern that the legal basis identified by the MPS may be overly ambiguous. Issues in this regard are discussed in greater detail in Section 3.2. of the report. Similar concerns regarding the absence of a clear legal basis have been raised by Liberty and Big Brother Watch when interviewed for this report, and in academic commentary.
Ultimately, this report concludes that the implicit legal authorisation claimed by the MPS for the use of LFR appears inadequate when compared with the ‘in accordance with the law’ requirement established under human rights law. The absence of publicly available guidance clearly circumscribing its circumstances of use – thereby facilitating foreseeability – reinforces this point. Without explicit legal authorisation in domestic law it is highly possible that police deployment of LFR technology – as a particularly invasive surveillance technology directly affecting a number of human rights protections, including those relevant to democratic participation – may be held unlawful if challenged before the courts.
The Absence of Effective Analysis Addressing the ‘Necessity in a Democratic Society’ Determination for LFR
Determining the necessity in a democratic society of any measure that interferences with human rights protections isessential in order to ensure overall rights compliance. In this context this requirement is intended to ensure that measures useful to the protection of public order and the prevention of crime do not inappropriately undermine other rights, including those necessary to the effective functioning of a democratic society, such as the right to private life, the right to freedom of expression, and/or the right to freedom of assembly and association. The test itself involves a number of different elements. An interference will be considered necessary in a democratic society ‘if it answers to a “pressing social need”, if it is proportionate to the legitimate aim pursued and if the reasons adduced by the national authorities to justify it are relevant and sufficient.’
In order to determine whether LFR is ‘necessary in a democratic society’ in the circumstances of the MPS’ test deployments, impact or risk assessments should be conducted prior to deployment in order to identify and understand any potential human rights harm. This conclusion is supported by the Surveillance Camera Commissioner’s recent guidance on ‘Police Use of Automated Facial Recognition Technology with Surveillance Camera Systems’.
The MPS did prepare a number of impact/risk assessment documents. However, these documents are regarded as inadequate with respect to engagement with human rights law requirements.
No MPS documents have been seen that clearly set out the justification underpinning the deployment of LFR technology in a manner capable of addressing whether such deployments may be considered ‘necessary in a democratic society’. Of particular concern is the lack of effective consideration of alternative measures, the absence of clear criteria for inclusion on the watchlist, including with respect to the seriousness of the underlying offence, and the failure to conduct an effective necessity and proportionality analysis. For these reasons, and as discussed in greater detail in Sections 3.3.1. and 3.3.2., it is highly possible that the MPS’ test deployments of LFR technology would not be regarded as ‘necessary in a democratic society’ if challenged before the courts.
Operational Factors
Alerts
Overall, the LFR system generated 46 matches over the course of observed test deployments, involving 45 separate individuals. 42 matches were deemed eligible for analysis. Adjudicating officers judged 16 (38.1%) of these 42 computer generated matches to be ‘non-credible’; that is, officers did not believe the image recorded by the LFR technology match the image on the watchlist. MPS officers considered the LFR match sufficiently credible to stop individuals and perform an identity check on 26 occasions. Four of these attempted interventions were unsuccessful, as individuals were lost in the crowd.
Of the remaining 22 stops, 14 (63.64%) were verified as incorrect matches following an identity check. Eight (36.36%) were verified as correct matches following an identity check. This means that across all six observed trials, and from all computer- generated alerts, face recognition matches were verifiably correct on eight occasions (eight of 42 matches, 19.05%).
Watchlist Construction
The condition of being ‘wanted’ was consistently stated as a criterion for being enrolled on a watchlist. However, ambiguity exists regarding the definition of ‘wanted’ adopted by the MPS, and documentation indicates that this included both ‘wanted by the courts’ and ‘wanted by the police’. Those included on the watchlist thus apparently ranged from individuals wanted by the courts to those wanted for questioning, across a range of different offences.
categories of ‘wanted’ persons. he identification of ‘individuals shown as wanted by the police and the courts’ was not the only watchlist criterion in use. MPS documentation also highlighted the use of LFR to identify individuals who present a risk of harm to themselves and others; support ongoing policing activity with regards to a specific problem or location; and assist police in identifying individuals who may be ‘at risk or vulnerable’.
The category of ‘wanted’ (re ‘to identify individuals shown as wanted by the police and the courts’) was relied on in creating watchlists for all observed test deployments. Refinements to the threshold for inclusion under this criterion were made from December 2018 onwards. Nonetheless, significant ambiguity remains regarding the criteria used for watchlist construction. This directly affects the ‘foreseeability’ of MPS activity regarding the use of LFR technology. The size of the watchlists varied considerably across the observed test deployments. No discernible direct relationship existed between the watchlist size and number of alerts.
Watchlist Accuracy
Legacy data handling systems meant data relevant to watchlists was spread across different databases and each watchlist entry needed to be assembled by manually extracting and merging records from each of these locations.
Ensuring accurate and up-to-date information from across these different data sources posed a significant challenge. Such difficulties made compliance with overall standards of good practice complex and placed a significant burden on officers. Issues to do with the accuracy of the watchlist played out when individuals were stopped on the basis of outdated information. On occasion, individuals were flagged by the LFR technology in relation to a serious offence, but this had already been dealt with by the criminal justice system. However, they were wanted in relation to more minor offences and were arrested accordingly. It is unlikely this lesser offence would have been sufficiently serious to be included in the initial watchlist. This raises additional concerns when LFR is deployed on a necessity calculation intended to address serious crime but is then also used for more minor offences.
Matching Intelligence to LFR Deployments
Most ethical guidance, and legal and oversight provisions governing surveillance also require a clearly prescribed application. Police uses of surveillance measures are directed towards protecting the public from crime and upholding public order. However, the legitimacy of any measure must still be determined in relation to the ‘necessary in a democratic society’ requirement. One key element when evaluating issues of necessity and, by extension, proportionality, is a consideration of the stated purpose of LFR test deployments and, crucially, analysis of the extent to which the use of the technology is ‘rationally connected’ to this purpose.
In the first test deployments observed (Stratford), LFR use was regularly justified in briefings on the basis of several perceived benefits including detection, deterrence, intentional crime displacement and disruption. These applications involve clear differences in the purpose of LFR, requiring distinct necessity calculations.
Public Consent
The role of public consent constituted a contentious debate surrounding the LFR test deployments. Like CCTV, LFR is classified by the MPS as a form of overt surveillance and the consent of affected individuals is seen as fundamental. The importance of consent is also emphasised in regulatory instruments. Measures undertaken by the MPS that bear on consent overlapped with attempts to promote public reassurance and to test public opinion.
For consent to be meaningful, several conditions are important:
1. Informed consent.
The MPS pursued a number of strategies intended to ensure that public consent for LFR constituted informed consent, including the use of uniformed officers to explain the role of the technology to the public, leafleting and signage boards. A key question emerges over the degree to which consent can be considered informed on the basis of the information supplied by the MPS. Information provided transparency regarding the time and location of the LFR test deployments yet less clarity over the purpose of the deployment, who was likely to be the subject of surveillance, and how additional information could be ascertained. With the exception of the morning of the first Soho trial an individual reading or standing next to a sign was out of camera range for each LFR deployment. A key conclusion is the importance of being clear about why information is provided to the public. What might be appropriate in respect to issues of public support is not necessarily sufficient or well targeted enough to support individual consent. During test deployments individuals were required to make an informed decision regarding consent in a very short time-frame, a factor exacerbated by the limits on prior knowledge amongst the public.
2. Consent and opportunities to exercise a different choice.
Opportunities for pedestrians to bypass the cameras and continue walking towards the same destination varied across the test deployments. These ranged from simply crossing the street to a walking detour of an additional 18 minutes to reach the same point.
3. Capacity to refuse or withdraw consent without penalty.
Treating LFR camera avoidance as suspicious behaviour undermines the premise of informed consent. In addition, the arrest of LFR camera avoiding individuals for more minor offences than those used to justify the test deployments raise clear issues regarding the extension of police powers and of ‘surveillance creep’. These issues highlight the distinctions between gaining consent for research (e.g. trials) and consent for police operations, and the tensions that will inevitably arise during live test deployments.
The Adjudication Process
All observed officer briefings were clear on the importance of human discretion in the LFR process. Officers were consistently instructed that a computer derived match was not sufficient to confirm an identity in and of itself. This is appropriate given the significant error rates associated with LFR and legal stipulations concerning meaningful human intervention in digital decision-making processes. Adjudication practices varied across the deployments. These can be placed within three distinct categories:
1. Multiple adjudicators in the control room.
Multiple operators brought additional scrutiny to the process but also raised the likelihood of contrasting approaches within the same adjudication team.
2. Simultaneous adjudication and street engagement.
This involved intelligence officers radioing through a description of a LFR match while they were still in the process of deliberating over the credibility of the alerted match. A decision to trigger the street intervention team to start looking for the matched individual may have sound operational reasons. However, every instance in which this simultaneous approach was followed led to an attempt to engage a matched individual. This forms part of a wider and discernible ‘presumption to intervene’. Greater clarity is possible over whether communications to intervention teams are instructions to maintain observation or an instruction to intervene.
3. Mobile devices and simultaneous adjudication on the street and in the control room.
During the second Romford test deployment, the decisive choice to intervene with a matched individual was made by street-based officers equipped with handheld devices capable of receiving LFR alerts on at least five occasions. Decisions of the control room-based intelligence teams to engage a subject were never rejected by mobile-equipped officers. Decisions by the control room-based intelligence units not to intervene were frequently ‘overruled’ by street-based mobile-equipped officers on the basis of their separate access to imaging information. These processes also contribute towards a presumption to intervene.
Physical Factors Relating to Deployments
The physical characteristics of a particular area, along with the spatial location of intervening officers, had significant bearing on the adjudication process and subsequent street intervention. The spatial deployment of officers to provide the best opportunity to locate a matched individual on the street constricted the time available to adjudicators to reach a decision. Conversely, situating officers further from cameras afforded more time for control room adjudication but increased the likelihood of losing track of individuals.
