Australian Legal Bibliometrics

The incisive and important 'A Report into Methodologies Underpinning Australian Law Journal Rankings. Prepared for the Council of Australian Law Deans (CALD)' (UNSW Law Research Paper No. 2016-30) by Kathy Bowrey comments

Law schools face significant institutional pressure to adopt journal ranking lists that are used to inform comparative assessment of the Faculty, School and individual researcher performance.

CALD has commissioned a written report that:

1. critically evaluates the methodology of up to eight Law journal lists or rankings agreed to by the parties; 

2. makes recommendations about the suitability of the lists to act as a proxy for academic research quality, including suggesting revisions or modifications to methodology and reference to how to maintain the currency of any proposed list, as appropriate; 

3. comments on the utility of the lists in view of the suggested purposes for which they may be used.

This report is in four parts.

Part One provides a brief overview of bibliometric databases and indices currently in use in the higher education sector to assess research productivity, quality and influence. New non-citation based alternative metrics for the Humanities and perception studies are also discussed. There is also discussion of the Washington & Lee journal ranking list, which underpinned the original CALD/ERA lists. An updated version informed the Deakin List.

This part also contains comments on difficulties in applying existing bibliometrics to the output of Australian legal researchers.

Part Two provides analysis of the following Australian law journal ranking methodologies and lists: CALD list (2009); ERA 2010; Australian Business Deans Council Journal Quality List 2013; Deakin University Law Journal Rankings; University Of Tasmania Law Journal Rankings. This part also includes tables that allow for review of the performance of particular law journals across the various ranking lists provided.

Part Three addresses new developments in research assessment and current critical literature on the use and misuse of metrics.

Part Four provides recommendations to guide future discussion of the use of metrics to assess legal research.

Metadata

'Evaluating the privacy properties of telephone metadata' by Jonathan Mayer, Patrick Mutchlera, and John C. Mitchell in 113(20) Proceedings of the National Academy of Sciences 5536–5541 comments

Privacy protections against government surveillance are often scoped to communications content and exclude communications metadata. In the United States, the National Security Agency operated a particularly controversial program, collecting bulk telephone metadata nationwide. We investigate the privacy properties of telephone metadata to assess the impact of policies that distinguish between content and metadata. We find that telephone metadata is densely interconnected, can trivially be reidentified, enables automated location and relationship inferences, and can be used to determine highly sensitive traits.

Since 2013, a stream of disclosures has prompted reconsideration of surveillance law and policy. One of the most controversial principles, both in the United States and abroad, is that communications metadata receives substantially less protection than communications content. Several nations currently collect telephone metadata in bulk, including on their own citizens. In this paper, we attempt to shed light on the privacy properties of telephone metadata. Using a crowdsourcing methodology, we demonstrate that telephone metadata is densely interconnected, can trivially be reidentified, and can be used to draw sensitive inferences.

Communications privacy law, in the United States and many other nations, draws a distinction between “content” and “metadata”. The former category reflects the substance of an electronic communication; the latter includes all other information about the communication, such as parties, time, and duration.

When a government agency compels disclosure of content, the agency must usually comply with extensive substantive and procedural safeguards. Demands for metadata, by contrast, are often left to the near-total discretion of authorities. In the United States, for instance, a law enforcement officer can request telephone calling records with merely a subpoena—essentially a formal letter from the investigating agency. An intelligence program by the National Security Agency (NSA) has drawn particular criticism; under the business records provision of the USA PATRIOT Act, the agency acquired a substantial share of all domestic telephone metadata.

In this paper, we empirically investigate factual assumptions that undergird policies of differential treatment for content and metadata. Using crowdsourced telephone logs and social networking information, we find that telephone metadata is densely interconnected, susceptible to reidentification, and enables highly sensitive inferences.

The balance of the paper is organized into three parts. First, we discuss our data collection methodology and properties of our participant population. We next present our results. Finally, we discuss implications for policy and future quantitative social science research.