The Analysis of degree classifications over time Changes in graduate attainment from 2010-11 to 2017-18 report states
The proportion of UK-domiciled, full-time first degree graduates attaining a first class honours degree from an English higher education provider has increased from 16 per cent in 2010-11 to 29 per cent in 2017-18, an overall increase of roughly 80 per cent over the period. For the same graduate population, the proportion attaining a first or an upper second class degree has increased from 67 per cent in 2010-11 to 79 per cent in 2017-18.
In December 2018, the Office for Students (OfS) published ‘Analysis of degree classifications with time: Changes in graduate attainment’ (OfS 2018.54), which uses statistical modelling to investigate changes in the proportions of graduates attaining first or upper second class degrees over the academic years from 2010-11 to 2016-17 . The UK Standing Committee for Quality Assessment also published an analysis investigating this issue in November 2018 . Both reports concluded that the growing proportion of the first and upper second class degrees awarded cannot be fully explained by factors linked with degree attainment, whether graduate or higher education provider characteristics.
This report builds on OfS 2018.54 by expanding the time and the domicile scope of the graduate population considered to all UK-domiciled graduates who qualified in academic years from 2010-11 to 2017-18 inclusive. Improvements in classifying the equivalent entry qualifications of individuals have also been made. These changes mean that the results presented in this report differ in some cases from those published in OfS 2018.54. While the results in each report are thus not directly comparable, the changes do not affect the overall conclusions of either report.
Our new analysis finds that in 2017-18, across the 148 providers considered, 13.9 percentage points’ worth of first class degree attainment is unexplained4 by changes in the graduate population since 2010-11, an increase of 2.4 percentage points from the unexplained attainment in 2016-17.
In respect of first class degree attainment, the new analysis shows that in 2017-18:
a. 71 per cent of providers (105 providers) show a statistically significant unexplained increase relative to both the sector and their own level in 2010-11. In 2016-17, 55 per cent of those providers (82) showed a statistically significant unexplained increase relative to both the sector and their own level in 2010-11.
b. A further 8 per cent of providers (12) show a statistically significant unexplained level of attainment above that of the sector in 2010-11, but no significant change relative to their own level in 2010-11. In 2016-17, this number was 16 per cent (23).
c. Another 15 per cent of providers (22) show a statistically significant unexplained increase relative to their own level in 2010-11, but attainment not significantly above the sector level in 2010-11. In 2016-17, this number was 18 per cent (26).
d. The remaining 6 per cent of providers (nine) showed no significant unexplained increase in attainment relative to their own level in 2010-11 and attainment not significantly above that in the sector in 2010-11. In 2016-17, this number was 11 per cent (17).
We performed additional analysis to try to address comments made after the release of OfS 2018.54. Its findings include:
a. Sector-level results of additional modelling where ‘additional contextual variables’ for graduates (ethnicity, sex, declared disability status and Participation of Local Areas (POLAR4) quintile) are omitted as explanatory variables in the modelling.
b. The effect of closing (reducing to zero) attainment gaps between individuals of differing sex, ethnicity, declared disability status and POLAR4 quintile on the sectorlevel unexplained attainment of first or upper second class degrees combined, and first class degrees alone. In a hypothetical sector where attainment gaps within the aforementioned groups do not exist the sector-level unexplained attainment of first and upper second class degrees combined in 2017-18 is estimated to be 6.0 percentage points (compared with 13.3 percentage points for the sector where attainment gaps exist). In this same scenario, for first class degrees alone the unexplained sector-level attainment in 2017-18 is estimated to be 10.9 percentage points (compared with 13.9 percentage points for the sector where attainment gaps exist).The OfS comments
This report sets out the results of our analysis of changes in the proportion of first and upper second class degrees awarded between 2010-11 and 2017-18. We report on how graduate attainment has changed over this time period, and the extent to which these changes can be accounted for by changes in certain characteristics of the graduate population. This analysis has been undertaken at both the sector level and the provider level. The sector-level analysis and a summary of the provider-level analysis are in the main body of this report, with detailed results of the provider-level analysis available separately (in Annex A and in full provider tables available at www.officeforstudents.org.uk/publications/analysis-of-degree-classifications-over-time/).
The graduate population considered in this report comprises UK-domiciled first degree graduates who studied full-time, were registered at higher education providers in England and graduated in the academic years from 2010-11 to 2017-18.
We further limit the graduate population included in this analysis to only include those who qualified from English providers awarding at least 10 classified honours degrees in each of the academic years considered, in total including 1,954,445 graduates from 148 providers (hereafter collectively referred to as ‘the sector’, for the purposes of this report only). Comparisons of key aspects of this population with the population of graduates from all English providers are presented in Annex B.
We have used statistical modelling of individual-level graduate data to predict expected patterns in degree classification attainment between 2010-11 to 2017-18, accounting for following graduate characteristics (explanatory variables): the provider at which the graduate was registered year of graduation subject studied qualifications on entry into higher education age additional contextual variables: declared disability status ethnicity sex Participation of Local Areas (POLAR4) quintile.
We present sector-level results from modelling where the additional contextual variables have been included (the ‘full model’) and omitted (the ‘simplified model’) as explanatory variables (see Annex D for details).
The modelling predicts little variation in the proportion of students attaining first and upper second class degrees between 2010-11 and 2017-18, meaning that the sectorlevel increase of 13.6 percentage points in first class degree attainment over this time period is considered unexplained by these factors alone.
The term ‘unexplained’ in this context means that changes in attainment over the time period cannot be accounted for by changes in the characteristics of the graduating cohort in terms of the explanatory variables included in the statistical modelling. It is not possible to deduce from this analysis what factors not included in the modelling (such as improvements in teaching quality, more diligent students or changes to assessment approaches) are driving the observed changes in degree attainment.
We first present a sector-level overview of the changes in the observed proportion of graduates who attained a first or an upper second class degree, and of those who attained a first class degree, in the academic years 2010-11 to 2017-18. We then disaggregate these changes and present the attainment of graduates by qualifications held on entry into higher education.
We then present sector-level findings from statistical modelling and the associated analysis that allows us to determine how much of the observed increase in attainment of first and upper second degree classifications over time can be attributed to changes in the characteristics (explanatory variables) of the graduate population, and how much remains unexplained, as defined in paragraph 13.
Next, we present estimates of the unexplained attainment of first and upper second class degrees combined, and of first class degrees alone, in a ‘hypothetical’ sector where attainment gaps between individuals of differing sex, ethnicity, declared disability status and POLAR4 quintile do not exist in the academic year 2017-18 (see Annex C for details).
. Finally, summaries of the provider-level analysis are presented (based on the full model, see paragraph 11), where for all academic years and for first and upper second class degrees combined, and for first class degrees alone, we have flagged providers where the unexplained attainment levels are statistically significantly below or above, firstly the average level of the sector in 2010-11, and secondly the level at the same provider in 2010-11.'Law Schools and the Burden of Bureaucracy: Release the Yoke (A Plea from the Coalface). Part 1: Over-regulation in Australia' by Olivia Rundle and Lynden Griggs in (2019) 93 Australian Law Journal 389 argues
Compliance work is stifling the capacity of Australian law schools and legal academics to engage in the core activities of research and teaching. There are multiple bodies who accredit, monitor and/or impose reporting requirements upon legal educators, and the legal profession is involved in more than one of these regulatory systems. In this two-part article, we make a plea from the coalface of academia, that the burden of regulation, certification, and accreditation of law schools and law teaching be lightened. In the first part, which is this article, we overview the legal profession’s role in legal education, the current and proposed Law Admissions Consultative Committee (LACC) regulatory systems, and critique the LACC Standards. In Part 2, (a future article), we draw from international experiences to compare with the Australian regulatory regime and make a proposal for change.The authors comment
Law schools are drowning in bureaucracy. Multiple bodies accredit, certify, regulate and/or monitor the “what” and “how” in Australian legal education. Currently imposed on law faculties include: compliance with the Higher Education Standards Framework (HESF) undertaken via the Tertiary Education Quality Standards Agency (TESQA), the Australian Qualifications Framework (AQF), the Threshold Learning Outcomes for Law (TLOs, with this incorporating the 11 core topic areas that must be successfully completed before someone can enter the legal profession), the standards of the Council of Australian Law Deans (CALD), internal reporting to School, Faculty, College and/or institutional administrators, and jurisdictional specific requirements of legal admitting authorities in the States and Territories of Australia as well as meeting the demands of overseas jurisdictions where this forms part of the student cohort. Yet another layer of monitoring and oversight is currently under trial by the Law Admissions Consultative Committee (LACC) through the application of its Standards. In the writers’ view, this level of duplication and overlay is unnecessary, and results in a diversion of resources from the core functions of a law school – teaching and research. It is a problem not unique to law. As noted in a report prepared for the Higher Education Standards Panel on Professional Accreditation, “[T]he aggregate effect of coping with idiosyncratic and excessive or unreasonable demands for information and compliance from some accrediting agencies is significant, expensive and problematic.” The detrimental effect of duplication of regulatory processes is magnified by both the lack of conformity and overlap in the subject matter of accreditation assessments. For example, some accreditation bodies focus upon subject areas taught, while others are concerned with the skills acquired by graduates, processes of teaching, qualifications of teachers, and law school environmental factors. The cycle of accreditation also differs from each authority. This results in the 11 core subject areas of legal study and TLOs being monitored several times across separate regulatory systems applying different criteria for assessment.
Because of this the regulatory system for legal education in Australia suffers from lack of co-ordination, unnecessary duplication and lack of clarity about the distinction between certification, accreditation and monitoring. This has a detrimental effect on law school resources; on people in decanal roles in law schools whose time and effort is redirected from other activities, as well as individual law lecturers, whose teaching practice is affected by demands that they evidence how, what, and why they facilitated student learning in the way they did. The coalface legal academic is burdened by extreme research performance expectations yet expected to respond as part of big picture compliance requirements, to requests for unit reports, course reports, student evaluation and peer review feedback, graduate quality statements, cohort engagement activities, course experience outcome questionnaires, as well as less formal student-initiated instruments, as part of their teaching performance expectations.
This article is in two parts. Part 1 (this article) critiques the LACC Standards in particular, within the context of the relationship between the legal profession and academe, and the other regulatory frameworks that already apply to Australian law schools. We focus on the LACC standards as they are the most recent of the bureaucratic impositions, yet curiously, the legal academe has remained largely silent to their introduction. That silence is something we seek to redress with our aim to begin a discourse as to how all stakeholders can better understand and be informed by the interests and expectations of others, as well as seeking to promote a collaborative and consensus based model to reduce the burden of compliance. Part 2 (our next article) compares the regulation of legal education by the legal profession in international jurisdictions and proposes a way forward for LACC’s role in the regulation and accreditation of Australian law degree programs. In saying this there will always be a role for collaboration between the legal profession and the academe about legal education. That relationship ought to be a constructive and co-operative one, grounded in mutual respect and trust. We urge a renegotiation of what is necessary in regard to the monitoring of qualifications held by people seeking to enter the Australian legal profession. Our suggestion is that all parties need to work together to forge a consensus to alleviate this bureaucratic burden and free the coalface academic (as well the departmental leaders within law faculties), together with the numerous agencies, from the yoke of unnecessary compliance.
In this Part 1, we start by examining the legal profession’s influence on legal education in Australia, historically, at present, and into the future. We will then overview the existing regulatory and accreditation frameworks before focusing in detail upon the LACC Standards.In their 'Law Schools and the Burden of Bureaucracy: Release the Yoke (A Plea from the Coalface). Part 2: International Comparators and a Proposal' in (2019) 93 Australian Law Journal 499 they state
we draw from international experiences to compare the Australian regulatory regime; we then focus upon the underlying needs of the legal profession and law schools, and make a proposal for change. Legal academics must be released from the burden of unnecessary and duplicative administrative work. Law schools must retain ultimate control over what and how law is taught. The legal profession has a collaborative and constructive role to play with the relationship between the academe and the profession grounded in trust and respect.
In Part 1 of this article, we explored in detail the regulation of law degree programs by government bodies, higher education authorities, and the legal profession. Our particular focus was the proposed Standards of the Law Admissions Consultative Committee (LACC). We questioned whether they were necessary given the existing compliance framework, and we also explained the burden that they would place upon law schools if the accreditation regime was widely implemented. We concluded that the resource implications of this proposal for both LACC and law schools would be enormous. Given the already established certification and monitoring to which law schools are subject, we had serious reservations about the need for these standards. But our identified problem needed to be married with a solution; and this is what Part 2 does. We outline a proposal which we believe delivers what the profession seeks to do, but which leaves law schools and the quality assurance programs of the Higher Education Standards Framework (HESF) as the means by which law school teaching and research can be judged. In structuring this part, we first focus upon the ways that the legal profession has exercised a regulatory role over legal education in international jurisdictions of the United States, England and Wales, Canada, India and Hong Kong. We identify some ways that the current Australian regime differs from those comparative experiences. We then conduct a needs analysis of both the legal profession and law schools in relation to the regulation of legal education. Finally, we present our proposal to address the problems identified in Parts 1 and 2 of our article.
