Counting gigs: How can we measure the scale of online platform work? by Agnieszka Piasna (European Trade Union Institute Working Paper 6/2020) comments
The potential transformation of labour markets by the emergence of online labour platforms has triggered an intense academic, media and policy debate, but its true scale remains speculation. Nevertheless, adequate policy responses hinge on a good understanding of dynamics – something that will only grow in importance with the labour market crisis created by the COVID-19 pandemic. With technologically enabled remote work, growing demand for services such as food delivery or care, as well as rising unemployment and the financial strain on many workers, platform work may resume its rapid growth. Therefore, there is a need for good quality data on the prevalence of platform and other forms of precarious work in society.
This working paper provides a critical assessment of different approaches to counting gigs; that is, estimating the scale of engagement in platform work in the general population. The aim is to examine the main obstacles encountered in previous studies, the reasons for surprising or contradictory results and possible sources of error, but also the lessons that can be learned for future research. This is illustrated with key research in this area, ranging from large projects conducted by national statistical offices to smaller scale independent research, from national to (nearly) global scale.
Piasna notes
Over recent years, the emergence of online labour platforms that use digital technologies to match workers with clients on a per-task basis has sparked an intense debate about their economic and social implications. Research in this area has exploded equally rapidly, primarily in the form of qualitative or case study investigations, on the issues that are most captivating of the imagination, such as algorithmic management, extremely flexible work models, the dismantling of long fought-for worker protections, legal cases or worker struggles (for example, Berg and De Stefano 2017; Drahokoupil and Piasna 2019; Graham et al. 2017; Vandaele et al. 2019; Wood et al. 2019). However, little is still known about the true scale of the phenomenon of platform work which is especially puzzling given that, as opposed to the traditional informal sector, all transactions mediated by online platforms are digitally recorded. Thus, questions on the proportion of workers engaged in platform work, whether they differ from the general workforce and the countries in which they are more common, remain largely unanswered (Codagnone and Martens 2016; Healy et al. 2017). Existing official labour market statistics are not well- suited to measuring the online platform economy as they are generally not sufficiently sensitive to capture sporadic or secondary employment, while they also fail to distinguish it from other economic activities. Ad hoc modules added to national employment surveys tend to use very different questions and are thus difficult to compare, while rare cross-national surveys provide such divergent results that they raise even more questions than they set out to answer (see discussion in Piasna and Drahokoupil 2019).
This paper provides a critical assessment of the different approaches to counting gigs, seeking to come to an estimation of the scale of engagement in platform work within the general population (see also Piasna 2021). The aim is to examine the main obstacles which have previously been encountered, the factors which explain surprising or contradictory results, and the potential errors involved, but also to explore the lessons learned for future research. This is illustrated with key research studies in this area, conducted by national statistical offices and independent researchers, and on a national and (nearly) global scale. The analysis ranges from various examples of the use of secondary data, produced in abundance by simple virtue of the operations of the platforms, to the collection of primary data through dedicated surveys. It is not an exhaustive review of all the studies carried out to date, but rather an analytical review of various approaches illustrated with a selection of examples.
The paradox in measuring the platform economy is that, although its opera- tions generate a wealth of data, with all transactions being digitally recorded, one of the biggest unknowns is still the scale of platform work (Codagnone et al. 2016). Every gig mediated by online labour platforms leaves a digital trace containing information such as the nature of the task, the compensation pro- vided, the number of hours worked or tasks completed, and the identity both of the requester or client and of the worker. A good starting point for a review of methods for measuring the platform economy are thus initiatives that have attempted to access such data, either directly from the platforms themselves or by tapping into other sources of big data generated by their operations.
In general, platforms are highly protective of their proprietary databases on work and compensation flows and thus research that uses such data is scarce. One of the early examples is a study by Hall and Krueger (2018), who used anonymised administrative data from Uber on the number of drivers and their work histories, schedules and earnings covering the period 2012–2014 in the US market. Its strength undoubtedly lies in charting in great detail the extent of work for one of the largest platforms. However, as the study was carried out at Uber’s request and one of the authors worked for Uber Technologies at the time, it remains unattainable for independent researchers to replicate such an analysis over time or in other countries. Another example of the use of administrative data is a study of Deliveroo riders in Belgium carried out by Dra- hokoupil and Piasna (2019). In this case, a rare opportunity to access comprehensive administrative records containing information on hours worked and the pay, age, gender and student status of workers was based on co-operation with SMart, an additional intermediary that hired Deliveroo riders and billed the platform on their behalf. However, Deliveroo ended its agreement with SMart soon after the research was carried out, so such data collection cannot now be repeated.
Insofar as access to the administrative records of one platform provides the precise number of workers on that particular platform, and usually allows the separation of registered users from active ones, it can serve as a basis for estimates of the size of the platform economy at national level. Nevertheless, such estimates are extremely rough. A complete picture of the platform workforce would require information from all platforms and some indication on the scale of overlap; that is, how many workers are registered on more than one platform (for example, Aleksynska et al. 2019 showed that, among platform workers in Ukraine, only about one-quarter of those registered were in fact active while many were registered on several platforms). As this is currently unattainable, other sources of data can be used to impute missing information. Kuek et al. (2015) complemented the publicly available data disclosed by online labour platforms with expert interviews; while Harris and Krueger (2015) supplemented data from Uber on the number of workers with the fre- quency of Google searches for the names of selected labour platforms. Their approach rested on the assumption that the number of workers providing services through a platform is proportionate to the frequency of its Google searches, even though the latter may be driven by a variety of factors, includ- ing media interest, litigation or academic research, and are likely to be skewed in favour of the most recognised platforms. Nonetheless, Harris and Krueger’s (2015) conclusion that labour platforms accounted for 0.4 per cent of total employment in the US was very close to the results from other studies of that period.
Digitally mediated transactions also leave records outside the platform, such as in financial institutions or, at least in theory, in tax records. A rare example of the use of tax returns data is a study by Collins et al. (2019), tracing independent work mediated by the 50 biggest online labour platforms in the US between 2010 and 2016. It revealed that, by 2016, about one per cent of the US workforce registered income from platform work, even though it could not, by design, include informal revenues and those falling below a certain threshold. An interesting illustration of the use of financial records is a report by Farrell and Greig (2016) from JPMorgan Chase Institute. Having access to a full database of the clients of a major bank in the US, they counted how many accounts received any payments from one of 30 online platforms (ex- panded to include 128 platforms in a follow-up study by Farrell et al. (2018)). Their analysis revealed that, by 2015, one per cent of adults earned income from online platforms in the current month (0.4 per cent on labour platforms) and 4.2 per cent had done so in the past three years. The clear advantage of such approaches lies in the large number of platforms that can be included in the analysis and the possibility of replicating and repeating measurements over time. However, such studies will miss payments not coming directly from platforms’ accounts (i.e. through PayPal or Amazon vouchers) and, in the case of bank records, produce data not strictly at an individual level as families may have joint bank accounts, also raising ethical concerns where data are used without clients’ explicit consent.
Another approach to gathering the data produced by platforms, which in principle is not contingent on access to exclusive sources such as banks and does not raise ethical concerns, is web ‘scraping’ – automatically accessing and downloading publicly available data from the platform’s web user inter- face. The most comprehensive initiative of this sort to date is probably the Online Labour Index (OLI) produced by the Oxford Internet Institute (Kässi and Lehdonvirta 2018). The index tracks in near-real time the number of new vacancies (i.e. projects or tasks) posted on five major English-speaking online labour platforms. It is possible to determine from which country the vacancy was posted and in which occupational category it falls, while continuous up-dating of the figures provides a consistent time series. However, as the OLI and other similar projects (see, for example, Ipeirotis 2010) count posted job offers and not the number of workers completing them, they might confuse an increasing fragmentation of tasks for an increase in the size of the plat- form economy. It is also difficult to grasp the actual extent of platform work without information on compensation for posted tasks, as single tasks can vary greatly in the amount of labour input required and pay levels, while some tasks might also be completed by multiple workers. Finally, the authors of the OLI acknowledge that this measure of online labour utilisation is incomplete as it fails to capture all new vacancies, and thus they choose to present it as an indexed trend rather than in terms of the absolute numbers of vacancies. Consequently, while valuable in mapping trends in online gig work and its occupational heterogeneity, the OLI does not provide answers to the scale of platform work.
Therefore, the use of secondary data generated by platforms’ operations seems a good way to sketch the contours of the platform economy, although it is not best suited for mapping the prevalence of platform work at an individual (worker) level. To investigate how widespread are experiences with platforms, how often and to what extent individuals engage in platform work and the role of this type of work in supporting their livelihoods, a collection of primary data is necessary. ....
