The rise of social media has provoked both optimism about potential societal benefits and concern about harms such as addiction, depression, and political polarization. We present a randomized evaluation of the welfare effects of Facebook, focusing on US users in the runup to the 2018 midterm election. We measured the willingness-to-accept of 2,844 Facebook users to deactivate their Facebook accounts for four weeks, then randomly assigned a subset to actually do so in a way that we verified. Using a suite of outcomes from both surveys and direct measurement, we show that Facebook deactivation (i) reduced online activity, including other social media, while increasing offline activities such as watching TV alone and socializing with family and friends; (ii) reduced both factual news knowledge and political polarization; (iii) increased subjective well-being; and (iv) caused a large persistent reduction in Facebook use after the experiment. We use participants’ pre-experiment and post-experiment Facebook valuations to quantify the extent to which factors such as projection bias might cause people to overvalue Facebook, finding that the magnitude of any such biases is likely minor relative to the large consumer surplus that Facebook generates.The authors state
Social media have had profound impacts on the modern world. Facebook, which remains by far the largest social media company, has 2.3 billion monthly active users worldwide (Facebook 2018). As of 2016, the average user was spending 50 minutes per day on Facebook and its sister platforms Instagram and Messenger (Facebook 2016). There may be no technology since television that has so dramatically reshaped the way people communicate, get information, and spend their time. Speculation about social media’s welfare impact has followed a familiar trajectory, with early optimism about potential benefits giving way to widespread concern about possible harms. At a basic level, social media dramatically reduce the cost of connecting, communicating, and sharing information with others. Given that interpersonal connections are among the most important drivers of happiness and well-being (Myers 2000; Reis, Collins, and Berscheid 2000; Argyle 2001; Chopik 2017), this could be expected to bring widespread improvements to individual welfare. Many have also pointed to wider social benefits, from facilitating protest and resistance in autocratic countries, to encouraging activism and political participation in established democracies (Howard et al. 2011; Kirkpatrick 2011).
More recent discussion has focused on an array of possible negative impacts. At the individual level, many have pointed to negative correlations between intensive social media use and both subjective well-being and mental health.1 Adverse outcomes such as suicide and depression appear to have risen sharply over the same period that the use of smartphones and social media has expanded.2 Alter (2018) and Newport (2019), along with other academics and prominent Silicon Valley executives in the “time well-spent” movement, argue that digital media devices and social media apps are harmful and addictive. At the broader social level, concern has focused particularly on a range of negative political externalities. Social media may create ideological “echo chambers” among like-minded friend groups, thereby increasing political polarization (Sunstein 2001, 2017; Settle 2018). Furthermore, social media are the primary channel through which fake news and other types of misinformation are spread online (Allcott and Gentzkow 2017), and there is concern that coordinated disinformation campaigns can affect elections in the US and abroad.
In this paper, we report on a large-scale randomized evaluation of the welfare impacts of Facebook, focusing on US users in the run-up to the November 2018 midterm elections. We recruited a sample of 2,844 users through Facebook display ads, and elicited their willingness-to-accept (WTA) to deactivate their Facebook accounts for a period of four weeks ending just after the election. We then randomly assigned the 58 percent of these subjects with WTA less than $102 to either a Treatment group that was paid to deactivate, or a Control group that was not. We verified compliance with deactivation by regularly checking participants’ public profile pages. We measured a suite of outcomes using text messages, surveys, emails, direct measurement of activity on Facebook and Twitter, and administrative records on voting and electoral contributions. Less than two percent of the sample failed to complete the endline survey, and the Treatment group’s compliance with deactivation exceeded 90 percent.
Our study offers the largest-scale experimental evidence available to date on the way Facebook affects a range of individual and social welfare measures. We evaluate the extent to which time on Facebook substitutes for alternative online and offline activities, with particular attention to crowd out of news consumption and face-to-face social interactions. We study Facebook’s broader political externalities via measures of news knowledge, awareness of misinformation, political engagement, and political polarization. We study the impact on individual utility via measures of subjective wellbeing, captured through both surveys and text messages. Finally, we analyze the extent to which behavioral forces like addiction and misprediction may cause sub-optimal consumption choices, by looking at how usage and valuation of Facebook change after the experiment.
Our first set of results focuses on substitution patterns. A key mechanism for effects on individual well-being would be if social media use crowds out face-to-face social interactions and thus deepens loneliness and depression (Twenge 2017). A key mechanism for political externalities would be if social media crowds out consumption of higher-quality news and information sources. We find evidence consistent with the first of these but not the second. Deactivating Facebook freed up 60 minutes per day for the average person in our Treatment group. The Treatment group actually spent less time on both non-Facebook social media and other online activities, while devoting more time to a range of offline activities such as watching television alone and spending time with friends and family. The Treatment group did not change its consumption of any other online or offline news sources and reported spending 15 percent less time consuming news.
Our second set of results focuses on political externalities, proxied by news knowledge, political engagement, and political polarization. Consistent with the reported reduction in news consumption, we find that Facebook deactivation significantly reduced news knowledge and attention to politics. The Treatment group was less likely to say they follow news about politics or the President, and less able to correctly answer factual questions about recent news events. Our overall index of news knowledge fell by 0.19 standard deviations. There is no detectable effect on political engagement, as measured by voter turnout in the midterm election and the likelihood of clicking on email links to support political causes. Deactivation significantly reduced polarization of views on policy issues and a measure of exposure to polarizing news. Deactivation did not statistically significantly reduce affective polarization (i.e. negative feelings about the other political party) or polarization in factual beliefs about current events, although the coefficient estimates also point in that direction. Our overall index of political polarization fell by 0.16 standard deviations. As a point of comparison, prior work has found that a different index of political polarization rose by 0.38 standard deviations between 1996 and 2018 (Boxell 2018).
Our third set of results looks at subjective well-being. Deactivation caused small but significant improvements in well-being, and in particular on self-reported happiness, life satisfaction, depression, and anxiety. Effects on subjective well-being as measured by responses to brief daily text messages are positive but not significant. Our overall index of subjective well-being improved by 0.09 standard deviations. As a point of comparison, this is about 25-40 percent of the effect of psychological interventions including self-help therapy, group training, and individual therapy, as reported in a meta-analysis by Bolier et al. (2013). These results are consistent with prior studies suggesting that Facebook may have adverse effects on mental health. However, we also show that the magnitudes of our causal effects are far smaller than those we would have estimated using the correlational approach of much prior literature. We find little evidence to support the hypothesis suggested by prior work that Facebook might be more beneficial for “active” users—for example, users who regularly comment on pictures and posts from friends and family instead of just scrolling through their news feeds.
Our fourth set of results considers whether deactivation affected people’s demand for Facebook after the study was over, as well as their opinions about Facebook’s role in society. As the experiment ended, participants reported planning to use Facebook much less in the future. Several weeks later, the Treatment group’s reported usage of the Facebook mobile app was about 12 minutes (23 percent) lower than in Control. The Treatment group was more likely to click on a post-experiment email providing information about tools to limit social media usage, and five percent of the Treatment group still had their accounts deactivated nine weeks after the experiment ended. Our overall index of post-experiment Facebook use is 0.61 standard deviations lower in Treatment than in Control. In response to open-answer questions several weeks after the experiment ended, the Treatment group was more likely to report that they were using Facebook less, had uninstalled the Facebook app from their phones, and were using the platform more judiciously. Reduced post-experiment use aligns with our finding that deactivation improved subjective well-being, and it is also consistent with the hypotheses that Facebook is habit forming in the sense of Becker and Murphy (1988) or that people learned that they enjoy life without Facebook more than they had anticipated. Deactivation caused people to appreciate Facebook’s both positive and negative impacts on their lives. Consistent with our results on news knowledge, the Treatment group was more likely to agree that Facebook helps people to follow the news. The great majority of the Treatment group agreed that deactivation was good for them, but they were also more likely to think that people would miss Facebook if they used it less. In free response questions, the Treatment group wrote more text about how Facebook has both positive and negative impacts on their lives. The opposing effects on these specific metrics cancel out, so our overall index of opinions about Facebook is unaffected.
Our work also speaks to an adjacent set of questions around how to measure the economic gains from free online services such as search and media.
In standard models with consumers who correctly optimize their allocation of time and money, researchers can approximate the consumer surplus from these services by measuring time use or monetary valuations, as in Brynjolfsson and Oh (2012), Brynjolfsson, Eggers, and Gannamaneni (2018), Corrigan et al. (2018), and others. But if users do not understand the ways in which social media could be addictive or make them unhappy, these standard approaches could overstate consumer surplus gains. Sagioglu and Greitemeyer (2014) provide suggestive evidence: while their participants predicted that spending 20 minutes on Facebook would make them feel better, it actually caused them to feel worse.
To quantify the possibility that a period of deactivation might help the Treatment group to understand ways in which their use had made them unhappy, we elicited WTA at three separate points, using incentive-compatible Becker-DeGroot-Marschak (1964, “BDM”) mechanisms. First, on October 11th, we elicited willingness-to-accept to deactivate Facebook between October 12th and November 8th, which we loosely call “month 1.” We immediately told participants the amount that they had been offered to deactivate ($102 for the Treatment group, $0 for Control), and thus whether they were expected to deactivate over that period. We then immediately elicited WTA to deactivate Facebook for the next four weeks after November 8th, which we call “month 2.” When November 8th arrived, we then re-elicited WTA to deactivate in month 2. The Treatment group’s change in valuation for month 2 reflects a time effect plus the unanticipated effect of spending time off of Facebook. The Control group’s parallel valuation change reflects only a time effect. Thus, the difference between how Treatment vs. Control change their WTAs for deactivation in month 2 reflects projection bias, learning, and similar unanticipated experience effects, which we collectively call “misprediction.”
After weighting our sample to match the average US Facebook user on observables, the median and mean willingness-to-accept to deactivate Facebook for the initial four weeks were $100 and $180, respectively. These valuations are larger than most estimates in related work by Brynjolfsson, Eggers, and Gannamaneni (2018), Corrigan et al. (2018), Mosquera et al. (2018), and Sunstein (2019). Aggregated across an estimated 172 million US Facebook users, this could be interpreted to mean that Facebook generates several hundred billion dollars of consumer surplus per year in the US alone. Consistent with our other results that deactivation reduced demand for Facebook, deactivation caused month 2 WTA to drop by 13 percent, although this may be an upper bound on misprediction for reasons we discuss later. While such misprediction may be substantial in absolute terms, it would not reverse the conclusion that Facebook generates enormous flows of consumer surplus.
Our results should be interpreted with caution, for several reasons. First, effects could differ with the duration or scale of deactivation. A longer period without Facebook might have less impact on news knowledge as people find alternative news sources, and either more or less impact on subjective well-being. Furthermore, a larger-scale experiment in which a greater share of the population deactivated could have a different impact due to network effects and equilibrium adjustments. Second, our sample is not fully representative. Our participants are relatively young, well-educated, and left-leaning compared to the average Facebook user, and we included only people who reported using Facebook more than 15 minutes per day. In addition, although we went as far as possible to avoid telegraphing the experimental design and research questions, deactivation could have different effects on the average Facebook user than on the type of person who was willing to participate in our experiment. Third, many of our outcome variables are self-reported, adding scope for both measurement error and experimenter demand effects. This latter concern is mitigated somewhat by the fact that the non-self-reported outcomes we measure (e.g., post-experiment Facebook use) paint a similar picture to the survey responses.
The causal impacts of social media have been of great interest to researchers in economics, psychology, and other fields. We are aware of 12 existing randomized impact evaluations of Facebook.6 The most closely related is the important paper by Mosquera et al. (2018), which was made public the month before ours. They also use Facebook deactivation to study news knowledge and well being, finding results broadly consistent with those reported here. Appendix Table A1 details these experiments in comparison to ours. Our deactivation period is substantially longer and our sample size an order of magnitude larger than most prior work, including Mosquera et al. (2018). We measure impacts on a relatively comprehensive range of outcomes, and we are the only one of these randomized trials to have submitted a pre-analysis plan. Given the effect sizes and residual variance in our sample, we would have been unlikely to have sufficient power to detect any effects if limited to the sample sizes in previous experiments.
Sections 2 and 3 detail the experimental design and empirical strategy. Section 4 presents the impact evaluation, and Section 5 presents measurements of the consumer surplus generated by Facebook.