Information and Adjudication

'Information and the Aim of Adjudication: Truth or Consequences?' by Louis Kaplow in (2015) Stanford Law Review (Forthcoming) comments 

Adjudication is fundamentally about information, usually concerning individuals’ previous or proposed behavior. Legal system design is challenging because information ordinarily is costly and imperfect. This Article analyzes a broad array of system features, asking throughout whether design should aim at the truth or at consequences, how these approaches may differ, and what general lessons may be drawn from the comparison. It will emerge that the differences in approach are often large and their character is sometimes counterintuitive. Accordingly, system engineers concerned with social welfare need to aim explicitly at consequences. This message is not one opposed to truth per se but rather a strong admonition: it is dangerous to be attached to the alluring view that adjudication is primarily about generating results most in accord with the truth of the matter at hand.

Kaplow argues that in asking whether system designers should aim at the truth or at consequences

truth is not taken [in this article] as an abstract concept or a normative principle. Nor is it taken to have a unitary meaning in different realms. Instead, the notion is to be understood as a proxy criterion (or even a metaphor) that seems appealing in particular domains and thereby is often taken as an appropriate target by policy analysts when they assess pertinent elements of the legal system. This Article will assume, until the final Part, that system engineers are in fact concerned entirely with consequences for social welfare. The question then is whether, in undertaking their work, it is a plausible strategy to aim at the truth with the expectation that this protocol will ordinary lead to good consequences. Perhaps there are some moderate deviations, occasional exceptions, and limitations, but nevertheless the applicable notion of truth might typically provide a workable guide or at least a sensible starting point. A competing view is that system engineers need to focus quite explicitly on consequences themselves, largely setting truth to the side; indeed, perhaps they need to strive to ignore truth’s siren call.

The Article examines how these two methodologies differ and what general lessons may be drawn from the comparison. It will emerge that a large divergence often exists and that its character can be counterintuitive. Accordingly, system engineers really do need to concentrate on consequences. This message is not one that is opposed to truth per se but rather a strong admonition: it is dangerous to be attached to the alluring view that adjudication is primarily about generating results most in accord with the truth of the matter at hand.

To frame the inquiry, it is useful to consider the underlying reason that truth and consequences can produce such divergent prescriptions. Start in an idealized world in which it is possible to achieve perfect accuracy in adjudication—such that every outcome corresponds to the truth—at zero cost. In such a world, in most instances and with some further simplifications, social welfare would be maximized by going with the truth. Adjudication should aim at the truth, it would in fact hit the bull’s-eye every time, and the best consequences would be achieved. Harmful acts would always result in the appropriate sanction, benign acts would not be discouraged at all by the prospect of the mistaken imposition of sanctions, and system costs would be nonexistent. This is a first-best setting (or close to it), an analogue to a frictionless universe. Here, we do not see much reason for truth and consequences to conflict.

Our actual system design problem, of course, is in the world of the second best. Nontrivial system costs must usually be incurred to obtain even an approximation of the truth. Attempting to move closer is increasingly costly, and perfect truth is unobtainable. Should we nevertheless pretty much always aim at the truth?

The obvious answer is in the negative. The presence of costs alone tells us that we will need to make tradeoffs. Spending the entire GDP to get as close as possible to the truth in a single torts dispute would destroy society, not maximize social welfare. This simple point instructs us to avoid excess, but it fails to illuminate a course of analysis that can prescribe what moderation would look like and what it should depend on.

Moreover, once complete truth is off the table, we confront significant hurdles: the lack of an obvious metric for degrees of truth or of a way to place a value on truth units, whatever they may be. Inquiries are sometimes conducted as if there were some sort of Platonic truth measure, but little reflection is required to appreciate the need to dig deeper. More explicit treatments may invoke various criteria, such as the command to minimize the number of errors in adjudication or to aim at some ratio of true positives to false positives. Such guidelines, however, are ad hoc, and conflicting, and they can have absurd implications. For example, it is obvious that adjudicative errors are minimized by eliminating adjudication, and further analysis indicates that some proposed performance ratios are improved by raising the flow of innocent acts into the system (because such may well improve the system’s batting average). To foreshadow a bit, we can see from these examples that it is grossly insufficient to consider only what happens in adjudicated cases; underlying behavior and determinants of what enters the legal system will be central. The main purpose of the legal system is not for adjudication to look good according to some abstract standard but rather for its operation—including the anticipation thereof—to foster productive activity, restrain harmful conduct, and avoid undue expense. More fundamentally, such precepts, whether focused entirely on some notion of truth or on related considerations involving types of errors, are ungrounded. As already mentioned, the approach adopted in this Article is that legal system engineers should be guided by the maximization of social welfare. That is, adjudication should, in principle, aim at consequences. Whether, how, and to what extent truth is important will emerge in the course of the analysis. Any truth metric or valuation of truth will be a byproduct of the inquiry, not its driver. Aiming at the truth may sometimes be a good summary or proxy for part of what matters, but it is never the entire story (if for no other reason than cost) and is often a misleading guidepost. In complex systems, this sort of perspective is familiar. Indeed, even when the setting is simpler, one does not always aim directly as one would in an idealized world. A marksman might optimally aim high and to the left to account for distance and wind. But that involves just a modest refinement: the maxim that one should aim true is approximately correct. When building a road to the top of a mountain, however, aiming straight for the top—following the precept that the shortest distance between two points is a straight line—is a prescription for disaster. Switchbacks will be required, so that much of the time the road is actually going in the wrong direction by reference to the ultimate endpoint. Moreover, depending on the conditions, it might be best to go down, not up; around to the other side; and only then begin a zig-zagged ascent.

The foregoing should lead us to wonder whether adjudication design that aims primarily at the truth will perform poorly, but it does not tell us how worried we should be. That depends on whether this domain is more like a gradual incline with a few bumps or a treacherous mountainside with imposing obstacles. There are two general reasons to expect our challenge to be more like the latter.

The first is the presence of costs. Not only will we want to stop short of the top, but once we know this, it is impossible to determine how far to go without an explicit determination of the social value of moving closer. One can contemplate the meaning of truth until the end of days without illuminating that question. The value of truth in adjudication depends on its consequences, and valuing various outcomes is outside the realm of truth per se. As a comparison, how can we value an additional medical test of a stated precision without assessing the consequences of one or another course of treatment under different medical conditions? In this type of setting, truth is indeed something that matters, but it is only one element of a larger calculus. It is a start to recognize that tradeoffs must be made, but this recognition alone tells us little of their anatomy.

The second reason is that the design of adjudication in many settings influences behavior. We are centrally concerned about deterring harmful acts and avoiding the chilling of benign conduct. Such primary behavior, and also litigation itself, is endogenous; social welfare depends on the operation and feedbacks of the system as a whole. In such a complex and interactive environment, moving somewhat closer to truth in adjudication, by any simple metric, need not improve social welfare even without regard to costs. In addition, seemingly more expensive systems can be cheaper (for example, via deterrence, reducing the frequency of adjudication) and less expensive ones more costly. Due to these multiple and moving targets, the optimal design of adjudication may be more roundabout than building a road up a treacherous mountain: at least the mountain stands still.

This Article explores multiple dimensions of legal system design with a focus on information and, in particular, how information costs and limitations bear on the nexus between truth and consequences. Its scope is broad and, accordingly, the analysis is limited and often selective. In the process, however, we will see how key aspects of the interaction play out in many contexts and also identify some systematic similarities and differences across domains. As mentioned at the outset, one should keep in mind that the core message here is not anti-truth. Truth, after all, is usually the right guide in an idealized world, which suggests that sometimes we should expect it to point us in a good direction. As a motivation for analysis, at least, thinking about truth is useful. Moreover, truth may be consequential for social welfare for additional reasons. The claim throughout this Article is that careful analysis must aim at consequences, not at truth, because the dictates of truth are almost always seriously incomplete and often enough misleading that we must be careful not to have our imagination, investigation, and prescription distorted by an infatuation with truth in adjudication.

Part I begins by examining information and substantive legal commands because the purpose of adjudication is to effectuate substantive law. The analysis of information and the aim of adjudication are inevitably about the relationship between procedure and substance. Accordingly, it is important to start with central design features of substantive legal commands, specifically, those most directly implicating the core informational dimensions that will be emphasized later. Notably, primary behavior is influenced by the expected consequences of adjudication, which accordingly is the core information about the legal system, procedure and substance, that proves to be consequential. Of course, administrative costs matter as well. Specifically, Part I addresses the two dimensions of a taxonomy employed in some of the literature. One involves the distinction between rules and standards, wherein a rule for this purpose refers to specification of the content of a legal command ex ante, before parties engage in the primary behavior governed by the pertinent law, whereas under a standard the content is determined ex post, in adjudication, after parties have acted. The second dimension involves the precision—specificity or level of detail—with which the legal command is given content: that is, the extent to which it makes finer distinctions rather than placing conduct in broader categories. Both dimensions implicate information in several ways. They obviously govern the intensity of effort (and thus cost) of supplying legal content both ex ante and ex post. Moreover, they influence the law’s consequences for individuals’ behavior in the interim because such behavior depends critically on the extent to which individuals choose to become informed about the law before they act. One of the recurring themes of this Article emerges in both analyses: truth at the conclusion of adjudication—understood in this Part to refer to the alignment of outcomes with the substantive ideal—does not directly translate into ex ante behavior in conformance thereto. Indeed, the gulf can be wide: a regime closer to the truth in adjudication can result in individuals’ actions in the world being less in accord with it than under an optimal regime aimed at consequences.

Part II analyzes the treatment of errors in adjudication—conventionally understood as setting a burden of proof or other decision threshold—taking as given the quality of information, a matter deferred to Part III. Examined first is a simpler context, set to the side in much of this Article, in which adjudication concerns the regulation of proposed conduct: license applications, zoning variances, or the approval of mergers or new drugs. Here, the optimal evidence threshold involves standard cost-benefit analysis for decisions under uncertainty, just as in the medical testing illustration above. The truth of the matter—the likelihood that the applicant, say, proposes an activity of a harmful rather than of a benign type—is certainly relevant, but one must also consider the possible social gains and losses from the proposed activity. When harm is great and the benefit of the benign act is small, prohibition is optimal even when the likelihood that the proposed action is a harmful one is low, and conversely when harm is slight and benign activity is highly beneficial, even though the truth of the matter is that it is most probably harmful. Better to make many mistakes of little consequence than a few that are momentous. In this basic setting, truth is an element of consequences but it is far from the entire story. Next, the analysis returns to the setting examined in most of the Article, where individuals’ actions (torts, contract breaches, and so forth) precede adjudication, in which case their behavior is influenced by their anticipation of outcomes in adjudication. Here, truth—in the sense of the likelihood that the individual before the tribunal in fact committed a harmful act—is not even a component of the more elaborate calculus that determines what evidence threshold is optimal with regard to the consequences it engenders. The explanation is that the truth of the matter at hand concerns the static, descriptive question of how best to characterize the case before the tribunal, whereas the consequences of setting an evidence threshold somewhat higher or lower turn on the dynamic question of how such a modification would change individuals’ ex ante behavior. As suggested previously, the endogeneity of behavior can greatly obscure the relationship between truth and consequences. In general terms, this phenomenon carries over to the determination of optimal decision criteria at earlier stages of adjudication, including formal pretrial terminations and the informal conduct of investigations, such as by government agencies. Part II also examines a particular relationship between procedure and substance, namely, whether concerns for errors involving the mistaken imposition of sanctions on (or application of prohibitions to) benign conduct are best dealt with through restrictions on substantive legal commands or with more demanding decision criteria in adjudication. The latter tends to be favored on informational grounds because raising, say, the burden of proof tends to remove the weakest cases from the system. In this instance, welfare is best advanced by keeping substantive law aligned with truth, in the sense of conformity to the substantive ideal, and relegating any needed adjustments to the burden of proof, perhaps by moving it in a direction less aligned with truth in the sense of whether outcomes accord with the fact of the matter in the case under adjudication.

Part III shifts the focus to the accuracy of adjudication. Because greater accuracy—attempting to move closer to the truth—comes at a cost, it is necessary to place a value on accuracy, which can be done only by assessing its consequences. The value of accuracy varies greatly on a number of dimensions. Raising the degree of accuracy in the determination of liability improves the error tradeoff that was taken as given in Part II’s discussion of evidence thresholds and accordingly has a social value that reflects the corresponding consequences. Improving the accuracy with which damages are assessed has a relationship to ex ante behavior that is similar to that of making substantive legal commands more precise or refined. Specifically, the value of moving closer to the truth is not automatic but depends on the extent to which individuals will anticipate and thus react to the greater precision of adjudication. Interestingly, in some important settings, greater accuracy will be a pure waste of resources because the added ex post specification—concerning, for example, just how much a particular auto accident victim’s future earnings are diminished by an injury—cannot plausibility be predicted ex ante. To the extent that more accurate damage awards improve the precision of compensation for risk-averse victims, accuracy has social value, in this instance measured by a risk premium. In all, whether the truth has any consequences at all and the social value of the consequences it does have vary greatly by the issue, the context, and in many settings the information possessed by actors before adjudication occurs.

The relationship between truth and consequences becomes even more complex and in some instances further attenuated when the analysis is extended to take account of the endogeneity of behavior in adjudication: which cases are pursued and how much information parties choose to generate. Part IV examines some of the possibilities that may arise. For example, private litigation is initiated when plaintiffs anticipate an expected recovery in excess of their costs. Anything that influences the costs or outcomes of adjudication affects these decisions, which in turn have feedback effects on primary actors’ behavior that, as emphasized repeatedly, is predicated on their own expectations about adjudication, including how often it will occur. As one indication of the potential implications, it is explained that a reform that lowers the evidence threshold, so plaintiffs win more often, and simultaneously imposes a filing fee, to an extent that (altogether) keeps deterrence constant, will in some settings lower system costs and also reduce the extent to which benign conduct is chilled. That is, a concern for errors involving the mistaken imposition of liability may favor a reform that reduces, not raises, the evidence threshold. This situation arises when litigants possess information superior to the tribunal’s. Truth in the outcomes of adjudication can be less consequential than the knowledge that motivates self-interested plaintiffs’ filing decisions.

Part IV also considers parties’ incentives to generate information in adjudication. In certain key settings—notably, some of those explored in Part III—litigants’ incentives are excessive. Their private benefit from influencing the outcome favorably, even though truthfully, exceeds the social value, so the overall consequences for social welfare can be negative, precisely because too much truth is generated. Accordingly, one mechanism that might address the problem involves the tribunal being committed to ignore some truthful information that parties might present. In this instance, aiming at truth is directly in opposition to good consequences. The final two Parts of the Article step back from the system-design methodology employed thus far to consider a broader perspective on the legal system’s objectives. First, continuing to assume that the purpose of the legal system is to advance social welfare, Part V asks whether there are additional consequences of the degree to which adjudication generates the truth, specifically, consequences regarding perceived legitimacy, abuse of power and corruption, participation and other process values, and preferences for the truth per se. Then, Part VI briefly examines the pursuit of truth independent of its consequences for social welfare.

This Article does not attempt to be exhaustive either with regard to all the ways that truth and consequences may or may not diverge from each other, taking an optimal system design perspective,1 or all the reasons that aiming at the truth may be important after all. It seeks to present enough of the former to illustrate the range of possibilities and to demonstrate the significance of potential divergences, and to discuss enough of the latter to instigate further reflection. Information is indeed central to adjudication, and the wide variation and complexity of the subject indicate the need for explicit, ground-up analysis that clearly focuses on the system’s objectives rather than an approach that starts in the middle and employs an appealing 1Brief remarks are in order regarding two familiar categories of truth/consequences divergence. First, there are isolated pockets of recognized exceptions. For example, the exclusionary rule and the requirement of proof beyond a reasonable doubt are usually seen as deviations from the norm that are justified by special considerations. See infra note 109. Second, as already noted in this Introduction, the need for cost tradeoffs is widely acknowledged, as reflected in many features of system design, from judges regulating the length of trials to enforcers employing randomized rather than dragnet strategies (traffic control and tax audits). Cost tradeoffs motivate Part III’s analysis of how to place a value on truth, which requires a systematic tracing of consequences that, as mentioned, is not much illuminated either by invocations of truth or by concessions of the need for moderation in light of costs. but potentially misleading proxy criterion: aiming at truth.