**The Error of Equal Error Rates**

©1999 D.H. Kaye^{(*)}

The final version of this papers is published in Law, Probability & Risk, Vol. 1, No. 1, July 2002, at 3-8.

In most civil litigation, plaintiffs are required to prove their cases by a bare
"preponderance of the evidence," and this is usually taken to mean that the
probability that their version of the disputed events exceeds one-half. This p > ½
rule can be shown to minimize the expected "losses,"^{(1)}
"disutilities,"^{(2)}
"regret,"^{(3)} "costs,"^{(4)} or "errors"^{(5)} in verdicts that turn on these events.^{(6)} This analysis has been called the Bayesian
Decision Theory justification or explanation of the p > ½ rule,^{(7)} and it has been presented, perhaps too
cavalierly, as the "reigning theory" of the law's burdens of persuasion.^{(8)}

Some distinguished commentators have defended the p > ½ rule on other grounds. They
see it as advancing a different objective--allocating erroneous verdicts across plaintiffs
and defendants in equal numbers. In a rich and thoughtful textbook on the law of evidence,^{(9)} Professors Allen, Kuhns, and Swift identify
"the premises underlying the preponderance rule"^{(10)}
as follows:

The preponderance rule incorporates an underlying assumption concerning the participants in litigation: that plaintiffs as a class and defendants as a class generally ought to be treated equivalently. The reason for this assumption is that before a case is resolved, one cannot know who should win; it is as likely that the defendant should win as the plaintiff. . . . Without knowing the facts, it seems just as likely that the defendant is refusing to pay what is owed as that the plaintiff is attempting to obtain an undeserved benefit.

The preponderance of the evidence standard generalizes this basic point into a statement that the law should treat plaintiffs as a class and defendants as a class equivalently (with defendants and the status quo favored if there is a "tie" -- that is, if the evidence does not preponderate in either direction). As Professor Kaye has demonstrated algebraically, if certain conditions are met the preponderance of the evidence standard should result in about the same numbers of errors being made for plaintiffs as for defendants.^{(11)}

I fear that this passage misstates the underpinnings and workings of the burden of
persuasion.^{(12)} There is no obvious reason to
pursue equal numbers of actual errors, and even if that policy had a sound foundation, it
would tend to support departing from the p > ½ rule rather than adhering to it.^{(14)} Accordingly, the equal-error theory of the
civil burden of persuasion is singularly unpersuasive. To establish these points, I
proceed in three short, but decisive steps. Part I questions the premise that jurors do or
should presume that the prior probabilities for plaintiffs and defendants are equal. Part
II probes the connection between equal prior probabilities and the p > ½ rule.
Finally, Part III identifies the aspect of equality that underlies the p > ½ rule.

**I. Equality as Equal Prior Odds**

According to Professor Allen and his colleagues, the reason to treat plaintiffs and
defendant equally is that "before a case is resolved, one cannot know who should win;
it is as likely that the defendant should win as the plaintiff."^{(14)} It is difficult to know what to make of this
suggestion. It might be an empirical assertion that prior to introducing any evidence, the
probability that the facts are such that a plaintiff should prevail is one-half. This
would be so if "in the set of all cases going to trial there are approximately as
many deserving plaintiffs as deserving defendants."^{(15)}
However, no data are available to test this hypothesis,^{(16)}
and even if it were a fact, what would it prove? No obvious inference as to how a case
ought to be decided after all the evidence is in seems to flow from the fact that a case
is in equipoise before the trial begins.

Perhaps, then, the claim is not empirical, but normative.^{(17)}
Maybe Professor Allen and his colleagues are claiming that ignorance of the facts of a
case *should* mean that the probabilities are in equipoise. Again, no argument for
this claim is made, and none is apparent. Suppose plaintiff sues in tort, claiming that he
bought an automobile with a defective steering column that caused the vehicle to crash.
The defendant manufacturer, let us assume, denies only part of the complaint--that there
was a defect in the steering. Even if one knows nothing else about the case, one should
not assume that the car's steering wheel is as likely be to defective as it is to be
non-defective. Equal treatment does not mean ignoring the reality that most cars that are
involved in accidents do not have defective steering columns. Plaintiff may be able to
prove that his car had this unusual characteristic, but until he produces some admissible
evidence of that fact, a juror need not believe that the claim has a 50-50 chance of being
true. Laplace's principle of insufficient reason hardly seems compelling in this context.^{(18)}

**II. Equal Prior Odds and the p > ½ Rule**

Even if some convincing argument were available to show that jurors should enter the
jury box with prior probabilities of one-half, the statement that "the preponderance
of the evidence standard generalizes this basic point into a statement that the law should
treat plaintiffs as a class and defendants as a class equivalently"^{(19)} would remain mysterious. According to
Profesor Allen and his colleagues, the generalization consists of an article that
"demonstrated algebraically" that "if certain conditions are met, the
preponderance of the evidence standard should result in about the same numbers of errors
being made for plaintiffs as for defendants."^{(20)}
But the article contains no such demonstration.^{(21)}
Instead, it shows that in cases with a single plaintiff, a single defendant, and a single
disputed factual issue, the p > ½ rule minimizes the expected number of dollars that
come out of the wrong pockets.^{(22)}

Professor Allen and his colleagues misread the article, suggesting that its analysis
"can be understood without the mathematics," simply by assuming "that in
the set all cases going to trial there are approximately as many deserving plaintiffs as
deserving defendants," that jurors "will make a rough probability assessment of
the strength of each case," and that "the probability assessments for these two
sets are in a normal distribution over the range of 0.0 to 1.0."^{(23)} They think that "then the number of
errors made for plaintiffs will approximate the number of errors made for defendants, and
the preponderance of the evidence standard will have done its job."^{(24)} Of course, the conclusion does not follow
even from these artificial premises,^{(25)} and
the premises are neither used nor required to establish the expected-error-minimizing
quality of the p > ½ rule.^{(26)}

**III. Equality and the p > ½ Rule**

Although the equal-error analysis does not offer an adequate positive or normative
account of the p > ½ rule, this is not because the p > ½ rule neglects the norm of
equality. To the contrary, as Professors Allen, Kuhns, and Swift assert at the outset of
their discussion, the rule implements the principle of equal treatment of plaintiffs and
defendants. However, it does this not by allocating errors in any particular way, but
rather by attending to the expected loss associated with the decision rule. The basic
premise is not that we should equalize the numbers of errors of each type, but rather that
we should be equally concerned with each type of error. When the loss resulting from a
mistaken verdict for plaintiff is set equal to that for a mistaken verdict for defendant,
the p > ½ rule immediately follows.^{(27)}

This form of equality is appropriate because every error is equally serious--
regardless of who it favors or hurts--and because errors for different parties in
different cases do not cancel each other out. When these conditions hold, we should strive
to minimize the *total* number of errors without worrying about whose ox is gored
by any asymmetry in the distribution of errors.

In adopting and applying the p > ½ rule, therefore, the law treats plaintiffs and
defendants equally. The prior probabilities may not be equal, and the numbers of errors of
each type may not be equal. But to demand that the burdens of factual mistakes at trials
fall with mathematical equality on plaintiffs and defendants would be to pursue an
outcome-based egalitarianism that has no defensible foundation. By giving equal weight to
each type of error and letting the errors fall where they may, we afford plaintiffs and
defendants equal concern and respect.^{(28)}

NOTES

* Regents Professor, Arizona State University, College of Law.

1. D.H. Kaye, *Clarifying the Burden of Persuasion: What Bayesian
Decision Rules Do and Do Not Do*, 3 Int'l J. Evid. & Proof 1 (1999).

2. John Kaplan, *Decision Theory and the Factfinding Process*,
20 Stan. L. Rev. 1065 (1968).

3. Richard Lempert & Stephen Saltzburg, A Modern Approach to
Evidence 163 (2d ed. 1983); Richard Lempert, *Modeling Relevance*, 75 Mich. L. Rev.
1021 (1977).

4. Richard A. Posner, *An Economic Approach to the Law of Evidence*,
51 Stan. L. Rev. 1477, 1504 (1999) (asserting that the p > ½ rule follows from the
presumed fact that, on average, the costs of an erroneous verdict are equal as between
civil plaintiffs and defendants).

5. David Kaye, *Naked Statistical Evidence*, 89 Yale L.J. 601
(1980) (book review).

6. *See infra* note 26; authorities cited, *supra* notes
1-3 & *infra* note 7.

7. *See, e.g*., D.H. Kaye, *What is Bayesianism?*, *in*
Probability and Inference in the Law of Evidence: The Limits and Uses of Bayesianism 1
(Peter Tillers & Eric D. Green eds., 1988).

8. D.H. Kaye, *Apples and Oranges: Confidence Coefficients Versus
the Burden of Persuasion*, 73 Cornell L. Rev. 54 (1987).

10. Ronald J. Allen et al., Evidence: Text, Cases, and Problems 828 (2d ed. 1997).

10. *Id.*

11. *Id.* at 828. The reference is to David Kaye, *The
Limits of the Preponderance of the Evidence Standard: Justifiably Naked Statistical
Evidence and Multiple Causation*, 1982 Am. B. Found. Res. J. 487, reprinted in
Evidence and Proof (W. Twining & A. Stein, eds., 1992).

12. A related mistake appears in an early edition of the influential
textbook by Professor (now judge) Richard Posner. *See* Richard A. Posner, Economic
Analysis of Law 432 (2d ed. 1977) (asserting that the p > ½ rule "implies that of
cases decided erroneously, about half will be won by undeserving plaintiffs and about half
lost by deserving plaintiffs").

13. *See* Michael Finkelstein, Quantitative Methods in Law:
Studies in the Application of Mathematical Probability and Statistics to Legal Problems
(1978) (asserting that error-equalization sometimes justifies increasing the threshold of
the civil burden to some higher probability than one-half). This may be the authors'
point. *See* Allen et al., *supra* note 8, at 829 ("We can use these
same graphs to demonstrate why alternative, higher burdens of persuasion are occasionally
relied on in civil cases."). If so, however, the argument is odd. It begins by
asserting that equalization is the reason for a broadly applicable p > ½ rule, but
specifies a very narrow range of circumstances in which the p > ½ rule achieves
equality. If these difficult-to-satisfy conditions were the only such circumstances in
which the rule leads to equal error rates, however, the equalization rationale would not
be much of an explanation for the rule.

14. Allen et al., *supra* note 8, at 828.

15. *Id.*

16. Such data would have to include information from which to determine the true facts in each case--surely, a Herculean task.

17. *Cf.* Posner, *supra* note 4 (making such a claim).

18. *Cf.* David Kaye, *Playing Games with Justice: Rawls
and the Maximin Rule*, 6 Social Theory and Practice 33 (1980).

19. Allen et al., *supra* note 8, at 828.

20. *Id.*

21. Furthermore, in other writings I have contrasted the
expected-error-minimizing p > ½ rule to an expected-error-equalizing standard and have
disavowed the idea that error equalization should influence burdens of persuasion or the
use of statistical evidence. *See* Kaye, *supra* note 4; Kaye, *supra*
note 8, at 72-73; D.H. Kaye, *Hypothesis Testing in the Courtroom*, *in*
Contributions to the Theory and Application of Statistics (Alan E. Gelfand ed.,
1987).

22. By "dollars that come out of the wrong pockets," I mean the sum of (a) dollars paid to plaintiffs who should not have been compensated, and (b) dollars retained by defendants who should have compensated plaintiffs.

23. *Id.*

24. *Id.* at 828-29.

25. For example, normality is neither a necessary nor a sufficient
condition for error equalization. Allen et al. recognize the latter point when they speak
of "shaded areas under the two graphs [being] of approximately equal size." *Id.*
at 829. For a more precise statement of the conditions required for error equalization,
see Kaye, *supra* note 4.

26. *See* Kaye, *supra* note 1; *infra* note 26.

27. Although many derivations are available in the literature, it may
be convenient to provide a version tailored to make this point. Let *p* be the
probability that the true facts are such as to warrant recovery for plaintiff, and let *L*
be the loss associated with a false positive (a verdict of liability when the facts do not
warrant recovery) as well as that of a false negative (a verdict of no liability when the
facts do warrant recovery). Because 1-*p* is the probability that the facts do *not*
warrant recovery, and the expected loss is the loss discounted by the probability that it
will occur, the expected loss of a verdict for plaintiff is (1-*p*) × *L*.
Likewise, because p is the probability that the facts justify a verdict for plaintiff, the
expected loss of a verdict for defendant is *p* × *L*. Thus, the expected
loss of a verdict for defendant is greater than that of a verdict for plaintiff when *pL*
> (1-*p*)*L*, i.e., when *p* > ½. In all these cases, a
verdict for plaintiff minimizes the expected loss. Similar reasoning shows that in all
cases in which *p* < ½, a verdict for defendant minimizes expected loss. Hence,
the p > ½ rule readily follows from the assumption of the equality of the costs of
false positive and false negative errors. (The case of *p* = ½ has no effect on
minimizing expected losses; some supplementary argument, such as avoiding transaction
costs or a preference for the status quo, is required to choose between a p>½ rule and
a p ½ rule.)

28. As this phrase suggests, one might argue that if potential
plaintiffs and defendants were to choose, behind a veil of ignorance, between an
expected-error-minimizing and an expected-error-equalizing burden of persuasion, they
would adopt the former. *See* Ronald Dworkin, Taking Rights Seriously 150 (1977);
John Rawls, A Theory of Justice (1972).

*updated 9 March 2000*