Adversarial Economics in
Jurimetrics, The Journal of Law, Science, and Technology
Volume 43, No. 3, Spring 2003, pp. 343-352
© 2003 D.H. Kaye. The author served as a consultant to attorneys representing United States Tobacco Co., Inc., in the appeals described here.
ABSTRACT: This article describes a questionable statistical study used to estimate large antitrust damages in United States Tobacco Co. v. Conwood Co., 290 F.3d 768 (6th Cir. 2002). It suggests that the district and appellate courts did not recognize that the study was incapable of separating the effects of legal conduct from illegal conduct and therefore failed in performing their role as gatekeepers for scientific evidence.
CITATION: D.H. Kaye, Adversarial Econometrics in United States Tobacco Co. v. Conwood Co., 43 Jurimetrics J. 343-352 (2003).
In 1820, Chief Justice Lord Dallas summarized the testimony of England's leading chemists and technologists as to the safety of a new process for refining sugar in the following lament: "It must be a matter of general regret to find the respectable witnesses . . . drawn up, not on one side, and for the maintenance of the same truths, but, as it were, in martial and hostile array against each other." (1) He advised the jury that "the two days during which the results of [the chemical] experiments had been brought into comparison, were days, not of triumph, but of humiliation to science." (2) <---------- p. 344 ---------->
In the past decade, the U.S. Supreme Court responded to the "humiliation" of adversarial science in three important decisions. In 1993, the Court in Daubert v. Merrell Dow Pharmaceuticals, Inc. (3) relied on the words "science" and "knowledge" in the Federal Rule of Evidence governing scientific testimony to incorporate a requirement that the method used by the expert be scientifically valid, and it suggested that trial courts ascertain validity by considering (1) the extent to which the expert's method has been tested, (2) its error rate and the presence or absence of "controlling standards" for applying the method, (3) its use in peer reviewed publications, and (4) its general acceptance in the relevant scientific community. Four years later, in General Electric Co. v. Joiner, (4) the Court held that a trial judge's exclusion of scientific testimony was reviewable only for abuse of discretion. In affirming the exclusion of expert opinions that workplace exposure to polychlorinated biphenyls caused an electrician to develop lung cancer, the Court reasoned that not only must the data on which an expert relies be generated from a valid method, but also that the expert's conclusion be "connected to existing data . . . by [something more than] the ipse dixit of the expert." (5) Two years after Joiner, the Court decided in Kumho Tire Co. v. Carmichael (6) that the factors listed in Daubert for ascertaining scientific validity could be applied to technical (and perhaps other) types of expert testimony. (7)
This Term, the Supreme Court declined to review the application of these "Daubert factors" in United States Tobacco Co. v. Conwood Co. (8) In this antitrust case, the district court allowed plaintiff's expert to estimate lost profits on the basis of a novel theory implemented in the form of a standard statistical technique known as "regression." Essentially, the district court reasoned that inasmuch as both sides used the technique of regression, all was well. The court of appeals affirmed the resulting jury verdict and award of $1.05 billion--the largest in the history of antitrust law--on the understanding that plaintiff's study was of the type routinely used to ascertain antitrust violations or damages.
This article describes the expert testimony in Conwood and considers how it should have fared under Daubert. It shows that, contrary to the Sixth Circuit's opinion, the study departs from the normal econometric approach to establishing <---------- p. 345 ----------> causation or damages. (9) The article then offers suggestions for improving the judicial response to contrived statistical studies of damages.
I. CONWOOD'S COMPLAINT
Snuff is a smokeless tobacco product that is placed in small amounts between the cheek and the gums. The major producers of moist snuff are United States Tobacco Company, Inc. (USTC) (10) and Conwood Company, L.P. (11) In 1998, Conwood filed a complaint in the United States District Court for the Western District of Kentucky alleging that USTC monopolized the moist snuff market in the U.S. in violation of Section 2 of the Sherman Act. Conwood's theory, as developed at a four-week trial, was that:
In 1990, [USTC] began an orchestrated campaign to choke off the distribution of rivals' products. Disdaining competition on the merits--which [USTC] feared would erode its market share and profit margin--[USTC] used its power to exclude competitors' display racks, advertising, and products. [USTC]'s representatives tossed as many as 20,000 Conwood [sales] racks [in retail stores] into dumpsters each month. (12)
USTC denied engaging in systematic, exclusionary conduct of this (or any other) sort. It also moved to exclude expert testimony designed to prove that USTC's allegedly illegal conduct gravely suppressed Conwood's sales of its brands of snuff, and it sought summary judgment. The district court denied these motions. At trial, USTC cross-examined Conwood's expert and presented its own expert, who dismissed the damages study as worthless.
After deliberating for under four hours, a jury awarded Conwood $350 million in damages. Trebling this figure, (13) the district court entered judgment of $1.05 billion, and it issued a permanent injunction. The Court of Appeals for the Sixth Circuit affirmed the judgment and denied a petition for rehearing. (14) With respect to the extraordinary damages, it stated that Conwood's expert employed "regression analyses, a yardstick test and a before-and-after test. All three are <---------- p. 346 ----------> generally accepted methods for proving antitrust damages." (15) It offered the following cursory explanation of these methods:
The before and after theory compares the plaintiff's profit record prior to the violation with that subsequent to it [and] the yardstick test . . . consists of a study of the profits of business operations that are closely comparable to the plaintiff's. . . . A regression analysis looks at the relationship between two variables. . . . The point of a regression analysis is to determine whether the relationship between the two variables is statistically meaningful. (16)
As we shall see, however, Conwood did not compare its "profit record prior to the violation with that subsequent to it." Neither did it study "the profits of business operations that are closely comparable to the plaintiff's." And, while it did employ a "regression analysis," this is a bit like saying that it used arithmetic or algebra. The real question under Daubert is whether the analysis was validly designed to measure lost profits. Because antitrust laws only impose liability for lost profits attributable to conduct that violates the antitrust laws, no study validly measures lost profits unless it filters out the effects of other factors likely to have influenced profits. In appropriate circumstances, the statistical methods listed by the court of appeals can do this.
II. "BEFORE AND AFTER" STUDIES
A "before and after" study attempts to isolate the effect of the illegal conduct by comparing sales revenue in two or more time periods. A crude approach would be to demonstrate a sudden drop in sales in the wake of the anticompetitive conduct. Perhaps this is what the court of appeals had in mind when it defined a "before and after theory" as a comparison of the "profit record prior to the violation with that subsequent to it." (17) However, Conwood made no such comparison. Rather than consider profits, revenues, or sales, its expert examined Conwood's share of the market. This quantity did not decline in the alleged violation period--it increased, and it did so in a greater amount than it had in a comparable preceding period. (18) Although a simplistic "before and after" study does not seem to support the damage award (and was not presented by Conwood), the fact that a plaintiff does better during the alleged violation period does not establish that there are no damages. Perhaps the plaintiff would have done better still had there been no misconduct. (19) <---------- p. 347 ---------->
Just as an increase in revenue or market share during the violation period is not logically conclusive, neither is a decline in such quantities. The problem with jumping from the observation of a decline to the conclusion that it was the product of illegal conduct is obvious. There could be other, legitimate explanations. To use slightly more technical language, the mere association between two variables--the presence of illegal conduct and a decline in sales--could be the result of a fluctuation in a third, unmeasured variable. For instance, if another competitor had introduced a new product that cut into plaintiff's sales more than defendant's, it could be the cause of any decline in plaintiff's sales or market share. In other words, the observed association could be spurious in the sense that it is actually the result of a confounding variable. Daubert requires the expert to take reasonable steps to eliminate these confounders and thus confine the estimated damages to the result of the allegedly illegal conduct.
If one could quantify the effects of all the possibly confounding variables, then one could ascertain whether there is any "left over" effect that should be attributed to the illegal conduct. Suppose, then, that a convincing mathematical model describes plaintiff's sales in the period before the illegal conduct. The predictions of this model in the period of illegality could be contrasted with the actual sales in that period. If all the important variables really were in the model, and if their relationships did not change over time, then the difference between the predicted and true sales would give a reasonable estimate of damages. Typically, the statistical model of the determinants of damages that is applied in the two time periods is known as a regression model. In this more sophisticated "before and after" study, regression is simply a part of the study. It is not an independent "generally accepted method"--any more than arithmetic is.
Constructing an accurate econometric model of sales is a demanding task. Deciding which variables might affect sales and how they could be related requires substantive economic theory. Deriving values for the coefficients of the variables in the model demands statistical methods. But, in principle, the approach is valid. If implemented correctly, it handles the problem of confounding. Unfortunately, Conwood never tried to construct and apply such a model. (20) As a result, the court of appeals was mistaken in its comforting view that the damage award rested on a "before and after" study of the sort accepted in previous antitrust cases.
III. "YARDSTICK" STUDIES
The court of appeals was equally misguided in thinking that the damages award was supported by a "yardstick test[, which is] a study of the profits of <---------- p. 348 ----------> business operations that are closely comparable to the plaintiff's." Conwood's expert thought that sales of smokeless, looseleaf tobacco might provide a useful comparison because USTC did not compete with Conwood in this market and therefore could not have influenced its growth in this market. A simplistic comparison, analogous to the crude comparison of sales in the two time periods, would have shown that Conwood lost market share in this "yardstick market" during the alleged violation period. (21) Thus, the outcome in the yardstick market is the opposite of what one would have expected if Conwood's claim of spectacular damages in the moist snuff market were true. A simplistic yardstick study lends no support to Conwood's claim of damages.
Moreover, even if Conwood had fared better in another market than in the tainted one, the reason would not necessarily have been the illegal conduct. The difference could have been due to the violation, but it also could have been due to other differences between the two markets. Again, a solid econometric model would be required to remove the effects of the many possible confounding variables. The result would be a valid comparison in this more sophisticated "yardstick study." Again, an adequate regression (or other) model of performance typically is central to such a study, but Conwood produced no such model. (22)
IV. CONWOOD'S REGRESSION
If Conwood employed neither a standard "before and after" study nor a conventional "yardstick" study, what did it do? Conwood relied heavily on an analysis prepared by Dr. Richard Leftwich, a professor of accounting and finance. Instead of pursuing the approaches outlined above, Dr. Leftwich hypothesized that USTC's behavior had a greater impact on Conwood's sales in states where Conwood had a low market share in 1990. To test this hypothesis (which came from conversations with Conwood personnel), Dr. Leftwich compiled a table showing Conwood's percentage of moist snuff sales in each state in 1990 and 1997. He then regressed the share growth by state in the1990-1997 period on the 1990 shares. This "standard economic method," as he called it, revealed that "there was a highly reliable relationship between Conwood's growth in the period [from 1990 to] 1997 and its market share in 1990." Indeed, he added, "the results were highly reliable, or statistically significant in . . . that there was more than a 95% chance that these results were, in fact, reflective of systematic patterns in the data." (23) <---------- p. 349 ---------->
The "systematic pattern" that Leftwich perceived was a linear association between starting share and subsequent growth. If, for each state, we plot the 1990 share against 1990-1997 growth, we obtain the cloud of data points shown in Figure 1.
Figure 1. The Regression Line for
1990 Market Shares in Each State
Versus Increases in Shares for 1990-97
[Graph to be added]
The regression procedure merely finds the one straight line that best fits the data. (24) For the market share data at Leftwich's disposal, the best-fitting line is the sloping line shown in Figure 1.
To derive any damages from this loosely fitting straight-line relationship, Leftwich relied on the hypothesis that USTC's illegal conduct had an impact in the low-share states on the left-hand side of the graph, and not in the high-share states on the right-hand side. In one calculation, for instance, he assumed that only the right-most three states were not affected, so that the lower growth in the preponderance of states to the left of the dotted line was entirely the result of the alleged Section 2 violations.
The Daubert question here is not, as the Sixth Circuit seemed to imply, whether fitting a straight line to some data points is a valid procedure for finding that certain quantities are related to each other. It is whether the computation of damages from such a line isolates the effect of the antitrust violation. And the <---------- p. 345 ----------> answer to this question is clear: The exercise in curve-fitting does not accomplish this essential task. Let us suppose, as Conwood's expert assumed, that Conwood was unable to resist USTC's allegedly illegal efforts to keep its stock off retail shelves because it lacked clout in stores in those states where it did not possess over 20% of the market. It then would be equally plausible to suppose that Conwood would be unable to resist perfectly legal efforts by USTC--and all other competitors--to displace Conwood's product with their own in these same stores. Because the difference between the left-hand and right-hand states (25) could be due to legitimate competition along with (or instead of) any conceivable antitrust violation, the regression study does not provide a valid comparison. The expert's conclusion that the sole cause of changes in Conwood's sales had to be a subset of USTC's conduct was, in the words of a distinguished group of economists and statisticians who recently reviewed the regression study, "circular reasoning" and "unacceptable science." (26)
V. IMPROVING THE JUDICIAL RESPONSE
TO STATISTICAL PROOF
Under the Daubert trilogy, it is no longer appropriate for the trial judge simply to "let it all in" in the traditional expectation that the jury somehow will get it right. (27) Rather, the clear message of the last decade of Supreme Court cases on expert testimony is that federal courts must exercise independent and careful judgment before admitting expert testimony that is challenged as "junk science." (28) <---------- p. 351 ----------> In United States Tobacco Co. v. Conwood Co., the lower courts did not rise to this challenge. These courts apparently paid more attention to the labels of "regression," "before and after studies," and "yardstick studies" than to the substance of the procedures actually used to generate huge damage estimates.
To be sure, courts cannot be expected to probe the hidden premises and infirmities of statistical studies without assistance. With proper judicial management, however, the parties can supply some of the necessary assistance. Pretrial hearings permit a thoughtful judge to question the experts, and these hearings can be structured to allow the parties' experts to engage in a more constructive dialogue with one another than typically occurs in bitterly disputatious written reports. In Conwood, the district court held no such hearing.
Another strategy is to bring in a court-appointed expert or advisor. While no panacea, an independent expert might reveal the approximate center of gravity of scientific opinion, as distinguished from the courtroom extremes that advocates are tempted or hired to present. Thus, in Joiner, Justice Breyer endorsed the view that "[j]udges should be strongly encouraged to make greater use of their inherent authority . . . to appoint experts." (29) In light of the amount at stake and the substantial questions about Dr. Leftwich's application of his novel and ad hoc theory that retailers in high-share states were resistant only to illegal competition, Conwood was an attractive case for the exercise of this judicial authority.
Not only was this opportunity lost at the trial level, but the court of appeals imposed barriers to an exposition of the many aspects of the damages study that made it an instance of "expertise that is fausse and science that is junky." (30) It denied defendant's requests to waive the usual page limit on its brief and to expand the time for oral argument. While these decisions were well within the court's discretion--What party would not like extra time and pages?--they severely limited counsel's ability to detail meaningfully the flaws in the damages study. In part, this may explain why the court of appeals mischaracterized the nature of the study and its findings. The parties had no realistic opportunity to give the court the depth of understanding necessary to look beneath the surface of the scientific-sounding words that papered over the gap between the data and the expert's conclusions.
Even so, these impediments to a satisfactory judicial analysis might have been overcome had the court of appeals considered the views of an expert not retained by either party. The Sixth Circuit, however, refused to accept an informative amicus brief outlining the requirements for a valid econometric analysis of damages and explaining why Conwood's study did not measure up to these standards. (31) <---------- p. 3452 ---------->
All too often, the conflicting testimony of hired experts produces, in the words of Lord Dallas, neither enlightenment nor "triumph, but . . . only humiliation to science." (32) The opinions and events in Conwood indicate that if the courts are not open to better sources of information and advice when they must "make subtle and sophisticated determinations about scientific methodology and its relation to the conclusions an expert witness seeks to offer," (33) they risk similar humiliation to the legal system.
1. Severn, King & Co. v. Imperial Ins. Co., Times (London), Apr. 14, 1820, at 3.
2. Id. For a more complete account of the trial, see June Z. Fullmer, Technology, Chemistry, and Law in Early 19th Century England, 21 Technology & Culture 1 (1980).
3. 509 U.S. 579 (1993).
4. 522 U.S. 136 (1997).
5. Id. at 146.
6. 526 U.S. 137 (1999).
7. Using the rubric of "reliability," the teachings of these cases have been explicitly incorporated into Federal Rule of Evidence 702. Catherine E. Brixen & Christine M. Meis, Note, Codifying the "Daubert Trilogy": The Amendment to Federal Rule of Evidence 702, 40 Jurimetrics J. 527 (2000).
8. 123 S. Ct. 876 (2003) (mem.). The petition for a writ of certiorari also raised issues, not discussed here, as to the nature of the Sherman Act's prohibition of "monopolization."
9. For a more complete analysis published before the court of appeals' opinion, see D.H. Kaye, The Dynamics of Daubert: Methodology, Conclusions, and Fit in Statistical and Econometric Studies, 87 Va. L. Rev. 1933 (2001).
10. U.S. Tobacco Company, Inc., was created in the court-ordered dissolution of the Duke Tobacco Trust in 1911. Two of its brands occupy approximately 75% of moist snuff sales in the United States. Brief for Appellees at 5-6, Conwood Co. v. United States Tobacco Co., 290 F.3d 768 (6th Cir. 2002) (No. 00-6267) [hereinafter Appellees' Brief].
11. "Conwood Company L.P. is a limited partnership which manufactures moist and dry snuff and loose leaf, plug and twist chewing tobacco." Edward Knight et al., The U.S. Tobacco Industry in Domestic and World Markets, Cong. Research Serv., Rep. No. 98-506 E, at CRS 21, 210 (updated June 9, 1998), available at http://www.ncseonline.org/NLE/CRSreports/Agriculture/ag-86.cfm.
12. Appellees' Brief, supra note 10, at 2.
13. 15 U.S.C. § 15(a) (2000) (allowing a prevailing plaintiff to "recover threefold the damages . . . sustained, and the cost of suit, including a reasonable attorney's fee").
14. Conwood Co. v. United States Tobacco Co., 290 F.3d 768 (6th Cir. 2002).
15. Id. at 793.
16. Id. at 793 n.8 (citations and internal quotation marks omitted).
18. Reply Brief for Petitioners at 8, United States Tobacco Co. v. Conwood Co., 123 S. Ct. 876 (2003) (No. 02-603) [hereinafter Reply Brief].
19. The district court was persuaded that this is what had occurred. In denying defendants' post-trial motion for judgment as a matter of law, the district judge wrote that "[a]lthough Conwood experienced some growth during the period of antitrust activity, its growth was sharply curtailed between 1990 and trial." Conwood Co. v. United States Tobacco Co., No. 5:98-CV-108-R, 2000 WL 33176054, at *5 (W.D. Ky., Aug 10, 2000). Apparently referring to this alleged diminution in potential--not actual--growth, the court also wrote that "[t]he high profit margins of moist snuff sales, the length of the antitrust activity and the decline in Conwood's growth support [the plaintiffs' statistical study of the] range of damages for lost profits." Id. at *6.
20. The only "before and after" aspect of Conwood's study was a faulty comparison of the correlation between starting and ending shares in the 1983-90 period to the correlation in the 1990-97 period. See Kaye, supra note 9, at 2001-02.
21. Reply Brief, supra note 18, at 8.
22. The only "yardstick" aspect of its study was a comparison of the correlation between starting and ending shares in the looseleaf and moist snuff markets.
23. These characterizations are unjustified. See, e.g., David H. Kaye & David A. Freedman, Reference Guide on Statistics, in Reference Manual on Scientific Evidence 83, 131 n.167 (Federal Judicial Center ed., 2d ed. 2000) (cautioning against the "transposition fallacy" reflected in this testimony). The 0.05 significance level suggests that there is some nonzero association, but neither the probability of this conclusion nor the extent of the association can be derived from the observed significance level. Id.
24. The "least-squares" regression used in Conwood treats the sum of the squared deviations from the fitted line as the measure of "best fit."
25. This shorthand description of the damage study as a simple comparison of high- and low-share states is an oversimplification. Dr. Leftwich did not compare the growth in low-share states to the growth in high-share states. The court of appeals thought that "Leftwich found a statistically significant difference in Conwood's market share between those states in which Conwood had a foothold and those in which it did not." 290 F.3d at 793. However, this is not what Leftwich found. Leftwich compared the growth predicted by the straight-line relationship for a hypothetical state in which Conwood had a 20% share in 1990 to the growth in states with less than this starting share. A direct comparison of the percentage-point growth in high- and low-share states reveals no statistically significant difference. Kaye, supra note 9, at 2003 n.399.
26. Motion for Leave to File Brief and Brief of Washington Legal Foundation, Stephen E. Fienberg, Franklin M. Fisher, Daniel L. McFadden, and Daniel L. Rubinfeld as Amici Curiae in Support of Petitioners at 9, United States Tobacco Co. v. Conwood Co., 123 S. Ct. 876 (2003) (No. 02-603).
27. In Conwood, the district court evinced deep faith in the jury's ability to do what Conwood's expert could not--namely, separate out the sales lost to illegal conduct from those lost to legitimate competition from all sources as well as other market influences. See Conwood, 2000 WL 33176054, at *6 (declining to set aside the admittedly high damage award because "[t]he fact that the jury awarded damages in the mid-range indicates that the jury adequately considered only those damages that were a direct result or likely consequence of [USTC]'s unlawful conduct.").
28. See, e.g., Gen. Elec. Co. v. Joiner, 522 U.S. 136, 147, 147-48 (1997) (Breyer, J., concurring) ("This requirement will sometimes ask judges to make subtle and sophisticated determinations about scientific methodology and its relation to the conclusions an expert witness seeks to offer . . . . [But] neither the difficulty of the task nor any comparative lack of expertise can excuse the judge from exercising the 'gatekeeper' duties that the Federal Rules of Evidence impose.").
29. Id. at 149-50 (Breyer, J., concurring).
30. Kumho Tire Co. v. Carmichael, 526 U.S. 137, 159 (1999) (Scalia, J., joined by O'Connor, Thomas, JJ., concurring).
31. The brief was from Daniel McFadden, the recipient of the 2000 Nobel Prize in economic science and the director of the econometrics laboratory at the University of California at Berkeley. Some of the points raised in this brief are described in Kaye, supra note 9, at 1997 n.294, 1999 n.301, 2000-03, 2010 n.325, 2012.
32/ See supra note 2.
33. Joiner, 522 U.S. at 147 (Breyer, J., concurring).