Click to return home

The Forensic Debut of the NRC's DNA Report:
Population Structure, Ceiling Frequencies,
and the Need for Numbers

D.H. Kaye

Note: This article appeared in Genetica, Vol. 96, pp. 99-105, 1995, and in the book Human Identification, Bruce Weir ed., Kluwer, 1995

Key words: Forensic DNA, Ceiling Frequencies, NRC Report, Courts

Abstract: This paper reviews judicial opinions that have discussed the April 1992 recommendations of a committee of the U.S. National Research Council concerning the statistics of forensic DNA profiles obtained with single-locus VNTR probes. It observes that a few courts have held "ceiling frequencies" (as opposed to less "conservative" estimates) admissible, but that the implications of the scientific criticisms of the ceiling procedures have yet to be addressed adequately in court opinions. It urges courts to distinguish between policy judgments and scientific assessments in both the NRC report and the scientific literature, and to defer less to the former than to the latter.


In deciding whether the results of DNA tests are admissible in criminal cases, courts in the United States have relied on the testimony of scientists about the state of the science, on the scientific literature, and on other judicial decisions and opinions. These inputs and the resulting caselaw -- have not been static. Since 1988, a fusillade of objections to the admission of DNA tests have been raised in court. These include questions about the possible effects of contaminants on forensic samples, the use of radioactive isotopes in exposing X-ray film, the addition of ethidium bromide, ad hoc corrections for band shifting, the records of laboratories on proficiency tests, the criteria and procedures for deciding whether two VNTR fragments "match," the size and sources of databases used to assess the significance of matching bands, and the method of calculating the frequency of matching DNA patterns within a reference population. Three phases in the judicial responses to DNA evidence can be discerned: an initial period of enthusiastic acceptance of the evidence, a damping counterreaction, and the current period of mixed outcomes and opinions expressing grave concern over certain aspects of DNA evidence. For reviews of these legal developments, see Kaye (1993); Thompson (1993).

In April 1992, a committee of the National Research Council (NRC, 1992) entered this free fire zone. This paper examines the courts' use of the NRC report in their opinions. I confine the discussion to one issue -- the statistics of DNA "matches" with single-locus VNTR probes -- for the committee's views on this topic have been the most controversial. I do not purport to show that the report has caused the courts to reach any particular results; it is possible, and often probable, that the outcomes of cases in which the report is cited would have been the same had no report been written. Neither do I assess the contribution the report has made to clarifying the issues, resolving doubts about other matters, and prompting reforms in forensic work. I merely seek to describe how one major part of the report has been treated in opinions and to offer some suggestions to avoid its being misunderstood or misused.

Proving the Existence of a Controversy

In some jurisdictions, the lack of general acceptance within the scientific community precludes admission of scientific evidence. In others, it weighs against admissibility but is not invariably fatal. See, for example, McCormick (1992, sec. 203). Pretrial hearings lasting months and generating thousands of pages of testimony have addressed the question of scientific consensus, and courts frequently cite the NRC report to support a finding that a major scientific controversy is raging over the proper method for ascertaining the frequency of a "match."

Prior to the NRC report, the standard practice was to estimate multi-locus frequencies by multiplying single-locus frequencies, which in turn, were obtained by multiplying the two "allele" frequencies at each locus. The allele frequencies were derived from histograms of fragment weights within a database. Under the protocol developed by the Federal Bureau of Investigation, the histograms involved very large class intervals to ensure that the single-locus frequencies would not be underestimated. The similarly sized fragments falling into to each large "bin," which are analogous to an allele, may be called a "binelle," and the set of binelles, which is analogous to a genotype, may be called a "binotype." Compare Devlin et al. (1992). The computations, in other words, presupposed Hardy-Weinberg equilibrium and linkage equilibrium: if the estimated binelle frequencies of a heterozygote at the jth locus are pj and qj, then the estimated binotype frequency P for a match at n loci is

(1) P = (2n)(p1 q1)(p2 q2 ... (pn qn)

In court, the most potent criticism of this simple calculation[1] has been the possibility of population structure -- the presence of subgroups, with varying binelle frequencies, that tend to mate among themselves. One exposition of this criticism often cited in judicial opinions is Lewontin & Hartl (1991). The critics argue that, until direct studies of subpopulations are completed, there is no way to be certain that the departures from equilibrium do not make the standard frequency estimates too small (or for that matter, too large) by several orders of magnitude. Other population geneticists and statisticians maintain that direct studies of subpopulations are unnecessary and unlikely to be productive. See, for example, Devlin et al. (1994); Roeder (1994).

This debate is not easy for the courts to penetrate. In People v. Pizarro, 12 Cal. Rptr. 436, 456 (Ct. App. 1992), for instance, a California court of appeals quoted at length from early scientific publications, and lamented:

The difficulty is, where does this place us? It places us in the middle of the conflict as to whether or not the basic theory of population genetics involving broad racial and ethnic groups as opposed to the argument of substructure has any general acceptance in the relevant scientific community -- a conflict which we cannot resolve on the present record.

The NRC report can only have reinforced the perception of dissension. If there was any doubt left in early 1992 that prominent scientists were divided over the adequacy of the forensic computations, the NRC committee put it to rest. Starting with People v. Barney, 10 Cal. Rptr. 2d 731, 741 (Ct. App. 1992), court after court has noted the committee's report of "considerable dispute" and a "substantial controversy." A New Mexico appellate court was especially impressed:

The report discusses the debate over the need for subpopulation databases, and concludes that they indeed are necessary. This report is indicative of the absence of general acceptance. There is not just one author trying to make a point, but rather a group of people that has reached a consensus in rejecting one aspect of the current methods of forensic use of DNA evidence.

State v. Anderson, 853 P.2d 135, 146 (N.M. Ct. App. 1993).

Indeed, a few judges have reacted to the NRC report like sharks scenting blood. Despite the report's endorsement of the principles underlying VNTR studies, these judges perceive vulnerability without pinpointing its location or extent. In State v. Futch, 860 P.2d 264 (Or. Ct. App. 1993), prosecution and defense experts presented binotype frequencies ranging from 10-10 to 10-5, and a majority of the appellate court concluded that such disagreements were matters for the jury to sort out. But a dissenting judge, citing only the first page of the report's summary, insisted that:

The National Research Council, an equally august body [as the Office of Technology Assessment], reports that important questions have been raised about the reliability and validity of forensic RFLP. . . . [T]he technique has not yet achieved general acceptance.

In short, the most common use of the NRC report has been to support the perception of a schism within the field of population genetics on the validity of using equation (1) with binelle frequencies estimated from racial databases. However, the disagreements may be more subtle than both the NRC report and the court opinions suggest. The NRC committee did not come down in favor of either side in the debate on the forensic implications of population structure. Assuming arguendo that population structure is a problem, it offered a compromise. In jumping to a solution without fully examining the problem, the report the does not address all the implications of its convenient assumption that the racial populations are sharply structured. In many cases, the reference population in which to estimate the binotype frequency is a general population, like Caucasians or African-Americans, rather than any special ethnic subpopulations. As Chakraborty et al. (1992), and Kaye (1993) have observed, even if binelle frequencies differ among the subpopulations, using equation (1) with binelle frequencies estimated without regard to substructure typically results in overestimates of the true general population frequency.[2] Since the appropriate reference population in most cases is a general population, the controversy over possible departures from equilibrium is less troubling than might first appear.[3]

Hitting the Ceiling

The Anderson court's claim that the NRC committee "reached a consensus in rejecting" the use of equation (1) with binelle frequencies estimated from racial databases flies in the face of the report itself and news accounts of the committee and its deliberations. The Barney court was far closer to the truth when it acknowledged (10 Cal. Rptr. 2d at 741) that "[t]he report does not, however, choose sides in the debate"; it merely "assume[s] for the sake of discussion that population substructure may exist." Unable to agree that population structure leads to dramatic errors in estimating VNTR binotype frequencies, the committee "decided to assume that population substructure might exist" and to propound one particularly "conservative" solution to this hypothesized problem the notorious "ceiling principle" (NRC committee, 1992, pp. 80, 94).

The ceiling procedure seeks one plausible upper bound on the binotype frequency -- both in the population and any of its subpopulations. Like the standard method, it uses equation (1); however, instead of taking actual binelle frequencies from the appropriate reference population, it substitutes for each such binelle frequency in (1) the largest frequency seen in any subpopulation or population. (This is merely the rough idea the method as presented in the NRC report is more involved; even so, it retains ambiguities that have been exploited in litigation. For criticisms of the general approach and the NRC committee's implementation, see Devlin et al. (1993, 1994); Morton (1992); Weir (1992, 1993a, 1993b). For rejoinders, see Lander (1993); Lempert (1993).)

Courts have responded to the proposal to use ceiling frequencies in several ways. Some have cited the proposal as a basis for excluding the standard calculations. Commonwealth v. Lanigan, 596 N.E.2d 311, 316 (Mass. 1992), is one of the first such cases. There, the Supreme Judicial Court of Massachusetts noted that the prosecution's failure to follow the "considered, conservative" prescription of the NRC committee "underscored the wisdom of . . . excluding the test evidence."

Other courts have intimated that ceiling frequencies may be admissible. On the basis of the NRC report and other materials, the Colorado Supreme Court in Fishback v. People, 851 P.2d 884 (Colo. 1993), took notice of the "substantial controversy" about population structure but sidestepped the issue on the theory that since the controversy materialized well after the trial court admitted standard calculations, that court did not err. At the same time, the Supreme Court went out of its way to the suggest that "this dispute may be rendered essentially moot if [the "ceiling principle" is] determined to be generally accepted."

An appellate court in Illinois, agreeing with a lower court that general scientific acceptance of the standard method was lacking, nevertheless directed the trial court to consider admitting ceiling estimates. The appellate court pointedly explained in People v. Watson, 629 N.E.2d 634 (Ill. App. 1994), that "[a]t least in our view, the NRC Report, which was not previously available to the trial court, suggests that the DNA evidence should be admitted on the basis of this more conservative probability calculation for which the requisite consensus may now exist." The same disposition, accompanied by a virtually identical observation concerning the NRC report, can be found in an earlier case, United States v. Porter, 618 A.2d 629 (D.C. App. 1992).

Vermont's Supreme Court responded to a conviction in State v. Vandebogart, 616 A.2d 483, 494 (Vt. 1992), involving DNA evidence similarly. It too remanded the case to the trial court, which it said "must conduct a hearing in order to determine whether the NRC's recommended ceiling principle has gained general acceptance in the relevant scientific community . . . ."

So, too, some Justices of the Massachusetts Supreme Judicial Court are contemplating the admissibility of ceiling frequencies. In Commonwealth v. Daggett, 622 N.E.2d 272, 277 (Mass. 1993), a plurality opinion stated that "[i]f the NRC's conclusion regarding the ability of the 'ceiling frequency' approach to correct for the errors that might result from the use of a general population database reflects that of the general scientific community, a numerical assessment concerning the likelihood that a given DNA match is false which relies on a general population database could be admissible if the 'ceiling frequency' method of calculation is employed by the expert."

The Supreme Court of Washington was less cautious in its endorsement of ceiling frequencies. The court was so "encouraged" by the NRC report that it commented in State v. Cauthron, 846 P.2d 502 (Wash. 1993), that "[a]lthough we lack the scientific expertise to either assess or explain the methodology, its adoption by the Committee indicates sufficient acceptance within the scientific community . . . ." On remand, the trial court in Cauthron was so impressed with these remarks that it decided that the issue of general acceptance of the interim ceiling principle was settled. Consequently, it ordered a hearing on whether Cellmark adhered to this procedure, and ruled that "the statistical evidence in a DNA case is no longer subject to a Frye hearing if it is founded on the NRC approved Interim Ceiling Principle." State v. Cauthron, No. 88-1-01253-3 (Super. Ct. Snohomish County Aug. 20, 1993) (Report of Proceedings).

Still, such pronouncements are dicta they are not essential parts of the opinions and are not strictly binding in later cases. A handful of appellate courts, however, have held ceiling frequencies admissible. In State v. Alt, 504 N.W.2d 38 (Minn. Ct. App. 1993), the Minnesota Court of Appeals held that "the statistical frequencies of individual loci should be admitted in this case if calculated according the NRC modified ceiling principle." Since the defendant did not dispute the general acceptance of the NRC's ceiling procedure, the court of appeals reversed the exclusion of such numbers, but expressly left open the possibility that defendants might demonstrate the lack of general acceptance in later cases.

Despite a state statute purporting to make probability calculations of genotype frequencies admissible in criminal cases, Alt stopped short of admitting the multi-locus binotype frequency because of several idiosyncratic decisons of the Minnesota Supreme Court forbidding testimony about the product of probabilities of independent events. In State v. Bloom, 516 N.W. 2d 159 (1994), however, the Supreme Court abruptly decided that this rule need not apply to DNA evidence. In previous opinions, the same court had fashioned a rule excluding the probability that is of most interest to the jury for fear that jurors would misuse even a properly computed joint probability; there was no suggestion in those cases that the probabilities, which involved traditional genetic markers and microscopic comparisons of hair fibers, were inaccurate.[4] Nevertheless, the Bloom court distinguished its earlier decisions on the basis of an affidavit from Daniel Hartl. As an expert for the prosecution, Hartl expressed his "belief that the [exclusionary] rule was correct at a time when genetic typing could be performed only with blood groups and other types of genetic systems that are not highly polymorphic," since these systems produce "a long chain of multiplication of probabilities yielding a small, and unjustified, match probability for the whole set of genes." In the case at bar, Hartl was prepared to testify that a nine-locus match found from a search of a database of convicted sexual offenders constituted "overwhelming evidence that, to a reasonable degree of scientific certainty, the DNA from the victim's vaginal swab came from [defendant], to the exclusion of all others," and that using the "interim ceiling method," there was a 1 in 634,687 chance of a random match across the five loci first tested. Evidently persuaded that such calculations for a DNA binotype are less of a problem because of "the very conservative nature of of the probability figures obtained using the NRC's approach," the court concluded that "a DNA exception to the rule against admission of quantitative, statistical probability evidence . . . is justified." It held that "any properly qualified prosecution or defense expert may . . . give an opinion as to random match probability using the NRC's approach to computing that statistic," and it approved as well of certain forms of qualitative characterizations of the meaning of a "match," although its treatment of this issue seems garbled.[5]

The only other definitive adoption of the ceiling procedure to date may have come in Springfield v. State, 860 P.2d 435 (Wyo. 1993). There, ceiling figures not only were held admissible in their own right, but they seem to have swept in more standard computations on their coattails. In 1988, a knife-wielding, masked man forced his way into a woman's home and raped her. In a trial four years later, the state introduced the testimony of an FBI agent that the binotype frequency of the rapist was 1/250 thousand among Native Americans (using the FBI's Native American database) and much less among African-Americans (1/150 million), Caucasians (1/250 million), and Hispanics (1/25 million). It also offered testimony about the NRC report, including ceiling frequencies of 1/17 million for African-Americans and 1/221 thousand for Native Americans. The Wyoming Supreme Court upheld the admission of these estimates, on the ground that they "will assist the trier of fact and that the . . . ceiling principle has provided the most conservative statistical estimate as recommended by the NRC report."

These opinions fail to mention the sometimes strident criticisms of the ceiling method (noted above) by some population geneticists and statisticians. The post-NRC report literature is just beginning to diffuse into the caselaw. In People v. Wallace, 17 Cal. Rptr. 2d 721 (Ct. App. 1993), the author of the Barney opinion cautioned the scientific community:

While we are in no position to choose sides in this ongoing debate, we note that its persistence threatens the admissibility of an extremely important forensic tool. This is no time for purist insistence that DNA evidence should be admitted on one's own terms or not at all. . . . Our hope is that the key players in this debate will . . . agree to a compromise on statistical calculation. Otherwise they risk preventing any general acceptance at all, thus precluding the admissibility of DNA analysis evidence.

This risk materialized in State v. DeFroe, No. 92-1-03699-8 (Super Ct. King County June 23, 1993), and State v. Hollis, No. 92-104603-9 (Super Ct. King County June 23, 1993) (Findings of Fact and Conclusions of Law). The trial court ruled that even with ceiling frequencies, DNA evidence was not generally accepted and therefore inadmissible. After reviewing post-NRC articles on the ceiling principle and affidavits and testimony from many expert witnesses, the court found the "interim-ceiling principle" to be "a statistical technique without scientific basis, contrived by compromise or pressure from the law enforcement community ... ."[6]

The view that the dispute over the ceiling principle demonstrates a fatal lack of general scientific acceptance springs from a failure to appreciate the nature and purpose of the ceiling procedure. As Lempert (1993) makes clear, the proposal was not devised for purely scientific applications and cannot be defended on strictly scientific grounds. It is intended to supply numbers that are useful in court hardly a scientific concern. It involves debatable policy judgments that these numbers should be conservative and that they should be as conservative as the ceiling binotype frequencies are.

Scientists have been drawn into this debate over policy matters, and their views may well be valuable, but courts have little reason to defer to their opinions. The scientific issue -- where scientific consensus is important -- is whether the method actually produces figures no larger than those that would be obtained with perfect knowledge of the reference population and its structure. The NRC committee, according to Lander (1993), thought "that the chosen upper bound sufficed to eliminate serious scientific objections," and there seems to be little dispute over the proposition that for a structured population with each subpopulation in equilibrium, the ceiling methods produce generous estimates of binotype frequencies.[7] In fact, the very perception that the methods are enormously generous evokes antagonism on the part of the scientists and statisticians who see the ceiling computations as inappropriate.

Several courts have recognized this distinction between the scientific question of whether the method gives conservative results and the policy question of whether these results are too conservative. The Porter court explained that:

[I]t is not necessary for the prosecution to prove . . . that there is a scientific consensus as to the precise probability of a coincidental match. So long as there is a consensus that the chances of such a match are no greater than some very small fraction, then the evidence is probative and should be admitted on an appropriately conservative basis.

More recently, in State v. Alt, 504 N.W.2d at n. 21, the Minnesota court noted that "the recent debate does not affect the admissibility in this case of DNA evidence under the NRC approach" because the report "does not purport to yield a statistical frequency which is more scientifically defensible than either the DNA proponents or opponents would advocate." The court thus concluded that "Insofar as the recent debate reflects disagreement with the policy judgments of the NRC Committee, or reiterates the view of DNA proponents that population subgroups are not a significant problem, the debate does not deprive the report of sufficient scientific acceptance . . . ."

Demanding Numbers

Another part of the NRC report that courts have attended to is the statement (p. 74) that "[t]o say that two patterns match, without providing any scientifically valid estimate (or, at least, an upper bound) of the frequency with which such matches might occur by chance, is meaningless." State v. Cauthron, 846 P.2d 502 (Wash. 1993), is illustrative. At trial, Cellmark's experts testified not only that defendant's DNA matched that in semen samples taken from rape victims, but also that "the DNA could not have come from anyone else on earth." On appeal, citing other opinions and quoting the NRC report, the Washington Supreme Court reversed the conviction on the ground that no "probability statistics" accompanied this opinion. (It relied as well on the NRC committee's recommendation (p. 92) that "[r]egardless of the calculated frequency, an expert should -- given the relatively small number of loci used and the available population data -- avoid assertions in court that a particular genotype is unique in the population.")[8]

As a legal matter, a completely unexplained statement of a "match" should be inadmissible because it is too cryptic to be weighed fairly by the jury, and an overblown characterization like the one in Cauthron should be excluded as, on balance, unfairly prejudicial. Conclusions like these follow from the policies behind the pertinent rules of evidence. They are not scientific judgments. Yet, the NRC report seems to have prompted some courts to think that the admissibility of nonnumerical characterizations must be resolved by reference to scientific norms. The dominant Daggett plurality, for instance, disparaged expert testimony that used phrases like "highly likely" but did not include numbers because the state "cited no authorities and presented no testimony . . . that the use of such terms is generally accepted by the scientific community in evaluating the significance of a match." 622 N.E.2d at 275 n.4. "The point is," the plurality insisted, "not that this court should require a numerical frequency, but that the scientific community clearly does." Id. Likewise, in State v. Bible, 858 P.2d 1152 (Ariz. 1993), the Arizona Supreme Court, misconstruing Cauthron, suggested that the NRC report's "meaningless" remark demonstrates that the law's general acceptance requirement precludes presenting evidence of a match without an estimate of the binotype frequency.

This view is plainly mistaken. The general acceptance standard addresses the validity and reliability of the methodology that produces evidence of identity. The fact of a match is scientifically valid evidence of identity as long as it can be shown from theory and data that the binotype is not ubiquitous in the relevant population. But how to present to a jury valid scientific evidence of a match is a legal rather than a scientific issue falling far outside the domain of the general acceptance test and the fields of statistics and population genetics. Thus, it would not be "meaningless" to inform the jury that two samples match and that this match makes it more probable, in an amount that is not precisely known, that the DNA in the samples comes from the same person. Nor, when all estimates of the frequency are in the millionths or billionths, would it be meaningless to inform the jury that there is a match that is known to be extremely rare in the general population.[9] Courts may reach differing results on the legal propriety of qualitative as opposed to quantitative assessments, but they only fool themselves when they act as if scientific opinion automatically dictates the correct answer.


Berry, D., 1991. Inferences using DNA profiling in forensic identification and paternity cases. Statistical Science 6: 175-205.

Chakraborty, R., et al., 1992. Effects of population subdivision and allele frequency differences on interpretation of DNA typing data for human identification, pp. 205-222 in Proceedings of the Third International Symposium on Human Identification. Promega, Madison, Wisconsin.

Devlin. B., N. Risch, & K. Roeder, 1994. Comments on the statistical aspects of the NRC's report on DNA typing. Journal of Forensic Science 39: 28-40.

Devlin, B., N. Risch, Neil & K. Roeder, 1993. Statistical evaluation of DNA fingerprinting: a critique of the NRC's report. Science 259: 748-749, 837.

Devlin, B., N. Risch, & K. Roeder, 1992. Forensic inference from DNA fingerprints. Journal of the American Statistical Association 87: 337-350.

Evett, I.W., J. Scranage, & R. Pinchin, 1993. An illustration of efficient statistical methods for RFLP analysis in forensic science. American Journal of Human Genetics 52: 498-505.

Panel on Statistical Assessments as Evidence in the Courts, 1989. The Evolving Role of Statistical Assessments as Evidence in the Courts, edited by S.E. Fienberg. Springer-Verlag, N.Y.

Kaye, D.H., 1993. DNA evidence: probability, population genetics, and the courts. Harvard Journal of Law & Technology 7: 101-172.

Lewontin, R.C. & D.L. Hartl, 1991. Population genetics in forensic DNA typing. Science 254: 1745-1750.

Lander, E.S., 1993. Letter: DNA fingerprinting: the NRC report. Science 260: 1221.

Lempert, R., 1993. DNA, science and the law: two cheers for the ceiling principle. Jurimetrics Journal 34: 41-57.

Morton, N.E., 1992. Genetic structure of forensic populations. Proceedings of the National Academy of Sciences (U.S.A.) 89: 2556-2560.

McCormick, C., 1992. McCormick on Evidence, volume 1, 4th edition, edited by J. Strong. West Publishing Company: Minneapolis, MN.

Roeder, K., 1994. DNA fingerprinting: a review of the controversy, Statistical Science 9 (in press).

Slimowitz, J.R. & J.E. Cohen, 1993. Violations of the ceiling principle: exact conditions and statistical evidence. American Journal of Human Genetics 53: 314-323.

Thompson, W.C., August 1994. DNA evidence in criminal law: new developments. Trial 35-40.

Thompson, W.C., 1993. Evaluating the admissibility of new genetic identification tests: lessons from the "DNA war". Journal of Criminal Law and Criminology 84: 22-104.

Weir, B.S., 1992. Population genetics in the forensic DNA debate. Proceedings of the National Academy of Sciences (U.S.A.) 89: 11654-11659.

Weir, B.S., 1993a. Letter: forensic population genetics and the NRC. American Journal of Human Genetics 52: 437.

Weir, B., 1993b. Letter: DNA fingerprinting report. Science 260:473.


This work was supported by a stipend from the Arizona State University College of Law Research Fund. James M. Whisman, Elizabeth Thompson, and Mark Prothero kindly provided information about the proceedings in State v. DeFroe, No. 92-1-03699-8 (Super. Ct. King County 1993), and State v. Hollis, No. 92-104603-9 (Super. Ct. King County 1993). A related version of this paper appears in Jurimetrics Journal 34:369-382 (1994).


1. Some courts think that P can be computed even more simply, as the product of all the binelle frequencies. State v. Cauthron, 846 P.2d 502, 513 (Wash. 1993).

2. This result assumes that the database from which binelle frequencies are taken is representative of the population and its subpopulations and that each subpopulation is in equilibrium.

3. For a legal argument for admissibility based on this idea, see Kaye (1993).

4. In actuality, the hair comparison frequencies are difficult to defend, but for reasons that have nothing to do with any independence assumptions. See Panel on Statistical Assessments as Evidence in the Courts (1989, pp. 64-65).

5. The court insisted that "the expert [should not] be allowed to say that defendant is the source to the exclusion of all others or to express an opinion as to the strength of the evidence. But should a properly qualified expert, assuming adequate foundation, be allowed to express an opinion that, to a reasonable scientific certainty, the defendant is (or is not) the source? We believe so."

6. The material considered by the court in excluding the DNA evidence included an affidavit from Elizabeth Thompson branding the ceiling principle "data-driven, interest-ridden, pseudo- statistical, ad hoc methodology, to which no statistician or scientist should be a party." Thompson (1994, note 21). (Dr. Thompson informs me that her criticism was directed not just at the ceiling principle, but also at various other attempts by both sides to adjust the uses of the ceiling principle to their perceived goals.) In a more recent case, Eric Lander provided an affidavit stating that:

As a scientist, the [Hollis] court's analysis is troubling to me. The court focuses on the fact that scientists disagree about the best method of estimating population frequencies, but seems to miss the point that scientists can disagree about the best estimate of a frequency, while agreeing that a formula (or even several alternative formulas) provide an upper bound on the frequency. This is the case for the ceiling principle, which makes adequate allowance for variation among populations and results in estimates that would not overestimate the case against the defendant.

State v. Dyer, No. 93-1-00489-0 (Super. Ct. King County) (Affidavit of Dr. Eric S. Lander, Sept. 20, 1993).

7. Slimowitz & Cohen (1993) question the premise that all the subpopulations are in equilibrium and discuss the conditions under which the ceiling binotype frequency is an upper bound.

8. Other opinions citing the NRC report's demand for quantitative assessments include Daggett, Watson, and still others cited by the plurality of justices who prevailed in Daggett, 622 N.E.2d at 274 n.2.

9. This is not to say that binotype frequencies are the best quantitative expression of the probative value of the evidence. To the contrary, they have shortcomings that the presentation of a suitably computed likelihood ratio or posterior probability might overcome. (Berry, 1991; Devlin et al., 1992; Evett et al., 1993; Kaye, 1993.)

updated 2/26/97