The Law of Small Numbers
The Impact of Small Samples on Reliability of Conclusions
System 1 vs. System 2 and Intuitive Statistics: The chapter begins with a consideration of a situation where kidney cancer incidences are unusually high or low in certain U.S. counties. This scenario activates a thought process primarily through System 2, where memory is searched and hypotheses are formed about causes. System 1 supports by retrieving associative memories and suggesting connections that may not actually be causal.
Misinterpretations of Statistical Information: The chapter introduces the concept that small populations in rural areas show misleading statistics regarding kidney cancer rates. Both very high and very low incidences are reported, not due to lifestyle or environmental factors, but because small sample sizes in these sparsely populated areas lead to statistical anomalies.
Statistical Explanation vs. Causal Explanation: Contrary to intuitive causal explanations that human minds tend to generate, the disparities in kidney cancer rates across different counties are explained statistically. Small samples from smaller populations are more likely to show extremes. This statistical principle contrasts with causal explanations, demonstrating that what looks like significant cancer rate differences is merely a variation typical of small samples.
Real-World Misinterpretations of Randomness: The chapter highlights common real-world scenarios where randomness is misunderstood, shaping narrative bias. For instance, during World War II, the patterns observed in the bombing of London, thought to be non-random due to apparent clustering, were in fact consistent with random distribution.
The Law of Small Numbers in Psychological Research
Flawed Intuitions About Sample Sizes: Kahneman and his colleague Amos Tversky challenge the optimistic belief held by some psychologists that individuals without statistical training have good intuitive grasp of statistics. They discover through research that even statistically educated psychologists are prone to mistakes in understanding the influence of sample sizes on research validity.
Inadequate Sensitivity to Sample Size: Many research psychologists tend to choose smaller samples than necessary, significantly risking the validity of the research findings without realizing it due to the pervasive but mistaken trust in the Law of Small Numbers.
General Bias Toward Causal Explanations: Humans are predisposed to favor certainty and causal explanations over statistical explanations involving chance. This tendency leads to frequent misclassification of random events as systematic, influencing decision-making in various fields including finance, policy, and scientific research.
Statistical Insight and Rational Thinking
Misinterpretations and Bias in Everyday Decisions: The narrative extends beyond academic research, affecting everyday decisions and actions where randomness is often perceived as pattern. People, including experts, routinely fail to appreciate how much random variation can affect outcomes.
Recommendations for Correcting Intuitive Biases: The chapter concludes with strong recommendations to treat intuitive statistical beliefs with skepticism and to base conclusions and decision-making processes on mathematical computations rather than intuition, to avoid errors associated with the Law of Small Numbers.
In summary, this chapter delves into the systematic errors that arise from the intuitive but incorrect application of the Law of Small Numbers, emphasizing the importance of adopting a more statistically informed approach in both scientific research and everyday decision-making.