- General Overview
- Central Thesis
- Math as thinking tool: mathematics is the science of not being wrong and the extension of common sense by other means.
- Hidden structure: math supplies X-ray specs revealing invisible patterns under messy real-world appearances.
- Judgment over calculation: real mathematics means deciding what to divide by what before dividing.
- Everyone can use it: everyday reasoning already contains mathematical instincts; formal training amplifies them.
- Linearity, Curves, and Calculus
- False linearity: political debates like the Sweden question fail when they assume every good thing scales in a straight line.
- Laffer curve: tax revenue bends with tax rates; valid nonlinearity, but policy conclusions often overreach.
- Straight locally, curved globally: derivatives make curves locally linear; Archimedes’ exhaustion matured into calculus.
- Extrapolation risk: fitting a line through a few points tempts false certainty, as obesity projections and river lengths show.
- Regression to the mean: extreme observations drift toward average whenever skill and luck mix, not because mediocrity triumphs.
- Statistical Vision and Bias
- Survivorship bias: Wald’s missing bullet holes show armor belongs where returning planes show no damage.
- Dead funds matter: mutual-fund averages ignore vanished funds, flattering surviving performance.
- Small samples mislead: tiny populations and short streaks produce extreme rates that are noise, not signal.
- Rates and baselines: comparing atrocities or deaths requires choosing proportions carefully; different denominators give different answers.
- Law of large numbers: large samples converge toward truth, while small samples scatter wildly around it.
- Evidence, Significance, and Inference
- Null hypothesis framework: significance tests ask whether data would be surprisingly extreme if nothing real were happening.
- P-values are conditional: p is P(result | null), not P(null | result); confusing the two is the prosecutor’s fallacy.
- Multiple testing: dead salmon voxels and Bible codes appear when researchers ignore how many chances they gave chance.
- Replication is the immune system: science advances when failed experiments are published, not hidden in file drawers.
- Significance is not importance: statistically detectable effects can be tiny, like doubled thrombosis risk from a rare baseline.
- Probability, Utility, and Bayes
- Expected value: a $2 lottery ticket usually returns under a dollar; break-even jackpots must account for sharing and taxes.
- Utility bends value: a ducat means less to the rich; St. Petersburg’s infinite expectation collapses under logarithmic utility.
- Bayesian updating: priors plus evidence yield posteriors; base rates make most flagged terrorists innocent.
- Priors shape everything: the same evidence reads differently with different starting beliefs; pure evidence-only inference is a myth.
- Conspiracy Bayesian armor: auxiliary theories absorb evidence, letting crackpot beliefs survive where simpler ones would die.
- Geometry, Codes, and Structure
- Projective geometry: train tracks meet at vanishing points; adding points at infinity makes parallel lines intersect.
- Fano plane: seven points and seven lines encode lottery designs, error-correcting codes, and geometry as one object.
- Hamming codes: seven-bit code words separated by Hamming distance turn cosmic-ray bit flips into correctable errors.
- Structure versus randomness: random codes are more efficient; structured codes like Hamming’s are easy to decode.
- Formalism: non-Euclidean geometry showed axioms need not match intuition; Hilbert’s dream ended with Gödel’s incompleteness.
- Democracy, Voting, and Collective Choice
- Public opinion is fragile: contradictory majority preferences mean “the people” often have no coherent will.
- Voting paradoxes: with three options, plurality, instant runoff, and pairwise majority can elect three different winners.
- Condorcet’s axiom fails: no voting rule satisfies every intuitive fairness condition, as Arrow proved.
- Formalism in law: Scalia’s creed treats procedure and text as decisive, even when intentions or consequences are plain.
- Social mathematics: Condorcet’s legacy is applying quantitative reasoning to government, despite the paradoxes.
- How to Be Right
- Principled uncertainty: “not sure, why, and roughly how not-sure” is action, not fence-sitting.
- Probability, not certainty: forecasters should report chances honestly; a 40% rain forecast isn’t wrong if it rains.
- Productive failure: contradiction and failed disproofs reveal structure, from Bolyai’s geometry to daily argument.
- Common sense by other means: math amplifies intuition through a formal exoskeleton; you already use it all the time.
- Central Thesis
- Deep Dive
- When Am I Going to Use This?
- Hidden Structures and Missing Holes (When Am I Going to Use This? · I)
- The Student's Question
- The standard answer is a lie: integrals won’t be used directly by most adults; the teacher and student both know it.
- Drills, not soccer moves: integrals are calisthenics; they build strength, speed, and flexibility for real mathematical play.
- X-ray specs: math reveals hidden structures under messy appearances and is a science of not being wrong.
- Everyone can play: math is woven into everyday reasoning; you don’t need a math career to benefit.
- Abraham Wald and the Wartime Statisticians
- Abraham Wald: Austrian-Jewish abstract mathematician; Nazi takeover pushed him to Columbia and the Statistical Research Group.
- The SRG: a Manhattan Project of equations; statisticians’ recommendations sent pilots into combat with life-or-death stakes.
- Wald’s bent: abstraction over applications; he looked through plane-and-gun details to the mathematical struts beneath.
- Winning edge: wars are won by 5% fewer planes lost, 5% less fuel, 5% more nutrition—math at every step.
- The Missing Bullet Holes
- The military’s plan: armor the fuselage, where returning planes showed the most bullet holes.
- Wald’s reversal: armor the engines, where holes were missing—the missing holes are on missing planes.
- Survivorship bias: hospital recovery rooms hold leg wounds, not chest wounds; engine-hit planes don’t come back.
- Hidden assumption: officers treated surviving planes as a random sample, ignoring that survival correlates with hit location.
- Zeroing a variable: set engine-hit survival to zero and the empty engine data reads as total vulnerability, not German marksmanship.
- Survivorship Bias Beyond the Battlefield
- Pattern recognition: once named, survivorship bias becomes visible in multiple contexts—the payoff of mathematical abstraction.
- Mutual funds: Morningstar’s Large Blend funds averaged 178.4% growth (1995–2004); Savant’s study casts a colder light.
- Missing funds: financial averages, like plane damage, are misleading if they count only survivors and forget vanished funds.
- The Student's Question
- Common Sense, Survivorship, and Simple Profundity (When Am I Going to Use This? · II)
- Survivorship Bias Hides Failure
- Survivorship bias: judging mutual funds by living survivors ignores dead funds, so performance looks artificially strong.
- Dead funds matter: including them drops the decade return to 134.5%, an ordinary 8.9% per year.
- Modern confirmation: a 2011 study of nearly 5,000 funds found survivors’ excess return runs about 20% higher than including the dead.
- Wald’s planes: bullet holes on returning planes mark survivable damage; armor the places that show no holes.
- Math Is Common Sense by Other Means
- Common sense: mathematics formalizes what we already know intuitively, like commutativity of addition.
- Multiplication insight: seeing six rows of eight holes as eight columns of six makes commutativity undeniable.
- Clausewitz paraphrased: mathematics is the extension of common sense by other means.
- Iron Man: the armor supplies the force, but the act is still punching—math amplifies ordinary thinking.
- Prosthesis analogy: math is an atomic-powered prosthesis vastly multiplying common sense’s reach and strength.
- The Danger of Detached Formalism
- Von Neumann warning: mathematics far from empirical roots degenerates into aestheticizing and insignificant branches.
- Two-way street: common sense without rigor misleads; rigor without common sense is sterile rule-following.
- Needed interplay: abstraction must stay connected to intuitions about quantity, time, space, motion, and uncertainty.
- Four Quadrants of Mathematical Ideas
- Simple and shallow: arithmetic facts like 1+2=3, basic identities, and the quadratic formula; little conceptual heft.
- Complicated and shallow: large computations and technical results that do not enrich your understanding of the world.
- Complicated and profound: celebrity theorems like Fermat’s Last Theorem or the Riemann Hypothesis; not this book.
- Simple and profound: directly usable principles that extend far beyond arithmetic and help you not be wrong.
- This Book’s Hands-On Approach
- Author’s path: pure number theory research gradually revealed simple, deep math at work in the everyday world.
- No royal road: understanding mathematics requires doing some mathematics, not just admiring from a distance.
- Concrete tools: school topics appear in real contexts—trig, calculus, quadratic formula, set theory, lottery designs.
- No homework: no tests, but expect to compute, meet formulas, and get your hands a little dirty.
- Survivorship Bias Hides Failure
- Hidden Structures and Missing Holes (When Am I Going to Use This? · I)
- PART I: Linearity
- One. LESS LIKE SWEDEN
- False Linearity in the Sweden Debate
- Cato’s argument: prosperity falls in a straight line as “Swedishness” rises, so the best policy is an extreme
- Nonlinearity changes the verdict: with a humped curve, America can sensibly add welfare while Sweden trims
- Direction depends on location: whether to move toward or away from Sweden depends on where you already are
- Wald’s armor: too little armor lets planes get shot down; too much keeps them from flying—optimum lies between
- Ancient precedent: Aristotle in the Nicomachean Ethics and Horace’s “est modus in rebus”—too much or too little both wrong
- The Laffer Curve
- Origin: Arthur Laffer sketched the curve on a napkin for Cheney, Rumsfeld, and Wanniski in 1974
- Shape: tax revenue is zero at 0% and 100% tax rates, so the relationship must bend
- Core insight: revenue vs. tax rate is necessarily nonlinear; no simple “more is better” rule holds
- Limits: the curve cannot show whether an economy is overtaxed; that requires hard empirical work
- From Voodoo to Reaganomics
- Politics: Ben Stein parodied the curve in Ferris Bueller’s Day Off; Bush called it “voodoo economics”
- Reagan’s proof: high wartime taxes made him stop after four pictures; low taxes caused more work
- Mankiw’s evidence: after Reagan’s cuts, personal income tax revenue fell 9% from 1980 to 1984 while average income grew 4%
- Rich vs. middle class: top earners may pay more after rate cuts; middle class, with fewer options, may carry the revenue
- Friedman’s stance: cut taxes under any excuse, since government money tends to be spent badly
- The Allure and Misuse of the Curve
- Easy sell: “explain it to a Congressman in six minutes and he can talk about it for six months”—Hal Varian
- False syllogism: lowering taxes could raise revenue; I want that to be true; therefore it is true
- Crackpot pattern: Wanniski compared himself to Edison, Galileo, and Leibniz; such self-comparisons are never right
- Valid math, invalid conclusions: the Laffer curve’s nonlinearity is sound; the policy overreach built on it is not
- False Linearity in the Sweden Debate
- Local Lines, Global Curves (Two. STRAIGHT LOCALLY, CURVED GLOBALLY · I)
- The Allure of Linear Thinking
- Linear reasoning: more of a good thing is always better; any stance implies its extreme — the engine of political straw men.
- Obvious falsehood: not all curves are straight, yet the error is everywhere in everyday argument.
- Mathematical foothold: in a sense curves are straight, and Archimedes is where that story starts.
- Natural default: we already expect things to keep moving straight unless something curves them.
- Exhaustion and the Circle
- Circle's area: a standard formula today, but a vexing open question for Archimedes.
- Number crisis: the Pythagorean Theorem reveals √2 as irrational, a length no ratio of whole numbers can express.
- Pythagorean dogma: Hippasus was said to be drowned for proving irrationality; numbers were bound to ideology.
- Exhaustion's lineage: Eudoxus devised it, Euclid recorded it in Book 12, Archimedes perfected it.
- Archimedes' squeeze: squares bound π between 2 and 4; octagons narrow it to 2.83–3.31; iteration tightens the trap.
- Straight locally, curved globally: tiny arcs approximate straight lines, as Earth's surface feels flat; a many-sided polygon stands in for the circle.
- The Page Where I Teach You Calculus
- Newton's infinitesimal zoom: shrink the view until the near-line becomes exactly a line at a single moment.
- Derivative: the slope of that exact line; Newton called it fluxion.
- Archimedes' limit: he let polygons approach the circle but never identified the circle with an infinite polygon; Newton completed the leap.
- Smoothness requirement: any curve without sharp corners becomes straight when magnified enough.
- Why calculus works: release a rock from a circular swing and it flies off along the straight path the derivative predicts.
- Rooted intuition: we already expect objects to move straight unless forced to curve.
- Evanescent Increments and Unnecessary Perplexities
- Legitimate critique: Newton's derivative was not yet rigorous; the infinitely small was a weak point.
- Berkeley's mockery: infinitesimals are "ghosts of departed quantities" — neither finite, nor infinitely small, nor nothing.
- Zeno's paradox: reaching any destination requires covering half the remaining distance forever, so motion should be impossible.
- Reductio effect: the argument rules out every motion, so the flaw is in the premise, not in walking.
- Limit concept: exhaustion treats the infinite as a single finite value approached without being reached.
- Historical residue: the infinitely small embarrassed mathematicians for millennia, yet the straight-line method endured.
- The Allure of Linear Thinking
- Limits Tame the Infinite (Two. STRAIGHT LOCALLY, CURVED GLOBALLY · II)
- Zeno and the 0.999… Puzzle
- Zeno’s paradox: walking to the store is an infinite halving journey; Cauchy later shows the sum is exactly 1.
- Parallel conundrum: 0.999… also creeps toward 1 forever without obviously arriving, making people suspect it is smaller.
- Algebraic proofs: multiplying 0.333… by 3, or 0.999… by 10, forces equality — yet feels like intimidation more than insight.
- Infinite Sums Say Strange Things
- 1 + 2 + 4 + 8 + …: the same algebraic moves seem to prove the sum is −1, a warning that symbol-shuffling can mislead.
- Grandi’s series: 1 − 1 + 1 − 1 + … can be bracketed to equal 0, regrouped to equal 1, or solved to equal 1/2.
- Historical credulity: Grandi called the 1/2 value creation from nothing; Leibniz and Euler accepted the computation without accepting the theology.
- The Infinitesimal Alternative
- Nonstandard analysis: Abraham Robinson made infinitesimals rigorous, vindicating Berkeley’s “evanescent increments” with infinitely small and large numbers.
- Brian’s numbers: one student reinvented the theory independently, then lost interest when his name wasn’t attached to it.
- A real choice: 0.999… could be 1 minus an infinitesimal — but only if we adopt an exotic, sprawling number system.
- Cauchy’s Definition Settles It
- Unask the question: “What is 0.999… really?” has no answer until we decide what an infinite sum means.
- The limit idea: Cauchy defined an infinite sum as the number the partial sums approach and never leave; .9 + .09 + … is therefore 1.
- Hardy’s lesson: pre-Cauchy mathematicians asked “What is 1 − 1 + 1 − …?”; modern mathematics assigns meanings instead of discovering them.
- The price paid: decimal expansions need not be unique — 1 and 0.999… are two names for the same number, like synonyms.
- Aftermath of the Revolution
- Rigor for calculus: Cauchy’s framework made Newton’s calculus fully rigorous without invoking vanishing increments or infinitesimals.
- Divergent series survive: Grandi’s series lies outside Cauchy’s theory; Abel called such series “the invention of the devil,” but Hardy urged context-dependent tolerance.
- Cauchy’s character: conservative in politics, revolutionary in mathematics; he rewrote his syllabus at the École Polytechnique and defied the note-takers.
- The joy of certainty: once you understand mathematics the right way, all the way down, you cannot bear to explain it the wrong way.
- Zeno and the 0.999… Puzzle
- Three. EVERYONE IS OBESE
- Linear Regression: The Universal Tool
- Linear regression: the statistical technique that is to social science as the screwdriver is to home repair.
- Fit line: comes closest to all data points; in the tuition/SAT example, each SAT point predicts $28 more tuition.
- Versatility: works with two variables or a thousand; any spreadsheet can run the regression in one click.
- Table-saw warning: it operates on any data set, so careless use can produce gruesome results.
- The Perils of Extrapolation
- Not every curve is a line: but every curve is nearly a line locally, which tempts us to extrapolate.
- Missile path: a straight-line fit through a few points says the missile climbs forever, though its path is a parabola.
- Twain's Life on the Mississippi: 242 miles shortened in 176 years predicts a 1.3-million-mile river a million years ago.
- False exactness: "wholesale returns of conjecture out of a trifling investment of fact" mocks blind linear extrapolation.
- The Guiding Human Hand
- Partial-credit rule: a correct derivation ending in an absurd −4 grams earns half credit; circling it earns zero.
- Guiding hand: computers calculate, but judging sense is human—no ideas but in things—else math courses train slow, buggy Excel.
- Math wars: traditionalists defend algorithms and exact answers; reformists prize meaning, discovery, approximation.
- Multiplication table: fluency is needed for mental flow; you can't write a sonnet if you must look up every word.
- Balanced math: teach precise answers and intelligent approximation, existing algorithms and on-the-fly horse sense.
- Plane geometry: last redoubt of proof; Fields Medalist David Mumford suggests replacing it with a programming course.
- The Obesity Apocalypse Revisited
- NHANES evidence: overweight prevalence rose from under 50% in early 1970s to almost 75% by 2008.
- Media panic: headlines about an "obesity apocalypse" recapitulate Mirman's joke: "I read that I was!"
- Wang's projection: Obesity paper's linear regression says all Americans will be overweight by 2048.
- Bounded curve: prevalence bends toward 100% as fewer thin remain; the line would hit 109% by 2060, later NHANES slowed.
- Subgroup contradiction: black men's slower growth leaves 20% not overweight in 2048—where would they be?
- Zero credit: the internal inconsistency is the epidemiological version of −4 grams in the bucket.
- Linear Regression: The Universal Tool
- Proportional Rhetoric and Small Samples (Four. HOW MUCH IS THAT IN DEAD AMERICANS? · I)
- The Rhetoric of Dead-American Equivalents
- Lineocentrism: judging foreign tragedies by their proportional size in American population is pure lineocentrism.
- Pundit arithmetic: intifada deaths are “50,000 Americans,” Nicaragua’s toll “3 million Americans,” Vietnam’s “27 million Americans.”
- Hidden assumption: proportional equivalence assumes impact scales linearly with population, as if deaths are fungible percentages.
- Absurdity test: proportional logic makes one bar punch equal to 150 million Americans punched; Rwanda cannot be “nine Holocausts.”
- Rates Matter and Baselines Multiply
- Rates vs. raw counts: per-capita rates correct for population; raw numbers just crown California, Texas, New York.
- Madrid red flag: the same bombing equals 1,300, 463, or 600 American deaths depending on whether you scale by nation, city, or province.
- Method hygiene: if different computations give different answers, something is wrong with the method.
- Pinker’s rates: The Better Angels of Our Nature: rates show violence declined; big 20th-century numbers stem from more people.
- Thirty Years’ War: killing 1 in 100 people then would require 70 million deaths today, more than both world wars combined.
- Small Samples Create False Extremes
- Brain-cancer leaderboard: top per-capita states are rural small-population states; bottom states are also small.
- Dakota puzzle: South Dakota tops and North Dakota bottoms, with no plausible local cause — chance is the likely culprit.
- Sample size: small populations are thin reeds whipped by chance; big states are oaks that barely bend.
- Coin-flip game: small teams regularly produce an 80% head-flipper; big teams almost never do, despite equal fairness.
- The Law of Large Numbers
- Convergence: the more coins flipped, the more tightly the proportion of heads is squeezed toward 50%.
- History: Cardano stated the idea informally, Poisson named it, Bernoulli proved the Law of Large Numbers.
- Kerrich’s experiment: interned in Denmark, a mathematician flipped a coin 10,000 times and watched the fraction converge.
- NBA illusion: early shooting leaders are bench players who took one or a few shots and made all of them.
- Ranking lesson: extreme rates in tiny samples are expected noise, not evidence of exceptional skill, risk, or toxicity.
- The Rhetoric of Dead-American Equivalents
- Small Samples, Square Roots, False Laws (Four. HOW MUCH IS THAT IN DEAD AMERICANS? · II)
- Small Samples Distort Rankings
- NBA leaderboards: restrict to players meeting playing-time thresholds; otherwise part-time shooters with tiny samples top the list.
- North Carolina's school contest: small schools swept the top-25 improvement rankings—not because they were better, just more variable.
- Kane and Staiger: 28% of smallest schools made top 25 vs 7% overall; small schools also drew more assistance teams.
- Extreme scores swing small averages: in big schools, a few prodigies or slackers barely budge the mean.
- Shrinkage: read a small state's cancer spike as luck and pull it toward the national rate; weighting is an art.
- De Moivre's Square-Root Law
- De Moivre's question: Law of Large Numbers says proportions approach 50%; de Moivre asked how close they get.
- Square-root rule: typical absolute discrepancy grows with √N, but shrinks as a proportion because √N grows slower than N.
- Examples: 60 heads in 100 is one √100 away; 538 in 1,000 is more unusual though closer to 50%.
- Standard error: halving a poll's error bar requires four times the sample.
- Normal distribution: discrepancies follow a bell curve; chance of being within one √N of parity is about 95.45%.
- The Law of Averages Is False
- Law of averages fallacy: after five heads or three sons, people expect the next trial to balance the record.
- Coins have no memory: the next flip is always 50-50; three sons don't make a daughter likelier.
- LLN dilutes, not compensates: early imbalance becomes proportionally negligible as new data pile up.
- Danger of divine hand: de Moivre saw regularities as God's hand, a feeling that invites the false law of averages.
- Comparing Atrocities
- Small-country bias in body counts: ranking atrocities by proportion puts Herero, Cambodia, Congo above Hitler, Stalin, Mao.
- Survivor rule: use proportion where "survivor" fits—Rwanda's 75% of Tutsi, not 9/11's 0.001% of Americans.
- Partially ordered set: some disasters compare clearly, others don't—"worse" isn't like "bigger."
- Moral arithmetic: to feel 26 bombing deaths, imagine 26 in your own city—no calculator required.
- Small Samples Distort Rankings
- Five. MORE PIE THAN PLATE
- Percentages of Net Changes
- Slogan: don't talk about percentages of numbers when the numbers might be negative.
- Spence-Hlatshwayo study: 98% of 1990–2008 U.S. job growth was nontradable, but tradable could have declined.
- 2000–2008 example: nontradable added 7M while tradable lost 3M jobs; as a share of net gain, nontradable was 175%.
- Fake total story: finance and computer design each added over 600,000 jobs; manufacturing losses shrank the net total to 620,000.
- Negative Numbers Short-Circuit Intuition
- Historical debate: Cardano called negative solutions "ficta"; Renaissance algebraists doubted their reality.
- Coffee shop illusion: $500 coffee loss plus $750 pastry and CD profits lets 75% come from each.
- Positive-only constraint: percentages of people sum to 100%; negative numbers let shares overlap and exceed 100%.
- More Pie Than Plate
- Piketty-Saez 2010: top 1% got 93% of income gains, while the next 9% got 17%; bottom 90% lost ground.
- Yearly anomalies: 1992 top 1% took 131%; in 1982–83, next 9% took 91% and top 1% took 63%.
- Actual pattern: income recovery mainly improved the upper 10%; the bottom 90% kept sliding.
- Political Percentages Are "True but False"
- Wisconsin's claim: 9,500 state jobs were half the national 18,000, but Minnesota, Texas, and others added more.
- Romney's 92.3%: women bore 92.3% of net job losses under Obama; one month later the figure becomes 3,000%.
- Actual gender pattern: recession hit men almost twice as hard; recovery added more jobs to men than women.
- True but false: a correct percentage can answer the wrong question, making it politically misleading.
- Mathematics Is Choosing What to Divide
- Word problems mislead: real-world questions aren't calculator inputs; context and causation matter first.
- Right computation: applying math requires deciding which ratio is meaningful before dividing.
- Real mathematics: figuring out what to divide by what, not the division itself.
- Percentages of Net Changes
- One. LESS LIKE SWEDEN
- PART II: Inference
- Bible Codes and Statistical Skepticism (Six. THE BALTIMORE STOCKBROKER AND THE BIBLE CODE · I)
- Mathematics Reaches for the Divine
- Territorial ambition: mathematics claims questions beyond the cosmic, even God.
- Maimonides' Abraham: rational observation, not revelation, led Abraham to one God.
- Modern turn: design arguments faded, but data and computers open new routes.
- The Torah as a Data Set
- Original digital signal: the Torah is a finite string of characters faithfully transmitted.
- Equidistant letter sequences: plucking letters at fixed intervals mines hidden strings.
- Pattern inevitability: with 304,805 letters, some ELSs will spell words by chance.
- Weissmandl's find: ELS for Mishneh Torah at steps 50 and 613; suggestive but not decisive.
- The Witztum-Rips-Rosenberg Experiment
- Systematic test: 32 famous rabbis; names and dates searched as ELSs in Genesis.
- Null-hypothesis shuffle: random pairings set the chance baseline; true match ranked 453rd of a million.
- Placebo texts: no effect in War and Peace, Isaiah, or scrambled Genesis.
- Published puzzle: Statistical Science presented the anomaly, baffling its referees.
- Sensation and Theological Resistance
- Drosnin's leap: The Bible Code read every cluster as prophecy, warning Rabin a year early.
- Authors' distance: Witztum, Rips, and Rosenberg denounced Drosnin's ad hoc method.
- Mathematical schism: Sternberg versus Kazhdan split Orthodox mathematicians; tea became awkward.
- Rabbinical caution: a code changing the Sabbath would force rethinking, so the method was rejected.
- The Baltimore Stockbroker Parable
- The setup: unsolicited weekly stock tips; ten straight predictions all come true.
- The pitch: the eleventh week asks you to invest with a hefty commission.
- The trap: apparent market genius can be manufactured by selective reporting of past guesses.
- Mathematics Reaches for the Divine
- Improbable Events and Wiggle Room (Six. THE BALTIMORE STOCKBROKER AND THE BIBLE CODE · II)
- The Baltimore Stockbroker Con
- Newsletter con: 10,240 receive opposing tips; only correct-prediction recipients continue.
- Ten straight picks: chance per recipient is 1/1024, but ten perfect records emerge by design.
- Incubated funds: firms launch only winners from many secret trial funds, then results regress.
- Investor lesson: resist hot new funds; choose low-fee index funds instead.
- Improbable Things Happen All the Time
- Newsletter arithmetic: 1/1024 per recipient, but ten winners expected from 10,240 mailings.
- Lottery coincidence: same numbers twice is rare, but many games and draws make it unremarkable.
- Perspective shift: ask "how many chances?" not just "how unlikely?"
- Surprise is the error: the mistake is being surprised to encounter the improbable.
- Aristotle: "it is probable that improbable things will happen."
- Fisher: "one chance in a million" occurs with its appropriate frequency, however surprising.
- Wiggle Room in the Bible Code
- Rabbinic appellations: no official names; many acceptable variants open choices.
- McKay and Bar-Natan: alternate name lists make the Torah's apparent code vanish.
- War and Peace test: Tolstoy's novel "predicts" rabbis as well as Genesis did.
- Neutral list: an independent scholar's list drops the Torah to near-chance performance.
- Moby Dick codes: Drosnin's challenge answered with assassinations, Trotsky, and Drosnin himself.
- Wiggle room: many acceptable name choices guarantee some list hides a code.
- Statistical Tests Under Suspicion
- Standard methods: the Bible code paper passed peer review in Statistical Science.
- Honest preselection: a fixed name list still leaves the miracle of choosing the scoring one.
- Wiggle room: hidden choices are the enemy of reliable inference from improbable events.
- Implication: accepted statistical tests can be gamed by hidden multiple testing.
- Known worry: statisticians had already doubted standard tests before this case.
- The Baltimore Stockbroker Con
- Statistical Traps and Reverse Inference (Seven. DEAD FISH DON’T READ MINDS · I)
- Dead-Fish fMRI and Multiple Comparisons
- Salmon study: Craig Bennett scanned a dead salmon and found voxels “reading” human emotion—a deadpan methodological rebuke.
- Voxels: fMRI divides the brain into tens of thousands of regions; random noise guarantees a few fake “hits.”
- Multiple comparisons correction: safeguards against chance hits are indispensable in massive data sets, yet many fMRI studies skip them.
- Threshold rule: the more chances you give yourself to be surprised, the higher your threshold for surprise must be.
- Survivorship bias: excited diet testimonials surface; silent failures don’t, so anecdotal evidence is weak.
- Stockbroker con: ignoring wrong predictions while celebrating lucky calls fools scientists as much as investors.
- Reverse Engineering, or, Why Algebra Is Hard
- Fractions: first conceptual break; numbers shift from “how many” to “what portion of.”
- Algebra: computation backward—given the output, infer what went into the box.
- Missile problem: 100 + 200x − 5x² = 0 asks when height is zero, forcing reverse inference.
- Trial and error: approximate impact time is practically enough; precise models overstate accuracy.
- Quadratic formula: yields two solutions, the negative one meaningfully represents the missile’s earlier launch moment.
- Cubic milestone: reverse engineering is so hard that Renaissance algebraists fought public duels over formulas.
- Defeating the Null: Improbability and p-values
- P-values: the numerical answer to “how surprised should I be?” is powerful but fraught.
- Frequentist probability: formal theory arrived late; coin flips approach 1/2 by the Law of Large Numbers, but not all questions fit.
- Non-repeatable events: rain tomorrow, human extinction, authorship, and miracle claims strain the coin-flip model of chance.
- Relative improbability: an event is improbable only under specific hypotheses about how the world works.
- Alternative hypotheses: Bible codes look miraculous unless Torah is assumed to contain all knowledge; rigged lotteries explain “coincidences.”
- Scientific questions: “Is something going on, or not?” is the yes/no null-style test behind rigor.
- Dead-Fish fMRI and Multiple Comparisons
- Statistical Significance and Its Traps (Seven. DEAD FISH DON’T READ MINDS · II)
- The Null Hypothesis Framework
- Null hypothesis: the intervention being tested has no effect; ruling it out is the researcher's central task.
- Significance test: Fisher's early-20th-century protocol, still the backbone of psychological research, for judging whether data refute the null hypothesis.
- p-value: probability, if the null is true, of getting results at least as extreme as those observed.
- 0.05 cutoff: a Fisherian convention, not a principled threshold.
- What the Test Requires
- Data must refute the null: evidence consistent with your theory is worthless if the null predicts it too.
- The sunrise example: claiming to raise the sun proves nothing because it happens under the null; stopping it would convince.
- Modest differences mean little: under the null, fewer drug patients than placebo patients die 43% of the time.
- Extreme results are compelling: all fifty drug patients surviving against five placebo deaths has p≈0.005.
- Historical and Theological Misuses
- Arbuthnot's proof: 82 consecutive years of more boys than girls in London; p≈1 in 4 septillion under equal-chance null.
- Unreasonable null: Bernoulli's 18/35 male chance fits the data perfectly; 82 heads means "coin is biased," not "God loves heads."
- Legacy: Arbuthnot's spirit carries on in Bible codes and creation scientists' arguments for design.
- Darwin's echo: natural selection's explanatory success was defended as "a false theory would not explain so much."
- The Dangerous Word "Significance"
- Statistical ≠ important: "statistically significant" only means the measured effect is not zero.
- 1995 pill scare: "doubled risk of thrombosis" meant 2 in 7,000 versus 1 in 7,000 affected women.
- Panic had real costs: 26,000 extra conceptions and 13,600 extra abortions followed; estimated deaths prevented: possibly one.
- Risk ratios mislead: twice a tiny risk is tiny; sevenfold day-care risk was 1.6 vs 0.23 per 100,000—both near zero.
- Null Hypotheses Are Almost Always False
- Null hypothesis is usually false: every intervention has some minuscule effect somewhere in the body's feedback loops.
- Power creates false importance: larger studies can make trivial effects statistically detectable.
- Context matters: a longer commute to a safer day care can cost more lives than the safety gain saves.
- Better term: "statistically noticeable" or "detectable" would capture what the test actually does.
- The Null Hypothesis Framework
- Statistical Power and Invisible Effects (Seven. DEAD FISH DON’T READ MINDS · III)
- Significance Tests Are Instruments
- Significance test: an instrument, like a telescope — weak instruments cannot see what is there.
- Absence of evidence: failing to reject the null is not evidence of absence.
- Underpowered study: too crude to detect an effect of the expected size; binoculars cannot reveal moons.
- Power’s flip side: high-powered trials overhype small effects; underpowered trials dismiss real ones.
- Skinner’s Statistical Assault on Shakespeare
- Skinner: failed novelist turned behaviorist, using Fisher’s p-value test to demolish literature.
- Null result: in a hundred sonnets, alliteration frequencies matched shuffled chance; Shakespeare “drew words out of a hat.”
- Restraint: alliteration is used sparingly and avoided where unwanted, so no surplus shows up in a crude test.
- Cabbage rule: overpacking alliteration would be “cabbage served twice is death”; the poet’s art is discipline.
- The Hot Hand, Tested and Retested
- GVT 1985: 76ers shot records showed no streak effect, despite nine in ten fans believing in the hot hand.
- Four-shot sequences: good, moderate, and bad blocks matched chance; no significant deviation.
- Simulations: with a real 90% hot hand built in, GVT’s test still failed to reject the null in over three-quarters of runs.
- Real warm hand: weak defenses create mild hot spells, but they are too small for GVT’s instruments.
- Right question: not “does it exist?” but “how much does ability vary, and can observers detect it live?”
- When Belief in Skill Misleads
- Investing: a fund beating the market five straight years is more likely lucky than skilled; past performance guarantees nothing.
- Overconfidence: players take harder shots after makes, especially after three-pointers.
- Self-canceling: belief in the hot hand changes shot selection and can make the edge disappear.
- Free-throw nuance: a make slightly raises the next free-throw chance, yet real-time hot-hand evidence stays weak.
- Significance Tests Are Instruments
- Reductio, improbability, and hidden clusters (Eight. REDUCTIO AD UNLIKELY · I)
- The Logic of Reductio
- Reductio ad absurdum: assume what you aim to deny, then watch it collapse under reality.
- Aristotelian pedigree: proof by contradiction is a time-honored, non-circular method.
- Mechanics: if hypothesis H implies a false fact F, and F is true, then H is false.
- Gun-deaths example: claim of 200 DC children killed was falsified by total homicides of 88.
- Irrationality by Contradiction
- √2 proof: assume √2 = m/n in lowest terms, with m and n sharing no factor.
- Evenness trap: squaring yields 2n² = m², forcing m even, then n even too.
- Contradiction: a lowest-terms fraction cannot have both numerator and denominator even; so the hypothesis fails.
- Result: √2 is irrational—an ancient proof that broke a paradigm.
- Reductio ad Unlikely
- Significance test: suppose the null is true; if the observed outcome would be very improbable, reject the null.
- Fuzzy reductio: not absurdity but improbability—conclude the null is very improbable.
- Michell's Pleiades: six stars this tightly clustered had about a 1 in 500,000 chance under random distribution.
- Fisher's disjunction: either an exceptionally rare chance occurred, or the theory of random distribution is false.
- Real cluster: the Pleiades are indeed a physical cluster, not an optical coincidence.
- Pitfalls of Probabilistic Contradiction
- Berkson's albino: one albino among 50 subjects gives p < 0.0025, yet the subjects are human—method fails.
- Improbable ≠ impossible: improbable things happen a lot; impossible things never happen.
- Lottery fallacy: any specific two-draw outcome has odds around 1 in 300 billion; that alone doesn't imply rigging.
- Don't be that person: don't fire off an angry email to the lottery commissioner every Thursday.
- Prime Clusters and Structureless Structure
- Random clusters expected: apparent groupings in random data are inevitable, not evidence of hidden cause.
- Numb3rs pilot: unclustered crime sites pointed to one intentional killer—mathematically sound.
- Prime gaps: primes are deterministic, yet cluster-like patterns appear in their distribution.
- Bounded gaps conjecture: Zhang's 2013 proof stunned mathematics; the former Subway sandwich seller had been written off.
- Belief before proof: mathematicians already believed bounded gaps and the twin primes conjecture from overwhelming evidence.
- The Logic of Reductio
- Primes Act As If Random (Eight. REDUCTIO AD UNLIKELY · II)
- The Atomic Structure of Numbers
- Unique factorization: every positive integer is a product of primes in exactly one way.
- Excluding 1: treating 1 as prime would shatter uniqueness by permitting arbitrary extra factors.
- Empty product: 1 is the product of no primes, just as the sum of no numbers is 0.
- Primeness is indivisibility: primes are number theory's atoms, the basic entities from which all numbers are built.
- Euclid's infinitude: primes never run out, but mere infinitude says nothing about how they are scattered.
- Distribution and the Logarithm
- Prime Number Theorem: about N/log N of the first N integers are prime, proved by Hadamard and de la Vallée Poussin.
- Rarity spectrum: primes sit between powers of 2 and even numbers—rarer than evens, commoner than sparse powers.
- Flogarithm intuition: digit count approximates the logarithm; a billion's flogarithm is only 10.
- Slow thinning: doubling digit length only halves a random number's chance of being prime.
- Bounded Gaps and Twin Primes
- Zhang's bounded gaps: infinitely many prime pairs lie within 70 million, proved with deep modern mathematics.
- Polymath improvements: collective tinkering cut the bound to 5,414; Maynard then reached 600.
- No prime gravity: random scatter naturally creates close pairs and visible clusters.
- Twin-prime count: random model predicts about N/(log N)^2 twin pairs up to N.
- Hardy–Littlewood correction: dependencies raise the estimate ~32%, matching counted twin primes near a quadrillion.
- Miraculous absence: if twin primes stopped, some unknown force would be pushing primes apart.
- Randomness as a Number-Theory Lens
- Deterministic randomness: primes are fixed, unrandom facts, yet in many respects they behave as if random.
- Dirichlet's residue theorem: large primes split evenly among allowed remainders, exactly as random integers do.
- Goldbach and Green–Tao: the random model makes Goldbach plausible and predicts arbitrary-length progressions, later proved by Green–Tao.
- Fermat proved by rarity: perfect powers are so scarce that random scattering makes nontrivial solutions nearly impossible; Wiles proved it.
- Beal's million-dollar prize: the generalized Fermat equation should have no large-exponent solutions without a common factor.
- Random is random: Zhang linked random-looking remainders to random-looking gaps, hinting at a theory of structurelessness.
- The Atomic Structure of Numbers
- Significance, False Positives, and Replication Crisis (Nine. THE INTERNATIONAL JOURNAL OF HARUSPICY · I)
- The Haruspicy Parable
- Haruspicy: sheep-entrail predictions create a journal of statistically significant results even when the null hypothesis is always true.
- p-value threshold: by design, 1 in 20 null experiments will pass, filling journals with false positives.
- Scientific resemblance: modern medicine and social science may harbor more "entrail reading" than researchers admit.
- Ioannidis's Null Fields
- Ioannidis: Why Most Published Research Findings Are False argues entire specialties can be null fields with no real effects.
- Schizophrenia gene scan: among 100,000 genes, 5,000 false positives swamp 10 real effects at p<.05.
- Low power worsens it: if real genes pass only half the time, false positives outnumber true positives a thousand to one.
- Box-and-circle diagram: the significance test works correctly; preponderance of null hypotheses makes false positives dominate.
- The Winner's Curse
- Winner's curse: significant results in small studies are often large only because noise inflated them.
- Ovulation/Romney study: implausibly large 17-point swing among 228 women passes p=.03, but likely mostly noise.
- Replication failure: Chabris's team found thirteen reported IQ-associated SNPs vanish to insignificance in larger data sets.
- IQ heritability: effects are many tiny genetic contributions, so single-gene "discoveries" arise at the 1-in-20 rate.
- File Drawer Problem
- File drawer: null results stay unpublished, so published evidence is a biased, nonrandom sample.
- Baltimore stockbroker analogy: seeing only lucky successes blinds the scientist just as it fools the naive investor.
- Wald's planes: what you don't see—failed experiments, downed planes—matters for judging evidence honestly.
- Self-inflicted con: the scientific community hides its own failures, playing both con man and victim.
- p-Hacking and Replicability Crisis
- Replicability crisis: Amgen could reproduce only 6 of 53 landmark cancer biology findings.
- p-hacking: tweaking analyses, removing outliers, or adding controls can push .06 to .04.
- Torturing data: scientists know forced confessions from data are about as reliable as extracted ones.
- P-value distribution: haruspicy journals show flat p-value curves; real journals reveal suspicious excess just below .05.
- The Haruspicy Parable
- Beyond the .05 Threshold (Nine. THE INTERNATIONAL JOURNAL OF HARUSPICY · II)
- The Arbitrary .05 Threshold
- P-hacking curve: p-values bunch just below .05 because researchers prod data across the publishable line.
- False binary: significance tests are graded evidence, not true/false verdicts; threshold thinking is a category error.
- Weak evidence counts: a failing p-value still adds to the “total mix,” as the Zicam anosmia ruling held.
- Selective significance: Heritage Foundation dismissed an insignificant race effect but touted an insignificant abstinence gap.
- Arbitrary convention: .05 is Fisher’s chosen boundary, useful for discipline but not a real border in nature.
- Confidence Intervals Do More
- Range of plausible effects: confidence interval collects hypotheses that reductio ad unlikely does not force you to discard.
- Richer than p-values: intervals distinguish “+3% to +17%” (positive, maybe small) from “+9% to +11%” (positive, sizable).
- Zero inside still informs: a narrow interval around zero means little effect; a wide one means ignorance, not absence.
- Neyman’s contribution: confidence intervals shift focus from pass/fail to how much evidence and how precise.
- Two Philosophies of Inference
- Fisher’s detective: significance is a clue, not a conviction; a fact is established only if experiments rarely fail.
- Neyman-Pearson’s judge: statistics tells us what to do, not what to believe; significance tests are decision rules.
- Trial analogy: courts seek justice by rules, not truth; “guilty” means convicted under procedure, not actual guilt.
- Fisher’s objection: rigid critical regions ignore the researcher’s background knowledge and “Ought I to take notice?”
- No fixed threshold: Fisher rejected a single significance level for all circumstances; judgment must adapt to evidence.
- Replication as the Immune System
- Replication standard: novel significant findings only begin the inquiry; repeated trials decide whether they hold.
- File drawer problem: null replications often go unpublished, suppressing science’s immune response.
- Registered Replication Reports: journals accept replication protocols before results, publishing failures too.
- Many Labs project: multinational replications confirmed 10 of 13 high-profile psychology effects.
- Public nulls required: the system works only when failed experiments are reported, not hidden.
- The Arbitrary .05 Threshold
- Big Data, P-values, and Bayes (Ten. ARE YOU THERE, GOD? IT’S ME, BAYESIAN INFERENCE · I)
- Big Data's Limits
- Algorithmic promise: data-fed algorithms can infer what people can’t—and that feels scary.
- Target's pregnancy guess: lotion, supplements, and cotton balls flagged a teen’s pregnancy before her father knew.
- Asteroid vs. weather: more data sharpens some predictions, but chaos caps others.
- Chaotic ceiling: Lorenz showed a sea gull's flap can cascade; weather tops out near two weeks.
- Human behavior harder: we lack weather’s physical model, so prediction remains massively harder.
- Netflix Prize: a 10% edge took three years, was worth millions, and was obsolete by the finish.
- The Terrorist Red List
- Facebook hypothetical: score users for terror risk from public records, like Target scores pregnancies.
- Base-rate effect: terrorism is rare; doubling a tiny risk still leaves almost all flagged users innocent.
- Box arithmetic: 100,000 flagged, 10 terrorists → 99,990 innocent people on the list.
- False-positive paradox: a 0.05% false-positive rate passes the 1-in-20 bar, yet flagged neighbors are 99.99% innocent.
- Two conditional questions: P(flagged | innocent) ≈ 1/2,000, but P(innocent | flagged) = 99.99%.
- Big Data isn't magic: NSA-style metadata collection can build red lists; most people on them are innocent.
- The P-Value Trap
- Conditional probabilities: P(X | Y) is not P(Y | X).
- P-value meaning: probability of the result under the null hypothesis, not probability the null is true.
- Prosecutor's fallacy: DA quotes P(match | innocent); the jury needs P(innocent | match).
- Rampant confusion: scientists and courts keep mistaking the first conditional for the second.
- Not a paradox: keeping the two-by-two box in view dissolves the apparent contradiction.
- Radio Psychics and the Rule of Bayes
- Prior information: p-values ignore base rates; it matters that most people aren't terrorists.
- Fisher's light of evidence: judge each hypothesis in light of what you already know.
- 1937 ESP mania: Rhine's New Frontiers of the Mind and Sinclair's Mental Radio made telepathy a mainstream test case.
- Zenith's on-air trial: a roulette wheel and self-styled telepaths staged a mass experiment.
- Bayes's rule: update prior odds with observed evidence, not p-values alone.
- Big Data's Limits
- Evidence, Priors, and Perceived Randomness (Ten. ARE YOU THERE, GOD? IT’S ME, BAYESIAN INFERENCE · II)
- Perceived Randomness and Human Behavior
- RRRRR feels nonrandom but is exactly as likely as BBRBR; roulette outcomes are uniform.
- Radio psychics: listeners mailed color sequences, producing notably nonrandom patterns like BBRBR far too often.
- Random-number intuition: people favor 17 or 7, avoid 0 and 5, so imitating randomness visibly deviates.
- Human randomness fails: attempts to match random processes reveal predictable biases, not random responses.
- Voting Digits as Evidence
- Iran 2009 election: official provincial vote counts showed last digits with too many 7s.
- Random last digits of genuine counts should be evenly distributed across 0–9.
- Human fabricators exaggerate odd-looking digits; the numbers testify against the count.
- Not proof of fraud, but statistical evidence that the official totals were not random.
- Bayesian Updating from Priors to Posteriors
- Prior probability: your degree of belief in a theory before seeing new evidence.
- Box method: split prior probability by outcome likelihoods; conditionalize on observed evidence.
- Posterior probability: updated belief after evidence, computed by renormalizing surviving boxes.
- Example: five reds raise RED from 5% to 12%, nearly wipe out BLACK at 1.5%, FAIR stays 86.5%.
- Ten thousand wheels analogy: only 325 wheels produce RRRRR; among them, fair wheels dominate.
- Priors Shape How Evidence Is Read
- Same evidence, different posteriors: cynics and trusting souls update to wildly different conclusions.
- Priors are inescapable: no one judges cancer-drug results and plastic-Stonehenge results identically.
- Neighbor example: Facebook evidence is real but the prior for terrorism is tiny, so posterior stays tiny.
- Objectivity myth: pure evidence-only inference ignores the prior beliefs every scientist actually holds.
- Bayesianism and Frequentist Rivalry
- Bayes’s Theorem advises how to update degrees of belief; Fisher rejected belief-based probability.
- Fisher’s view: "no scientific worker has a fixed level of significance"—context and ideas always matter.
- Bayesian statisticians prefer predictive models and openly discuss probabilities of hypotheses.
- Null-hypothesis testing treats drug and Stonehenge alike, ignoring priors.
- Probability as degree of belief: Fisher panned Keynes’s Treatise on Probability for defending it.
- Perceived Randomness and Human Behavior
- Bayesian Priors and Design Arguments (Ten. ARE YOU THERE, GOD? IT’S ME, BAYESIAN INFERENCE · III)
- Spiky Priors Explain What Feels Random
- Spiky priors: most theories get near-zero mental weight, so RBRRB feels random while RRRRR activates the loaded-wheel theory.
- Flat priors: treating every cockamamie hypothesis as live produces constant amazement, as Feynman’s license-plate reaction shows.
- Lottery suspicions: repeated winning numbers trigger a small-prior rigging theory; the equally improbable one-off pair activates no theory.
- Round-number cue: a number ending in 0 feels less random because it suggests an estimate, not an exact count.
- Conspiracy Theories Wear Bayesian Armor
- Bayesian coating: an auxiliary theory U shields theory T from incoming evidence, leaving T+U untouched while T alone would die.
- Winnowing process: ordinary crazy beliefs fade as inconsistent evidence arrives; survivable craziness explains away each blow.
- Information ecology: successful crackpot theories are like multi-drug-resistant E. coli — flexible enough to absorb any observation.
- More Than One Theory Can Explain the Same Data
- Cleanest man in school: a lazy entrepreneur wore brand-new unsold T-shirts daily; observers inferred he was dirty because they never considered it.
- Holmes’s maxim: excluding the impossible leaves the truth only if every possible hypothesis was on the list; otherwise it fails.
- More correct version: whatever remains, however improbable, must be the truth unless it didn’t occur to you to consider it.
- Design Arguments Depend on the Hypothesis List
- Paley’s watch: a complex watch implies a maker; living complexity was held to imply God in Natural Theology.
- Bayes box for design: with flat 50/50 priors, human existence appears vastly more probable under GOD than NO GOD.
- GODS alternative: if equal prior weight goes to a squabbling committee of gods, panda-like design makes GODS win.
- SIMS alternative: simulated worlds built by humans would almost surely contain humans, so SIMS dominates the posterior.
- Reductio ad unlikely: the Bible-coder style argument “no God makes this unlikely” only works if no other theory is in play.
- Numbers Have Limits in Ultimate Questions
- Queasy feeling: the GODS and SIMS results show quantitative reasoning has hit its limits, not that we are simulations.
- Reference-class problem: “20% chance of rain” is meaningful; “20% chance God created the universe” has no satisfying numeric grounding.
- Faith is non-quantitative: math is silent on God; Pascal’s Pensées says reason can decide nothing here.
- School board test: teaching design “reasonably” would push students to simulation or Greek gods, not single-creatorism.
- Spiky Priors Explain What Feels Random
- Bible Codes and Statistical Skepticism (Six. THE BALTIMORE STOCKBROKER AND THE BIBLE CODE · I)
- PART III: Expectation
- Lotto Math, Expected Value, and Loopholes (Eleven. WHAT TO EXPECT WHEN YOU’RE EXPECTING TO WIN THE LOTTERY · I)
- The Lottery's Long Shadow
- Genoa origins: lottery evolved from drawing city officials by lot; numbers replaced politicians by 1700.
- Early adopters: Continental Congress and Harvard ran lotteries to fund the Revolution and dormitories.
- Adam Smith's warning: in The Wealth of Nations, lotteries overvalue gain, but his more-tickets claim is mathematically false.
- Smith's missing tool: expected value formalizes why lottery tickets are usually bad bets without claiming every ticket is irrational.
- Expected Value Is Not What You Expect
- Expected value: multiply each outcome's probability by payoff and sum; a $1 ticket worth 60 cents is overpriced.
- Misleading name: an expected value need not be a possible outcome; a $10 expectation can come only from $100 or $0.
- Long-run average: repeated identical bets converge to expected value through the Law of Large Numbers.
- Halley's annuities: the 1692 Million Act priced annuities without age; Halley used mortality data to set fair lifetime values.
- Hindsight bias: "charge the young more" seems obvious now, yet such ideas arrived late and are not naturally obvious.
- Powerball's House Edge
- Base expected value: with a $100 million jackpot, a $2 Powerball ticket wins just under 94 cents on average.
- Jackpot threshold: algebra puts the break-even jackpot at about $285 million before accounting for sharing.
- Shared jackpots: with 75 million players, a 35% chance of splitting cuts a $337 million prize below the threshold.
- Hidden haircuts: taxes, annuity disbursement, and debt seizure reduce the real payout further.
- The Certified Strategy
- First rule: don't play Powerball; lotteries return part of sales to the state, so average tickets lose.
- Second rule: if you play, wait for a truly huge jackpot, but factor in ticket sales and splitting.
- Third rule: pick unpopular numbers; avoid birthdays, previous winners, patterns, and fortune-cookie numbers.
- Cash WinFall's "Scam"
- Suspicious order: an MIT student presented 14,000 handwritten slips for $28,000 in Cash WinFall tickets.
- Waiver wave: twelve Boston-area stores sought sales waivers before the July 14 drawing, including a cluster in Quincy.
- Plain-sight answer: the cause was in the game's rules, not in fraud.
- The Lottery's Long Shadow
- Expected Value, Additivity, and Lotteries (Eleven. WHAT TO EXPECT WHEN YOU’RE EXPECTING TO WIN THE LOTTERY · II)
- Roll-Down Mechanics
- Accidental edge: Massachusetts designed Cash WinFall to seem like a good deal—and it actually was one.
- Roll-down rules: unclaimed jackpots over $2 million rolled down to smaller prizes; the jackpot reset to $500,000.
- Ordinary days: expected value of a $2 ticket was about $0.80—a poor bet.
- Roll-down days: match-4 prizes could reach ~$2,385, making a ticket's expected value ~$5.53.
- The value you expect: a typical ticket still loses; expected value is only realized across many tickets.
- The Syndicates Move In
- James Harvey: MIT senior spotted the edge and founded Random Strategies, buying thousands of tickets.
- Doctor Zhang: Boston medical researcher quit his job and ran a club buying $300,000 per roll-down.
- Gerald Selbee: Michigan retiree added profit by bargaining for half the store's 5% sales commission.
- Saturation: heavy syndicate buying shrank match-4 prizes from ~$2,300 to ~$800.
- The State's Bottom Line
- State's cut: 40% of ticket revenue goes to the state, the other 60% to prizes.
- Break-even: 2.5 million roll-day tickets sold makes the state's 40% cut equal the $2 million pot; beyond that, players lose.
- Zero-sum: whatever the state makes, players on average lose, and vice versa.
- Additivity of Expected Value
- Additivity formula: E(X+Y) = E(X) + E(Y) for any uncertain quantities X and Y.
- Lottery insight: total payout on roll-down is fixed, so average ticket value equals that total divided by tickets sold.
- Subtlety: most-likely values don't add—each heir may be most likely to get nothing, yet the total is fixed.
- Buffon's Needle
- Buffon's needle: thrown on a slatted floor, it crosses a crack with probability 2/π, about 64%.
- Franc-carreau: symmetric coin-on-tile game with probability ((L−2r)/L)².
- Barbier's trick: solve the harder long-needle version first; additivity splits it into short needles.
- Additivity at work: expected crossings of a length-L needle = Lp; at length 1, that's exactly the probability p.
- Roll-Down Mechanics
- Needles, Noodles, and Lottery Cartels (Eleven. WHAT TO EXPECT WHEN YOU’RE EXPECTING TO WIN THE LOTTERY · III)
- Barbier’s Bent-Needle Proof
- Additivity: expected crossings add segment by segment, so bent needles obey the same length rule as straight ones.
- Length rule: any needle of length L in slat-width units has expected crossings Lp.
- Polygonal approximation: a 65,536-gon standing in for a circle is just many tiny needles; the floor can’t tell.
- Circle symmetry: a circular hoop crosses floor lines exactly twice, restoring symmetry lost with the coin.
- The Sea and the Stone
- Two mathematical styles: blast the unknown with dynamite, or let understanding rise like the sea around it.
- Grothendieck’s method: contemplation surrounds the resisting stone until, gradually, it is overtopped and gone.
- Deligne’s verdict: “Nothing seems to happen, and yet at the end a highly nontrivial theorem is there.”
- Barbier’s route: the bent-needle argument is sea-like, replacing brute-force computation with a slow rise in insight.
- Mathematicians and the Madness Myth
- Math Melodrama: popular culture casts mathematicians as Promethean geniuses whose brilliance is also a fatal flaw.
- Real mathematicians: ordinary people; math strengthens the mind more often than it breaks it.
- Barbier and Grothendieck: both left academic life—Barbier to an asylum, Grothendieck to the Pyrenees—but that’s not the norm.
- Math as meditation: in emotional extremity, a problem quiets the psyche and puts you in contact with the universe.
- The WinFall Cartels in Action
- Three cartels: Selbee, Doctor Zhang, and Random Strategies dominated every roll-down drawing with high-volume tickets.
- Cooperation refused: Selbee rejected sharing roll-downs with a rival; exploiting public rules felt fair, colluding did not.
- Grind, not glamour: hand-bubbling, scanning, sorting, and storing losing tickets for IRS audits made it a job.
- August 2010 play: Random Strategies quietly bought 700,000 tickets, triggered the roll-down alone, and made 50% profit.
- Game’s end: Boston Globe exposure led to retailer sales caps; the last WinFall drawing was a roll-down in 2012.
- The House, the State, and the Players
- Revenue flow: state keeps 80 cents per $2 ticket, puts $1.20 in prizes; roll-downs release accumulated jackpot money.
- Cartels as house: high-volume bettors didn’t beat the state; they collected the money ordinary players had lost.
- State as state: Massachusetts took its 40% cut of cartel tickets, like Nevada taxing casino winnings.
- Historical precedent: Voltaire and La Condamine exploited a mispriced French bond lottery with a ticket-buying cartel.
- Disputed harm: Inspector General saw no scam—odds were unchanged—but cartel volume sliced every roll-down prize into smaller pieces.
- State gamble: officials hoped 3.5 million ordinary players would show up; then the state profits and the loophole closes.
- Barbier’s Bent-Needle Proof
- Expected Value in Life's Tradeoffs (Twelve. MISS MORE PLANES! · I)
- Stigler's Airport Tradeoff
- Stigler's rule: never missing a plane means spending too much time in airports.
- Expected-value frame: weigh wasted terminal time against the cost of missing the flight.
- Illustrative options: with 6 utils per missed flight, 1.5 hours (5% miss) beats 2 hours (2%) or 1 hour (15%).
- Optimal risk: the best strategy always assigns some positive probability of missing a flight.
- Near zero ≠ zero: 1% risk may mean never missing in a lifetime; only not driving has zero crash risk.
- Subjective optimum: how much you hate missed flights moves the optimum, not the principle.
- The Optimal Rate of Government Waste
- Waste elimination has costs: vigilance, like early airport arrival, can cost more than it saves.
- SSA example: $31 million in improper payments to deceased beneficiaries is .004% of disbursed benefits.
- Already excellent: the agency rarely errs; squeezing out the last errors would be expensive.
- Right question: ask what amount of taxpayer waste is optimal, not why any exists.
- Stigler's corollary: if your government isn't wasteful, you're spending too much time fighting waste.
- Pascal's Wager as Expected Value
- Pascal's turn: co-founded probability with Fermat, then a mystical "night of fire" turned him to faith.
- The wager: belief offers infinite joy if God exists, finite cost if not; expected value is infinite.
- Nonzero probability suffices: any chance of God, however small, outweighs finite costs.
- Cat in the Hat flaw: possible gods who damn Christians add infinite downside, killing the wager.
- Voltaire's objection: interest in believing is no proof God exists; Pascal offers utility, not evidence.
- Fake it till you make it: Infinite Jest describes adopting slogans until belief arrives, like Pascal's counsel.
- Utils, Subjective Costs, and Decisions
- Utils measure annoyance: nonmonetary costs like waiting are converted into common units.
- Dollar values too: utility applies even to well-defined monetary outcomes; early puzzles revealed this.
- Calibration changes choice: missing a plane at 6 utils favors the middle option; 20 utils favors earlier arrival.
- Laffer-curve shape: too little and too much buffer both hurt; optimum lies between extremes.
- Decision without certainty: Pascal and Neyman-Pearson: act even when evidence cannot settle truth.
- Transition to St. Petersburg: utility extended to dollars through an early probability puzzle.
- Stigler's Airport Tradeoff
- Utility, Risk, and Uncertainty (Twelve. MISS MORE PLANES! · II)
- The St. Petersburg Paradox
- Game: Peter doubles Paul’s payoff until the first heads; expected value becomes 1/2 + 1/2 + … without bound.
- Paradox: the math says Paul should pay any entry fee, yet no sane person would.
- Origin: Nicolas Bernoulli devised the puzzle; Daniel Bernoulli resolved it thirty years later.
- Moral: a divergent answer signals a kink in math or intuition—not a place to stop.
- Bernoulli’s Utility Fix
- Ducats differ: a ducat in a rich man’s hand is worth less than in a peasant’s.
- Diminishing utility: two thousand ducats is less than twice as good as one thousand.
- Logarithmic value: utility grows like digit count; the 2^k-ducat prize is worth k utils.
- Tamed sum: triangular summation turns the infinite expected utility into exactly 2 utils.
- Utility Is Personal
- No universal curve: the utility of money bends differently across people and contexts.
- Mankiw effect: higher taxes make him work less, because lost family time outweighs diminished pay.
- Lebowitz effect: she stops driving once rent and food are covered, so taxes make her work more.
- Implication: economic predictions built on one person’s utility curve are fragile.
- Moral Considerations
- Cramer and Buffon independently saw that moral value, not money, drives rational choice.
- Buffon’s insight: money beyond certain limits has almost no real value; a mountain of gold adds nothing more than a cubic fathom.
- Laplace’s celebration: probability theory is common sense reduced to calculus, determining the most advantageous choice.
- Ellsberg’s Paradox
- Urn setup: thirty red balls; sixty black-or-yellow in unknown split; bets alternate on colors.
- Known beats unknown: people prefer RED over BLACK and NOT-RED over NOT-BLACK, preferring known odds.
- Same bundle: RED + NOT-RED pays $100, just like BLACK + NOT-BLACK — violating utility axioms.
- Diagnosis: expected utility treats known unknowns and unknown unknowns as interchangeable; people don’t.
- Limits of the Formal Model
- Risk vs uncertainty: risky choices have quantifiable probabilities; uncertain ones lack any probability basis.
- Ellsberg’s challenge: the axioms give wrong predictions and bad advice; subjects violate them deliberately as sensible.
- Cold War stakes: decision theory was revered as a war-winning science, so doubting it carried political weight.
- Ellsberg’s arc: that early mathematical doubt may have fed his later Pentagon Papers rebellion.
- The St. Petersburg Paradox
- Utility, Variance, and Geometry (Thirteen. WHERE THE TRAIN TRACKS MEET · I)
- Utility and Risk
- Expected value: a 50/50 bet on losing $100,000 or gaining $200,000 equals $50,000, but real people should not take it.
- Utility: gains and losses carry nonlinear worth; a ruinous loss can outweigh a life-changing win.
- Risk tolerance: the richer you are, the more risks you can afford to take.
- Steamroller trades: a 99% chance of profit and 1% chance of catastrophe is “picking up pennies in front of a steamroller.”
- Too big to fail: down a million is your problem; down five billion is the government’s problem.
- Variance and Diversification
- Variance: measures how widely possible outcomes spread, including how likely extremes are.
- Risk preference: when expected dollar value matches, most people prefer the lower-variance option.
- Bonds vs stocks: bonds trade some expected return for lower variance; stocks likely do better but may end up worse.
- Diversification: splitting holdings across many uncorrelated assets lowers portfolio variance.
- Index funds: A Random Walk Down Wall Street champions this dull strategy—it works.
- Negative expected value: when the bet is a sure loss, gamblers chase higher variance, not lower.
- Random vs Deliberate Lottery Strategies
- Cash WinFall: Quic Pic and hand-picked tickets have identical expected value; variance is another story.
- Duplicate tickets: buying 300,000 random tickets from 10 million options makes repeats nearly certain.
- Birthday problem: shared birthdays are likely with 30 people because pairs multiply: 435 pairs, over 70% chance.
- Transylvanian lottery: with seven balls, hitting two of three numbers scores a “deuce.”
- Selbee vs Harvey: both expect 2.4 deuces, but Harvey’s hand-picked set has far smaller variance.
- Deliberate selection: Harvey’s tickets guarantee either the jackpot or exactly three deuces—a minimum payoff.
- The Combinatorial Explosion
- Combinatorial explosion: simple operations can balloon manageable numbers into impossible ones.
- Brute force: picking among all 300,000-ticket sets from 10 million options would outlast the universe.
- Traveling salesman: routing through fifty states means about 30 vigintillion possibilities—a different scale.
- Train Tracks and Perspective
- Parallel lines: rails appear to meet at the horizon; perspective is the cost of fitting 3-D into 2-D.
- Renaissance painters: Brunelleschi and the Florentines joined art with mathematics and optics.
- Alhazen: the eleventh-century Cairene mathematician detailed vision by reflected light rays.
- Ancient theories: Alcmaeon and Plato thought the eye emitted fire, citing phosphenes.
- Plane geometry: the promised key to low-variance lottery tickets lies in the geometry of train tracks.
- Utility and Risk
- From Projective Geometry to Error-Correcting Codes (Thirteen. WHERE THE TRAIN TRACKS MEET · II)
- Sight, Canvas, and Vanishing Points
- Optical lineage: Kitab al-Manazir reached Latin readers, enabling systematic theory of sight and painting.
- Point-to-line mapping: a point P on the canvas represents one unique line in space — the one through P and the eye.
- Rails become planes: each rail R determines the plane through the eye and R; its cut across the canvas is the painted rail.
- Vanishing point: the rail planes' intersection line, parallel to the tracks, hits the canvas where painted tracks converge.
- All parallels converge: every pair of parallel prairie paths shares the same vanishing point; only canvas-parallel lines stay parallel.
- Points at Infinity and Projective Plane
- Conceptual shift: instead of points in the landscape, think of lines through the eye; some lines never meet the ground.
- Points at infinity: horizontal lines through the eye correspond to vanishing points — "infinitely far" directions on the ground.
- Projective plane: add one point at infinity per direction; parallel lines meet, so geometry needs no exceptions.
- Elegant tradeoff: hard to draw, but its axioms allow no exceptions — two points determine a line, two lines meet.
- Literary echoes: Churchill saw a quantity pass through infinity and change sign; Wallace's ending depends on projected parallel convergence.
- Finite Geometries and Fano's Plane
- Finite geometries: many systems satisfy the projective axioms beyond Brunelleschi's plane, including finite ones.
- Fano plane: seven points and seven three-point lines; the circle in its drawing counts as a line.
- Fano's habit: geometry is defined by behavior, not looks; if it walks and quacks like geometry, it is geometry.
- Lottery application: labeled Fano lines 124, 135, 167, 257, 347, 236, 456 cover every pair once — the Transylvanian lottery solution.
- From Pipes to Error-Correcting Codes
- Noisy communication: cosmic rays can flip bits; repeating each bit three times lets majority rule cancel errors.
- Shannon's capacity: A Mathematical Theory of Communication showed noise caps channel capacity; robustness costs speed.
- Hamming's motivation: weekend-only relay computer crashes annoyed him into designing self-correcting codes.
- Math expands: probability and infinity were once unthinkable mathematically; communication theory continues the expansion.
- Glory of math: correct ideas stay correct far outside their birth context — Fano's geometry found use in lotteries and codes.
- Sight, Canvas, and Vanishing Points
- Error-Correcting Codes Meet Geometry (Thirteen. WHERE THE TRAIN TRACKS MEET · III)
- Secret Geometry of the Hamming Code
- Hamming code: transforms 3-bit blocks into 7-bit code words; any other string on the wire is a sure error.
- Fano plane: the seven nonzero code words match the seven lines, written as 0/1 strings.
- Unified object: Hamming code, Fano plane, and optimal Transylvanian lottery ticket are the same math in different outfits.
- Decoding rule: one flipped bit leaves two or four points; complete the line or pick its only line.
- Uniqueness: Fano lines share at most one point, so a four-point error contains exactly one line.
- Hamming Distance and Information Geometry
- Hamming distance: number of bit changes needed to convert one block into another; closeness is literal.
- Code separation: any two code words differ in at least 4 places, so a single flip cannot be ambiguous.
- Hamming spheres: strings within distance 1 of a code word; error correction means no two spheres overlap.
- Efficiency: seven bits per three message bits beat repetition's factor of three.
- Packing analogy: code words maximize mutual distance, like electrons in a box or antisocial elevator riders.
- Error-Correcting Codes in the Wild
- Engineering shift: after Hamming and Shannon, systems need only make errors rare; codes absorb the noise.
- Applications: Mariner 9 used Hadamard code; CDs use Reed-Solomon; flash drives use BCH codes.
- Checksums: bank routing numbers detect one wrong digit but cannot correct it.
- Patent story: Bell patented the formulas; Golay published first; an antitrust settlement ended license fees.
- Language as an Error-Correcting Code
- Natural redundancy: English corrects typos like "lanvuage" because no other word is one letter away.
- Semantic distance: context disambiguates short words — "bit you" suggests dog, "fell off" suggests log.
- Ro: compact logical vocabulary crowds words, so one-letter slips create false meanings.
- Lojban: successful constructed language forbids phonetic closeness among roots, implementing Hamming's principle.
- Sphere Packing, from Oranges to the Leech Lattice
- Kepler's conjecture: densest 3D packing is face-centered cubic; each sphere touches twelve neighbors.
- Pomegranate seeds: twelve-sided seeds arise from pressing against twelve packed neighbors.
- Leech lattice: Golay's code yields a dense 24-dimensional packing; each sphere touches 196,560 neighbors.
- Near optimal: Cohn and Kumar showed any denser lattice beats Leech by at most 1.00000000000000000000000000000165.
- Symmetries: Conway's twelve-hour paper-roll calculation revealed the lattice's exotic symmetries.
- Proof, Computation, and Confidence
- Kepler resolved: Hales proved the conjecture in 1998 using a massive computer calculation.
- Computer proof unease: code cannot be checked like a proof; this discomfort drove Hales's turn.
- Formal verification: Hales envisions future math where complex, interdependent proofs are mechanically verified.
- Secret Geometry of the Hamming Code
- Proofs, Random Codes, and Lottery (Thirteen. WHERE THE TRAIN TRACKS MEET · IV)
- Machine-Verified Mathematics
- Classification of finite simple groups: Ten thousand pages across hundreds of papers; no human alive understands it all.
- Hales's solution: Rebuild the mathematical corpus inside a formal structure a machine can verify.
- Checkable checker: If the proof-checking program is itself checkable, proof controversies can permanently end.
- Machine ideas: Next, computers may construct proofs, or even have ideas, without human intervention.
- Mathematics survives: Short of total machine dominance, math persists; automated research gets reclassified as computation.
- Shannon's Random Codes
- Shannon's theorem: At one-per-thousand error rate, codes can add just 1.2% overhead and meet any reliability target.
- Randomness suffices: Almost every random set of code words is error-correcting; no design needed.
- Hamming's tribute: He praised Shannon's courage in asking what an average random code would do.
- Structure versus randomness: Structured codes like Hamming's are easy to decode but less efficient than random codes.
- Decoding trap: A quadrillion-word random code cannot be decoded by brute force; the practical search rides the boundary.
- Denniston's Lottery System
- Denniston's list: 285,384 six-number tickets from 48 numbers; no two share five numbers.
- Miracle property: Every five-number combination appears exactly once, making a code with Hamming distance at least four.
- Cash WinFall adaptation: Dropping tickets with 47 or 48 fits the 46-number game; 217,833 tickets remain.
- Guaranteed prizes: Buying all adapted tickets guarantees at least four of six five-out-of-six prize chances.
- Risk comparison: Random Quick Picks share expected value but risk fewer prizes; Denniston removes risk—but only with the right tickets.
- Explaining Lottery Demand
- Popularity puzzle: Lotteries thrive despite negative expected value; economists have puzzled since Adam Smith.
- Friedman–Savage curve: People value wealth by class; small losses barely hurt, jackpots change social stratum.
- Prospect theory: Kahneman and Tversky showed people overweight rare events, making jackpots more alluring than strict utility allows.
- Simple fun: A ticket buys anticipation; Pascal saw gamblers craving the sting of possible winning, not amusement alone.
- The Utility Defense and Its Limits
- Utility is not conserved: If a ticket is genuinely fun, both player and state gain; math gives permission.
- Meth caveat: The same win-win reasoning would bless meth dealers and clients; mutual enjoyment alone doesn't settle policy.
- Entrepreneurship analogy: Starting a business is usually a bad bet, yet society honors the attempt because value exceeds dollars.
- Harvey and Lu: The former WinFall players founded a startup, still chasing dreams beyond expected value.
- Machine-Verified Mathematics
- Lotto Math, Expected Value, and Loopholes (Eleven. WHAT TO EXPECT WHEN YOU’RE EXPECTING TO WIN THE LOTTERY · I)
- PART IV: Regression
- The Math Behind Mediocrity (Fourteen. THE TRIUMPH OF MEDIOCRITY · I)
- Secrist's Doomy Data
- Secrist's study: 468-page The Triumph of Mediocrity in Business tracked hundreds of firms from hardware to banking.
- The pattern: top-sextile stores lost advantage; worst stores improved — both regressed toward average.
- Universal effect: same regression appeared in clothing, groceries, hardware, by every metric Secrist tried.
- Contrast: July city temperatures showed no regression — hottest cities stayed hottest.
- Secrist's Moral: Competition Breeds Mediocrity
- Secrist's verdict: free competition perpetuates mediocrity; superior judgment is at the mercy of the unscrupulous and unwise.
- Schoolhouse analogy: ungraded one-room schoolhouse dilutes superiority — echoing the era's eugenicist anxieties.
- Galton's lineage: Secrist's views descend from Francis Galton, pioneering eugenicist and Darwin's cousin.
- Galton's Genetic Pursuit
- Hereditary Genius: natural abilities are inherited; judicious marriages could breed a highly-gifted race.
- Darwin's frenzy: Darwin called it the most interesting and original book he'd read — though cousinly bias helped.
- Height data: tall parents have tall children, but not as tall; short parents' children are short, but less so.
- Necessary fact: offspring stature must on the whole be more mediocre than parents' stature.
- The Mathematical Explanation
- Two ingredients: height and business success combine hereditary disposition with environment and chance.
- Extremes are lucky: the tallest people almost certainly had good genes plus an upward boost from luck.
- Luck doesn't transmit: children inherit genes, not lucky breaks, so they fall back toward the average.
- No mystery force: regression is not mediocrity-loving; it's a mathematical necessity of signal mixed with noise.
- Secrist's Doomy Data
- Regression Is Universal and Mathematical (Fourteen. THE TRIUMPH OF MEDIOCRITY · II)
- The Ubiquity of Regression
- Regression effect: any mix of stable skill and chance makes extreme observations drift toward the mean.
- Diet illusion: people usually start diets at a weight peak, so pounds can drop with or without the diet.
- Sophomore slump: breakout artistic success blends talent and luck, so follow-ups usually look weaker.
- Contract-year letdown: running backs' big years signal both skill and luck, making a return to normal likely.
- The "On Pace" Fallacy
- False linearity: projecting April's hot start over a full season ignores that the leader is partly lucky.
- Matt Kemp: "on pace" for 86 home runs in 2012; he finished with 23 after hamstring injury and regression.
- Second-half decline: first-half AL home run leaders averaged only 60% of that pace after the All-Star break.
- No Derby curse: Home Run Derby participants regress because they were chosen for unusually hot first halves.
- Secrist's Mistake
- Secrist's finding: stores at the top in 1916 drifted toward mediocrity by 1922, which he blamed on competition.
- Hotelling's rebuttal: the pattern is mathematically automatic whenever stable factors and chance both operate.
- Backward-time test: if competition caused regression, the 1922 best should also be mediocre in 1916—they weren't.
- Elephants analogy: proving a trivial mathematical truth with mountains of business data is like testing multiplication with elephants.
- Half-Understood for a Century
- Weldon's warning: regression also runs from children to parents, so it cannot be a one-way biological force.
- Bran study: both slow and fast transit times moved toward 48 hours—exactly the pattern chance alone produces.
- Scared Straight: falling arrest rates after the program mirror Secrist's low performers regressing upward without treatment.
- Randomized tests: when properly randomized, Scared Straight increased antisocial behavior instead of reducing it.
- The Ubiquity of Regression
- The Geometry of Correlation (Fifteen. GALTON’S ELLIPSE · I)
- From Numbers to Scatterplots
- Scatterplot: Galton plotted father-son heights as coordinate pairs, visualizing data via Descartes’ analytic geometry.
- Visual power: brains excel at pattern-finding in 2-D space, not columns of numbers.
- Golden age of visualization: Minard’s Napoleon chart and Nightingale’s coxcomb turned data into insight.
- Chance, Heredity, and Conditional Expectation
- No chance: if sons equal fathers, all points lie on the diagonal x = y.
- No heredity: if heights are independent, the scatterplot is round and conditional expectation equals the unconditional mean.
- Galton’s middle case: sons of tall fathers are shorter than Dad but taller than average — regression to the mean in action.
- Galton’s Ellipse and Correlation
- Elliptical isopleths: Galton drew density contours; all shared a center at average heights.
- Bivariate normal: the “mountain” behind the ellipses is de Moivre’s hat in two dimensions.
- Eccentricity: skinny ellipses mean strong heredity and weak regression; round ellipses mean weak correlation.
- Correlation defined: Galton’s geometric measure of association became his lasting statistical term.
- Empirical ubiquity: votes, stock prices, SAT/tuition, and state wealth all form rough ellipses.
- Why Ellipses? The Ubiquity of Quadrics
- Simple curves dominate: mathematics offers few simple objects, so nature’s approximate solutions often land on them.
- Conic sections: ellipses, parabolas, and hyperbolas are the three quadric classes, appearing from orbits to rainbows.
- Dickson’s proof: Galton concealed his data’s origin; the Cambridge mathematician showed theory demanded the ellipse.
- Bertillonage and the Reach of Correlation
- Bertillon system: numerical body measurements replaced haphazard suspect identification, spreading worldwide.
- Correlation insight: bodily measurements are not independent, so more measurements don’t guarantee sharper identification.
- Modern echo: Justice Kennedy likened DNA databases to twenty-first-century Bertillon cards.
- From Numbers to Scatterplots
- Correlation, Redundancy, and Information (Fifteen. GALTON’S ELLIPSE · II)
- Correlation and Redundant Measurements
- Redundant measures: if traits are correlated, recording both adds little new information
- Galton’s cubit plot: height versus cubit showed the same ellipse — correlation without strict determination
- Optimal choice: take measurements uncorrelated with every other entry on the card
- Dactyloscopy versus Bertillonage
- Bertillon’s weakness: correlated physical features shrank the effective number of identifying categories
- Fingerprints win: prints are often available where the criminal himself is not
- Peruggia’s theft: his Bertillon card was useless; the print on the Mona Lisa’s discarded frame would have named him
- Correlation, Information, and Compression
- Parsons code: melodies become strings of u, d, r symbols; ruurdddd identifies Ode to Joy
- Exponential discrimination: seventeen notes yield 43 million codes — more than all recorded melodies
- Shannon’s insight: stronger correlation means less information, in the precise information-theoretic sense
- Compression: images shrink because neighboring pixels are highly correlated with one another
- Signal, Noise, and Regression
- Wisconsin weather: similar stations regress toward the mean yearly — noise dominates signal
- California contrast: stable climate differences swamp chance — signal dominates, no regression appears
- Secrist’s real lesson: businesses resemble Wisconsin stations; luck and skill matter in roughly equal measure
- The Limits of Mathematical Rightness
- Galton’s eugenics: mathematical clarity did not prevent his appalling compulsory-sterilization views
- Math is partial: a way not to be wrong, but not a way not to be wrong about everything
- Confidence trap: analytical power can breed unjustified confidence in domains where we are still mistaken
- Pearson’s Generalized Correlation
- Agnostic association: Galton let researchers discuss association without committing to any underlying cause
- Darwinian echo: as Darwin separated progress from purpose, Galton separated association from cause
- Pearson’s formula: generalized correlation to arbitrary variables, with a clean geometric description
- Correlation and Redundant Measurements
- Correlation’s Hidden Geometry (Fifteen. GALTON’S ELLIPSE · III)
- Data Are High-Dimensional Vectors
- Shift invariance: adding the same constant to both variables doesn’t change correlation; it only shifts Galton’s ellipse.
- Data as vectors: a column of ten temperatures is a point in ten-dimensional space via Descartes’ coordinates.
- High-dimensional sight: we compute in n dimensions, but mental pictures stay two- or three-dimensional.
- Real-world dimensions: a four-megapixel photo is a 4-million-dimensional vector; video is a curve in that space.
- Correlation Is an Angle
- Geometric correlation: Pearson’s r is the cosine of the angle between the two mean-centered vectors.
- Sign by angle: acute angle means positive correlation; obtuse means negative; right angle means zero.
- Orthogonal: mathematicians use it for unrelated or irrelevant, and the usage is creeping into broader language.
- Trigonometric serendipity: Hipparchus invented cosine for eclipses; a right mathematical tool keeps finding new uses.
- Correlation Is Not Transitive
- Nontransitivity: correlation between A–B and B–C doesn’t force correlation between A–C.
- Geometric proof: two acute angles between adjacent vectors don’t guarantee the outer vectors point together.
- Political example: wealth correlates with rich-state status, rich-state status with Democratic voting, but rich individuals still lean Republican.
- Pundit bias: pundits generalize from rich liberal enclaves to the whole country.
- Mutual-fund DNA: Laura and Tim share half of Sara’s portfolio, but share no common stock with each other.
- Broken Medical Chains
- Niacin paradox: niacin raises HDL, high HDL predicts fewer heart events, but a large trial found no fewer events.
- HRT reversal: estrogen correlated with lower risk, yet hormone-replacement therapy appeared to raise risk in trials.
- Biomarker limits: chaining correlations can’t predict drug effects in an immensely complex human body.
- Trials required: plausible interventions must still be experimentally tested; most fail.
- Uncorrelated Doesn’t Mean Unrelated
- Linear scope: correlation only detects straight-line relationships between variables.
- Political poll: ideology and informedness were uncorrelated, yet the scatterplot formed a heart shape.
- Polarization insight: more informed voters move left or right; uninformed voters cluster in the center.
- Camera analogy: every tool detects some signals, not others; no linear correlation isn’t no relation.
- Data Are High-Dimensional Vectors
- Correlation, Causation, and Lung Cancer (Sixteen. DOES LUNG CANCER MAKE YOU SMOKE CIGARETTES? · I)
- Correlation and Binary Variables
- Negative correlation: married people are less likely than average to smoke — equivalently, smokers are less likely than average to be married.
- Mathematical equivalence: both inequalities reduce to the same product comparison: married smokers × all people < smokers × all married people.
- Ubiquitous correlation: without exact equality, every pair of binary variables correlates at least slightly; the null hypothesis is almost always false.
- Significance as signal: reported correlations are those strong enough to pass statistical tests, prompting us to ask what might be going on.
- Correlation Is Not Causation
- Indicative vs. subjunctive: "smokers are less likely to marry" states a fact; "if you were a smoker, you'd be less likely to marry" suggests an intervention.
- Causal inference gap: correlation has had rigorous mathematics since Galton and Pearson; causation still lacks a comparable foundation.
- Surrogate endpoint problem: HDL correlates with lower heart-attack risk, but an unmeasured factor could cause both; HDL-raising drugs may not help.
- Tim and Sara: correlated fund successes trace to a shared Honda stock, not to either one causing the other.
- The Smoking–Lung Cancer Debate
- Epidemic rise: lung cancer went from rare to a fifth of British male cancer deaths by 1947, too fast for better diagnosis to explain.
- Doll and Hill (1950): among 649 male lung cancer patients, only 2 were nonsmokers versus 27 among controls.
- Dose response: heavy smokers were overrepresented among lung cancer patients, strengthening the association.
- Correlation, not explanation: the authors conceded lung cancer causing smoking or a common cause as logical alternatives.
- Fisher's Skeptical Challenge
- Common-cause hypothesis: Fisher proposed that genetics or precancerous inflammation could drive both smoking and lung cancer.
- Sympathetic polemic: he likened taking cigarettes from a pre-cancerous smoker to taking a blind man's white stick.
- Twin evidence: identical twins match in smoking status more than fraternal twins, suggesting some heritable influence on smoking.
- A worthy opponent: Vandenbroucke found Fisher's tobacco papers impeccably logical — "if only the authors had been on the right side."
- Toward Consensus
- Burney's 1959 claim: the surgeon general declared the "weight of evidence" made smoking the principal factor in lung cancer.
- JAMA rebuttal: Talbott demanded definitive studies; both statements turned out to be ghostwritten by rival PHS scientists.
- Terry's 1964 commission: cigarette smoking "contributes substantially" to mortality and warrants remedial action, backed by consistent studies.
- Correlation and Binary Variables
- Action Under Uncertainty and Berkson's Fallacy (Sixteen. DOES LUNG CANCER MAKE YOU SMOKE CIGARETTES? · II)
- Policy Must Act on Probability, Not Proof
- Smoking evidence: dose response, site-specific cancers, and ex-smoker benefits led the surgeon general to declare causation.
- Science vs policy: scientists specify how uncertain we are; policymakers decide how to act under that uncertainty.
- Impossible trial: randomizing teenagers to fifty years of smoking is conceivable but ethically impossible.
- Cost of certainty: if tobacco were later exonerated, the campaign would still have been a reasonable bet ex ante.
- Expected Value Justifies Acting
- Slogan: it is not always wrong to be wrong; expected value, not certainty, should guide action.
- Eggplant example: 75% chance of saving 1,000 and 25% risk of 200 deaths yields 700 expected lives saved.
- Repetition: individual campaigns may fail, but repeated similar bets reliably save lives on average.
- Subjective probabilities: real health questions lack coin-flip odds; only degrees of belief are available.
- Stigler's maxim: never giving advice until certain means not giving enough advice.
- Berkson's Fallacy: Correlations from Common Effects
- Berkson's fallacy: correlations can arise from a shared effect, not only a shared cause.
- Old-school epidemiology: Berkson insisted cancer is a biologic problem, not a statistical one.
- Hospital illusion: among inpatients, high blood pressure looked protective against diabetes.
- Selection mechanism: hospitalization is a common effect; one condition's absence makes another more likely to explain admission.
- Phantom correlation: the hospital association is neither causal nor a hidden common cause.
- Everyday Selection Effects
- Great Square of Men: looks and niceness are uncorrelated in the population, but negatively correlated in your dating pool.
- Acceptable Triangle: only nice-or-handsome men enter the visible set; mean uglies are never considered.
- Fallacy in action: trying to make your boyfriend meaner to improve his looks mistakes selection for causation.
- Literary snobbery: popular novels look terrible because only popular or good books reach visibility.
- Great Hypercube: many dimensions, diverse preferences, and opinion aggregation make selection effects richer.
- Policy Must Act on Probability, Not Proof
- The Math Behind Mediocrity (Fourteen. THE TRIUMPH OF MEDIOCRITY · I)
- PART V: Existence
- Public Opinion Is Mathematically Incoherent (Seventeen. THERE IS NO SUCH THING AS PUBLIC OPINION · I)
- The Self-Contradicting Polls
- Budget paradox: Americans want spending cuts, yet oppose cutting nearly every program; only foreign aid and unemployment insurance get axed.
- State-level impasse: no single option for balancing budgets—cuts or taxes—draws majority support.
- Forest-and-trees summary: the public wants to cut down the forest but keep the trees.
- Free-lunch diagnosis: Caplan says voters want smaller government without touching main functions; Krugman: a mandate to repeal arithmetic.
- Rational Voters, Irrational Majority
- Word problem: three factions split on taxes, defense, and Medicare produce contradictory majority preferences.
- Individual coherence: each voter's stance is rational; the aggregate public position is nonsensical.
- Selfishness, not stupidity: most Americans protect programs that benefit them personally.
- No worthless-program consensus: voters agree waste exists, not on which programs are worthless.
- Majority Rule Breaks With Three Options
- Three-way choices: with more than two options, majority preferences become contradictory.
- Obamacare poll: repeal 37%, weaken 10%, leave 15%, expand 36%; each option opposed by most.
- Framing exploitation: Fox can report "majority oppose Obamacare"; MSNBC can report "majority preserve or strengthen."
- 1992 election: Clinton won 43% while majorities opposed Clinton, Bush, and Perot.
- Perot counterfactual: 13% Perot voters preferring Bush can make Bush the majority favorite over Clinton.
- Governing Without Public Opinion
- No public opinion: it exists only when a clear majority holds, as on terrorism or The Big Bang Theory.
- Politician's options: mediocre cites contradictions; good says lead, not watch polls; master exploits incoherence.
- Governor's two-choice trick: exclude the most popular tax hike, force localities to cut roads or schools.
- First-mover advantage: governor keeps majority support while mayor bears cost of unpopular cuts.
- Cruel and Unusual: The Atkins Case
- Atkins case: in Atkins v. Virginia, a murderer with IQ 59 challenged execution as mentally retarded.
- Eighth Amendment: Trop v. Dulles set "evolving standards of decency," not 1789 norms.
- Penry precedent: in Penry v. Lynaugh, O'Connor required legislative evidence, not polls, for standards of decency.
- 1989 vs 2002: only two states barred such executions; by 2002 many had, including Texas legislature blocked by veto.
- Scalia's dissent: 18 of 38 death-penalty states (47%) cannot amount to national consensus.
- Counting Consensus Differently
- Majority's math: 18 plus 12 abolitionist states equals 30 of 50, a substantial majority.
- Amar brothers' scenario: if 47 states abolished the death penalty, don't let 3 outliers dictate decency.
- Correct fraction: national standard should be 48/50, not 1/3 among nonconforming states.
- Real-life caveat: there is plainly no national consensus against the death penalty itself, complicating the count.
- The Self-Contradicting Polls
- Collective Choice and Irrelevant Alternatives (Seventeen. THERE IS NO SUCH THING AS PUBLIC OPINION · II)
- Aggregating Opinions Isn't Logic
- Aggregation paradox: majority opinions don't combine into a coherent collective view.
- Scalia's Atkins dissent assumes state positions on capital punishment and mental retardation can be summed logically.
- 1992 example: voters rejecting Bush and Clinton separately didn't imply a majority wanted neither.
- Counting State Positions
- Scalia's count: 18 states draw a legal distinction for mentally retarded criminals, not a majority.
- Amar brothers' count: 30 states outlaw the practice; the question changes the answer.
- Whether Scalia's conclusion is right is a legal question, not a mathematical one.
- Stevens, Scalia, and Capital Punishment
- Stevens cited only five executions of mentally retarded prisoners in thirteen years; Scalia noted expected base rate ~six or seven.
- Scalia saw no reluctance: "No Greek Orthodox bishop ever executed in Texas" doesn't prove unwillingness.
- Incremental abolition: Scalia fears judicial ratchet disables states from punishing future crimes.
- Author rejects the worry: punishment is a renewable resource, not something society can lose.
- Irrelevant Alternatives
- Independence of irrelevant alternatives: adding option C shouldn't change ranking between A and B.
- Slime mold: 3g dark and 5g light were tied; adding 1g dark made 3g dark win.
- Florida 2000: Nader was an irrelevant option; majority preferred Gore to Bush, but Bush won.
- Voting can yield paradoxical outcomes; "what voters really want" is hard to define.
- Borda Count
- Borda count: assign points by rank; use voter's full preference order, not first choice alone.
- Hypothetical Florida ballots: Gore wins Borda count while satisfying all pairwise majorities.
- Shift 2% of voters to make Bush first choice for a majority; Gore can still win Borda.
- Nader's presence penalizes Bush, while Gore is never ranked last.
- Decoy Effects in Nature and People
- Asymmetric domination effect: a clearly inferior option changes preference between two balanced options.
- Slime mold may aggregate "nuclei preferences" via a Borda-like rule.
- Jays, honeybees, and hummingbirds show the same seemingly irrational choice pattern.
- Human dating experiment: adding a dominated third option shifted undergraduates' choices.
- Aggregating Opinions Isn't Logic
- Voting Paradoxes and Majority Rule (Seventeen. THERE IS NO SUCH THING AS PUBLIC OPINION · III)
- Irrationality and the Inner Nation-State
- Decoy effect: adding a slightly worse option makes the original more attractive—irrelevant alternatives can shift choices.
- Slime-mold lesson: apparent collective irrationality can emerge from rational parts; even individuals are not single rational actors.
- Self as nation-state: each person brokers compromises among squabbling inner voices, producing messy but workable decisions.
- Democracy analog: like the slime mold, democracy is a mess—but it kind of works.
- Instant-Runoff Voting: The Whole Cow
- Australian ballot: voters rank all candidates instead of choosing just one.
- Instant runoff: eliminate last-place candidate, transfer their ballots to next choices, recount until majority emerges.
- Florida 2000 simulation: Nader eliminated; Gore gains his voters and beats Bush 51–49.
- Manipulation shift: moving 2% from Gore-first to Bush-first makes Bush win under IRV, though Borda still favors Gore.
- IRV appeal: Nader supporters can vote sincerely without helping their least favorite candidate.
- History and praise: 150+ years old; used in Australia, Ireland, Papua New Guinea; Mill called it a great improvement.
- Burlington 2009: An Election That Breaks Everything
- Initial count: Wright 3297, Kiss 2982, Montroll 2554; Montroll eliminated under IRV.
- IRV result: Kiss beats Wright 4314–4064, despite trailing in first choices.
- Head-to-heads: Montroll beats both Kiss and Wright by large majorities—yet he was eliminated first.
- IRV weakness: consensus centrists who are no one's first choice struggle to win.
- Vote inversion: giving Kiss more first-place votes can switch the outcome to Montroll, so Kiss loses from increased support.
- Three methods, three winners: plurality gives Wright, IRV gives Kiss, pairwise majority gives Montroll.
- Condorcet and the Paradox of Majority Will
- Condorcet paradox: Kiss beats Wright, Wright beats Montroll, Montroll beats Kiss—no majority candidate exists.
- Condorcet's identity: shy, combative Enlightenment aristocrat nicknamed "the rabid sheep"; believed in social mathematics.
- Jury theorem: a large enough jury with >50% individual accuracy is overwhelmingly likely to reach the truth.
- Trust the majority: Condorcet argued majority opinion should override even your own judgment.
- Voltaire and Pascal: Condorcet blended Pascal's mathematical reach with Voltaire's secular reason, despite Voltaire's jealousy.
- Legacy and critiques: he championed women's rights; Adams called him a "mathematical charlatan"; his constitution was never adopted.
- The Axiom That Failed
- Condorcet's axiom: if a majority prefers A to B, B cannot be the people's choice.
- Borda's sin: rank-based Borda counts can flip winners when a third alternative is added, violating the axiom.
- Euclidean ambition: Condorcet tried to build voting theory from self-evident axioms, like Euclid's geometry.
- Contradiction found: his paradox shows the axiom set cannot all be true—"the people's choice" may simply not exist.
- Banach-Tarski warning: intuitive axioms about volume fail on spheres; fairness axioms can fail for elections.
- Condorcet's response: he weakened his axiom and conceded majorities can err, yet still believed a true general will exists.
- Irrationality and the Inner Nation-State
- Truth vs. Formalism Everywhere (Eighteen. “OUT OF NOTHING I HAVE CREATED A STRANGE NEW UNIVERSE” · I)
- Truth vs. Procedure
- Condorcet's democracy: citizens as measurement instruments; majority rule as a way to find right answers.
- Modern democratic view: appeal is fairness and rights, not epistemic accuracy.
- Core divergence: all conceptual difficulty lives where true conclusions and rule-licensed conclusions separate.
- Criminal law: procedure may free the guilty or convict the innocent; justice becomes a question of which side to privilege.
- Statistics dispute: Fisher asked what hypotheses we should believe; Neyman-Pearson asked which to certify, regardless of truth.
- The Parallel Postulate Quest
- Euclid's fifth axiom: more complicated and less obvious; a stain geometers scrubbed for two thousand years.
- Farkas Bolyai's warning: he called the quest a bottomless night and begged his son to abandon it.
- János Bolyai's reversal: instead of proving the postulate, he assumed it false and discovered another geometry.
- "Out of nothing": Bolyai wrote he had created a strange new universe, amazed by what he found.
- Independent discovery: Lobachevskii and Gauss reached similar non-Euclidean ideas around the same time.
- Hyperbolic abundance: in Bolyai's geometry, infinitely many lines through a point are parallel to a given line.
- Spherical Geometry's Blow
- Riemann's redefinition: Points are antipodal pairs; Lines are great circles; first four axioms survive.
- No parallels exist: every two great circles meet, so the fifth axiom fails spectacularly.
- Area proof: a great circle encloses exactly half the sphere; a nonintersecting circle would enclose less.
- Same as projective plane: Brunelleschi's perspective geometry also has every pair of lines meeting.
- Independence established: because spheres exist, the fifth axiom cannot follow from the first four.
- Einstein's payoff: non-Euclidean geometry is not a game; it is how spacetime actually looks.
- Formalism in Mathematics
- Meaning is irrelevant: points can be frogs or kumquats; only logical deduction from axioms matters.
- Mathematical force multiplier: proofs from shared axioms apply to every structure satisfying them.
- Hardy's criterion: modern math asks what expressions should be defined to be, not what they were.
- Geometry as game: a theorem is any statement derivable from axioms, whatever the terms refer to.
- Caramello's toposes: theories in different fields classified by the same topos let theorems transfer for free.
- Formalism in Law and Voting
- Formalist democracy: the public will is defined as the most frequent mark on ballots, not actual preferences.
- Butterfly ballot: voters who meant Gore recorded Buchanan; the system counts marks, not intentions.
- Bush v Gore: Supreme Court stopped recounts, prioritizing formal protocol over accurate vote totals.
- Scalia's creed: "Long live formalism"—words and procedure, not spirit, make a government of laws.
- Troy Davis dissent: Scalia held the verdict defined guilt even if witnesses recanted and innocence was claimed.
- Roberts's umpire: judges apply rules, don't make them; a limited, procedural role.
- Truth vs. Procedure
- Formalism, Paradox, and Consistency (Eighteen. “OUT OF NOTHING I HAVE CREATED A STRANGE NEW UNIVERSE” · II)
- Formalism on the Field
- Baseball formalism: a thing is what an umpire declares it — “It ain’t nothin’ till I call it” (Bill Klem)
- Jeffrey Maier incident: Jeter’s catchable fly became a home run because umpire Rich Garcia called it one
- No correction expected: Jeter accepted the call although everyone in the stadium except Garcia saw the interference
- Video replay: official review since 2008 improves accuracy, but many fans feel it betrays baseball’s spirit
- Ellenberg’s sympathy: he counts himself among fans who find replay foreign to the formalist game
- Formalism and the Law
- Scalian formalism: law’s text and precedent decide verdicts; judges should not impose personal views
- Indeterminacy: words like “cruel and unusual,” and even Euclid’s axioms, leave space for interpretation
- Posner’s realism: Supreme Court cases are toss-ups that conventional legal reasoning cannot settle
- Pragmatic decision: when axioms run out, justices choose by consequences, as in Bush v. Gore
- Pascal’s constraint: judges must play the game even when reason cannot determine the move
- Hilbert’s Axiomatic Dream
- Hilbert’s vision: rewrite geometry so no intuition is needed; “tables, chairs, and beer mugs” replace points and lines
- Peano arithmetic: self-evident axioms about numbers generate rationals and much of mathematics by pure deduction
- Fresh start: formalism promised an incontrovertible foundation after nineteenth-century crises in analysis and geometry
- Second problem: prove that no finite chain of logical steps from axioms can ever yield a contradiction
- Finitary proof: Hilbert wanted consistency shown without relying on belief in infinite sets
- Problem legacy: his 23 questions shaped twentieth-century math; Riemann remains open, the tenth problem was answered by impossibility
- Contradiction, Paradox, and Gödel
- Absurdity rule: no scientist treats p=.03 the same for a promising treatment and for dead salmon
- Russell’s paradox: the set of non-self-containing sets contains itself iff it does not — Frege’s foundation crumbled
- Frege’s lament: Grundgesetze was already in press; Russell’s “difficulty” showed its groundwork had given way
- Hilbert’s nightmare: contradictory results proved from obvious axioms would invalidate the whole mathematical edifice
- Gödel’s theorem: no finitary proof of arithmetic’s consistency exists; Hilbert’s program ended in 1931
- Modern doubters: Nelson’s 2011 inconsistency proof foundered; Voevodsky still proposes new foundations
- Formalism on the Field
- Formalism, Genius, and Mathematical Progress (Eighteen. “OUT OF NOTHING I HAVE CREATED A STRANGE NEW UNIVERSE” · III)
- Hilbert’s Legacy
- Voevodsky’s group: geometry, newly defined via homotopy theory, may again be the foundation.
- Formalist program failed: Hilbert’s mathematical style survived Gödel’s demolition.
- Platonism: mathematical objects feel as real to mathematicians as mountains.
- Weekday/Sunday: working mathematicians are Platonists on weekdays, formalists on Sundays.
- Hilbert’s goal: make the world safe for Platonism, not replace intuition with symbol-pushing.
- Voevodsky’s verdict: “Ask me in twenty years” about immunity from skepticism.
- Genius Is a Thing That Happens
- Genius cult: tells students math isn’t worth doing unless they are best; it drives away valuable futures.
- Grit: sustained focused effort is essential; “hardworking” is not an insult.
- Poincaré’s insight: the omnibus step felt instantaneous, but came from weeks of background work.
- Ramanujan: a real prodigy, but an outlier whose isolation made him unrepresentative.
- Tao: modern mathematics advances cumulatively through hard work, literature, and luck, not mystic inspiration.
- Twain and football: the last stone gets credit, but the whole network built the arch.
- Political Logic
- Axiomatic sympathies: Hilbert wanted other sciences to adopt the axiomatic approach.
- Gödel’s constitutional proof: saw how the Constitution could legally become a dictatorship; “I can prove it.”
- Hilbert’s refusal: wouldn’t sign 1914 war declaration because he couldn’t verify “It is not true.”
- Noether defense: “We are a university, not a bathhouse,” rejecting sex-based exclusion.
- Blind spot: 1938 Hilbert couldn’t accept that Nazi Germany could dismiss a professor without crime.
- Limits of political logic: formal reasoning cannot always grasp political reality.
- The Progress of the Human Mind
- Condorcet cycles: majority preferences can be self-contradictory, dooming intuitive voting axioms.
- Arrow’s theorem: even weak fairness axioms yield paradox; elegant, Nobel-winning, disappointing.
- Sketch: reason and science would eliminate royalism, sex prejudice, hunger, and old age.
- Malthus: wrote his bleaker population account as a response to Condorcet’s optimism.
- Social mathematics: Condorcet’s real legacy is using quantitative social science in government.
- A choice: we still choose social mathematics; not inevitable, but right.
- Hilbert’s Legacy
- Public Opinion Is Mathematically Incoherent (Seventeen. THERE IS NO SUCH THING AS PUBLIC OPINION · I)
- How to Be Right
- The Power of Principled Uncertainty (How to Be Right · I)
- The Tuberculosis Forecast
- Model uncertainty compounds: local errors feed through the simulation until noise engulfs signal by 2050.
- No principled choice: with uncertainties percolating, the simulation could yield anything from zero cases to mass infection.
- False precision: the boss demanded a best guess, so the final number was quoted as math-backed truth.
- The Critic Who Counts
- Roosevelt's arena: "the critic who counts" is a sneer at sideline doubters; credit belongs to the doer.
- Wald's rebuttal: an unbloodied mathematician improved wartime decisions by telling doers how to do better.
- Library work counts: Condorcet's closet philosophy served France more than many practical men's deeds.
- For This Is Action
- Ashbery's axiom: "For this is action, this not being sure" — uncertainty is not cowardly fence-sitting.
- Math domesticates chance: since Pascal, mathematics makes gamblers' whims and cosmic odds tractable.
- Principled not-sureness: math lets you say "I'm not sure, why, and roughly how not-sure — and so should you."
- A Man Who Swings from Poll to Poll
- Silver's honesty: treat uncertainty as real and report probabilities, not winners.
- Pundits misread odds: movements from 85% to 67% look like hedging but reflect changing evidence.
- Weather-forecast logic: a 40% chance that rains doesn't make the forecast wrong.
- The Expected Wrong Count
- Additivity of error: expected wrong states = sum of each state's error probability — about 2.83.
- Beyond his promise: Silver predicted likely miss in about 3 states; he got all 50 right.
- Quine's stance: believe each belief true, yet expect some are false — "always think you're right, not that you're always right."
- The Tuberculosis Forecast
- Precision, Contradiction, and Productive Failure (How to Be Right · II)
- Against Precision
- Overprecise forecasts mislead: Silver's "73.1%" implies a measurement exactness his model lacks; day-to-day drift is noise.
- Elections can be too close to know: Florida 2000's few hundred votes were smaller than the error from spoiled, lost, miscounted ballots.
- Judges vs. mathematicians: Judges must pretend we know who won; mathematicians can say the truth: we don't know.
- Close contests are already chance: Franken–Coleman turned on "Lizard People" ballots; chance events, not "the people," decide razor-thin races.
- Coin-flip remedy: Seife's proposal keeps us honest—it admits the people said "I dunno."
- Necessary precision: Mathematicians compute to as many decimals as necessary, not as many as possible; pi's digits are uninteresting, pi's structure is not.
- Living with Contradiction
- Principle of explosion: One contradiction in a formal system makes every statement provable—Russell's paradox destroyed Frege's set theory, Kirk's paradox frazzles AIs.
- Fitzgerald's test: First-rate intelligence holds two opposed ideas and still functions; math uses it in reductio ad absurdum.
- Reductio as lucid dreaming: Mathematicians reason from a false proposition to its consequences without short-circuiting.
- Wallace's mathematical struggle: He held opposing hypotheses side by side, clearing brush until truth or the nearest truth came clear.
- Beckett's "Fail better": Contradiction and failure are not breakdowns but creative states, from "I can't go on, I'll go on" to "fail better."
- Productive Failure
- Prove by day, disprove by night: Attacking your own theorem nightly hedges against error and reveals its structure.
- Failure can build proof: Each failed disproof hits a wall; the walls assemble into the structure of why the theorem is true.
- Bolyai's new geometry: His failure to prove the parallel postulate revealed the geometry that blocked him—he created it by understanding his failure.
- Apply to all beliefs: Argue against your dearest convictions; if you can't disprove them, you learn why you hold them.
- Mathematical Thinking Is Common Sense
- Mathematicians create: Galton, Condorcet, Bolyai, Shannon, Hamming, Wald—all built new structures; all mathematical writing is creative writing.
- No psychedelic madness: Mathematical entities are defined and bound by reason; "touched by fire and bound by reason" is the condition.
- Intuition on tracks: Logic is a narrow channel through which intuition flows with vastly augmented force.
- Lessons without numbers: Structure exists; we can understand it; intuition is stronger with a formal exoskeleton.
- You already use it: Decisions about futures, regression, and group beliefs are mathematics—"common sense by other means."
- Against Precision
- The Power of Principled Uncertainty (How to Be Right · I)
- Acknowledgments
- From Idea to Book
- Eight-year gestation: a vague urge to praise math became a real book proposal.
- Agent Jay Mandel: yearly persuasion and concept-sharpening turned “yell about math” into a book.
- Penguin Press: Colin Dickerman acquired and guided; Scott Moyers finished the push.
- Authorial transformation: the final book evolved far beyond the original proposal.
- Editorial and Publishing Support
- Slate math column: editors Josh Levin, Jack Shafer, David Plotz fostered accessible math writing since 2001.
- Other outlets: pieces in New York Times, Washington Post, Boston Globe, Wall Street Journal fed the book.
- Long-form mentors: Heidi Julavits and Nicholas Thompson taught sustained mathematical narrative.
- Production care: Elise Craig fact-checked; Greg Villepique copyedited with a war on unnecessary hyphens.
- Intellectual and Personal Debts
- Barry Mazur: advisor and model for linking math to other modes of thinking.
- David Foster Wallace: left the Russell epigraph unused in notes for Everything and More.
- Readers and colleagues: many named friends and strangers gave suggestions and close readings.
- University support: UW–Madison sabbatical and Romnes Fellowship provided time and resources.
- Coffee and context: Barriques Coffee in Madison hosted much of the writing.
- Key Readers and Family
- Tom Scocca: unsparing full read with a keen eye.
- History keepers: Andrew Gelman and Stephen Stigler checked statistics history; Stephen Burt checked poetry.
- Close readers: Henry Cohn gave detailed notes; Lynda Barry authorized self-drawn pictures.
- Statistician parents: read everything and flagged excessive abstraction.
- Children: accepted working weekends; son contributed a drawing.
- Tanya Schlam: first and final reader; her support taught being right more than math did.
- From Idea to Book
- When Am I Going to Use This?
- Core Conclusion and Practical Takeaways
- The Universal Toolkit of Mathematical Thinking
- Math is common sense by other means: it formalizes intuition into a rigorous, powerful prosthesis for thought.
- Straight locally, curved globally: every curve looks linear up close, but extrapolating far from data misleads.
- Expected value governs decisions: multiply probabilities by payoffs and sum; lotteries and hot tips lose on average.
- Regression to the mean: extreme outcomes mix signal with luck, so they drift back toward average.
- Correlation is not causation: shared causes, selection effects, and chance can manufacture associations without causal links.
- Uncertainty is real: report probabilities and ranges, not false precision; a 40% rain forecast is not wrong.
- Daily Practices for Clearer Judgment
- Ask "compared to what?": base rates and reference classes decide whether a risk, effect, or ranking is meaningful.
- Beware survivorship bias: look for missing planes, failed funds, and unpublished studies before trusting observed winners.
- Check the conditionals: P(data | null) is not P(null | data); avoid the prosecutor's fallacy and the p-value trap.
- Update with Bayes's rule: fold prior beliefs and new evidence into a two-by-two box instead of reacting to isolated numbers.
- Raise the surprise threshold: the more chances you take to find an effect, the stronger the evidence required.
- Demand confidence intervals: ranges of plausible effect sizes say more than a pass/fail significance test.
- Statistical Thinking as a Defense Against Self-Deception
- Statistically significant ≠ important: large samples can make trivial effects detectable; "noticeable" is a better word.
- Small samples create false extremes: tiny schools, states, and shooting streaks top rankings by chance alone.
- File drawers corrupt evidence: hidden null results make published findings a biased, nonrandom sample.
- P-hacking is torture: tweaking analyses, outliers, or controls can force data into false confessions.
- Replication is the immune system: single studies are clues; repeated, published replications decide what survives.
- Reductio ad unlikely has limits: improbable data refute a hypothesis only when all plausible alternatives are considered.
- Mindset Shifts from the Book
- Embrace not-sureness: uncertainty is action, not fence-sitting; give reasons and degrees for your doubt.
- Judge decisions by process, not outcomes: a correct expected-value bet can lose; a wrong bet can win.
- Look for hidden structure: the Fano plane, Hamming code, and lottery system are one mathematical object in costume.
- Argue against your own beliefs: failed disproofs reveal why your convictions hold—or why they should not.
- Practice quantitative humility: math prevents many errors, but ultimate questions may escape numeric answers.
- Be a critic who counts: careful analysis can improve real decisions even without entering the arena.
- The Universal Toolkit of Mathematical Thinking
opening map…