Type I & Type II Errors and Statistical Power

Type I & Type II Errors and Statistical Power

1. Definition & overview — what the topic is about

• Hypothesis testing is the formal procedure of statistical inference by which we decide, from sample data, whether an observed difference (e.g. between two treatment means) is likely to reflect a real difference in the underlying populations or could plausibly have arisen by chance alone (Swinscow 10e Ch.5, pp.46–47).
• Because we work from samples rather than entire populations, every such decision carries a risk of being wrong in two opposite directions — declaring a difference that does not exist (Type I error), or missing a difference that does exist (Type II error) (Swinscow 10e Ch.5, pp.47–49; Medhi Ch.11, p.128).
• Statistical power is the flip-side of the Type II error: the probability that the test will detect a real difference when one truly exists. It is the central design parameter that governs whether a study is large enough to answer its question (Swinscow 10e Ch.5, p.49; Medhi Ch.11, pp.127–128).
• Two complementary frameworks coexist in the chapter: the hypothesis-testing (significance-test) approach, which yields a P value and a yes/no verdict, and the estimation approach, which yields a point estimate plus a confidence interval (CI) describing the precision of that estimate. The two are mathematically linked — a 95% CI that just excludes the null value corresponds to P = 0.05 on a two-sided test (Swinscow 10e Ch.5, p.47; Ch.4, pp.40–41).
• Why it matters in pharmacology: these concepts underpin the interpretation of every clinical trial and preclinical comparison — whether a new drug is "significantly" better, whether a "negative" trial was truly negative or merely under-powered, and how large an experiment must be to be worth running (Swinscow 10e Ch.5, p.49; Medhi Ch.11, p.127).

Foundational scaffolding (from Ch.3–4) needed before the error framework makes sense

• Population vs sample: a population is the entire aggregate of subjects/observations of interest; a sample is a subset drawn from it. Population characteristics (mean, SD) are parameters, conventionally written with Greek letters — population mean μ (mu), population SD σ (lower-case sigma); the corresponding sample statistics are written x̄ (sample mean) and s or SD (Swinscow 10e Ch.3, p.29; Medhi Ch.11, pp.123–125).
• The whole purpose of inference is to use the sample statistic to draw a valid conclusion about the unknown population parameter; the gap between a sample value and the true population value is the sampling error (Medhi Ch.11, p.123).
• A random sample requires that every individual in the population has a known, non-zero (ideally equal) chance of selection, with selections made independently — achieved via random-number tables or equivalent. "Random" describes the method of selection, not the sample itself (Swinscow 10e Ch.3, p.30).
• Standard error of the mean (SEM) = SD/√n — an estimate of how much sample means would vary from sample to sample if sampling were repeated; it shrinks as n grows. Crucially, the SEM can be estimated from a single sample — repeated sampling is not required (Swinscow 10e Ch.3, pp.33–34; Medhi Ch.11, p.126).
• Central limit theorem: the means of repeated samples follow a Normal (Gaussian) distribution, often even when the underlying observations do not — this is what licenses the use of Normal-based ±1.96 SE limits in large samples (Swinscow 10e Ch.3, pp.33–34).
• SD vs SEM (a perennial confusion): SD describes the variability of individual observations; SEM describes the precision of the estimate of the mean (variation of a statistic from one study to the next) — SEM = SD/√n. Use SD to describe data, SE/CI to describe an estimate of a study outcome (mnemonic: Deviation→Description, Error→Estimate) (Swinscow 10e Ch.3, pp.35–36; Medhi Ch.11, p.126).