Type I & Type II Errors and Statistical Power

Introduction

• What it is — Hypothesis testing decides, from sample data, whether an observed difference (e.g. between two treatment means) reflects a real population difference or could have arisen by chance alone.
• Two ways to be wrong — Because we work from samples, not whole populations, every decision risks error in two opposite directions — declaring a difference that does not exist (Type I error) or missing one that does (Type II error).
• Power as the flip-side — Statistical power is the complement of the Type II error — the probability the test will detect a real difference when one truly exists; it is the central parameter governing whether a study is big enough to answer its question.
• Two complementary frameworks — The significance-test route yields a P value and a yes/no verdict; the estimation route yields a point estimate plus a confidence interval (CI). They are linked — a 95% CI that just excludes the null corresponds to P = 0.05 two-sided.
• Why it matters — These concepts underpin every clinical trial and preclinical comparison — whether a drug is "significantly" better, whether a "negative" trial was truly negative or merely under-powered, and how large an experiment must be to be worth running.