P-value and Statistical Significance

P-value and Statistical Significance

1. Why inference is needed — populations, samples and the leap from one sample to a parameter

• In statistics a population is not restricted to people: it is any aggregate of creatures, things, events, procedures or observations of interest — e.g. "the systolic blood pressure of Englishmen aged 40–59", or every visit to a doctor (Swinscow & Campbell, Statistics at Square One 10e Ch.3, p.29).
• Summary measures computed on a whole population are population parameters and are written with Greek letters — the population mean is μ (mu) and the population standard deviation σ (lower-case sigma) (Swinscow & Campbell 10e Ch.3, p.29).
• A population usually contains too many individuals to study, so investigation is restricted to one or more samples drawn from it; a well-chosen sample contains most of the information about a particular parameter, provided the sample-to-population relation permits true inferences (Swinscow & Campbell 10e Ch.3, p.30).
  • The defining attribute of a valid sample: every individual in the source population must have a known, non-zero chance of being included — naturally these chances are made equal and the choices independent, achieved by a random process (coin-spin, or more usually a table of random numbers). Such a sample is a random sample; "random" describes how the sample is selected, not the sample itself (Swinscow & Campbell 10e Ch.3, p.30).
• Randomisation of treatment allocation in a clinical trial is a distinct use of random numbers: it ensures no bias in allocation and that, in the long run, treatment groups are comparable in both known and unknown prognostic factors; blocked randomisation (block sizes 2, 4, 6, 8, 10) keeps the groups numerically balanced at regular intervals (Swinscow & Campbell 10e Ch.3, p.32).
  • Critical caveat for inference: patients in a randomised trial are not a random sample of the disease population — they are a highly selected set of eligible, willing patients; randomisation guarantees internal comparability, not external representativeness (Swinscow & Campbell 10e Ch.3, p.32).
• Variation between samples drawn from one population is inevitable and depends on (i) the amount of variation in the source population and (ii) the size of the sample — a small sample is a much less certain guide to its population than a large one (Swinscow & Campbell 10e Ch.3, p.33).
• The whole apparatus of the p-value and significance testing exists to discipline this leap — to quantify, from a single sample, how plausibly the data could have arisen from a stated population — so the foundational sampling concepts above are not optional background but the logical substrate of every significance test (Swinscow & Campbell 10e Ch.3, pp.29–34; Ch.5, p.47).