Statistical Inference Essentials
Null-hypothesis testing, p-values, effect sizes, power, and the errors that come with each.
Slide 1 of 4
The logic of NHST
- A p-value is the probability of data at least as extreme as observed, assuming the null is true.
- It is not the probability that the null is true, nor the probability the result is due to chance.
- Statistical significance does not by itself indicate practical importance.
Slide 2 of 4
Type I and Type II errors
- Type I error (alpha): rejecting a true null — a false positive.
- Type II error (beta): failing to reject a false null — a false negative.
- Power = 1 − beta: the probability of detecting a true effect of a given size.
Slide 3 of 4
Effect sizes
- Standardized mean differences (Cohen's d) and correlations (r) summarize magnitude.
- Cohen's conventions: d ≈ .2/.5/.8 and r ≈ .1/.3/.5 for small/medium/large.
- Always report an effect size and a confidence interval alongside a test statistic.
Slide 4 of 4
Confidence intervals
- A 95% CI is a range of plausible parameter values consistent with the data.
- Over repeated samples, 95% of such intervals would contain the true parameter.
- Wide intervals signal imprecision — often from small samples.