Sample size, power and precision explained
An advanced guide explaining sample size, statistical power, precision, effect size, uncertainty and why planning should focus on estimation as well as hypothesis testing.
Structure
Problem, intuition, method, working, limitations and discussion.
Best for
Students preparing for coursework, analysis, interpretation or revision.
Use with
Learning Hub lessons, tutoring sessions or dissertation planning.
Resource guide
Problem
Students often think sample size is only about reaching a magic number or satisfying a supervisor, ethics form or software calculator. In reality, sample size affects precision, statistical power, uncertainty, confidence intervals and the credibility of the whole study. A study can be too small to detect meaningful effects, but it can also be large enough to make trivial effects statistically significant. Good planning requires understanding the difference between detecting an effect and estimating it precisely.
- Students often ask how many participants are needed without defining the main outcome.
- Power is confused with the probability that the hypothesis is true.
- A statistically significant result is treated as proof of importance.
- Small studies often produce wide confidence intervals and unstable estimates.
- Large studies can detect effects that are too small to matter.
- Sample size calculations are sometimes done after seeing the data.
- Precision is ignored even though confidence intervals are often more informative than p-values.
Resource guide
Intuition
Sample size controls how much information the study contains. More information usually means narrower confidence intervals and greater ability to detect effects, but sample size alone cannot fix poor measurement, confounding, bias or a weak research question. Power focuses on the probability of detecting a specified effect if it truly exists. Precision focuses on how close the estimate is likely to be to the true value.
- Power is about detecting an effect under assumed conditions.
- Precision is about how narrow or wide the uncertainty around an estimate is.
- Effect size is the size of difference or association the study is designed to detect or estimate.
- Confidence intervals show uncertainty more directly than p-values.
- A small sample can miss important effects because estimates are noisy.
- A large sample can make small effects statistically significant.
- The required sample size depends on the outcome, design, variability and target effect size.
Resource guide
Method
A sensible sample size discussion starts with the research aim. If the study is designed to test a hypothesis, power may be central. If the study is designed to estimate a mean, proportion, difference, odds ratio or regression coefficient, precision may be more relevant. The calculation should be based on the primary outcome, a meaningful effect size, acceptable uncertainty and realistic assumptions.
- Step 1: Define the primary research question.
- Step 2: Identify the primary outcome variable.
- Step 3: Decide whether the goal is testing, estimation, prediction or feasibility.
- Step 4: Specify the smallest effect size worth detecting or estimating.
- Step 5: Estimate variability, baseline risk or event rate from previous studies if possible.
- Step 6: Choose an acceptable significance level if hypothesis testing is used.
- Step 7: Choose desired power, commonly 80% or 90%, when relevant.
- Step 8: Consider expected missing data, dropout or loss to follow-up.
- Step 9: Inflate the target sample size if attrition is expected.
- Step 10: Report assumptions clearly rather than only giving the final number.
Resource guide
Working
Suppose a study compares mean blood pressure between two groups. The required sample size depends on the expected difference, the variability of blood pressure, the desired power and the significance level. If the expected difference is small or the variability is large, more participants are needed. If the study instead estimates the prevalence of hypertension, the sample size depends on the expected prevalence and desired confidence interval width.
- For a mean difference, sample size depends on the target difference and standard deviation.
- For a proportion, sample size depends on the expected proportion and desired precision.
- For logistic regression, the number of outcome events matters, not only total sample size.
- For survival analysis, the number of events often matters more than the number recruited.
- For cluster studies, sample size must account for similarity within clusters.
- For repeated measures, correlation within individuals affects information.
- For missing data, the planned sample should be larger than the final required complete sample.
Resource guide
Limitations
Sample size calculations are only as good as their assumptions. If the assumed effect size, variance, event rate or dropout rate is unrealistic, the calculation may be misleading. Power also does not protect a study from bias, confounding, poor measurement or inappropriate analysis. A large biased study can be confidently wrong.
- Power calculations depend on assumed effect sizes.
- Previous studies may provide poor estimates of variability or event rates.
- Power does not measure the probability that the result is true.
- A powered study can still be biased.
- A small pilot study is usually not designed to provide definitive evidence.
- Post-hoc power calculations are often unhelpful after results are known.
- Precision-based planning may be more appropriate for estimation-focused studies.
Resource guide
Discussion
A strong dissertation or research report should explain why the sample size is reasonable. If a formal calculation was performed, report the assumptions. If the study used an available dataset, explain the achieved sample size, the number of complete cases or events and the implications for precision. The discussion should connect sample size to uncertainty, not only statistical significance.
- Report the primary outcome used for the sample size calculation.
- State the assumed effect size, variance, event rate or baseline proportion.
- Report the significance level and power if used.
- Mention allowance for missing data or dropout.
- Discuss precision using confidence interval width.
- Avoid claiming that a non-significant result proves no effect in an underpowered study.
- Explain sample size limitations honestly.
Practical checklist
Before you apply this topic
- Have you defined the primary research question?
- Have you identified the primary outcome?
- Is the goal hypothesis testing, estimation, prediction or feasibility?
- Have you specified a meaningful effect size?
- Have you considered variability or baseline event rate?
- Have you considered power if testing is the main goal?
- Have you considered precision if estimation is the main goal?
- Have you allowed for missing data or dropout?
- Have you considered events rather than only participants where relevant?
- Have you reported assumptions clearly?
- Have you avoided post-hoc power interpretation?
- Have you discussed uncertainty and confidence intervals?
Common mistakes
What to avoid
- Choosing a sample size without defining the primary outcome.
- Reporting a sample size number without assumptions.
- Confusing power with the probability that the null hypothesis is false.
- Using post-hoc power to explain a non-significant result.
- Ignoring missing data and dropout.
- Ignoring the number of events in logistic or survival analysis.
- Assuming a larger sample fixes bias.
- Treating statistical significance as practical importance.
- Ignoring precision and confidence interval width.
- Using unrealistic effect sizes to make the required sample smaller.
How this connects to learning
Use the guide as a bridge between theory and application.
A resource guide should not replace a full course or live teaching session. Instead, it helps you organise your thinking. Use it to identify what you understand, what feels unclear, and what questions you should ask before applying a method to real data.
Before a lesson
Read the intuition and problem sections to prepare.
During analysis
Use the method and checklist to guide decisions.
When writing
Use limitations and discussion to improve interpretation.
Related guides