Introduction to meta-analysis

Problem

Individual studies often give different answers because they use different samples, settings, measures, designs and levels of precision. Meta-analysis provides a statistical framework for combining results across studies. However, students often think meta-analysis is simply an average of published p-values or a quick way to produce a forest plot. In reality, meta-analysis requires careful definition of the research question, eligibility criteria, effect size, heterogeneity and risk of bias.

• Studies are combined without checking whether they answer the same question.
• P-values are confused with effect sizes.
• Forest plots are produced without understanding weights and confidence intervals.
• Heterogeneity is ignored or treated as a minor technical detail.
• Fixed-effect and random-effects models are confused.
• Publication bias is not considered.
• Study quality and risk of bias are separated from statistical interpretation.

Intuition

Meta-analysis combines estimates, not raw conclusions. Each study contributes an effect estimate and a measure of uncertainty. More precise studies usually receive more weight. If studies are estimating the same underlying effect, a fixed-effect model may be considered. If true effects may vary across populations, settings or methods, a random-effects model is often more realistic.

• The unit of analysis is usually the study-level effect estimate.
• Larger or more precise studies usually have greater weight.
• A forest plot shows study estimates and their confidence intervals.
• The pooled estimate summarises the combined evidence.
• Heterogeneity describes variation in effects across studies.
• Random-effects models allow true effects to vary between studies.
• Meta-analysis is only meaningful when studies are sufficiently comparable.

Method

A meta-analysis workflow starts with a focused review question, often structured using population, exposure or intervention, comparator and outcome. Studies are selected using pre-specified criteria. For each study, an effect size and standard error are extracted or calculated. The analyst then chooses an appropriate pooling method, assesses heterogeneity and interprets the pooled estimate alongside study quality and clinical or methodological diversity.

• Step 1: Define the review question clearly.
• Step 2: Specify inclusion and exclusion criteria.
• Step 3: Identify the effect measure, such as mean difference, odds ratio, risk ratio or hazard ratio.
• Step 4: Extract effect estimates and uncertainty measures.
• Step 5: Choose fixed-effect or random-effects modelling based on the question and heterogeneity.
• Step 6: Produce a forest plot.
• Step 7: Assess heterogeneity using visual inspection and statistics such as I-squared.
• Step 8: Consider subgroup or sensitivity analyses if justified.
• Step 9: Assess publication bias where enough studies are available.
• Step 10: Interpret pooled results alongside risk of bias and study differences.

Working

Suppose a review examines whether a lifestyle intervention reduces systolic blood pressure. Each included study reports a mean difference between intervention and control groups. A forest plot displays the mean difference and confidence interval for each study. The pooled estimate gives an overall summary, but if the studies differ greatly in population, intervention intensity or follow-up time, heterogeneity must be discussed.

• Effect size: mean difference in systolic blood pressure.
• Each study contributes an estimate and standard error.
• More precise studies contribute more weight.
• The forest plot shows whether study effects are consistent.
• The pooled estimate summarises the average intervention effect.
• I-squared describes the proportion of variability due to heterogeneity rather than chance.
• A random-effects model may be suitable when effects differ across contexts.
• Subgroup analysis may explore whether effects differ by population or intervention type.

Limitations

Meta-analysis can produce a precise-looking result even when the underlying studies are biased, heterogeneous or poorly designed. Statistical pooling should not be automatic. If studies are too different, a narrative synthesis may be more appropriate. Publication bias, selective reporting and small-study effects can also distort conclusions.

• Poor-quality studies can produce a misleading pooled estimate.
• High heterogeneity can make the pooled estimate hard to interpret.
• Publication bias may overstate effects.
• Selective outcome reporting can bias evidence.
• Subgroup analyses can be underpowered or data-driven.
• Random-effects models do not solve all heterogeneity problems.
• Meta-analysis cannot correct poor original study design.

Discussion

A strong meta-analysis discussion should interpret the pooled estimate, uncertainty, heterogeneity and quality of evidence together. Students should avoid saying that the pooled result is the final truth. Instead, they should discuss consistency, clinical relevance, bias, generalisability and whether further research is needed.

• Interpret the pooled effect size in context.
• Discuss the width of the confidence interval.
• Describe heterogeneity and possible reasons for it.
• Consider whether the studies are clinically comparable.
• Discuss risk of bias and publication bias.
• Avoid overinterpreting subgroup findings.
• State whether the evidence is strong, limited or uncertain.

How this connects to learning

Use the guide as a bridge between theory and application.

A resource guide should not replace a full course or live teaching session. Instead, it helps you organise your thinking. Use it to identify what you understand, what feels unclear, and what questions you should ask before applying a method to real data.