Module 5 · Statistics Foundation

Regression foundations.

This module develops regression as a framework for describing, modelling and interpreting relationships. Students move from scatterplots and correlation to fitted lines, residuals, least-squares logic, adjusted regression, confounding and logistic regression for binary outcomes.

Start Lesson 5.1 →View all lessons

5

Lessons

14 hrs

Study time

0

Coding

Foundation → Advanced

Level

What this module builds

Regression thinking beyond mechanical line fitting.

The module treats regression as a conceptual framework. Students learn how models summarise relationships, how coefficients depend on the chosen variables, and why diagnostics and context matter.

Association before modelling

Regression begins with relationship thinking. Students first learn how scatterplots, direction, strength and correlation describe patterns between variables.

Linear prediction

Simple linear regression turns a relationship into an equation, giving fitted values, predictions, residuals, slope and intercept interpretations.

Least-squares reasoning

Students study how the fitted line is chosen by minimising squared residuals, and why residuals reveal what the model has failed to capture.

Adjustment and confounding

Multiple regression changes the comparison being made. Students learn adjusted coefficients, confounding, overadjustment and model-choice caution.

Binary outcome modelling

Logistic regression shows why probabilities, odds, log-odds and classification thresholds are needed when the outcome is yes/no.

Responsible interpretation

Throughout the module, regression is treated as a structured statistical argument, not an automatic causal machine.

By the end

Students should interpret regression with precision and caution.

1

Read scatterplots for form, direction, strength, clusters and outliers.

2

Define and interpret Pearson correlation.

3

Explain why correlation does not prove causation.

4

Write and interpret a simple linear regression equation.

5

Interpret slope, intercept, fitted values and residuals.

6

Explain how least squares chooses a fitted line.

7

Use residual plots to identify model limitations.

8

Interpret adjusted coefficients in multiple regression.

9

Explain confounding, overadjustment and model-choice risks.

10

Convert between probability, odds and log-odds.

11

Interpret logistic regression coefficients and odds ratios.

12

Separate predicted probabilities from classification decisions.

Interpretation workflow

Ask six questions before interpreting any coefficient.

Regression coefficients are not self-explanatory. Their meaning depends on the outcome, predictor scale, included variables, model form, assumptions and research goal.

What is the outcome?

Regression interpretation begins by identifying the response variable and whether it is continuous or binary.

What is the predictor?

The explanatory variable determines the comparison being made and the unit attached to the slope.

What scale is the coefficient on?

Linear regression coefficients are on the outcome scale; logistic coefficients are on the log-odds scale.

What is being held constant?

In multiple regression, each coefficient is interpreted conditional on the other variables in the model.

What do residuals show?

Residuals reveal missed structure, unusual observations, nonlinearity and changing spread.

Is causation justified?

Regression can describe associations. Causal claims require study design, timing, theory and assumptions.

Formula map

Regression formulae are connected by prediction error and coefficient interpretation.

Students see how the regression line, residuals, least squares, adjusted regression and logistic regression build one connected modelling story.

Pearson correlation

r = Σzₓzᵧ / (n − 1)

Measures direction and strength of linear association between two quantitative variables.

Simple regression

Y = β₀ + β₁X + ε

Models a continuous outcome using one explanatory variable and random error.

Fitted line

ŷ = b₀ + b₁x

Gives the predicted average outcome at a chosen value of x.

Residual

eᵢ = yᵢ − ŷᵢ

Measures the vertical prediction error for observation i.

Least squares

minimise Σeᵢ²

Chooses the fitted line with the smallest total squared residuals.

Multiple regression

Y = β₀ + β₁X₁ + β₂X₂ + ... + ε

Models adjusted associations using more than one predictor.

Logit model

log[p/(1 − p)] = β₀ + β₁X

Models binary outcomes by making log-odds linear in predictors.

Odds ratio

OR = eβ

Exponentiated logistic coefficient; multiplicative change in odds.

Common regression traps

This module teaches careful modelling judgement.

Regression is powerful, but it is also easy to overinterpret. Students learn to avoid common mistakes before moving into more applied modelling.

Correlation is not causation

A strong association may be due to confounding, reverse causation, selection or shared context.

A line can hide curvature

A straight-line model may fit poorly if the true relationship is curved or has clusters.

Outliers can move the model

Large residuals and high-leverage points can strongly affect slope and interpretation.

Adjustment can create bias

Adjusting for mediators or colliders can distort the relationship being studied.

Odds ratios are not risk ratios

CapstoneBinary outcomes

Logistic regression foundations

Learn why binary outcomes need a different regression framework and how probabilities, odds and log-odds connect.

Odds ratiosLogit linkClassification

Open lesson

Start →

How to study this module

Do not memorise coefficients. Interpret the comparison.

For every regression output, ask what outcome is being modelled, what one-unit change means, what variables are adjusted for, what residuals show and whether the model supports only association or a stronger explanation.

Course completion

Ready to move into applied modelling and real study questions.

After this module, students should understand regression as a careful modelling framework. They can now approach applied statistical modelling with stronger foundations in association, uncertainty, adjustment and binary outcomes.

Back to course →