• Part I: Statistical Metrics
  • 1. Descriptive & Distributional Statistics
  • 2. Statistical Inference
  • 3. Correlation & Association
  • 4. Information-Theoretic Metrics
  • 5. GLM & Count Model Metrics
  • 6. Model Selection Criteria
  • 7. Spatial Statistics
• Part II: Machine Learning Metrics
  • 8. Classification — Threshold-Based
  • 9. Classification — Threshold-Independent
  • 10. Multi-Class Extensions
  • 11. Calibration Metrics
  • 12. Regression Metrics
  • 13. Ranking & Retrieval Metrics
  • 14. Clustering — External (with Ground Truth)
  • 15. Clustering — Internal (No Ground Truth)
  • 16. Model Selection & Validation
  • 17. Feature Importance & Interpretability
  • 18. Distance & Similarity Metrics
• Appendix: Common Notation

Part I: Statistical Metrics

1. Descriptive & Distributional Statistics

Mean (Arithmetic)

• Definition: The sum of all values divided by the number of values.

• Range: Any real number.
• When to use: Default summary of central tendency for symmetric, roughly normal distributions. Avoid when data is heavily skewed or contains extreme outliers — use median instead.

Median

• Definition: The middle value when data are sorted. For even N, the average of the two middle values.

• Range: Any real number.
• When to use: Preferred over the mean when the distribution is skewed or contains outliers (e.g. income data, travel time distributions, collision counts with long tails).

Mode

• Definition: The most frequently occurring value in a dataset.
• Range: Any value in the dataset.
• When to use: Useful for categorical data or discrete distributions. Also relevant for identifying peaks in multimodal distributions.

Variance

• Definition: The average squared deviation from the mean.

• Range: [0, +∞).
• When to use: Measures spread. Foundation for many statistical tests. Use sample variance (N−1 denominator) when estimating from a sample.

Standard Deviation

• Definition: The square root of variance. In the same units as the data.

• Range: [0, +∞).
• When to use: The default measure of spread for normally distributed data. More interpretable than variance due to same-unit property.

Skewness

• Definition: Measures the asymmetry of a distribution about its mean.

• Range: (−∞, +∞). 0 = symmetric, positive = right-skewed, negative = left-skewed.
• When to use: Check before applying methods that assume normality. Heavily skewed data may need transformation (log, Box-Cox) before modelling.

Kurtosis

• Definition: Measures the “tailedness” of a distribution — how much of the variance comes from extreme values.

• Range: (−∞, +∞). Positive = heavy tails, negative = light tails.
• When to use: High kurtosis warns of outlier-prone data. Important when assessing whether standard error estimates are reliable.

Interquartile Range (IQR)

• Definition: The range between the 25th and 75th percentiles.

• Range: [0, +∞).
• When to use: Robust measure of spread that is not influenced by outliers. Used in box plots and for outlier detection (values beyond Q₁ − 1.5·IQR or Q₃ + 1.5·IQR).

Coefficient of Variation (CV)

• Definition: The ratio of the standard deviation to the mean, expressed as a percentage.

• Range: [0, +∞)%.
• When to use: Comparing variability across datasets with different units or scales. Undefined when mean is zero.

2. Statistical Inference

p-value

• Definition: The probability of observing a test statistic at least as extreme as the one computed, assuming the null hypothesis is true.

• Range: [0, 1]. Smaller = stronger evidence against H₀. Reject H₀ if p < α (commonly 0.05).
• When to use: Standard hypothesis testing for any parametric or non-parametric test. Be aware that large N can produce small p for trivially small effects — always pair with effect size. Not the probability that H₀ is true.

Confidence Interval (CI)

• Definition: A range of values that, under repeated sampling, would contain the true parameter at a given confidence level.

• Range: Width depends on SE and confidence level. For 95%: z = 1.96.
• When to use: Always report alongside point estimates. More informative than p-values alone because it conveys both the magnitude and precision of an estimate.

Effect Size — Cohen’s d

• Definition: The standardised difference between two group means.

• Range: (−∞, +∞).

  d

  : 0.2 = small, 0.5 = medium, 0.8 = large.

• When to use: When comparing two groups and you want to quantify the practical significance of the difference, independent of sample size. Essential complement to p-values.

Effect Size — Eta Squared (η²)

• Definition: The proportion of total variance explained by a factor in ANOVA.

• Range: [0, 1]. 0.01 = small, 0.06 = medium, 0.14 = large.
• When to use: After ANOVA to understand how much of the outcome variation is attributable to the grouping variable.

Wald Test

• Definition: Tests whether an estimated parameter is significantly different from a hypothesised value (usually zero).

• Range: χ² statistic.
• When to use: Testing individual coefficient significance in GLMs and logistic regression. Faster to compute than the likelihood ratio test but less reliable for small samples.

Likelihood Ratio Test (LRT)

• Definition: Compares two nested models by testing whether the additional parameters significantly improve the fit.

• Range: [0, +∞). df = difference in number of parameters. Significant if p < α.
• When to use: When comparing a simpler (nested) model against a more complex one — e.g. testing whether adding a variable improves a logistic regression or Poisson GLM. Generally preferred over the Wald test for small samples.

Chi-Squared Test of Independence

• Definition: Tests whether two categorical variables are independent.

• Range: [0, +∞).
• When to use: Contingency table analysis with categorical variables. Requires expected cell counts ≥ 5; use Fisher’s exact test otherwise.

Fisher’s Exact Test

• Definition: An exact test for independence in a 2×2 contingency table, based on the hypergeometric distribution.
• Range: p-value in [0, 1].
• When to use: Small sample sizes where χ² approximation is unreliable, or when any expected cell count is below 5.

t-test (Independent / Paired)

• Definition: Tests whether the means of one or two groups differ significantly.

• Range: t statistic → p-value.
• When to use: Comparing means of two groups (independent) or before-and-after measurements (paired). Assumes approximate normality for small samples; robust for large N. Use Welch’s variant when variances are unequal.

Mann-Whitney U Test

• Definition: A non-parametric test comparing the distributions of two independent groups using ranks.
• Range: U statistic → p-value.
• When to use: Alternative to the independent t-test when the normality assumption is violated, data is ordinal, or the distribution is heavily skewed.

Kruskal-Wallis Test

• Definition: Non-parametric extension of one-way ANOVA for comparing three or more independent groups.
• Range: H statistic → p-value.
• When to use: When comparing more than two groups and the assumptions of ANOVA (normality, homoscedasticity) are not met.

Shapiro-Wilk Test

• Definition: Tests whether a sample comes from a normally distributed population.
• Range: W statistic in (0, 1]; associated p-value.
• When to use: Before applying parametric tests that assume normality. Most powerful normality test for small to moderate samples (N < 5000).

Levene’s Test

• Definition: Tests equality of variances across groups (homoscedasticity).
• Range: F statistic → p-value.
• When to use: Before ANOVA or t-test to check whether the equal-variance assumption holds. More robust to departures from normality than Bartlett’s test.

Durbin-Watson Test

• Definition: Tests for first-order autocorrelation in regression residuals.

• Range: [0, 4]. DW ≈ 2 = no autocorrelation. DW < 2 = positive, DW > 2 = negative.
• When to use: After fitting a regression model to time-series or sequentially ordered data. Violated autocorrelation inflates standard errors and invalidates inference.

3. Correlation & Association

Pearson Correlation (r)

• Definition: Measures the linear relationship between two continuous variables.

• Range: [−1, 1].

  r

  > 0.7 typically considered strong.

• When to use: Both variables are continuous and the relationship is approximately linear. Sensitive to outliers.

Spearman Rank Correlation (ρ)

• Definition: Measures monotonic (not necessarily linear) relationship between two variables using ranks.

• Range: [−1, 1].
• When to use: When the relationship is monotonic but not linear, when data is ordinal, or when outliers are present. Non-parametric alternative to Pearson.

Kendall’s Tau (τ)

• Definition: Measures ordinal association based on concordant and discordant pairs.

• Range: [−1, 1].
• When to use: Small samples, ordinal data, or when there are many tied ranks. More robust than Spearman in small datasets.

Point-Biserial Correlation

• Definition: Pearson correlation between a continuous and a dichotomous variable.
• Range: [−1, 1].
• When to use: Assessing the relationship between a binary variable (e.g. collision / no collision) and a continuous predictor.

Cramér’s V

• Definition: Measures the strength of association between two nominal (categorical) variables.

• Range: [0, 1]. 0 = no association, 1 = perfect.
• When to use: After a chi-squared test on a contingency table to quantify association strength. Works for tables larger than 2×2.

4. Information-Theoretic Metrics

Entropy

• Definition: Measures the uncertainty or disorder in a probability distribution.

• Range: [0, log₂(n)]. 0 = deterministic, max = uniform distribution.
• When to use: Quantifying uncertainty in a categorical variable. Used in decision tree splitting (information gain) and as a baseline for mutual information.

Cross-Entropy

• Definition: The average number of bits needed to encode data from distribution p using a code optimised for distribution q.

• Range: [H(p), +∞). Minimised when q = p.
• When to use: Standard loss function for classification models. Equivalent to negative log-likelihood.

KL Divergence (Kullback-Leibler)

• Definition: Measures how one probability distribution diverges from a reference distribution.

• Range: [0, +∞). 0 = identical distributions.
• When to use: Comparing a learned distribution against a reference (e.g. in variational inference, generative models). Note: asymmetric — D_KL(p‖q) ≠ D_KL(q‖p).

Mutual Information

• Definition: Quantifies the amount of information shared between two variables.

• Range: [0, min(H(X), H(Y))]. 0 = independent.
• When to use: Feature selection (detecting non-linear dependencies), clustering evaluation (NMI), and measuring information overlap. More general than correlation — captures any statistical dependence.

Jensen-Shannon Divergence

• Definition: A symmetric, bounded version of KL divergence.

• Range: [0, 1] (when using log₂). 0 = identical.
• When to use: When you need a symmetric measure of distributional difference. Its square root is a proper distance metric.

5. GLM & Count Model Metrics

Deviance

• Definition: Twice the difference between the log-likelihood of the saturated model and the fitted model. Generalises the residual sum of squares to GLMs.

• Range: [0, +∞). Lower = better fit.
• When to use: Assessing goodness-of-fit for GLMs (Poisson, logistic, negative binomial). The null deviance (intercept-only) minus residual deviance indicates variance explained.

Pearson Chi-Squared Statistic

• Definition: Sum of squared Pearson residuals. Tests whether observed counts deviate from model expectations.

• Range: [0, +∞).
• When to use: Assessing fit of count models. X²/df approximates the overdispersion parameter — if substantially > 1, the Poisson variance assumption may be violated.

Overdispersion Parameter (φ̂)

• Definition: Ratio of the Pearson χ² statistic to residual degrees of freedom.

• Range: [0, +∞). φ ≈ 1 = equidispersion.
• When to use: After fitting a Poisson model. φ̂ » 1 indicates overdispersion (variance > mean) — switch to Negative Binomial or Quasi-Poisson. φ̂ « 1 suggests underdispersion.

Pseudo-R² (McFadden)

• Definition: An R²-like goodness-of-fit measure for GLMs based on log-likelihood comparison with the null model.

• Range: [0, 1). Values of 0.2–0.4 are considered excellent.
• When to use: Reporting model explanatory power for logistic regression, Poisson GLMs, etc. Not directly comparable to OLS R². Other variants (Nagelkerke, Cox-Snell, Efron) exist — always specify which.

Hosmer-Lemeshow Test

• Definition: Groups predictions into deciles and compares observed vs expected outcomes to assess logistic regression calibration.

• Range: χ² statistic. Non-significant p = good calibration.
• When to use: After logistic regression to check whether predicted probabilities match observed frequencies. Losing favour to ECE in modern ML but still standard in epidemiology and public health.

Vuong Test

• Definition: A non-nested model comparison test.

• Range: Z-statistic.

  V

  > 1.96 suggests one model significantly better.

• When to use: Comparing non-nested count models — e.g. standard Poisson vs Zero-Inflated Poisson (ZIP), or Poisson vs Negative Binomial.

6. Model Selection Criteria

AIC (Akaike Information Criterion)

• Definition: Balances goodness-of-fit (log-likelihood) against model complexity.

• Range: (−∞, +∞). Lower is better (relative measure). k = number of estimated parameters.
• When to use: Comparing models fitted on the same dataset. Favours predictive accuracy; tends to select slightly more complex models than BIC. Standard in ecology, spatial modelling, and epidemiology.

BIC (Bayesian Information Criterion)

• Definition: Similar to AIC but with a stronger complexity penalty that depends on sample size.

• Range: (−∞, +∞). Lower is better.
• When to use: When you want a more parsimonious model than AIC selects, especially with large N. Asymptotically consistent (selects the true model as N → ∞). Penalises complexity more than AIC when N > 7.

AICc (Corrected AIC)

• Definition: AIC with a finite-sample correction.

• Range: (−∞, +∞). Lower is better.
• When to use: When the sample size is small relative to the number of parameters (rule of thumb: N/k < 40). Converges to AIC as N → ∞.

7. Spatial Statistics

Moran’s I (Global)

• Definition: A measure of global spatial autocorrelation — whether similar values cluster together in space.

• Range: Approx [−1, 1]. Positive = clustering, near 0 = random, negative = dispersion.
• When to use: Testing whether a spatial pattern (e.g. collision rates, traffic stress scores) exhibits significant clustering or dispersion. First step in exploratory spatial data analysis (ESDA). Requires defining a spatial weights matrix W.

Geary’s C

• Definition: An alternative to Moran’s I, more sensitive to local differences than global patterns.

• Range: [0, 2]. C < 1 = positive autocorrelation, C = 1 = random, C > 1 = negative.
• When to use: When you care about local dissimilarity rather than global pattern. Inversely related to Moran’s I but captures different spatial features.

Local Moran’s I (LISA — Local Indicator of Spatial Association)

• Definition: Identifies local clusters and spatial outliers at each location.

• Range: Positive = local cluster (HH or LL), negative = spatial outlier (HL or LH).
• When to use: Identifying where significant spatial clusters or outliers exist — e.g. collision hotspots (High-High), unexpectedly safe areas near dangerous ones (Low-High). Produces a cluster map with four quadrants.

Getis-Ord Gi* (Hotspot Statistic)

• Definition: Identifies statistically significant spatial clusters of high values (hotspots) and low values (coldspots).

• Range: Z-score.

  Gi*

  > 1.96 is significant at 95%.

• When to use: Hotspot mapping. Unlike LISA, Gi* identifies concentrations of high or low values (not dissimilarity). Standard tool for identifying collision hotspots, crime hotspots, disease clusters.

Ripley’s K Function

• Definition: Evaluates whether a point pattern is clustered, random, or dispersed at various spatial scales.

• Range: K(d) > πd² indicates clustering at distance d.
• When to use: Analysing point patterns (e.g. collision locations) across multiple distance scales. The L function — L(d) = √(K(d)/π) − d — is easier to interpret (L > 0 = clustered).

Semivariogram

• Definition: Describes how spatial dependence changes with distance. The foundation of geostatistics.

• Range: [0, +∞). Key parameters: nugget (intercept), sill (asymptote), range (distance at which spatial dependence vanishes).
• When to use: Before kriging or any geostatistical interpolation. Characterises the spatial structure of continuous data — determines how far apart points must be before they become independent.

Spatial Lag Model Diagnostics (LM Tests)

• Definition: Lagrange Multiplier tests for spatial dependence in regression residuals. LM-Lag tests for spatial lag (endogenous interaction), LM-Error tests for spatial error (correlated errors).
• Range: χ² statistic → p-value.
• When to use: After OLS regression on spatial data to determine whether a spatial lag model, spatial error model, or both are needed. The robust versions (RLM-Lag, RLM-Error) help distinguish between the two.

Spatial Weights Matrix (W)

• Definition: Encodes the spatial neighbourhood structure. Common types: contiguity (queen, rook), distance-based (k-nearest, distance band), kernel.
• When to use: Required for computing Moran’s I, LISA, Gi*, and spatial regression models. The choice of W affects results — always perform sensitivity analysis with alternative specifications.

Part II: Machine Learning Metrics

8. Classification — Threshold-Based

Accuracy

• Definition: The proportion of all predictions that are correct.

• Range: [0, 1]. Higher is better.
• When to use: Only when classes are roughly balanced. Avoid for imbalanced datasets — a model always predicting the majority class can score high accuracy while being useless.

Precision (Positive Predictive Value)

• Definition: Of all instances predicted positive, the proportion truly positive.

• Range: [0, 1].
• When to use: When the cost of false positives is high — e.g. spam detection, medical screening follow-up costs. Answers: “When the model says positive, how often is it correct?”

Recall (Sensitivity / True Positive Rate)

• Definition: Of all actual positive instances, the proportion correctly identified.

• Range: [0, 1].
• When to use: When missing a positive case is costly — e.g. disease screening, fraud detection, safety-critical systems. Answers: “Of all actual positives, how many did we catch?”

Specificity (True Negative Rate)

• Definition: Of all actual negatives, the proportion correctly identified.

• Range: [0, 1].
• When to use: When correctly identifying negatives matters — e.g. ruling out a condition. Paired with sensitivity in diagnostic test evaluation.

F1 Score

• Definition: Harmonic mean of precision and recall.

• Range: [0, 1].
• When to use: When you need a single balanced metric for precision and recall. Standard choice for imbalanced classification when both false positives and false negatives matter.

F-beta Score

• Definition: Generalisation of F1 that allows tuning the precision-recall trade-off via β.

• Range: [0, 1].
• When to use: β > 1 when recall matters more (e.g. F₂ for safety-critical applications where missing a dangerous case is worse than a false alarm). β < 1 when precision matters more (e.g. F₀.₅).

Matthews Correlation Coefficient (MCC)

• Definition: A correlation coefficient between observed and predicted binary classifications using all four confusion matrix cells.

• Range: [−1, +1]. +1 = perfect, 0 = random, −1 = total disagreement.
• When to use: Widely regarded as the best single metric for imbalanced binary classification. Unlike F1, it uses all four quadrants of the confusion matrix and is symmetric between positive and negative classes.

Cohen’s Kappa (κ)

• Definition: Agreement between predicted and actual labels, adjusted for agreement occurring by chance.

• Range: [−1, +1]. 1 = perfect, 0 = no better than chance. p_o = observed accuracy, p_e = expected random accuracy.
• When to use: Comparing a classifier against random chance or comparing two classifiers/raters. Also used in inter-rater reliability studies. Accounts for class prevalence.

Balanced Accuracy

• Definition: The arithmetic mean of recall for each class.

• Range: [0, 1].
• When to use: A simple adjustment to accuracy for imbalanced datasets. Equivalent to accuracy if classes are balanced. Less informative than MCC but more intuitive.

False Positive Rate (FPR / Fall-out)

• Definition: Of all actual negatives, the proportion incorrectly classified as positive.

• Range: [0, 1]. Lower is better.
• When to use: X-axis of the ROC curve. Important when the cost of false alarms is relevant.

False Negative Rate (FNR / Miss Rate)

• Definition: Of all actual positives, the proportion missed.

• Range: [0, 1]. Lower is better.
• When to use: Safety-critical systems where missed detections have severe consequences.

Negative Predictive Value (NPV)

• Definition: Of all instances predicted negative, the proportion truly negative.

• Range: [0, 1].
• When to use: When a negative prediction must be trustworthy — e.g. “this road segment is safe” needs to be reliable.

9. Classification — Threshold-Independent

AUC-ROC (Area Under the ROC Curve)

• Definition: Area under the Receiver Operating Characteristic curve (TPR vs FPR across all thresholds).

• Range: [0, 1]. 0.5 = random, 1.0 = perfect.
• When to use: Comparing overall discriminative ability of classifiers regardless of threshold. Works well when classes are balanced. For imbalanced data, AUC-PR is more informative.

AUC-PR (Area Under the Precision-Recall Curve)

• Definition: Area under the Precision-Recall curve across all thresholds.

• Range: [0, 1]. Baseline = class prevalence (not 0.5).
• When to use: When the positive class is rare — e.g. fatal collisions, fraud, rare events. More sensitive to performance differences on the minority class than AUC-ROC.

Average Precision (AP)

• Definition: Discrete approximation of AUC-PR — weighted mean of precisions at each threshold.

• Range: [0, 1].
• When to use: Same as AUC-PR. Standard metric in object detection (mAP) and information retrieval.

Log Loss (Binary Cross-Entropy)

• Definition: Penalises the deviation of predicted probabilities from actual labels. Heavily punishes confident wrong predictions.

• Range: [0, +∞). Lower is better. 0 = perfect.
• When to use: When the model outputs probabilities and you care about calibration, not just ranking. Standard training loss for logistic regression and neural network classifiers.

Brier Score

• Definition: Mean squared difference between predicted probabilities and actual binary outcomes.

• Range: [0, 1]. Lower is better.
• When to use: Evaluating probabilistic forecasts. Decomposes into calibration + refinement + uncertainty, enabling diagnosis of why a model performs poorly.

KS Statistic (Kolmogorov-Smirnov)

• Definition: Maximum vertical distance between the CDFs of positive and negative class scores.

• Range: [0, 1]. Higher = better separation.
• When to use: Credit scoring, risk modelling. Identifies the threshold with maximum class separation. Common in financial model validation.

10. Multi-Class Extensions

Macro-Averaged Precision / Recall / F1

• Definition: Compute the metric independently per class, then take the unweighted mean.

• Range: [0, 1].
• When to use: When all classes are equally important regardless of their size. Gives equal weight to rare and common classes.

Micro-Averaged Precision / Recall / F1

• Definition: Aggregate TP, FP, FN globally across all classes, then compute.

• Range: [0, 1].
• When to use: When you want an overall performance measure that is dominated by the majority class. Micro-precision = micro-recall = accuracy in multi-class settings.

Weighted-Averaged Precision / Recall / F1

• Definition: Per-class metric weighted by class support (count).

• Range: [0, 1].
• When to use: When classes are imbalanced and you want a single number that accounts for prevalence.

Confusion Matrix

• Definition: A C×C matrix where entry (i,j) counts samples with true class i predicted as class j.

• When to use: Always inspect this first. Reveals the full error pattern — which classes are confused with which. Foundation for all other classification metrics.

G-Mean (Geometric Mean)

• Definition: Geometric mean of sensitivity and specificity.

• Range: [0, 1].
• When to use: Imbalanced classification where you want both classes predicted well. Zero if either class has zero recall.

Multi-Class AUC (OvR / OvO)

• Definition: Extension of binary AUC-ROC to multi-class.

• Range: [0, 1].
• When to use: Multi-class ranking evaluation. OvR is simpler; OvO is more fine-grained for models that produce pairwise scores.

11. Calibration Metrics

Expected Calibration Error (ECE)

• Definition: Weighted average of the absolute gap between predicted probability and observed frequency across bins.

• Range: [0, 1]. Lower = better calibrated.
• When to use: When predicted probabilities are used for downstream decisions (resource allocation, risk scoring). A model can have high AUC but poor calibration. Common choice: 10–15 equal-width bins.

Maximum Calibration Error (MCE)

• Definition: The worst-case calibration gap across all bins.

• Range: [0, 1].
• When to use: Safety-critical applications where even one poorly calibrated probability region is unacceptable.

Calibration Curve (Reliability Diagram)

• Definition: Visual diagnostic plotting mean predicted probability vs observed fraction of positives per bin.
• When to use: Always plot alongside ECE. A perfectly calibrated model lies on the y = x diagonal. Reveals whether the model is over-confident (below diagonal) or under-confident (above).

12. Regression Metrics

Mean Absolute Error (MAE)

• Definition: Average of absolute differences between predictions and actual values.

• Range: [0, +∞). Lower is better. Same units as target.
• When to use: Default regression metric when you want robustness to outliers and equal treatment of all errors. More interpretable than MSE.

Mean Squared Error (MSE)

• Definition: Average of squared differences between predictions and actuals.

• Range: [0, +∞). Units are squared.
• When to use: When large errors are disproportionately bad and you want to penalise them. Differentiable, standard for optimisation.

Root Mean Squared Error (RMSE)

• Definition: Square root of MSE.

• Range: [0, +∞). Same units as target.
• When to use: When you want MSE’s outlier sensitivity but in interpretable units. The most commonly reported regression metric.

Mean Absolute Percentage Error (MAPE)

• Definition: Average of absolute percentage errors.

• Range: [0, +∞)%.
• When to use: When you want errors as relative percentages for easy communication. Avoid when actual values can be zero or near-zero.

Symmetric MAPE (sMAPE)

• Definition: A modified MAPE that treats over- and under-predictions more symmetrically.

• Range: [0, 200]%.
• When to use: Alternative to MAPE that partially addresses its asymmetry. Still problematic when both actual and predicted are near zero.

R² (Coefficient of Determination)

• Definition: Proportion of variance in the target explained by the model.

• Range: (−∞, 1]. 1 = perfect. Can be negative.
• When to use: Standard for reporting OLS regression fit. Negative R² means worse than predicting the mean. Always report for regression models but do not use as the sole criterion.

Adjusted R²

• Definition: R² penalised for the number of predictors.

• Range: (−∞, 1]. p = number of predictors.
• When to use: Comparing models with different numbers of predictors. Prevents selecting overfit models that add noise variables.

Median Absolute Error (MedAE)

• Definition: Median of absolute differences between predictions and actuals.

• Range: [0, +∞).
• When to use: When you want the typical error, uninfluenced by extreme outliers. More robust than both MAE and RMSE.

Max Error

• Definition: The largest absolute error across all predictions.

• Range: [0, +∞).
• When to use: Worst-case analysis — critical when you need to bound the maximum possible prediction error.

Explained Variance Score

• Definition: Proportion of target variance captured, allowing biased predictions.

• Range: (−∞, 1].
• When to use: Differs from R² only when predictions are biased (mean(y − ŷ) ≠ 0). Use alongside R² to diagnose systematic bias.

Huber Loss

• Definition: Loss function that is quadratic for small errors and linear for large ones.

• Range: [0, +∞).
• When to use: Training regression models when data has outliers but you still want some sensitivity to large errors. Combines benefits of MSE and MAE.

Quantile Loss (Pinball Loss)

• Definition: Asymmetric loss for quantile regression.

• Range: [0, +∞).
• When to use: When you want to predict a specific quantile rather than the mean — e.g. “the 90th percentile of travel time” or “the worst-case delay”. τ = 0.5 recovers MAE.

13. Ranking & Retrieval Metrics

Precision@k

• Definition: Fraction of the top-k predictions that are relevant.

• Range: [0, 1].
• When to use: Evaluating top-k recommendation systems or prioritised risk lists (e.g. “of the 10 road segments we flagged, how many are genuinely high-risk?”).

Recall@k

• Definition: Fraction of all relevant items appearing in the top-k.

• Range: [0, 1].
• When to use: When you need to know what proportion of true positives are captured in the top-k.

MAP (Mean Average Precision)

• Definition: Mean of average precision across all queries.

• Range: [0, 1].
• When to use: Standard metric for information retrieval and ranked recommendation evaluation.

NDCG (Normalised Discounted Cumulative Gain)

• Definition: Evaluates ranking quality using graded relevance, discounting items further down the list.

• Range: [0, 1].
• When to use: When relevance is graded (not binary) and the position of relevant items matters. Top results should be more relevant.

MRR (Mean Reciprocal Rank)

• Definition: Average of the reciprocal rank of the first relevant result across queries.

• Range: [0, 1].
• When to use: When you only care about the first correct result — e.g. search engines, QA systems.

Lift

• Definition: Ratio of the model’s positive rate in a selected group compared to the overall rate.

• Range: [0, +∞). Lift > 1 = better than random.
• When to use: Marketing, credit risk, targeted interventions. “How much better than random is our targeting?”

14. Clustering — External (with Ground Truth)

Adjusted Rand Index (ARI)

• Definition: Pairwise agreement between predicted clustering and true labels, adjusted for chance.

• Range: [−1, 1]. 1 = perfect, 0 = random agreement.
• When to use: When you have true cluster labels and want a chance-corrected measure. Preferred over raw Rand Index.

Normalised Mutual Information (NMI)

• Definition: Mutual information between clustering and ground truth, normalised to [0,1].

• Range: [0, 1].
• When to use: Comparing clusterings with different numbers of clusters. Information-theoretic and handles varying K well.

Homogeneity

• Definition: Each cluster contains only members of a single class.

• Range: [0, 1].
• When to use: When cluster purity matters most. Satisfied trivially by assigning each point to its own cluster.

Completeness

• Definition: All members of a given class are assigned to the same cluster.

• Range: [0, 1].
• When to use: When you want to ensure that entire classes are captured within single clusters.

V-Measure

• Definition: Harmonic mean of homogeneity and completeness.

• Range: [0, 1].
• When to use: Balanced evaluation of clustering that rewards both purity and completeness. Analogous to F1 for clustering.

Fowlkes-Mallows Index (FMI)

• Definition: Geometric mean of pairwise precision and pairwise recall.

• Range: [0, 1].
• When to use: Alternative to ARI that is more interpretable as a precision-recall analogue for pairs.

15. Clustering — Internal (No Ground Truth)

Silhouette Score

• Definition: How similar a point is to its own cluster vs the nearest neighbouring cluster.

• Range: [−1, 1]. Higher = better. Negative = likely misclassified.
• When to use: General-purpose internal validation for convex clusters. Easy to interpret. Less suitable for density-based or non-convex clusters.

Davies-Bouldin Index

• Definition: Average similarity ratio of each cluster with its most similar cluster.

• Range: [0, +∞). Lower = better.
• When to use: Comparing different values of K. Does not require computing pairwise distances for all points, so more efficient than silhouette for large datasets.

Calinski-Harabasz Index (Variance Ratio Criterion)

• Definition: Ratio of between-cluster to within-cluster dispersion.

• Range: [0, +∞). Higher = better.
• When to use: Fast to compute. Favours compact, well-separated globular clusters. Best for k-means-like methods.

Inertia (Within-Cluster SSE)

• Definition: Sum of squared distances from each point to its cluster centroid.

• Range: [0, +∞). Lower = better.
• When to use: Used in the elbow method for choosing K. Always decreases with more clusters — look for the “elbow” where marginal improvement diminishes.

DBCV (Density-Based Cluster Validation)

• Definition: Validity index for density-based clusters using mutual reachability distances.

• Range: [−1, 1].
• When to use: Evaluating DBSCAN or HDBSCAN clusters with arbitrary shapes. Unlike silhouette, does not assume convex clusters.

16. Model Selection & Validation

k-Fold Cross-Validation Score

• Definition: Data is split into k folds; model trains on k−1, tests on the held-out fold, repeated k times.

• When to use: Standard evaluation protocol to estimate generalisation performance and diagnose overfitting. Typical k = 5 or 10. Use stratified k-fold for imbalanced classification. Report both mean and standard deviation.

Leave-One-Out Cross-Validation (LOOCV)

• Definition: Special case of k-fold where k = N.

• When to use: Very small datasets where losing even a few samples to a test fold is costly. Unbiased but high-variance and computationally expensive.

Bias-Variance Decomposition

• Definition: Decomposes expected prediction error into irreducible noise + bias² + variance.

• When to use: Conceptual framework for diagnosing whether a model underfits (high bias) or overfits (high variance). Not directly computed, but diagnosed via learning curves.

Learning Curve

• Definition: Plots training and validation performance as a function of training set size.
• When to use: Diagnosing bias-variance trade-off visually. Converging low scores = underfit; diverging scores = overfit; converging high scores = good fit.

17. Feature Importance & Interpretability

Gini Importance (Mean Decrease in Impurity)

• Definition: Total reduction in impurity attributed to a feature across all tree splits.

• Range: [0, 1] when normalised.
• When to use: Quick feature ranking for tree-based models (Random Forest, Gradient Boosting). Biased towards high-cardinality and correlated features — use permutation importance for more reliable results.

Permutation Importance

• Definition: Drop in model performance when a feature’s values are randomly shuffled.

• Range: Can be negative (feature hurts model).
• When to use: Model-agnostic feature importance. Compute on the validation set. More reliable than Gini importance for correlated features.

SHAP Values (SHapley Additive exPlanations)

• Definition: Game-theoretic attribution of each feature’s contribution to each individual prediction.

• Range: (−∞, +∞). Sum of all φⱼ = f(x) − E[f(x)].
• When to use: When you need instance-level explanation — why did the model make this specific prediction? Satisfies desirable theoretical properties (local accuracy, missingness, consistency). Computationally expensive.

Variance Inflation Factor (VIF)

• Definition: Measures multicollinearity for predictor j.

• Range: [1, +∞). VIF > 5–10 = problematic.
• When to use: Before fitting regression models to detect and remove highly correlated predictors. High VIF inflates standard errors and destabilises coefficients.

18. Distance & Similarity Metrics

Euclidean Distance (L2)

• Definition: Straight-line distance in n-dimensional space.

• Range: [0, +∞).
• When to use: Default distance for k-means, kNN, and most geometric methods. Assumes all features are on comparable scales — standardise first if not.

Manhattan Distance (L1)

• Definition: Sum of absolute differences along each dimension.

• Range: [0, +∞).
• When to use: Grid-based environments, high-dimensional data (less affected by curse of dimensionality than Euclidean), or when features are on different scales and not standardised.

Cosine Similarity

• Definition: Cosine of the angle between two vectors.

• Range: [−1, 1]. 1 = identical direction.
• When to use: Text analysis (TF-IDF, embeddings), recommendation systems — anywhere direction matters more than magnitude.

Jaccard Index (IoU)

• Definition: Intersection over union of two sets.

• Range: [0, 1].
• When to use: Comparing sets or binary vectors — e.g. object detection bounding boxes (IoU), set similarity, binary feature overlap.

Mahalanobis Distance

• Definition: Scale-invariant distance accounting for correlations via the covariance matrix.

• Range: [0, +∞). S = covariance matrix.
• When to use: Outlier detection, classification with correlated features. Reduces to Euclidean when S = I. Accounts for the shape of the data distribution.

Hamming Distance

• Definition: Number of positions where corresponding elements differ.

• Range: [0, n].
• When to use: Categorical or binary data — e.g. comparing one-hot encoded features, DNA sequences, error-correcting codes.

Minkowski Distance (Lp)

• Definition: Generalised distance metric parameterised by p.

• Range: [0, +∞). p=1 → Manhattan, p=2 → Euclidean, p→∞ → Chebyshev.
• When to use: When you want to tune the distance sensitivity between Manhattan and Euclidean behaviour.

Appendix: Common Notation

Community detection — the task of finding groups of densely connected nodes in a network — has become a popular tool in urban analysis. Applied to street networks, it promises to reveal something about how cities are naturally structured: which streets belong together, where one neighbourhood ends and another begins, and how urban form shapes movement and flow.

But here is the catch: there is no single definition of what a “community” is. Different algorithms ask fundamentally different questions of the same network. Run three algorithms on the same city and you will get three different answers. Run one algorithm on three different cities and the results will shift again.

This post does both. Using OSMnx to extract street networks from three cities — the City of London, the Eixample district of Barcelona, and the San Polo sestiere of Venice — and cdlib to run three community detection algorithms, I ask: does the algorithm see the city, or does the city shape the algorithm?

Three Cities, Three Urban Archetypes

The three cities were chosen deliberately to represent three fundamentally different types of urban morphology.

The City of London is the result of organic, unplanned growth. Its street network reflects nearly two thousand years of incremental development — Roman foundations, medieval expansion, post-fire rebuilding, and Victorian additions layered on top of each other. The result is an irregular, asymmetric network with high variability in block size, street width, and connectivity. It has no dominant geometry.

Eixample, Barcelona is the opposite extreme. Designed by Ildefons Cerdà in 1860, it is one of the most celebrated examples of rational urban planning in history. Its defining feature is a near-perfect grid of octagonal blocks, with chamfered corners at every intersection. Every block is roughly the same size. Every intersection looks roughly the same. It is a network of radical, deliberate uniformity.

San Polo, Venice is constrained by something neither of the other two cities had to contend with: water. The street network of Venice’s historic centre is shaped entirely by the lagoon’s geography — canals replace roads, bridges create bottlenecks, and the result is a labyrinthine network of narrow pedestrian alleys (calli) punctuated by small squares (campi). It is fragmented by nature, not by plan.

These three cities were not chosen arbitrarily. They represent the three fundamental forces that shape urban street networks: organic growth, rational planning, and geographic constraint. If community detection algorithms are sensitive to urban morphology — and we should expect them to be — these three cities should produce meaningfully different results.

Three Algorithms, Three Ways of Seeing

Just as important as the choice of cities is the choice of algorithms. Community detection is not a single method but a family of approaches, each built on a different theoretical foundation and a different implicit definition of what makes a good community.

For this experiment, three algorithms were selected to cover three distinct paradigms.

Walktrap (Pons & Latapy, 2005) is built on the intuition of random walks. The core idea is simple: a random walker exploring a network will tend to get trapped within densely connected subgraphs, visiting the same nodes repeatedly before finding a bridge to another part of the network. Communities are identified by running many short random walks and clustering nodes that are frequently visited together. For street networks, this has a natural urban interpretation — it approximates the experience of someone wandering through a neighbourhood, more likely to loop around familiar streets than to cross into an entirely different area.

Leiden (Traag, Waltman & van Eck, 2019) takes a different approach, optimising a quantity called modularity — a measure of how much denser the connections within communities are compared to what you would expect from a random network with the same degree distribution. Leiden is an improvement over the earlier and widely-used Louvain algorithm, adding a critical refinement step that guarantees every detected community is internally connected. This matters for spatial networks: Louvain can produce communities that are topologically disconnected, which makes no sense in a geographic context. Leiden fixes this.

Infomap (Rosvall & Bergstrom, 2008) comes from information theory. Rather than maximising modularity, it minimises the description length of a random walk through the network — essentially asking: what partition of the network allows us to describe the path of a random walker most efficiently? Communities are places where information flow tends to circulate before escaping. Crucially, Infomap natively supports directed and weighted networks, making it the only algorithm in this set that can properly account for the directionality of one-way streets.

Together, these three algorithms ask three different questions of the same street network:

• Walktrap asks: where does a random walker get stuck?
• Leiden asks: where are connections denser than chance would predict?
• Infomap asks: where does information tend to circulate?

These are not equivalent questions, and we should not expect equivalent answers.

A note on implementation: all three algorithms were run on undirected versions of the OSMnx street graphs, with edge weights set to the inverse of street length (so that shorter streets — implying stronger local connectivity — carry higher weight). Modularity, coverage, and performance were computed for each partition to enable cross-algorithm and cross-city comparison.

Not all community detection algorithms are created equal — and not all of them are suitable for street networks. Here is a quick reference of the main algorithms I considered before settling on the three used in this experiment.

Results

Metrics Summary

Modularity measures how much denser the connections within communities are compared to a random network. Scores above 0.7 are generally considered strong. It has a known limitation — the resolution limit — where optimisation tends to merge small but genuine communities into larger ones, which is one reason Leiden consistently produces fewer, larger communities than Walktrap.

Coverage measures the proportion of edges that fall within communities rather than between them. A high coverage score means the partition is capturing most of the network’s real internal connections, rather than cutting across them.

Performance measures the fraction of all possible node pairs that are correctly classified — nodes in the same community that are actually connected, and nodes in different communities that are genuinely not. Infomap consistently achieves ~0.99 across all three cities, which sounds impressive until you realise it does so by producing hundreds of tiny, statistically clean but geographically meaningless communities.

The tension between these three metrics is itself one of the key findings of this experiment: a partition can score well on Performance while scoring poorly on Modularity. There is no single number that captures whether a community partition is good — it depends entirely on what you are trying to do with it.

Finding 1: The algorithm ranking is stable, but the city changes everything

The most striking result is how consistent the relative ranking of algorithms is across all three cities. In every case, Walktrap produces the highest modularity, Leiden produces the fewest and largest communities, and Infomap over-partitions while achieving near-perfect Performance (~0.99). The hierarchy does not change regardless of whether you are looking at medieval London, rational Barcelona, or labyrinthine Venice.

What does change is the absolute values and the community sizes. Eixample consistently produces the lowest modularity scores across all three algorithms — with Infomap dropping as low as 0.7803 — and the largest average community sizes under Leiden (avg=51.9 nodes per community, compared to 30.6 in London and 25.9 in Venice). The grid’s radical uniformity leaves the algorithm with very little structural signal to work with. When every intersection looks the same, there is less modularity to find.

San Polo produces the smallest communities overall, with Walktrap averaging just 15.5 nodes per community. This is consistent with what you would expect from Venice’s topology: the narrow alleys and frequent dead-ends create natural bottlenecks that the algorithms interpret as community boundaries, fragmenting the network into small, tight clusters.

Finding 2: High performance does not mean urban interpretability

Infomap achieves the highest Performance score in every city and every comparison. On paper, this makes it the best algorithm. In practice, it is the least useful for urban analysis.

Performance measures how accurately node pairs are classified — whether two nodes in the same community are actually well-connected, and whether two nodes in different communities are genuinely separated. Infomap’s micro-communities score well on this metric precisely because they are so small: a community of 8 nodes is almost guaranteed to be internally dense. But a community of 8 nodes in a medieval city or a Venetian campo does not correspond to any recognisable urban unit. It is statistically rigorous and geographically meaningless.

This is a broader lesson that applies beyond this experiment. Optimising for a metric is not the same as finding something real. The choice of algorithm should follow the research question, not the other way around.

Finding 3: Urban morphology is legible in the community structure

Perhaps the most visually compelling result is how different the community maps look for each city, even under the same algorithm.

In the City of London, communities under Leiden are large, irregular polygons that roughly correspond to recognisable clusters of streets — the kind of organic groupings you might expect from a city that grew without a plan. The convex hulls are uneven and overlapping, reflecting the network’s complexity.

In Eixample, the communities are more regular in shape, reflecting the underlying grid. Leiden in particular produces very large communities that span multiple blocks, suggesting that the algorithm cannot find strong internal boundaries in a network designed to have none. The uniformity of the grid is the point — and it makes community detection harder.

In San Polo, the communities are small and scattered, disconnected from each other in ways that mirror the physical fragmentation of the island’s topology. The bridges and bottlenecks that define movement in Venice show up clearly as community boundaries.

Conclusion

Running community detection on street networks is easy. Interpreting the results requires understanding both the algorithm and the city.

This experiment shows that the relative behaviour of algorithms is robust — Walktrap will always find more granular communities than Leiden, and Infomap will always over-partition — but the absolute results are shaped by urban morphology in ways that are both measurable and visually interpretable. A grid city like Eixample is harder to partition than an organic city like the City of London. A constrained network like Venice produces smaller, more fragmented communities than either.

The deeper lesson is about the relationship between method and meaning. No algorithm is universally better. Walktrap’s random walk intuition maps naturally onto pedestrian movement. Leiden’s connectivity guarantees make it appropriate for spatial analysis. Infomap’s direction-awareness makes it the right tool when one-way streets matter. The question is not which algorithm wins, but which question you are trying to answer.

The streets were there long before the algorithms. The algorithms just help us see them differently.

References

• Pons, P. & Latapy, M. (2005). Computing communities in large networks using random walks. Computer and Information Sciences.
• Traag, V.A., Waltman, L. & van Eck, N.J. (2019). From Louvain to Leiden: guaranteeing well-connected communities. Scientific Reports, 9, 5233.
• Rosvall, M. & Bergstrom, C.T. (2008). Maps of random walks on complex networks reveal community structure. PNAS, 105(4), 1118–1123.
• Boeing, G. (2017). OSMnx: New methods for acquiring, constructing, analyzing, and visualizing complex street networks. Computers, Environment and Urban Planning, 65, 126–139.
• Lancichinetti, A. & Fortunato, S. (2009). Community detection algorithms: a comparative analysis. Physical Review E, 80, 016118.

See also: Literature Reading

Log Entries

2026-03-05 — Supervisor Meeting #1

Meeting · Prof. Duncan Smith

• Spatial scale needs narrowing — confirm with TfWM that a scope adjustment is acceptable
• Road-level analysis cannot predict need for new infrastructure; frame limitations clearly
• Reference local council Low Traffic Neighbourhood (AKA.LTN) proposals and TfL Cycle Superhighways as context
• Quantifiable metrics for sub-criteria still unresolved
• ML methodology: role and application within the framework not yet defined

Action Items

• Follow up with TfWM on scope/scale adjustment
• Read Duncan and Philyoung’s prior co-authored papers
• Identify available datasets for the West Midlands

Open Questions

• What is the appropriate spatial scale for this study?
• Which sub-criteria can realistically be quantified?

2026-03-12 — Methodology Development

Writing · LTS Framework Adaptation

Drafted the core rationale for the UK-adapted LTS framework. Existing frameworks were developed for the North American road environment; the proposed adaptation replaces American-origin thresholds with LTN 1/20 design standards — producing a framework directly grounded in the standards against which West Midlands cycling schemes will be assessed.

Decisions Made

• LTN 1/20 thresholds replace North American equivalents as the primary source of design standards
• Framework remains road-segment based (consistent with original LTS methodology)

Still Unresolved

• Intersection stress: how to map LTN 1/20 guidance onto intersection-level indicators
• Weighted aggregation vs. parallel dimensions — aggregation logic not yet decided

Next Steps

• Build indicator crosswalk table: LTS indicators ↔ LTN 1/20 equivalents
• Review Furth & Nixon (2016) intersection stress methodology

2026-04-22 — Data Ethics & TfWM Data Receipt

Ethics · Admin

UCL ethics forms in progress: Form A (Screening), Form B (Data Protection), Website Application.

Received data from TfWM this week:

• Compass IoT Near Miss Data
• iRAP Road Safety Assessment Star Ratings Data
• TfWM Self-Serve portal

Next Steps

• Submit completed ethics forms
• Audit received datasets for completeness and coverage

2026-05-08 — Literature reading & Jeong’s LTS Reproducibility

Meeting · Coding

Reproducing Jeong’s LTS Methodology:

• Getting a initial LTS map of West Midland Area
• Finishing reading Mekuri/ Conveyal/ Jeong’s LTS (initially)

While further reading / work are needed, especially concerning the whole frame of methodology (how to frame with LTN 1/20). Figuring out what TfWM can benefit from, and making the methods easier to understand. Maybe I have to build it under a wider background, should not be limited within LTS?

Next Steps

• Making some sliders for TfWM (DDL: 20/5/2026)
• presentation and communication with Jeong, preparing some questions in advance (DDL 13/5/2026)!!
• Buffer Zone: Do i really need it? do some reading.

2026-05-20 — TfWM Meeting: Deliverables Confirmed & Next Steps

Meeting · Decision

Met with TfWM. Callum confirmed the three final deliverables for the dissertation, so the overall direction is now locked in. The ask is clearly industry-facing — a commercial report plus a presentation suggests TfWM wants something they can actually use and circulate internally, not just an academic analysis. This also means the rationale for which LTN 1/20 metrics are included vs excluded needs to be very explicit: what can be answered with data, what can’t, and why each one is in or out.

Decisions Made

• Final deliverables confirmed as three components:
  • a) A classification of LTN 1/20 metrics by operability — which can be quantified / improved through data, and which are inherently subjective and harder to fold into an automated network-scale assessment
  • b) A commercial report for TfWM
  • c) A final presentation to TfWM
• Next meeting (Fri 29/5) deliverables specified by Callum (see Next Steps)

Open Questions

• Format and depth of the commercial report — page count, structure, whether an executive summary is expected. Needs confirming with Callum.
• Criteria for including/excluding metrics — purely data availability, or also methodological rigour and TfWM’s intended use case?
• Audience for the final presentation — technical team only, or also policy / decision-makers? This shapes the framing.

Next Steps

• Prepare slides for next Friday’s meeting detailing progress so far (DDL: 29/5/2026)
• List which LTN 1/20 metrics will be answered, and which won’t
• Write justification for excluded metrics (data limitation / subjectivity / scope)
• Finalise the LTN 1/20-enhanced LTS framework and send to Duncan for review
• Draft a plan for the next few weeks of data analysis work
• Compile potential interview questions (for stakeholders)

Origins: The Gravity Model

The intellectual roots of spatial interaction models lie in an analogy with Newtonian physics. The first to apply gravitational logic to human movement was Henry Carey (1858), who observed that human migration between cities followed patterns resembling gravitational attraction. This idea was formalised mathematically by Ernst Ravenstein (1885) in his Laws of Migration, where he established empirically that migration intensity decreases with distance.

The first explicitly gravitational formulation came from E.G. Young (1924) and was later popularised by George Zipf (1946), who proposed the P₁P₂/D rule:

where $T_{ij}$ is the interaction between places $i$ and $j$, $P_i$ and $P_j$ are their populations, $d_{ij}$ is the distance separating them, and $k$ is a proportionality constant. This model treats population as the sole “mass” generating interaction, and assumes flows decay linearly with distance.

The astronomer John Stewart (1941) then introduced the concept of demographic potential — a measure of the cumulative gravitational pull exerted on one place by all others:

Stewart’s contribution was to generalise the distance exponent to $\beta$, recognising that the rate of distance decay was an empirical question rather than a physical constant. This $\beta$ parameter — the distance decay parameter — became the central object of interest in all subsequent SIM development. A high $\beta$ means flows are very sensitive to distance (steep decay); a low $\beta$ means people are willing to travel further.

The General Gravity Model

Building on Stewart and Zipf, the general unconstrained gravity model takes the form:

The parameters $\mu$, $\alpha$, and $\beta$ are estimated from observed flow data. No marginal totals are constrained — both origin and destination flows are free to vary. This makes the model maximally flexible but less predictively accurate, since it cannot guarantee that predicted flows sum to known totals.

Typical applications: inter-regional trade flows; early migration studies; telephone communication patterns — any context where neither origin totals nor destination totals are independently known.

Wilson’s Entropy-Maximising Framework and the SIM Family (1967–71)

The most important theoretical advance came from Alan Wilson (1967, 1971), who derived the entire family of spatial interaction models from first principles using entropy maximisation. Rather than relying on the physics analogy, Wilson asked: given what we know about total flows, what is the most probable distribution of individual trips? Maximising entropy subject to constraints on observed totals yields exactly the gravity model family — but now with a rigorous statistical foundation.

Crucially, Wilson showed that the appropriate model form depends entirely on which marginal totals are known and should be treated as constraints. This gave rise to three distinct models.

Unconstrained Model

Used when neither $O_i$ nor $D_j$ are known. Wilson showed this corresponds to maximising entropy with only a constraint on total travel cost.

Typical applications: exploratory analysis of interaction patterns; inter-regional trade; contexts where no marginal totals can be independently observed.

Production-Constrained Model

When the total flow leaving each origin $O_i$ is known (e.g. total shopping trips from a residential zone, derived from population), it should be held fixed. Wilson introduced a balancing factor $A_i$ to enforce this:

The constraint satisfied is:

Here $\alpha$ and $\beta$ are the only free parameters — origin mass is already encoded in $O_i$. The destination inflows $\sum_i T_{ij}$ are a model output, not an input.

Typical applications: retail catchment modelling (flows from residential zones to supermarkets); journey-to-work where census commuter totals by origin are available; school choice modelling.

Attraction-Constrained Model

Symmetrically, when total inflow to each destination $D_j$ is known but origin totals are not:

The constraint satisfied is:

Typical applications: modelling flows to a fixed-capacity facility (hospital, airport) where total admissions or passengers are known; migration to cities where in-migration totals are recorded.

Doubly Constrained Model

When both $O_i$ and $D_j$ are known, both sets of balancing factors are applied simultaneously:

Since $A_i$ depends on $B_j$ and vice versa, these are solved iteratively (the Furness/IPF procedure) until convergence. Both constraints are satisfied simultaneously:

The only free parameter is $\beta$ — both mass terms are fully determined by observed data, leaving only the distance decay to estimate.

Typical applications: transport planning trip distribution (four-step model); inter-regional migration where both in- and out-migration totals come from the census; freight movement between fixed origin and destination terminals.

The Role of the Cost Function

All three constrained models require specifying $f(c_{ij})$. Two standard forms are:

For supermarket shopping — short, frequent, and urban — the negative exponential is standard, as most people are unwilling to travel more than a few kilometres and the deterrence effect is sharp.

Summary

Wilson’s framework showed that these are not four separate models but one unified family, differentiated only by which empirical constraints are imposed. This theoretical unification, combined with the entropy-maximising derivation, transformed spatial interaction modelling from a physics analogy into a rigorous applied framework — and laid the foundation for every land use and transport model that followed.

References

Carey, H.C. (1858) Principles of Social Science. Philadelphia: J.B. Lippincott.

Fotheringham, A.S. and O’Kelly, M.E. (1989) Spatial Interaction Models: Formulations and Applications. Dordrecht: Kluwer Academic Publishers.

Furness, K.P. (1965) ‘Time function iteration’, Traffic Engineering and Control, 7(7), pp. 458–460.

Hansen, W.G. (1959) ‘How accessibility shapes land use’, Journal of the American Institute of Planners, 25(2), pp. 73–76.

Ravenstein, E.G. (1885) ‘The laws of migration’, Journal of the Statistical Society of London, 48(2), pp. 167–235.

Stewart, J.Q. (1941) ‘An inverse distance variation for certain social influences’, Science, 93(2404), pp. 89–90.

Wilson, A.G. (1967) ‘A statistical theory of spatial distribution models’, Transportation Research, 1(3), pp. 253–269.

Wilson, A.G. (1971) Entropy in Urban and Regional Modelling. London: Pion.

Zipf, G.K. (1946) ‘The P1 P2/D hypothesis: On the intercity movement of persons’, American Sociological Review, 11(6), pp. 677–686.