Files
ML-py/model_card.md
T
Noel Sabia 939a8db482 feat: complete ML pipeline, FastAPI service, small model, tests, image
- finish the global HistGB bundle, scenario forecasts and 28-day ablation
- export sensors/series/forecasts/metrics/context plus baked weather frames
- add FastAPI health/meta/predict/simulate and background /train jobs
- add the p75 busy classifier, retraining from CSV, and hot-swap persistence
- add pytest coverage, multi-stage non-root Dockerfile, README and model card

Holdout: weather 7d MAE 3.32 / WAPE 27.5% / R2 0.777 vs seasonal naive
MAE 4.23 / WAPE 35.0% / R2 0.656 (5,350 eval rows).
2026-11-14 05:01:16 +01:00

5.7 KiB
Raw Blame History

Model card — PulseCast footfall forecast

Model

One global HistGradientBoostingRegressor predicts hourly footfall density for all eight inner-city sensors. A separately trained logistic regression predicts a binary busy hour for the training demo. Both are batch-trained; no model tunes itself at inference time.

  • Forecast version: histgb-YYYY-MM-DD (baked), histgb-YYYYMMDD-HHMMSS after a runtime retrain.
  • Small classifier version: logit-YYYYMMDD-HHMMSS.
  • Framework: scikit-learn 1.9, pandas 3.1, Python 3.12.

Intended use

Decision support for a store manager: next 7 days of footfall, an uncertainty band, and a transparent staffing heuristic on top. The model is not a census of people, not a city-wide heatmap, and not an automated labor optimizer.

Features (forecast)

Group Features
Calendar hour, weekday, month, is_weekend, is_holiday, sensor_code
Seasonal lag_week (previous observed same weekday/hour), seasonal_mean_4w
Weather temp_c, precip_mm

The production 7-day model uses only features that are at least 168 hours old. Day-ahead ablation models additionally use lag_day, lag_2d, roll_24, roll_168.

Target and data

  • Target: mean of 15-minute densityData readings in a UTC hour, per sensor.
  • Training window: 2026-06-12 → 2026-10-10; holdout: 2026-10-10 → 2026-11-07.
  • Eight usable sensors after excluding maker-fair (73 rows, Hannover), niklas-01 (8 rows), hn-bollwerksturm-01 (5 rows).
  • density-01 merges two physical devices (863f75b0, f8c19730) into one logical location. The older device history is outside the 8-sensor window and is not used for the production model.
  • Weather: Open-Meteo historical archive for training, forecast for inference and scenarios, Heilbronn grid cell.
  • Holidays: explicit Baden-Württemberg 2025–2026 calendar.

Evaluation

Time-based 28-day holdout, no shuffling. Seasonal naive is the baseline. WAPE is sum(|error|) / sum(|actual|). R² is computed on the same 5,350 rows.

Model MAE WAPE R²
Seasonal naive (same local hour last week) 4.23 35.0% 0.656
Day-ahead, no weather 3.96 32.8% 0.706
Day-ahead, + holiday flag 3.84 31.8% 0.732
Day-ahead, + temperature/rain 3.36 27.8% 0.763
7-day-safe, no weather 4.08 33.8% 0.697
7-day-safe, + temperature/rain (production) 3.32 27.5% 0.777

Weather is the useful signal: adding temperature and rain cuts MAE by ~22% versus the 7-day no-weather model and ~38% versus seasonal naive. The holiday flag alone changes day-ahead MAE by −0.12 in this run; holidays are context, not a headline accuracy claim.

The mission's quick-baseline numbers (seasonal naive 5.40 / 44.7% / 0.462) came from a different window and feature setup. The direction and the weather gain reproduce; the exact decimals are not copied.

Uncertainty band

The lo/hi band is the 10th/90th percentile of holdout residuals added to the point forecast, clipped at zero. It is an empirical 80% band, not a calibrated guarantee.

Small classifier

  • Target: busy = hourly footfall strictly above that sensor's p75, where the p75 is computed on the training window only.
  • Features: hour, weekday, is_weekend, temp_c, precip_mm, is_holiday.
  • Model: standardized logistic regression, class_weight="balanced".
  • Split: last 20% of hourly timestamps, time-ordered.
  • Result: accuracy 0.708, precision 0.360, recall 0.574, F1 0.443, ROC-AUC 0.726.
  • Trains in ~0.25 s on the challenge CSV and under 1 s on a small upload.
  • The low precision reflects a hard p75 threshold and a deliberately tiny model; it is a demo artifact, not the production forecast.

No-leakage rules

  • All splits are time-based; no random shuffling.
  • Seasonal lags use the previous observed same weekday/hour; future lags are never filled from future targets.
  • Rolling means are shifted at least 24 hours back.
  • The production 7-day model uses only ≥168-hour-old information, so a 168-hour forecast needs no recursion.
  • Weather in the holdout is observed, not forecast. The demo uses forecast weather, so live accuracy will be lower than the holdout number.
  • The holdout is used for reporting the headline; the production bundle is refit on all observed rows only after the holdout is evaluated.

Limitations

  • PAX counters detect discoverable devices; penetration is unknown. Values are an index of estimated devices, not people.
  • Eight sensors cover ~580 m; the data does not support a city-wide surface or a store-location recommender.
  • Five months of multi-sensor history; no annual seasonality claim. density-01 has a longer single-sensor history, but that cannot represent the network.
  • Sensor gaps of 4–6 days remain. The model tolerates missing lags but cannot recover unobserved footfall.
  • No events, markets, school holidays, bridge days, or promotions are modeled. Scenario uplifts are explicit assumptions, not causal estimates.
  • Weather is observed in training and forecast in the demo; forecast error is not included in the reported metrics.
  • The staffing heuristic assumes visitors-per-staff-hour, shift length, safety factor, and labor cost. There is no POS, queue, sales, or labor-cost data.

Ethical and operational notes

  • No PII, MAC addresses, or individual tracking is stored or exposed.
  • Public API reads are open; training is token-protected and internal.
  • Model artifacts and derived aggregates are committed; the raw challenge CSV is not. No secrets are baked into the image.
  • On restart the service falls back to the baked model; retrained models persist only in /tmp and are not assumed to survive a pod restart.