Files
ML-py/README.md
Noel Sabia 070ecd2a78
ci / check (push) Successful in 1m12s
ci: fix Gitea runner setup and tighten docs
Replace setup-python with setup-uv: the self-hosted Gitea runner lacks the
toolcache and lsb_release dependency, so the job failed before any project
step. uv installs its own Python 3.12; the checks are unchanged.

Align the README and docs with the committed artifacts: quick start serves
from baked artifacts, rebuild steps are separate, and the reference-model
comparison no longer overstates the result. Drop stale external-doc
references and comments that restated the code.
2026-11-14 09:10:47 +01:00

8.6 KiB

PulseCast ML

Footfall pipeline, weather-aware 7-day forecast, scenario engine, and FastAPI service for the PulseCast staffing copilot. It turns raw 15-minute PAX counter rows into hourly series, joins weather and Baden-Württemberg holidays, trains one global model, evaluates it on a 28-day time-based holdout, and serves the resulting JSON artifacts.

Results

28-day time-based holdout: 2026-10-10 23:00 UTC → 2026-11-07 23:00 UTC, 5,350 evaluation rows, 8 sensors.

Model MAE WAPE R²
Seasonal naive (exact 168-row lag) 5.40 44.7% 0.462
Day-ahead, no weather 3.96 32.8% 0.706
Day-ahead, + holiday flag 3.84 31.8% 0.732
Day-ahead, + temperature/rain 3.36 27.8% 0.763
7-day-safe, no weather 4.08 33.8% 0.697
7-day-safe, + temperature/rain (production) 3.32 27.5% 0.777

The seasonal-naive row reproduces the mission reference exactly (5.40 / 44.7% / 0.462), and the day-ahead weather row matches 3.36 / 27.8%. Weather cuts error by ~19% versus the no-weather 7-day model and ~38% versus seasonal naive. The production 7-day model edges the reference day-ahead weather row (3.32 vs 3.36 MAE). The two use different features; lag_week uses the previous observed same weekday/hour across sensor gaps, while the seasonal-naive baseline uses the exact 168-row lag.

Small busy-hour classifier (baked for the "train new data" demo):

Accuracy Precision Recall F1 ROC-AUC Train rows Test rows
0.708 0.360 0.574 0.443 0.726 18,242 5,710

The classifier predicts busy = hourly footfall above the sensor's p75 using hour, weekday, weekend, temperature, rain, and holiday. It trains in under a second on the challenge CSV and is retrained from an uploaded CSV through POST /train. Full definition, confusion matrix, per-sensor table, and limits: docs/small-model.md. Paper tables and figures: docs/metrics.md.

Quick start

Requires Python 3.12 and uv.

uv sync
make serve     # uses the committed artifacts; http://localhost:8000
make check     # same gate as CI: sync --frozen, ruff, compileall, pytest

To rebuild the artifacts from raw data:

make data      # needs the raw CSV; first run also fetches Open-Meteo weather
make train     # writes artifacts/model.joblib, metrics.json, forecasts.json
make small     # prints the busy-hour classifier metrics + confusion matrix
make figures   # regenerates docs/figures/*.svg from metrics.json
make test

The raw challenge CSV is never committed. The default path is /home/noel/Documents/Hackathon_2026/data/raw/schwarz-pax.csv; set PULSECAST_RAW_CSV to use another file. data/raw/ and data/cache/ are git-ignored. The committed artifacts/ (JSON + joblib + compact weather frames) let the API and Docker image run without the raw file or network.

Artifacts

File Purpose
artifacts/sensors.json 8 sensors: id, name, lat/lon, mean, p95, battery
artifacts/series.json + series/<id>.json Hourly series per sensor
artifacts/forecasts.json 48h + 7d forecasts for 4 scenarios
artifacts/metrics.json Versions, coverage, headline metrics, ablation, per-sensor, small summary, limits
artifacts/small_metrics.json Busy classifier: metrics, confusion matrix, per-sensor, thresholds
artifacts/context.json Current + 7-day weather, BW holidays
artifacts/data_audit.json Row/device checks for the raw CSV
artifacts/model.joblib Baked HistGB forecast bundle
artifacts/small_model.joblib Baked busy classifier
artifacts/weather_*.parquet Baked weather frames for retraining/inference

API

Response shapes match the Go service contract. The service is internal; Go exposes it publicly.

Method Path Notes
GET /healthz {"ok":true}
GET /readyz {"ok":true,"model":true}, 503 if no model
GET /meta Versions, coverage, metrics, features, holiday effect
GET /bootstrap Snapshot for Go: sensors, hourly series, context, meta
POST /predict {sensor,horizon,scenario,weather?,is_holiday?} → points
POST /simulate Scenario forecast for all or selected sensors
POST /train Control token; multipart CSV or JSON path; 202 job
GET /train/{job_id} Status, metrics, model_version, error

Horizons: 15m (1 point), 1h (4 points), 48h (48 points), 7d (168 points). Scenarios: base, sunny-saturday-market, rainy-saturday, weekday-event.

curl -s localhost:8000/predict \
  -H 'content-type: application/json' \
  -d '{"sensor":"hn-stadtgalerie-01","horizon":"7d","scenario":"base"}'

curl -s localhost:8000/simulate \
  -H 'content-type: application/json' \
  -d '{"scenario":"sunny-saturday-market","sensors":"all"}'

# Training requires CONTROL_TOKEN and one job at a time.
CONTROL_TOKEN=change-me make serve
curl -s -X POST localhost:8000/train \
  -H "Authorization: Bearer change-me" \
  -F model=small -F file=@new-data.csv
curl -s localhost:8000/train/<job_id> \
  -H "Authorization: Bearer change-me"

Training runs in a background thread, hot-swaps the in-memory model on success, and persists to PULSECAST_MODELS (default /tmp/pulsecast/models). A restart falls back to the baked model. model=small is the fast demo; model=forecast retrains the global HistGB (slower, still background).

End-to-end check

scripts/e2e_train.py starts a real uvicorn process on port 18000 with a control token, then checks the snapshot, every /predict combination, every /simulate scenario, auth and error paths, one-job-at-a-time, and both real training jobs (small + forecast) including hot-swap and /predict afterward:

uv run python scripts/e2e_train.py --port 18000 --report /tmp/e2e.json

The unit and contract tests are make test; make check is the CI gate.

Docs

File Contents
docs/metrics.md Paper tables/figures: protocol, headline, ablation, per sensor, uncertainty, small model, claim limits
docs/small-model.md Busy-hour target, thresholds, split, metrics, confusion matrix, per sensor, limits
docs/train-e2e.md Exact live training-job evidence (small + forecast, hot-swap, errors)
docs/figures/*.svg Ablation MAE, per-sensor MAE, small-model confusion matrix (generated)
model_card.md Model card: features, split, baseline, ablation, leakage, limitations

Docker

make docker
docker run --rm -p 8000:8000 -e CONTROL_TOKEN=change-me pulsecast-ml:local

Multi-stage python:3.12-slim, numeric non-root user 10001:10001, model and artifacts baked in, writable paths only under /tmp.

Configuration

Variable Default Use
PULSECAST_ROOT first parent with pyproject.toml Project root
PULSECAST_RAW_CSV challenge path Raw CSV location
PULSECAST_ARTIFACTS ./artifacts Baked JSON/model output
PULSECAST_CACHE ./data/cache Parquet/weather cache
PULSECAST_WEATHER PULSECAST_ARTIFACTS Baked weather frame location
PULSECAST_MODELS /tmp/pulsecast/models Retrained model persistence
PULSECAST_UPLOADS /tmp/pulsecast/uploads Multipart upload landing
CONTROL_TOKEN unset Required by /train; service returns 503 if unset

Layout

src/pulsecast/
  config.py     paths, sensors, holidays, scenarios, feature sets
  parse.py      CSV validation, exclusions, density-01 device merge
  features.py   hourly aggregation, leakage-safe lags/rollups
  weather.py    Open-Meteo archive/forecast with cache
  pipeline.py   `make data`
  model.py      HistGB fit, inference frame, bands, scenario overlay
  train.py      `make train`, ablation, artifacts
  small.py      busy classifier + flexible CSV parser
  retrain.py    runtime forecast retraining from upload
  server.py     FastAPI app and training job manager
scripts/        train_small.py, plot_metrics.py, e2e_train.py
docs/           metrics.md, small-model.md, train-e2e.md, generated figures
tests/          schema, bootstrap, health, prediction, small-training job
model_card.md   features, split, baseline, ablation, limits, leakage

Limits

  • Values are estimated devices detected by PAX counters, not a count of people.
  • Eight sensors in a ~580 m inner-city cluster; no city-wide claim.
  • Five months of multi-sensor data; no annual-seasonality claim.
  • Weather is observed in training and forecast in the demo/scenarios.
  • Holidays are context and a small feature, not a large accuracy claim.
  • No POS, sales, queue, or labor-cost data; staffing is decision support with explicit assumptions.