# PulseCast ML Footfall pipeline, weather-aware 7-day forecast, scenario engine, and FastAPI service for the PulseCast staffing copilot. It turns raw 15-minute PAX counter rows into hourly series, joins weather and Baden-Württemberg holidays, trains one global model, evaluates it on a 28-day time-based holdout, and serves the resulting JSON artifacts. ## Results 28-day time-based holdout: 2026-10-10 23:00 UTC → 2026-11-07 23:00 UTC, 5,350 evaluation rows, 8 sensors. | Model | MAE | WAPE | R² | |---|---:|---:|---:| | Seasonal naive (exact 168-row lag) | 5.40 | 44.7% | 0.462 | | Day-ahead, no weather | 3.96 | 32.8% | 0.706 | | Day-ahead, + holiday flag | 3.84 | 31.8% | 0.732 | | Day-ahead, + temperature/rain | 3.36 | 27.8% | 0.763 | | 7-day-safe, no weather | 4.08 | 33.8% | 0.697 | | **7-day-safe, + temperature/rain (production)** | **3.32** | **27.5%** | **0.777** | The seasonal-naive row reproduces the mission reference exactly (5.40 / 44.7% / 0.462), and the day-ahead weather row matches 3.36 / 27.8%. Weather cuts error by ~19% versus the no-weather 7-day model and ~38% versus seasonal naive. The production 7-day model edges the reference day-ahead weather row (3.32 vs 3.36 MAE). The two use different features; `lag_week` uses the previous *observed* same weekday/hour across sensor gaps, while the seasonal-naive baseline uses the exact 168-row lag. Small busy-hour classifier (baked for the "train new data" demo): | Accuracy | Precision | Recall | F1 | ROC-AUC | Train rows | Test rows | |---:|---:|---:|---:|---:|---:|---:| | 0.708 | 0.360 | 0.574 | 0.443 | 0.726 | 18,242 | 5,710 | The classifier predicts `busy` = hourly footfall above the sensor's p75 using hour, weekday, weekend, temperature, rain, and holiday. It trains in under a second on the challenge CSV and is retrained from an uploaded CSV through `POST /train`. Full definition, confusion matrix, per-sensor table, and limits: [`docs/small-model.md`](docs/small-model.md). Paper tables and figures: [`docs/metrics.md`](docs/metrics.md). ## Quick start Requires Python 3.12 and `uv`. ```bash uv sync make serve # uses the committed artifacts; http://localhost:8000 make check # same gate as CI: sync --frozen, ruff, compileall, pytest ``` To rebuild the artifacts from raw data: ```bash make data # needs the raw CSV; first run also fetches Open-Meteo weather make train # writes artifacts/model.joblib, metrics.json, forecasts.json make small # prints the busy-hour classifier metrics + confusion matrix make figures # regenerates docs/figures/*.svg from metrics.json make test ``` The raw challenge CSV is **never committed**. The default path is `/home/noel/Documents/Hackathon_2026/data/raw/schwarz-pax.csv`; set `PULSECAST_RAW_CSV` to use another file. `data/raw/` and `data/cache/` are git-ignored. The committed `artifacts/` (JSON + joblib + compact weather frames) let the API and Docker image run without the raw file or network. ## Artifacts | File | Purpose | |---|---| | `artifacts/sensors.json` | 8 sensors: id, name, lat/lon, mean, p95, battery | | `artifacts/series.json` + `series/.json` | Hourly series per sensor | | `artifacts/forecasts.json` | 48h + 7d forecasts for 4 scenarios | | `artifacts/metrics.json` | Versions, coverage, headline metrics, ablation, per-sensor, small summary, limits | | `artifacts/small_metrics.json` | Busy classifier: metrics, confusion matrix, per-sensor, thresholds | | `artifacts/context.json` | Current + 7-day weather, BW holidays | | `artifacts/data_audit.json` | Row/device checks for the raw CSV | | `artifacts/model.joblib` | Baked HistGB forecast bundle | | `artifacts/small_model.joblib` | Baked busy classifier | | `artifacts/weather_*.parquet` | Baked weather frames for retraining/inference | ## API Response shapes match the Go service contract. The service is internal; Go exposes it publicly. | Method | Path | Notes | |---|---|---| | `GET` | `/healthz` | `{"ok":true}` | | `GET` | `/readyz` | `{"ok":true,"model":true}`, 503 if no model | | `GET` | `/meta` | Versions, coverage, metrics, features, holiday effect | | `GET` | `/bootstrap` | Snapshot for Go: sensors, hourly series, context, meta | | `POST` | `/predict` | `{sensor,horizon,scenario,weather?,is_holiday?}` → `points` | | `POST` | `/simulate` | Scenario forecast for all or selected sensors | | `POST` | `/train` | Control token; multipart CSV or JSON path; 202 job | | `GET` | `/train/{job_id}` | Status, metrics, model_version, error | Horizons: `15m` (1 point), `1h` (4 points), `48h` (48 points), `7d` (168 points). Scenarios: `base`, `sunny-saturday-market`, `rainy-saturday`, `weekday-event`. ```bash curl -s localhost:8000/predict \ -H 'content-type: application/json' \ -d '{"sensor":"hn-stadtgalerie-01","horizon":"7d","scenario":"base"}' curl -s localhost:8000/simulate \ -H 'content-type: application/json' \ -d '{"scenario":"sunny-saturday-market","sensors":"all"}' # Training requires CONTROL_TOKEN and one job at a time. CONTROL_TOKEN=change-me make serve curl -s -X POST localhost:8000/train \ -H "Authorization: Bearer change-me" \ -F model=small -F file=@new-data.csv curl -s localhost:8000/train/ \ -H "Authorization: Bearer change-me" ``` Training runs in a background thread, hot-swaps the in-memory model on success, and persists to `PULSECAST_MODELS` (default `/tmp/pulsecast/models`). A restart falls back to the baked model. `model=small` is the fast demo; `model=forecast` retrains the global HistGB (slower, still background). ## End-to-end check `scripts/e2e_train.py` starts a real uvicorn process on port 18000 with a control token, then checks the snapshot, every `/predict` combination, every `/simulate` scenario, auth and error paths, one-job-at-a-time, and both real training jobs (small + forecast) including hot-swap and `/predict` afterward: ```bash uv run python scripts/e2e_train.py --port 18000 --report /tmp/e2e.json ``` The unit and contract tests are `make test`; `make check` is the CI gate. ## Docs | File | Contents | |---|---| | `docs/metrics.md` | Paper tables/figures: protocol, headline, ablation, per sensor, uncertainty, small model, claim limits | | `docs/small-model.md` | Busy-hour target, thresholds, split, metrics, confusion matrix, per sensor, limits | | `docs/train-e2e.md` | Exact live training-job evidence (small + forecast, hot-swap, errors) | | `docs/figures/*.svg` | Ablation MAE, per-sensor MAE, small-model confusion matrix (generated) | | `model_card.md` | Model card: features, split, baseline, ablation, leakage, limitations | ## Docker ```bash make docker docker run --rm -p 8000:8000 -e CONTROL_TOKEN=change-me pulsecast-ml:local ``` Multi-stage `python:3.12-slim`, numeric non-root user `10001:10001`, model and artifacts baked in, writable paths only under `/tmp`. ## Configuration | Variable | Default | Use | |---|---|---| | `PULSECAST_ROOT` | first parent with `pyproject.toml` | Project root | | `PULSECAST_RAW_CSV` | challenge path | Raw CSV location | | `PULSECAST_ARTIFACTS` | `./artifacts` | Baked JSON/model output | | `PULSECAST_CACHE` | `./data/cache` | Parquet/weather cache | | `PULSECAST_WEATHER` | `PULSECAST_ARTIFACTS` | Baked weather frame location | | `PULSECAST_MODELS` | `/tmp/pulsecast/models` | Retrained model persistence | | `PULSECAST_UPLOADS` | `/tmp/pulsecast/uploads` | Multipart upload landing | | `CONTROL_TOKEN` | unset | Required by `/train`; service returns 503 if unset | ## Layout ```text src/pulsecast/ config.py paths, sensors, holidays, scenarios, feature sets parse.py CSV validation, exclusions, density-01 device merge features.py hourly aggregation, leakage-safe lags/rollups weather.py Open-Meteo archive/forecast with cache pipeline.py `make data` model.py HistGB fit, inference frame, bands, scenario overlay train.py `make train`, ablation, artifacts small.py busy classifier + flexible CSV parser retrain.py runtime forecast retraining from upload server.py FastAPI app and training job manager scripts/ train_small.py, plot_metrics.py, e2e_train.py docs/ metrics.md, small-model.md, train-e2e.md, generated figures tests/ schema, bootstrap, health, prediction, small-training job model_card.md features, split, baseline, ablation, limits, leakage ``` ## Limits - Values are estimated devices detected by PAX counters, not a count of people. - Eight sensors in a ~580 m inner-city cluster; no city-wide claim. - Five months of multi-sensor data; no annual-seasonality claim. - Weather is observed in training and forecast in the demo/scenarios. - Holidays are context and a small feature, not a large accuracy claim. - No POS, sales, queue, or labor-cost data; staffing is decision support with explicit assumptions.