Files
ML-py/README.md
T
Noel Sabia 939a8db482 feat: complete ML pipeline, FastAPI service, small model, tests, image
- finish the global HistGB bundle, scenario forecasts and 28-day ablation
- export sensors/series/forecasts/metrics/context plus baked weather frames
- add FastAPI health/meta/predict/simulate and background /train jobs
- add the p75 busy classifier, retraining from CSV, and hot-swap persistence
- add pytest coverage, multi-stage non-root Dockerfile, README and model card

Holdout: weather 7d MAE 3.32 / WAPE 27.5% / R2 0.777 vs seasonal naive
MAE 4.23 / WAPE 35.0% / R2 0.656 (5,350 eval rows).
2026-11-14 05:01:16 +01:00

164 lines
6.4 KiB
Markdown

# PulseCast ML
Footfall pipeline, weather-aware 7-day forecast, scenario engine, and FastAPI
service for the PulseCast staffing copilot.
The reusable asset is the pipeline: raw 15-minute PAX counters → clean hourly
series → observed weather + Baden-Württemberg holidays → global model →
honest holdout metrics → JSON artifacts → decision-ready API.
## Results
28-day time-based holdout: 2026-10-10 23:00 UTC → 2026-11-07 23:00 UTC,
5,350 evaluation rows, 8 sensors.
| Model | MAE | WAPE | R² |
|---|---:|---:|---:|
| Seasonal naive (same local hour last week) | 4.23 | 35.0% | 0.656 |
| Day-ahead, no weather | 3.96 | 32.8% | 0.706 |
| Day-ahead, + holiday flag | 3.84 | 31.8% | 0.732 |
| Day-ahead, + temperature/rain | 3.36 | 27.8% | 0.763 |
| 7-day-safe, no weather | 4.08 | 33.8% | 0.697 |
| **7-day-safe, + temperature/rain (production)** | **3.32** | **27.5%** | **0.777** |
Weather cuts error by ~22% versus the no-weather 7-day model and ~38% versus
seasonal naive. The mission reference numbers (5.40 / 44.7% / 0.462) come from a
different quick-baseline window; these numbers are reproducible from this repo.
Small busy-hour classifier (baked for the "train new data" demo):
| Accuracy | Precision | Recall | F1 | ROC-AUC | Train rows | Test rows |
|---:|---:|---:|---:|---:|---:|---:|
| 0.708 | 0.360 | 0.574 | 0.443 | 0.726 | 18,242 | 5,710 |
The classifier predicts `busy` = hourly footfall above the sensor's p75 using
hour, weekday, weekend, temperature, rain, and holiday. It trains in under a
second on the challenge CSV and is retrained from an uploaded CSV through
`POST /train`.
## Quick start
Requirements: Python 3.12 and `uv` (the repo also works with a plain venv).
```bash
uv sync
make data # needs the raw CSV, first run also fetches Open-Meteo weather
make train # writes artifacts/model.joblib, metrics.json, forecasts.json
make test
make serve # http://localhost:8000
```
The raw challenge CSV is **never committed**. Put it at:
```text
/home/noel/Documents/Hackathon_2026/data/raw/schwarz-pax.csv
```
or point `PULSECAST_RAW_CSV` at it. `data/raw/` and `data/cache/` are git-ignored.
The generated `artifacts/` (JSON + joblib + compact weather frames) are
committed so the API and Docker image work without the raw file or network.
## Artifacts
| File | Purpose |
|---|---|
| `artifacts/sensors.json` | 8 sensors: id, name, lat/lon, mean, p95, battery |
| `artifacts/series.json` + `series/<id>.json` | Hourly series per sensor |
| `artifacts/forecasts.json` | 48h + 7d forecasts for 4 scenarios |
| `artifacts/metrics.json` | Versions, coverage, headline metrics, ablation, limits |
| `artifacts/context.json` | Current + 7-day weather, BW holidays |
| `artifacts/data_audit.json` | Row/sensor checks against `mission/data.md` |
| `artifacts/model.joblib` | Baked HistGB forecast bundle |
| `artifacts/small_model.joblib` | Baked busy classifier |
| `artifacts/weather_*.parquet` | Baked weather frames for retraining/inference |
## API
Shapes follow `CONTRACT.md`. The service is internal; Go exposes it publicly.
| Method | Path | Notes |
|---|---|---|
| `GET` | `/healthz` | `{"ok":true}` |
| `GET` | `/readyz` | `{"ok":true,"model":true}`, 503 if no model |
| `GET` | `/meta` | Versions, coverage, metrics, features, holiday effect |
| `POST` | `/predict` | `{sensor,horizon,scenario,weather?,is_holiday?}` → `points` |
| `POST` | `/simulate` | Scenario forecast for all or selected sensors |
| `POST` | `/train` | Control token; multipart CSV or JSON path; 202 job |
| `GET` | `/train/{job_id}` | Status, metrics, model_version, error |
Horizons: `15m` (1 point), `1h` (4 points), `48h` (48 points), `7d` (168
points). Scenarios: `base`, `sunny-saturday-market`, `rainy-saturday`,
`weekday-event`.
```bash
curl -s localhost:8000/predict \
-H 'content-type: application/json' \
-d '{"sensor":"hn-stadtgalerie-01","horizon":"7d","scenario":"base"}'
curl -s localhost:8000/simulate \
-H 'content-type: application/json' \
-d '{"scenario":"sunny-saturday-market","sensors":"all"}'
# Training requires CONTROL_TOKEN and one job at a time.
CONTROL_TOKEN=change-me make serve
curl -s -X POST localhost:8000/train \
-H "Authorization: Bearer change-me" \
-F model=small -F file=@new-data.csv
curl -s localhost:8000/train/<job_id> \
-H "Authorization: Bearer change-me"
```
Training runs in a background thread, hot-swaps the in-memory model on success,
and persists to `PULSECAST_MODELS` (default `/tmp/pulsecast/models`). A restart
falls back to the baked model. `model=small` is the fast demo; `model=forecast`
retrains the global HistGB (slower, still background).
## Docker
```bash
make docker
docker run --rm -p 8000:8000 -e CONTROL_TOKEN=change-me pulsecast-ml:local
```
Multi-stage `python:3.12-slim`, non-root user, model and artifacts baked in,
writable paths only under `/tmp`.
## Configuration
| Variable | Default | Use |
|---|---|---|
| `PULSECAST_RAW_CSV` | challenge path | Raw CSV location |
| `PULSECAST_ARTIFACTS` | `./artifacts` | Baked JSON/model output |
| `PULSECAST_CACHE` | `./data/cache` | Parquet/weather cache |
| `PULSECAST_MODELS` | `/tmp/pulsecast/models` | Retrained model persistence |
| `PULSECAST_UPLOADS` | `/tmp/pulsecast/uploads` | Multipart upload landing |
| `CONTROL_TOKEN` | unset | Required by `/train`; service returns 503 if unset |
## Layout
```text
src/pulsecast/
config.py paths, sensors, holidays, scenarios, feature sets
parse.py CSV validation, exclusions, density-01 device merge
features.py hourly aggregation, leakage-safe lags/rollups
weather.py Open-Meteo archive/forecast with cache
pipeline.py `make data`
model.py HistGB fit, inference frame, bands, scenario overlay
train.py `make train`, ablation, artifacts
small.py busy classifier + flexible CSV parser
retrain.py runtime forecast retraining from upload
server.py FastAPI app and training job manager
tests/ schema, health, prediction, small-training job
model_card.md features, split, baseline, ablation, limits, leakage
```
## Limits
- Values are estimated devices detected by PAX counters, not a count of people.
- Eight sensors in a ~580 m inner-city cluster; no city-wide claim.
- Five months of multi-sensor data; no annual-seasonality claim.
- Weather is observed in training and forecast in the demo/scenarios.
- Holidays are context and a small feature, not a large accuracy claim.
- No POS, sales, queue, or labor-cost data; staffing is decision support with
explicit assumptions.