Build artifacts/prezero_priority.json from the anonymized glass fill-level export (11,701 rows, 449 containers, 1,338 location/material series). The contract tier rule yields high 88, medium 248, low 113 with 1,192 overfull readings.
Add /prezero/summary, /prezero/priority, and /prezero/containers/{id} to the internal FastAPI service, with startup validation, filtering/pagination, recent reading history, and 503 isolation when the artifact is missing. Raw CSV stays gitignored; the derived 1.1 MB artifact is the source of truth. This is a priority proof, not a route: the dataset has no coordinates.
260 lines
11 KiB
Markdown
260 lines
11 KiB
Markdown
# PulseCast ML
|
|
|
|
Footfall pipeline, weather-aware 7-day forecast, scenario engine, and FastAPI
|
|
service for the PulseCast staffing copilot. It turns raw 15-minute PAX counter
|
|
rows into hourly series, joins weather and Baden-Württemberg holidays, trains
|
|
one global model, evaluates it on a 28-day time-based holdout, and serves the
|
|
resulting JSON artifacts.
|
|
|
|
## Results
|
|
|
|
28-day time-based holdout: 2026-10-10 23:00 UTC → 2026-11-07 23:00 UTC,
|
|
5,350 evaluation rows, 8 sensors.
|
|
|
|
| Model | MAE | WAPE | R² |
|
|
|---|---:|---:|---:|
|
|
| Seasonal naive (exact 168-row lag) | 5.40 | 44.7% | 0.462 |
|
|
| Day-ahead, no weather | 3.96 | 32.8% | 0.706 |
|
|
| Day-ahead, + holiday flag | 3.84 | 31.8% | 0.732 |
|
|
| Day-ahead, + temperature/rain | 3.36 | 27.8% | 0.763 |
|
|
| 7-day-safe, no weather | 4.08 | 33.8% | 0.697 |
|
|
| **7-day-safe, + temperature/rain (production)** | **3.32** | **27.5%** | **0.777** |
|
|
|
|
The seasonal-naive row reproduces the mission reference exactly
|
|
(5.40 / 44.7% / 0.462), and the day-ahead weather row matches 3.36 / 27.8%.
|
|
Weather cuts error by ~19% versus the no-weather 7-day model and ~38% versus
|
|
seasonal naive. The production 7-day model edges the reference day-ahead
|
|
weather row (3.32 vs 3.36 MAE). The two use different features; `lag_week` uses
|
|
the previous *observed* same weekday/hour across sensor gaps, while the
|
|
seasonal-naive baseline uses the exact 168-row lag.
|
|
|
|
Small busy-hour classifier (baked for the "train new data" demo):
|
|
|
|
| Accuracy | Precision | Recall | F1 | ROC-AUC | Train rows | Test rows |
|
|
|---:|---:|---:|---:|---:|---:|---:|
|
|
| 0.708 | 0.360 | 0.574 | 0.443 | 0.726 | 18,242 | 5,710 |
|
|
|
|
The classifier predicts `busy` = hourly footfall above the sensor's p75 using
|
|
hour, weekday, weekend, temperature, rain, and holiday. It trains in under a
|
|
second on the challenge CSV and is retrained from an uploaded CSV through
|
|
`POST /train`. Full definition, confusion matrix, per-sensor table, and limits:
|
|
[`docs/small-model.md`](docs/small-model.md). Paper tables and figures:
|
|
[`docs/metrics.md`](docs/metrics.md).
|
|
|
|
## Quick start
|
|
|
|
Requires Python 3.12 and `uv`.
|
|
|
|
```bash
|
|
uv sync
|
|
make serve # uses the committed artifacts; http://localhost:8000
|
|
make check # same gate as CI: sync --frozen, ruff, compileall, pytest
|
|
```
|
|
|
|
To rebuild the artifacts from raw data:
|
|
|
|
```bash
|
|
make data # needs the raw CSV; first run also fetches Open-Meteo weather
|
|
make train # writes artifacts/model.joblib, metrics.json, forecasts.json
|
|
make small # prints the busy-hour classifier metrics + confusion matrix
|
|
make prezero # rebuilds artifacts/prezero_priority.json from the fill-level CSV
|
|
make figures # regenerates docs/figures/*.svg from metrics.json
|
|
make test
|
|
```
|
|
|
|
The raw challenge CSV is **never committed**. The default path is
|
|
`/home/noel/Documents/Hackathon_2026/data/raw/schwarz-pax.csv`; set
|
|
`PULSECAST_RAW_CSV` to use another file. `data/raw/` and `data/cache/` are
|
|
git-ignored. The committed `artifacts/` (JSON + joblib + compact weather frames)
|
|
let the API and Docker image run without the raw file or network.
|
|
|
|
The PreZero fill-level export is also git-ignored. For `make prezero`, put
|
|
`glass_fill_levels_anonymized_f.csv` at `data/raw/` or point
|
|
`PULSECAST_PREZERO_CSV` at it. Only the derived
|
|
`artifacts/prezero_priority.json` is committed.
|
|
|
|
## Artifacts
|
|
|
|
| File | Purpose |
|
|
|---|---|
|
|
| `artifacts/sensors.json` | 8 sensors: id, name, lat/lon, mean, p95, battery |
|
|
| `artifacts/series.json` + `series/<id>.json` | Hourly series per sensor |
|
|
| `artifacts/forecasts.json` | 48h + 7d forecasts for 4 scenarios |
|
|
| `artifacts/metrics.json` | Versions, coverage, headline metrics, ablation, per-sensor, small summary, limits |
|
|
| `artifacts/small_metrics.json` | Busy classifier: metrics, confusion matrix, per-sensor, thresholds |
|
|
| `artifacts/context.json` | Current + 7-day weather, BW holidays |
|
|
| `artifacts/data_audit.json` | Row/device checks for the raw CSV |
|
|
| `artifacts/prezero_priority.json` | 449-container priority snapshot + last 10 readings each |
|
|
| `artifacts/model.joblib` | Baked HistGB forecast bundle |
|
|
| `artifacts/small_model.joblib` | Baked busy classifier |
|
|
| `artifacts/weather_*.parquet` | Baked weather frames for retraining/inference |
|
|
|
|
## API
|
|
|
|
Response shapes match the Go service contract. The service is internal; Go
|
|
exposes it publicly.
|
|
|
|
| Method | Path | Notes |
|
|
|---|---|---|
|
|
| `GET` | `/healthz` | `{"ok":true}` |
|
|
| `GET` | `/readyz` | `{"ok":true,"model":true}`, 503 if no model |
|
|
| `GET` | `/meta` | Versions, coverage, metrics, features, holiday effect |
|
|
| `GET` | `/bootstrap` | Snapshot for Go: sensors, hourly series, context, meta |
|
|
| `GET` | `/prezero/summary` | Priority counts and material totals |
|
|
| `GET` | `/prezero/priority` | Ranked containers; `priority`, `limit`, `offset` |
|
|
| `GET` | `/prezero/containers/{id}` | One container + its 10 newest readings |
|
|
| `POST` | `/predict` | `{sensor,horizon,scenario,weather?,is_holiday?}` → `points` |
|
|
| `POST` | `/simulate` | Scenario forecast for all or selected sensors |
|
|
| `POST` | `/train` | Control token; multipart CSV or JSON path; 202 job |
|
|
| `GET` | `/train/{job_id}` | Status, metrics, model_version, error |
|
|
|
|
Horizons: `15m` (1 point), `1h` (4 points), `48h` (48 points), `7d` (168
|
|
points). Scenarios: `base`, `sunny-saturday-market`, `rainy-saturday`,
|
|
`weekday-event`.
|
|
|
|
```bash
|
|
curl -s localhost:8000/predict \
|
|
-H 'content-type: application/json' \
|
|
-d '{"sensor":"hn-stadtgalerie-01","horizon":"7d","scenario":"base"}'
|
|
|
|
curl -s localhost:8000/simulate \
|
|
-H 'content-type: application/json' \
|
|
-d '{"scenario":"sunny-saturday-market","sensors":"all"}'
|
|
|
|
# Training requires CONTROL_TOKEN and one job at a time.
|
|
CONTROL_TOKEN=change-me make serve
|
|
curl -s -X POST localhost:8000/train \
|
|
-H "Authorization: Bearer change-me" \
|
|
-F model=small -F file=@new-data.csv
|
|
curl -s localhost:8000/train/<job_id> \
|
|
-H "Authorization: Bearer change-me"
|
|
```
|
|
|
|
Training runs in a background thread, hot-swaps the in-memory model on success,
|
|
and persists to `PULSECAST_MODELS` (default `/tmp/pulsecast/models`). A restart
|
|
falls back to the baked model. `model=small` is the fast demo; `model=forecast`
|
|
retrains the global HistGB (slower, still background).
|
|
|
|
## PreZero container priority
|
|
|
|
`artifacts/prezero_priority.json` ranks the 449 anonymous container locations in
|
|
the 11,701-row glass fill-level export (2025-10-17 → 2026-09-30, 1,338
|
|
location/material series). The rule uses only reported states:
|
|
|
|
- **high:** `overfull_count >= 5` or (`overfull_count >= 2` and
|
|
`overfull_rate >= 0.25`)
|
|
- **medium:** `overfull_count >= 1`, not high
|
|
- **low:** `overfull_count == 0`
|
|
|
|
The generated snapshot has **high 88, medium 248, low 113**, with 1,192 overfull
|
|
readings. Sorted by tier, overfull count/rate, staleness (`reference_date` is
|
|
the dataset max date, 2026-09-30, not today), then location id. The artifact
|
|
also holds each container's 10 newest readings for the history route.
|
|
|
|
**The dataset has no coordinates.** This is a service-priority proof, not a
|
|
route; there is no capacity or depot data, and `overfull` is a reported state,
|
|
not a measured volume. The committed artifact is the API's source of truth. If
|
|
it is missing, only `/prezero/*` returns 503; the forecast/training routes keep
|
|
working.
|
|
|
|
Rebuild from the raw, git-ignored export:
|
|
|
|
```bash
|
|
cp glass_fill_levels_anonymized_f.csv data/raw/ # or set PULSECAST_PREZERO_CSV
|
|
make prezero
|
|
# equivalently:
|
|
uv run python scripts/build_prezero.py \
|
|
--csv data/raw/glass_fill_levels_anonymized_f.csv \
|
|
--out artifacts/prezero_priority.json
|
|
```
|
|
|
|
```bash
|
|
curl -s localhost:8000/prezero/summary
|
|
curl -s 'localhost:8000/prezero/priority?priority=high&limit=3'
|
|
curl -s localhost:8000/prezero/containers/<location_id>
|
|
```
|
|
|
|
Detail, counts, and limitations: [`docs/prezero.md`](docs/prezero.md).
|
|
|
|
## End-to-end check
|
|
|
|
`scripts/e2e_train.py` starts a real uvicorn process on port 18000 with a
|
|
control token, then checks the snapshot, every `/predict` combination, every
|
|
`/simulate` scenario, auth and error paths, one-job-at-a-time, and both real
|
|
training jobs (small + forecast) including hot-swap and `/predict` afterward:
|
|
|
|
```bash
|
|
uv run python scripts/e2e_train.py --port 18000 --report /tmp/e2e.json
|
|
```
|
|
|
|
The unit and contract tests are `make test`; `make check` is the CI gate.
|
|
|
|
## Docs
|
|
|
|
| File | Contents |
|
|
|---|---|
|
|
| `docs/metrics.md` | Paper tables/figures: protocol, headline, ablation, per sensor, uncertainty, small model, claim limits |
|
|
| `docs/small-model.md` | Busy-hour target, thresholds, split, metrics, confusion matrix, per sensor, limits |
|
|
| `docs/train-e2e.md` | Exact live training-job evidence (small + forecast, hot-swap, errors) |
|
|
| `docs/prezero.md` | Container-priority rule, counts, artifact/API shape, no-coordinates limits |
|
|
| `docs/figures/*.svg` | Ablation MAE, per-sensor MAE, small-model confusion matrix (generated) |
|
|
| `model_card.md` | Model card: features, split, baseline, ablation, leakage, limitations |
|
|
|
|
## Docker
|
|
|
|
```bash
|
|
make docker
|
|
docker run --rm -p 8000:8000 -e CONTROL_TOKEN=change-me pulsecast-ml:local
|
|
```
|
|
|
|
Multi-stage `python:3.12-slim`, numeric non-root user `10001:10001`, model and
|
|
artifacts baked in, writable paths only under `/tmp`.
|
|
|
|
## Configuration
|
|
|
|
| Variable | Default | Use |
|
|
|---|---|---|
|
|
| `PULSECAST_ROOT` | first parent with `pyproject.toml` | Project root |
|
|
| `PULSECAST_RAW_CSV` | challenge path | Raw CSV location |
|
|
| `PULSECAST_ARTIFACTS` | `./artifacts` | Baked JSON/model output |
|
|
| `PULSECAST_CACHE` | `./data/cache` | Parquet/weather cache |
|
|
| `PULSECAST_WEATHER` | `PULSECAST_ARTIFACTS` | Baked weather frame location |
|
|
| `PULSECAST_PREZERO_ARTIFACT` | `./artifacts/prezero_priority.json` | PreZero priority snapshot |
|
|
| `PULSECAST_PREZERO_CSV` | `./data/raw/glass_fill_levels_anonymized_f.csv` | Raw fill-level export for `make prezero` |
|
|
| `PULSECAST_MODELS` | `/tmp/pulsecast/models` | Retrained model persistence |
|
|
| `PULSECAST_UPLOADS` | `/tmp/pulsecast/uploads` | Multipart upload landing |
|
|
| `CONTROL_TOKEN` | unset | Required by `/train`; service returns 503 if unset |
|
|
|
|
## Layout
|
|
|
|
```text
|
|
src/pulsecast/
|
|
config.py paths, sensors, holidays, scenarios, feature sets
|
|
parse.py CSV validation, exclusions, density-01 device merge
|
|
features.py hourly aggregation, leakage-safe lags/rollups
|
|
weather.py Open-Meteo archive/forecast with cache
|
|
pipeline.py `make data`
|
|
model.py HistGB fit, inference frame, bands, scenario overlay
|
|
train.py `make train`, ablation, artifacts
|
|
small.py busy classifier + flexible CSV parser
|
|
retrain.py runtime forecast retraining from upload
|
|
prezero.py load/validate/filter/page/lookup the priority snapshot
|
|
server.py FastAPI app and training job manager
|
|
scripts/ train_small.py, plot_metrics.py, build_prezero.py, e2e_train.py
|
|
docs/ metrics.md, small-model.md, train-e2e.md, prezero.md, figures
|
|
tests/ schema, bootstrap, health, prediction, prezero, small-training job
|
|
model_card.md features, split, baseline, ablation, limits, leakage
|
|
```
|
|
|
|
## Limits
|
|
|
|
- Values are estimated devices detected by PAX counters, not a count of people.
|
|
- Eight sensors in a ~580 m inner-city cluster; no city-wide claim.
|
|
- Five months of multi-sensor data; no annual-seasonality claim.
|
|
- Weather is observed in training and forecast in the demo/scenarios.
|
|
- Holidays are context and a small feature, not a large accuracy claim.
|
|
- No POS, sales, queue, or labor-cost data; staffing is decision support with
|
|
explicit assumptions.
|
|
- PreZero has no coordinates, capacity, or depot data: it ranks collection
|
|
priority, it does not produce or optimize routes.
|