The small-model page said make small reproduced every number, but the threshold table and several derived values only appear in the JSON payload. Say what the commands actually print, fix the stale weather-cache docstring, and lead the e2e evidence page with what it checks.
PulseCast ML
Footfall pipeline, weather-aware 7-day forecast, scenario engine, and FastAPI service for the PulseCast staffing copilot. It turns raw 15-minute PAX counter rows into hourly series, joins weather and Baden-Württemberg holidays, trains one global model, evaluates it on a 28-day time-based holdout, and serves the resulting JSON artifacts.
Results
28-day time-based holdout: 2026-10-10 23:00 UTC → 2026-11-07 23:00 UTC, 5,350 evaluation rows, 8 sensors.
| Model | MAE | WAPE | R² |
|---|---|---|---|
| Seasonal naive (exact 168-row lag) | 5.40 | 44.7% | 0.462 |
| Day-ahead, no weather | 3.96 | 32.8% | 0.706 |
| Day-ahead, + holiday flag | 3.84 | 31.8% | 0.732 |
| Day-ahead, + temperature/rain | 3.36 | 27.8% | 0.763 |
| 7-day-safe, no weather | 4.08 | 33.8% | 0.697 |
| 7-day-safe, + temperature/rain (production) | 3.32 | 27.5% | 0.777 |
The seasonal-naive row reproduces the mission reference exactly
(5.40 / 44.7% / 0.462), and the day-ahead weather row matches 3.36 / 27.8%.
Weather cuts error by ~19% versus the no-weather 7-day model and ~38% versus
seasonal naive. The production 7-day model edges the reference day-ahead
weather row (3.32 vs 3.36 MAE). The two use different features; lag_week uses
the previous observed same weekday/hour across sensor gaps, while the
seasonal-naive baseline uses the exact 168-row lag.
Small busy-hour classifier (baked for the "train new data" demo):
| Accuracy | Precision | Recall | F1 | ROC-AUC | Train rows | Test rows |
|---|---|---|---|---|---|---|
| 0.708 | 0.360 | 0.574 | 0.443 | 0.726 | 18,242 | 5,710 |
The classifier predicts busy = hourly footfall above the sensor's p75 using
hour, weekday, weekend, temperature, rain, and holiday. It trains in under a
second on the challenge CSV and is retrained from an uploaded CSV through
POST /train. Full definition, confusion matrix, per-sensor table, and limits:
docs/small-model.md. Paper tables and figures:
docs/metrics.md.
Quick start
Requires Python 3.12 and uv.
uv sync
make serve # uses the committed artifacts; http://localhost:8000
make check # same gate as CI: sync --frozen, ruff, compileall, pytest
To rebuild the artifacts from raw data:
make data # needs the raw CSV; first run also fetches Open-Meteo weather
make train # writes artifacts/model.joblib, metrics.json, forecasts.json
make small # prints the busy-hour classifier metrics + confusion matrix
make figures # regenerates docs/figures/*.svg from metrics.json
make test
The raw challenge CSV is never committed. The default path is
/home/noel/Documents/Hackathon_2026/data/raw/schwarz-pax.csv; set
PULSECAST_RAW_CSV to use another file. data/raw/ and data/cache/ are
git-ignored. The committed artifacts/ (JSON + joblib + compact weather frames)
let the API and Docker image run without the raw file or network.
Artifacts
| File | Purpose |
|---|---|
artifacts/sensors.json |
8 sensors: id, name, lat/lon, mean, p95, battery |
artifacts/series.json + series/<id>.json |
Hourly series per sensor |
artifacts/forecasts.json |
48h + 7d forecasts for 4 scenarios |
artifacts/metrics.json |
Versions, coverage, headline metrics, ablation, per-sensor, small summary, limits |
artifacts/small_metrics.json |
Busy classifier: metrics, confusion matrix, per-sensor, thresholds |
artifacts/context.json |
Current + 7-day weather, BW holidays |
artifacts/data_audit.json |
Row/device checks for the raw CSV |
artifacts/model.joblib |
Baked HistGB forecast bundle |
artifacts/small_model.joblib |
Baked busy classifier |
artifacts/weather_*.parquet |
Baked weather frames for retraining/inference |
API
Response shapes match the Go service contract. The service is internal; Go exposes it publicly.
| Method | Path | Notes |
|---|---|---|
GET |
/healthz |
{"ok":true} |
GET |
/readyz |
{"ok":true,"model":true}, 503 if no model |
GET |
/meta |
Versions, coverage, metrics, features, holiday effect |
GET |
/bootstrap |
Snapshot for Go: sensors, hourly series, context, meta |
POST |
/predict |
{sensor,horizon,scenario,weather?,is_holiday?} → points |
POST |
/simulate |
Scenario forecast for all or selected sensors |
POST |
/train |
Control token; multipart CSV or JSON path; 202 job |
GET |
/train/{job_id} |
Status, metrics, model_version, error |
Horizons: 15m (1 point), 1h (4 points), 48h (48 points), 7d (168
points). Scenarios: base, sunny-saturday-market, rainy-saturday,
weekday-event.
curl -s localhost:8000/predict \
-H 'content-type: application/json' \
-d '{"sensor":"hn-stadtgalerie-01","horizon":"7d","scenario":"base"}'
curl -s localhost:8000/simulate \
-H 'content-type: application/json' \
-d '{"scenario":"sunny-saturday-market","sensors":"all"}'
# Training requires CONTROL_TOKEN and one job at a time.
CONTROL_TOKEN=change-me make serve
curl -s -X POST localhost:8000/train \
-H "Authorization: Bearer change-me" \
-F model=small -F file=@new-data.csv
curl -s localhost:8000/train/<job_id> \
-H "Authorization: Bearer change-me"
Training runs in a background thread, hot-swaps the in-memory model on success,
and persists to PULSECAST_MODELS (default /tmp/pulsecast/models). A restart
falls back to the baked model. model=small is the fast demo; model=forecast
retrains the global HistGB (slower, still background).
End-to-end check
scripts/e2e_train.py starts a real uvicorn process on port 18000 with a
control token, then checks the snapshot, every /predict combination, every
/simulate scenario, auth and error paths, one-job-at-a-time, and both real
training jobs (small + forecast) including hot-swap and /predict afterward:
uv run python scripts/e2e_train.py --port 18000 --report /tmp/e2e.json
The unit and contract tests are make test; make check is the CI gate.
Docs
| File | Contents |
|---|---|
docs/metrics.md |
Paper tables/figures: protocol, headline, ablation, per sensor, uncertainty, small model, claim limits |
docs/small-model.md |
Busy-hour target, thresholds, split, metrics, confusion matrix, per sensor, limits |
docs/train-e2e.md |
Exact live training-job evidence (small + forecast, hot-swap, errors) |
docs/figures/*.svg |
Ablation MAE, per-sensor MAE, small-model confusion matrix (generated) |
model_card.md |
Model card: features, split, baseline, ablation, leakage, limitations |
Docker
make docker
docker run --rm -p 8000:8000 -e CONTROL_TOKEN=change-me pulsecast-ml:local
Multi-stage python:3.12-slim, numeric non-root user 10001:10001, model and
artifacts baked in, writable paths only under /tmp.
Configuration
| Variable | Default | Use |
|---|---|---|
PULSECAST_ROOT |
first parent with pyproject.toml |
Project root |
PULSECAST_RAW_CSV |
challenge path | Raw CSV location |
PULSECAST_ARTIFACTS |
./artifacts |
Baked JSON/model output |
PULSECAST_CACHE |
./data/cache |
Parquet/weather cache |
PULSECAST_WEATHER |
PULSECAST_ARTIFACTS |
Baked weather frame location |
PULSECAST_MODELS |
/tmp/pulsecast/models |
Retrained model persistence |
PULSECAST_UPLOADS |
/tmp/pulsecast/uploads |
Multipart upload landing |
CONTROL_TOKEN |
unset | Required by /train; service returns 503 if unset |
Layout
src/pulsecast/
config.py paths, sensors, holidays, scenarios, feature sets
parse.py CSV validation, exclusions, density-01 device merge
features.py hourly aggregation, leakage-safe lags/rollups
weather.py Open-Meteo archive/forecast with cache
pipeline.py `make data`
model.py HistGB fit, inference frame, bands, scenario overlay
train.py `make train`, ablation, artifacts
small.py busy classifier + flexible CSV parser
retrain.py runtime forecast retraining from upload
server.py FastAPI app and training job manager
scripts/ train_small.py, plot_metrics.py, e2e_train.py
docs/ metrics.md, small-model.md, train-e2e.md, generated figures
tests/ schema, bootstrap, health, prediction, small-training job
model_card.md features, split, baseline, ablation, limits, leakage
Limits
- Values are estimated devices detected by PAX counters, not a count of people.
- Eight sensors in a ~580 m inner-city cluster; no city-wide claim.
- Five months of multi-sensor data; no annual-seasonality claim.
- Weather is observed in training and forecast in the demo/scenarios.
- Holidays are context and a small feature, not a large accuracy claim.
- No POS, sales, queue, or labor-cost data; staffing is decision support with explicit assumptions.