Noel Sabia ac0d8984bb
ci / check (push) Successful in 28s
ci / check (pull_request) Successful in 30s
docs: correct reproduction and cache claims
The small-model page said make small reproduced every number, but the
threshold table and several derived values only appear in the JSON payload.
Say what the commands actually print, fix the stale weather-cache docstring,
and lead the e2e evidence page with what it checks.
2026-11-14 09:12:29 +01:00
2026-11-14 04:16:22 +01:00
2026-11-14 04:16:22 +01:00

PulseCast ML

Footfall pipeline, weather-aware 7-day forecast, scenario engine, and FastAPI service for the PulseCast staffing copilot. It turns raw 15-minute PAX counter rows into hourly series, joins weather and Baden-Württemberg holidays, trains one global model, evaluates it on a 28-day time-based holdout, and serves the resulting JSON artifacts.

Results

28-day time-based holdout: 2026-10-10 23:00 UTC → 2026-11-07 23:00 UTC, 5,350 evaluation rows, 8 sensors.

Model MAE WAPE R²
Seasonal naive (exact 168-row lag) 5.40 44.7% 0.462
Day-ahead, no weather 3.96 32.8% 0.706
Day-ahead, + holiday flag 3.84 31.8% 0.732
Day-ahead, + temperature/rain 3.36 27.8% 0.763
7-day-safe, no weather 4.08 33.8% 0.697
7-day-safe, + temperature/rain (production) 3.32 27.5% 0.777

The seasonal-naive row reproduces the mission reference exactly (5.40 / 44.7% / 0.462), and the day-ahead weather row matches 3.36 / 27.8%. Weather cuts error by ~19% versus the no-weather 7-day model and ~38% versus seasonal naive. The production 7-day model edges the reference day-ahead weather row (3.32 vs 3.36 MAE). The two use different features; lag_week uses the previous observed same weekday/hour across sensor gaps, while the seasonal-naive baseline uses the exact 168-row lag.

Small busy-hour classifier (baked for the "train new data" demo):

Accuracy Precision Recall F1 ROC-AUC Train rows Test rows
0.708 0.360 0.574 0.443 0.726 18,242 5,710

The classifier predicts busy = hourly footfall above the sensor's p75 using hour, weekday, weekend, temperature, rain, and holiday. It trains in under a second on the challenge CSV and is retrained from an uploaded CSV through POST /train. Full definition, confusion matrix, per-sensor table, and limits: docs/small-model.md. Paper tables and figures: docs/metrics.md.

Quick start

Requires Python 3.12 and uv.

uv sync
make serve     # uses the committed artifacts; http://localhost:8000
make check     # same gate as CI: sync --frozen, ruff, compileall, pytest

To rebuild the artifacts from raw data:

make data      # needs the raw CSV; first run also fetches Open-Meteo weather
make train     # writes artifacts/model.joblib, metrics.json, forecasts.json
make small     # prints the busy-hour classifier metrics + confusion matrix
make figures   # regenerates docs/figures/*.svg from metrics.json
make test

The raw challenge CSV is never committed. The default path is /home/noel/Documents/Hackathon_2026/data/raw/schwarz-pax.csv; set PULSECAST_RAW_CSV to use another file. data/raw/ and data/cache/ are git-ignored. The committed artifacts/ (JSON + joblib + compact weather frames) let the API and Docker image run without the raw file or network.

Artifacts

File Purpose
artifacts/sensors.json 8 sensors: id, name, lat/lon, mean, p95, battery
artifacts/series.json + series/<id>.json Hourly series per sensor
artifacts/forecasts.json 48h + 7d forecasts for 4 scenarios
artifacts/metrics.json Versions, coverage, headline metrics, ablation, per-sensor, small summary, limits
artifacts/small_metrics.json Busy classifier: metrics, confusion matrix, per-sensor, thresholds
artifacts/context.json Current + 7-day weather, BW holidays
artifacts/data_audit.json Row/device checks for the raw CSV
artifacts/model.joblib Baked HistGB forecast bundle
artifacts/small_model.joblib Baked busy classifier
artifacts/weather_*.parquet Baked weather frames for retraining/inference

API

Response shapes match the Go service contract. The service is internal; Go exposes it publicly.

Method Path Notes
GET /healthz {"ok":true}
GET /readyz {"ok":true,"model":true}, 503 if no model
GET /meta Versions, coverage, metrics, features, holiday effect
GET /bootstrap Snapshot for Go: sensors, hourly series, context, meta
POST /predict {sensor,horizon,scenario,weather?,is_holiday?} → points
POST /simulate Scenario forecast for all or selected sensors
POST /train Control token; multipart CSV or JSON path; 202 job
GET /train/{job_id} Status, metrics, model_version, error

Horizons: 15m (1 point), 1h (4 points), 48h (48 points), 7d (168 points). Scenarios: base, sunny-saturday-market, rainy-saturday, weekday-event.

curl -s localhost:8000/predict \
  -H 'content-type: application/json' \
  -d '{"sensor":"hn-stadtgalerie-01","horizon":"7d","scenario":"base"}'

curl -s localhost:8000/simulate \
  -H 'content-type: application/json' \
  -d '{"scenario":"sunny-saturday-market","sensors":"all"}'

# Training requires CONTROL_TOKEN and one job at a time.
CONTROL_TOKEN=change-me make serve
curl -s -X POST localhost:8000/train \
  -H "Authorization: Bearer change-me" \
  -F model=small -F file=@new-data.csv
curl -s localhost:8000/train/<job_id> \
  -H "Authorization: Bearer change-me"

Training runs in a background thread, hot-swaps the in-memory model on success, and persists to PULSECAST_MODELS (default /tmp/pulsecast/models). A restart falls back to the baked model. model=small is the fast demo; model=forecast retrains the global HistGB (slower, still background).

End-to-end check

scripts/e2e_train.py starts a real uvicorn process on port 18000 with a control token, then checks the snapshot, every /predict combination, every /simulate scenario, auth and error paths, one-job-at-a-time, and both real training jobs (small + forecast) including hot-swap and /predict afterward:

uv run python scripts/e2e_train.py --port 18000 --report /tmp/e2e.json

The unit and contract tests are make test; make check is the CI gate.

Docs

File Contents
docs/metrics.md Paper tables/figures: protocol, headline, ablation, per sensor, uncertainty, small model, claim limits
docs/small-model.md Busy-hour target, thresholds, split, metrics, confusion matrix, per sensor, limits
docs/train-e2e.md Exact live training-job evidence (small + forecast, hot-swap, errors)
docs/figures/*.svg Ablation MAE, per-sensor MAE, small-model confusion matrix (generated)
model_card.md Model card: features, split, baseline, ablation, leakage, limitations

Docker

make docker
docker run --rm -p 8000:8000 -e CONTROL_TOKEN=change-me pulsecast-ml:local

Multi-stage python:3.12-slim, numeric non-root user 10001:10001, model and artifacts baked in, writable paths only under /tmp.

Configuration

Variable Default Use
PULSECAST_ROOT first parent with pyproject.toml Project root
PULSECAST_RAW_CSV challenge path Raw CSV location
PULSECAST_ARTIFACTS ./artifacts Baked JSON/model output
PULSECAST_CACHE ./data/cache Parquet/weather cache
PULSECAST_WEATHER PULSECAST_ARTIFACTS Baked weather frame location
PULSECAST_MODELS /tmp/pulsecast/models Retrained model persistence
PULSECAST_UPLOADS /tmp/pulsecast/uploads Multipart upload landing
CONTROL_TOKEN unset Required by /train; service returns 503 if unset

Layout

src/pulsecast/
  config.py     paths, sensors, holidays, scenarios, feature sets
  parse.py      CSV validation, exclusions, density-01 device merge
  features.py   hourly aggregation, leakage-safe lags/rollups
  weather.py    Open-Meteo archive/forecast with cache
  pipeline.py   `make data`
  model.py      HistGB fit, inference frame, bands, scenario overlay
  train.py      `make train`, ablation, artifacts
  small.py      busy classifier + flexible CSV parser
  retrain.py    runtime forecast retraining from upload
  server.py     FastAPI app and training job manager
scripts/        train_small.py, plot_metrics.py, e2e_train.py
docs/           metrics.md, small-model.md, train-e2e.md, generated figures
tests/          schema, bootstrap, health, prediction, small-training job
model_card.md   features, split, baseline, ablation, limits, leakage

Limits

  • Values are estimated devices detected by PAX counters, not a count of people.
  • Eight sensors in a ~580 m inner-city cluster; no city-wide claim.
  • Five months of multi-sensor data; no annual-seasonality claim.
  • Weather is observed in training and forecast in the demo/scenarios.
  • Holidays are context and a small feature, not a large accuracy claim.
  • No POS, sales, queue, or labor-cost data; staffing is decision support with explicit assumptions.
S
Description
Data Analysis + Machine Learning (wow i create my own AI, buzzwords covered :) )
Readme
2.5 MiB
0 Stars 1 Watchers 0 Forks
Languages
Python 98.5%
Makefile 0.8%
Dockerfile 0.7%