Replace setup-python with setup-uv: the self-hosted Gitea runner lacks the toolcache and lsb_release dependency, so the job failed before any project step. uv installs its own Python 3.12; the checks are unchanged. Align the README and docs with the committed artifacts: quick start serves from baked artifacts, rebuild steps are separate, and the reference-model comparison no longer overstates the result. Drop stale external-doc references and comments that restated the code.
This commit is contained in:
1 parent
9c7f77f66b
commit
070ecd2a78
13 files changed
+81
-75
No files matched your search
@@ -18,13 +18,14 @@ jobs:
|
||||
- name: Check out
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
# setup-python needs the runner toolcache and lsb_release; this Gitea
|
||||
# runner has neither. uv installs its own Python 3.12.
|
||||
- name: Set up uv
|
||||
uses: astral-sh/setup-uv@v6
|
||||
with:
|
||||
version: "0.10.4"
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install uv
|
||||
run: python -m pip install "uv==0.10.4"
|
||||
enable-cache: false
|
||||
|
||||
# Same checks as local `make check`, split into steps for readable CI
|
||||
# failures. No secrets, no image push, no cluster access.
|
||||
|
||||
@@ -1,11 +1,10 @@
|
||||
# PulseCast ML
|
||||
|
||||
Footfall pipeline, weather-aware 7-day forecast, scenario engine, and FastAPI
|
||||
service for the PulseCast staffing copilot.
|
||||
|
||||
The reusable asset is the pipeline: raw 15-minute PAX counters → clean hourly
|
||||
series → observed weather + Baden-Württemberg holidays → global model →
|
||||
honest holdout metrics → JSON artifacts → decision-ready API.
|
||||
service for the PulseCast staffing copilot. It turns raw 15-minute PAX counter
|
||||
rows into hourly series, joins weather and Baden-Württemberg holidays, trains
|
||||
one global model, evaluates it on a 28-day time-based holdout, and serves the
|
||||
resulting JSON artifacts.
|
||||
|
||||
## Results
|
||||
|
||||
@@ -24,9 +23,10 @@ honest holdout metrics → JSON artifacts → decision-ready API.
|
||||
The seasonal-naive row reproduces the mission reference exactly
|
||||
(5.40 / 44.7% / 0.462), and the day-ahead weather row matches 3.36 / 27.8%.
|
||||
Weather cuts error by ~19% versus the no-weather 7-day model and ~38% versus
|
||||
seasonal naive. The production model is slightly better than the reference
|
||||
because its seasonal feature uses the previous *observed* same weekday/hour
|
||||
across sensor gaps; the baseline uses the exact 168-row lag.
|
||||
seasonal naive. The production 7-day model edges the reference day-ahead
|
||||
weather row (3.32 vs 3.36 MAE). The two use different features; `lag_week` uses
|
||||
the previous *observed* same weekday/hour across sensor gaps, while the
|
||||
seasonal-naive baseline uses the exact 168-row lag.
|
||||
|
||||
Small busy-hour classifier (baked for the "train new data" demo):
|
||||
|
||||
@@ -37,34 +37,35 @@ Small busy-hour classifier (baked for the "train new data" demo):
|
||||
The classifier predicts `busy` = hourly footfall above the sensor's p75 using
|
||||
hour, weekday, weekend, temperature, rain, and holiday. It trains in under a
|
||||
second on the challenge CSV and is retrained from an uploaded CSV through
|
||||
`POST /train`. Full definition, confusion matrix, per-sensor table, and honest
|
||||
limits: [`docs/small-model.md`](docs/small-model.md). Paper tables and figures:
|
||||
`POST /train`. Full definition, confusion matrix, per-sensor table, and limits:
|
||||
[`docs/small-model.md`](docs/small-model.md). Paper tables and figures:
|
||||
[`docs/metrics.md`](docs/metrics.md).
|
||||
|
||||
## Quick start
|
||||
|
||||
Requirements: Python 3.12 and `uv` (the repo also works with a plain venv).
|
||||
Requires Python 3.12 and `uv`.
|
||||
|
||||
```bash
|
||||
uv sync
|
||||
make data # needs the raw CSV, first run also fetches Open-Meteo weather
|
||||
make serve # uses the committed artifacts; http://localhost:8000
|
||||
make check # same gate as CI: sync --frozen, ruff, compileall, pytest
|
||||
```
|
||||
|
||||
To rebuild the artifacts from raw data:
|
||||
|
||||
```bash
|
||||
make data # needs the raw CSV; first run also fetches Open-Meteo weather
|
||||
make train # writes artifacts/model.joblib, metrics.json, forecasts.json
|
||||
make small # prints the busy-hour classifier metrics + confusion matrix
|
||||
make figures # regenerates docs/figures/*.svg from metrics.json
|
||||
make test
|
||||
make check # CI gate: sync --frozen, ruff, compileall, pytest
|
||||
make serve # http://localhost:8000
|
||||
```
|
||||
|
||||
The raw challenge CSV is **never committed**. Put it at:
|
||||
|
||||
```text
|
||||
/home/noel/Documents/Hackathon_2026/data/raw/schwarz-pax.csv
|
||||
```
|
||||
|
||||
or point `PULSECAST_RAW_CSV` at it. `data/raw/` and `data/cache/` are git-ignored.
|
||||
The generated `artifacts/` (JSON + joblib + compact weather frames) are
|
||||
committed so the API and Docker image work without the raw file or network.
|
||||
The raw challenge CSV is **never committed**. The default path is
|
||||
`/home/noel/Documents/Hackathon_2026/data/raw/schwarz-pax.csv`; set
|
||||
`PULSECAST_RAW_CSV` to use another file. `data/raw/` and `data/cache/` are
|
||||
git-ignored. The committed `artifacts/` (JSON + joblib + compact weather frames)
|
||||
let the API and Docker image run without the raw file or network.
|
||||
|
||||
## Artifacts
|
||||
|
||||
@@ -76,14 +77,15 @@ committed so the API and Docker image work without the raw file or network.
|
||||
| `artifacts/metrics.json` | Versions, coverage, headline metrics, ablation, per-sensor, small summary, limits |
|
||||
| `artifacts/small_metrics.json` | Busy classifier: metrics, confusion matrix, per-sensor, thresholds |
|
||||
| `artifacts/context.json` | Current + 7-day weather, BW holidays |
|
||||
| `artifacts/data_audit.json` | Row/sensor checks against `mission/data.md` |
|
||||
| `artifacts/data_audit.json` | Row/device checks for the raw CSV |
|
||||
| `artifacts/model.joblib` | Baked HistGB forecast bundle |
|
||||
| `artifacts/small_model.joblib` | Baked busy classifier |
|
||||
| `artifacts/weather_*.parquet` | Baked weather frames for retraining/inference |
|
||||
|
||||
## API
|
||||
|
||||
Shapes follow `CONTRACT.md`. The service is internal; Go exposes it publicly.
|
||||
Response shapes match the Go service contract. The service is internal; Go
|
||||
exposes it publicly.
|
||||
|
||||
| Method | Path | Notes |
|
||||
|---|---|---|
|
||||
@@ -160,9 +162,11 @@ artifacts baked in, writable paths only under `/tmp`.
|
||||
|
||||
| Variable | Default | Use |
|
||||
|---|---|---|
|
||||
| `PULSECAST_ROOT` | first parent with `pyproject.toml` | Project root |
|
||||
| `PULSECAST_RAW_CSV` | challenge path | Raw CSV location |
|
||||
| `PULSECAST_ARTIFACTS` | `./artifacts` | Baked JSON/model output |
|
||||
| `PULSECAST_CACHE` | `./data/cache` | Parquet/weather cache |
|
||||
| `PULSECAST_WEATHER` | `PULSECAST_ARTIFACTS` | Baked weather frame location |
|
||||
| `PULSECAST_MODELS` | `/tmp/pulsecast/models` | Retrained model persistence |
|
||||
| `PULSECAST_UPLOADS` | `/tmp/pulsecast/uploads` | Multipart upload landing |
|
||||
| `CONTROL_TOKEN` | unset | Required by `/train`; service returns 503 if unset |
|
||||
@@ -182,7 +186,7 @@ src/pulsecast/
|
||||
retrain.py runtime forecast retraining from upload
|
||||
server.py FastAPI app and training job manager
|
||||
scripts/ train_small.py, plot_metrics.py, e2e_train.py
|
||||
docs/ metrics.md, small-model.md, generated figures
|
||||
docs/ metrics.md, small-model.md, train-e2e.md, generated figures
|
||||
tests/ schema, bootstrap, health, prediction, small-training job
|
||||
model_card.md features, split, baseline, ablation, limits, leakage
|
||||
```
|
||||
|
||||
+10
-9
@@ -1,10 +1,10 @@
|
||||
# Metrics — paper support
|
||||
|
||||
All values in this file are generated by `make train` and stored in
|
||||
`artifacts/metrics.json` (forecast + ablation + per-sensor + small model) and
|
||||
`artifacts/small_metrics.json` (small-model detail). No number here is typed by
|
||||
hand: to refresh a table, rerun `make train && make figures` and copy the new
|
||||
JSON, or quote the JSON directly.
|
||||
Every value in this file comes from the committed artifacts:
|
||||
`artifacts/metrics.json` (forecast + ablation + per-sensor + small model),
|
||||
`artifacts/small_metrics.json` (small-model detail), and
|
||||
`artifacts/data_audit.json` (raw-input counts). To refresh a table, rerun
|
||||
`make train && make figures` and quote the new JSON.
|
||||
|
||||
## Run provenance
|
||||
|
||||
@@ -49,9 +49,10 @@ JSON, or quote the JSON directly.
|
||||
- WAPE improvement: 17.2 percentage points.
|
||||
- The mission reference numbers for seasonal naive (5.40 / 44.7% / 0.462) are
|
||||
reproduced exactly.
|
||||
- The production row is slightly better than the reference 7-day model
|
||||
(4.37 / 36.2% / 0.672) because `lag_week` uses the previous *observed* same
|
||||
weekday/hour across sensor gaps instead of recursion.
|
||||
- The production row also beats the mission reference 7-day no-weather model
|
||||
(4.37 / 36.2% / 0.672): weather supplies most of the gain, and `lag_week`
|
||||
uses the previous *observed* same weekday/hour across sensor gaps instead of
|
||||
recursion.
|
||||
|
||||
## Ablation
|
||||
|
||||
@@ -174,7 +175,7 @@ Per-sensor small-model holdout:
|
||||
- The small classifier as the production forecast. It is a demo artifact with
|
||||
six features and a per-window p75 target.
|
||||
- Tuned hyperparameters. The model configuration is fixed; the 28-day holdout
|
||||
is the honest signal, not a search result.
|
||||
is the held-out result, not a search result.
|
||||
|
||||
## Reproduce
|
||||
|
||||
|
||||
+3
-5
@@ -1,10 +1,8 @@
|
||||
# Small model — busy-hour classifier
|
||||
|
||||
This is the small, fast, explainable model that powers the "train new data"
|
||||
demo and the "I coded an AI" artifact. It is deliberately separate from the
|
||||
production 7-day forecaster: six cheap features, a linear model, a time-based
|
||||
split, and under a second of fitting. It is not the model behind the dashboard
|
||||
forecast.
|
||||
A logistic regression that labels an hour `busy` when footfall exceeds that
|
||||
sensor's p75. It powers the `model=small` retraining demo, not the dashboard
|
||||
forecast: six cheap features, a time-based split, and under a second to fit.
|
||||
|
||||
Every number below comes from the baked artifact
|
||||
`artifacts/small_metrics.json` (version `logit-20261114-052723`, trained
|
||||
|
||||
+3
-3
@@ -60,9 +60,9 @@ Input: 36,846 density rows, 8 sensors, 2026-09-08T23:53Z → 2026-11-07T23:51Z.
|
||||
|
||||
This is a different window from the baked full-window model in
|
||||
`docs/small-model.md`, so the numbers are not directly comparable: the p75
|
||||
target and the 80/20 split are recomputed inside the 60-day upload. The
|
||||
low recall illustrates the honest limit noted in the small-model doc — a short,
|
||||
recent window with a different threshold is a different classification problem.
|
||||
target and the 80/20 split are recomputed inside the 60-day upload. The low
|
||||
recall illustrates the limit noted in the small-model doc: a short, recent
|
||||
window with a different threshold is a different classification problem.
|
||||
|
||||
Post-conditions verified:
|
||||
|
||||
|
||||
+5
-4
@@ -68,10 +68,11 @@ a headline accuracy claim.
|
||||
|
||||
The seasonal-naive row reproduces the mission reference exactly
|
||||
(5.40 / 44.7% / 0.462), and the day-ahead weather row matches 3.36 / 27.8%. The
|
||||
production model is slightly better than the reference because `lag_week` uses
|
||||
the previous *observed* same weekday/hour across sensor gaps, while the baseline
|
||||
uses the exact 168-row lag. R² differs slightly from the reference 0.779
|
||||
(0.777 here) because of that feature choice.
|
||||
production 7-day model edges that day-ahead reference on MAE and WAPE
|
||||
(3.32 / 27.5%) but is slightly lower on R² (0.777 vs 0.779). The two use
|
||||
different features; `lag_week` uses the previous *observed* same weekday/hour
|
||||
across sensor gaps, while the seasonal-naive baseline uses the exact 168-row
|
||||
lag.
|
||||
|
||||
Per-sensor holdout MAE improves over each sensor's own seasonal naive by
|
||||
27.7%–43.9%; the full table, figures, and provenance are in
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Configuration: paths, sensors, holidays, scenarios, contract versions.
|
||||
"""Configuration: paths, sensors, holidays, features, scenarios.
|
||||
|
||||
All timestamps are UTC internally; Europe/Berlin is used only for calendar
|
||||
features and display-oriented payloads.
|
||||
@@ -65,8 +65,8 @@ def control_token() -> str:
|
||||
|
||||
# --------------------------------------------------------------------------- data
|
||||
|
||||
# Multi-sensor window documented in mission/data.md. density-01 has older
|
||||
# history but the other seven locations start on 2026-06-12.
|
||||
# Multi-sensor window of the published dataset. density-01 has older history,
|
||||
# but the other seven locations start on 2026-06-12.
|
||||
MULTI_SENSOR_START = "2026-06-12T00:00:00Z"
|
||||
|
||||
# 28-day time-based holdout, measured back from the last observed hour.
|
||||
@@ -134,8 +134,8 @@ WEATHER_COLUMNS = {
|
||||
"relative_humidity_2m": "humidity_pct",
|
||||
}
|
||||
|
||||
# Baden-Württemberg public holidays, 2025-2026 (mission/data.md). Kept explicit
|
||||
# so the build is deterministic and needs no holiday library.
|
||||
# Baden-Württemberg public holidays, 2025-2026. Kept explicit so the build is
|
||||
# deterministic and needs no holiday library.
|
||||
HOLIDAYS: dict[str, str] = {
|
||||
"2025-01-01": "Neujahr",
|
||||
"2025-01-06": "Heilige Drei Könige",
|
||||
@@ -185,7 +185,6 @@ FEATURE_WEATHER = ["temp_c", "precip_mm"]
|
||||
FEATURE_DAY_NO_HOLIDAY = FEATURE_CALENDAR_NO_HOLIDAY + FEATURE_DAY[6:]
|
||||
FEATURE_DAY_NO_WEATHER = FEATURE_DAY
|
||||
FEATURE_DAY_WEATHER = FEATURE_DAY + FEATURE_WEATHER
|
||||
FEATURE_DAY_HOLIDAY = FEATURE_DAY + ["is_holiday"] # is_holiday already present; kept for clarity
|
||||
FEATURE_WEEK_NO_WEATHER = FEATURE_WEEK
|
||||
FEATURE_WEEK_WEATHER = FEATURE_WEEK + FEATURE_WEATHER
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ from . import config
|
||||
from .features import SENSOR_CODES, future_calendar
|
||||
|
||||
# Fixed, deliberately small model. No tuning beyond a short sanity pass: the
|
||||
# 28-day holdout is the honest signal, not the training loss.
|
||||
# 28-day holdout decides, not the training loss.
|
||||
HISTGB_PARAMS = {
|
||||
"loss": "squared_error",
|
||||
"learning_rate": 0.06,
|
||||
@@ -58,7 +58,7 @@ def metrics(y_true, y_pred) -> dict:
|
||||
|
||||
|
||||
def residual_quantiles(eval_frame: pd.DataFrame, pred: np.ndarray) -> dict:
|
||||
"""80% prediction band offsets from the honest holdout residuals."""
|
||||
"""80% prediction band offsets from the holdout residuals."""
|
||||
residual = np.asarray(eval_frame["y"], dtype=float) - np.asarray(pred, dtype=float)
|
||||
return {
|
||||
"p10": round(float(np.quantile(residual, 0.10)), 4),
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
"""Parse and clean the raw PAX CSV.
|
||||
|
||||
Verifies the claims in mission/data.md: 192,464 rows, 12 device IDs, 8 usable
|
||||
logical locations, three excluded test devices, and the density-01 merge.
|
||||
The raw file is never written to git; a cached parquet copy lives in data/cache/.
|
||||
The audit checks the counts the pipeline depends on: 192,464 rows, 12 device
|
||||
IDs, 8 usable logical locations, three excluded test devices, and the
|
||||
density-01 merge. The raw file is never written to git; a cached parquet copy
|
||||
lives in data/cache/.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""FastAPI ML service: /healthz, /readyz, /meta, /bootstrap, /predict, /simulate, /train.
|
||||
|
||||
Shapes follow CONTRACT.md. The service is internal: only Go exposes it publicly,
|
||||
and the training routes require a bearer control token.
|
||||
The service is internal: only Go exposes it publicly, and the training routes
|
||||
require a bearer control token.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -79,10 +79,9 @@ def _load_runtime(runtime: Runtime) -> None:
|
||||
json.loads(meta_path.read_text()) if meta_path.exists() else None
|
||||
)
|
||||
|
||||
# Baked snapshot artifacts served by GET /bootstrap. Go caches this once at
|
||||
# startup, so loading them at boot keeps the endpoint a pure in-memory read.
|
||||
# Project to the contract fields; artifacts carry extra audit fields
|
||||
# (last_seen) that Go does not need.
|
||||
# Go caches /bootstrap once at startup, so load it at boot and keep the
|
||||
# endpoint a pure in-memory read. Project to contract fields because the
|
||||
# artifacts carry extra audit fields (last_seen).
|
||||
sensors_path = artifacts / "sensors.json"
|
||||
if sensors_path.exists():
|
||||
sensors = json.loads(sensors_path.read_text())
|
||||
@@ -653,6 +652,8 @@ def create_app() -> FastAPI:
|
||||
state = runtime()
|
||||
model, path = await _parse_train_request(request)
|
||||
|
||||
# Serialize check-and-start so two requests cannot both pass the
|
||||
# "no active job" check.
|
||||
with state.schedule_lock:
|
||||
with state.jobs_lock:
|
||||
if any(job["status"] in ACTIVE_STATUSES for job in state.jobs.values()):
|
||||
|
||||
@@ -93,8 +93,8 @@ def _attach_weather(hourly: pd.DataFrame, archive: pd.DataFrame | None) -> pd.Da
|
||||
frame[column] = frame[column].where(frame[column].notna(), frame[archive_column])
|
||||
frame = frame.drop(columns=[archive_column])
|
||||
|
||||
# Honest fallback: no weather available for the uploaded window. Constant
|
||||
# values keep the other features useful; the job metrics show the effect.
|
||||
# No weather for the uploaded window: constant values keep the other
|
||||
# features usable, and the job metrics show the effect.
|
||||
frame["temp_c"] = frame["temp_c"].fillna(14.0)
|
||||
frame["precip_mm"] = frame["precip_mm"].fillna(0.0)
|
||||
return frame
|
||||
|
||||
@@ -170,7 +170,7 @@ def train_bundle(
|
||||
}
|
||||
)
|
||||
|
||||
# Production model: refit on all observed rows after the honest holdout.
|
||||
# Production model: refit on all observed rows after the holdout is scored.
|
||||
production = fit(features, config.FEATURE_WEEK_WEATHER)
|
||||
residual = residual_quantiles(eval_frame, preds["week_weather"])
|
||||
sensor_mean = hourly.groupby("sensor", observed=True)["v"].mean().round(4).to_dict()
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
"""Open-Meteo weather: historical archive for training, forecast for scenarios.
|
||||
|
||||
No API key. Responses are cached under data/cache/ so `make train` and the
|
||||
Docker image need no network. The raw CSV and cache stay out of git; the final
|
||||
context.json is committed.
|
||||
No API key. Responses are cached under data/cache/ so `make train` needs no
|
||||
network after the first fetch. The raw CSV and cache stay out of git; compact
|
||||
frames and context.json are committed under artifacts/.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
Reference in new issue
Block a user