diff --git a/.gitea/workflows/ci.yml b/.gitea/workflows/ci.yml index 1706dd4..1e82345 100644 --- a/.gitea/workflows/ci.yml +++ b/.gitea/workflows/ci.yml @@ -18,13 +18,14 @@ jobs: - name: Check out uses: actions/checkout@v4 - - name: Set up Python - uses: actions/setup-python@v5 + # setup-python needs the runner toolcache and lsb_release; this Gitea + # runner has neither. uv installs its own Python 3.12. + - name: Set up uv + uses: astral-sh/setup-uv@v6 with: + version: "0.10.4" python-version: "3.12" - - - name: Install uv - run: python -m pip install "uv==0.10.4" + enable-cache: false # Same checks as local `make check`, split into steps for readable CI # failures. No secrets, no image push, no cluster access. diff --git a/README.md b/README.md index 860cc67..d39ff7a 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,10 @@ # PulseCast ML Footfall pipeline, weather-aware 7-day forecast, scenario engine, and FastAPI -service for the PulseCast staffing copilot. - -The reusable asset is the pipeline: raw 15-minute PAX counters → clean hourly -series → observed weather + Baden-Württemberg holidays → global model → -honest holdout metrics → JSON artifacts → decision-ready API. +service for the PulseCast staffing copilot. It turns raw 15-minute PAX counter +rows into hourly series, joins weather and Baden-Württemberg holidays, trains +one global model, evaluates it on a 28-day time-based holdout, and serves the +resulting JSON artifacts. ## Results @@ -24,9 +23,10 @@ honest holdout metrics → JSON artifacts → decision-ready API. The seasonal-naive row reproduces the mission reference exactly (5.40 / 44.7% / 0.462), and the day-ahead weather row matches 3.36 / 27.8%. Weather cuts error by ~19% versus the no-weather 7-day model and ~38% versus -seasonal naive. The production model is slightly better than the reference -because its seasonal feature uses the previous *observed* same weekday/hour -across sensor gaps; the baseline uses the exact 168-row lag. +seasonal naive. The production 7-day model edges the reference day-ahead +weather row (3.32 vs 3.36 MAE). The two use different features; `lag_week` uses +the previous *observed* same weekday/hour across sensor gaps, while the +seasonal-naive baseline uses the exact 168-row lag. Small busy-hour classifier (baked for the "train new data" demo): @@ -37,34 +37,35 @@ Small busy-hour classifier (baked for the "train new data" demo): The classifier predicts `busy` = hourly footfall above the sensor's p75 using hour, weekday, weekend, temperature, rain, and holiday. It trains in under a second on the challenge CSV and is retrained from an uploaded CSV through -`POST /train`. Full definition, confusion matrix, per-sensor table, and honest -limits: [`docs/small-model.md`](docs/small-model.md). Paper tables and figures: +`POST /train`. Full definition, confusion matrix, per-sensor table, and limits: +[`docs/small-model.md`](docs/small-model.md). Paper tables and figures: [`docs/metrics.md`](docs/metrics.md). ## Quick start -Requirements: Python 3.12 and `uv` (the repo also works with a plain venv). +Requires Python 3.12 and `uv`. ```bash uv sync -make data # needs the raw CSV, first run also fetches Open-Meteo weather +make serve # uses the committed artifacts; http://localhost:8000 +make check # same gate as CI: sync --frozen, ruff, compileall, pytest +``` + +To rebuild the artifacts from raw data: + +```bash +make data # needs the raw CSV; first run also fetches Open-Meteo weather make train # writes artifacts/model.joblib, metrics.json, forecasts.json make small # prints the busy-hour classifier metrics + confusion matrix make figures # regenerates docs/figures/*.svg from metrics.json make test -make check # CI gate: sync --frozen, ruff, compileall, pytest -make serve # http://localhost:8000 ``` -The raw challenge CSV is **never committed**. Put it at: - -```text -/home/noel/Documents/Hackathon_2026/data/raw/schwarz-pax.csv -``` - -or point `PULSECAST_RAW_CSV` at it. `data/raw/` and `data/cache/` are git-ignored. -The generated `artifacts/` (JSON + joblib + compact weather frames) are -committed so the API and Docker image work without the raw file or network. +The raw challenge CSV is **never committed**. The default path is +`/home/noel/Documents/Hackathon_2026/data/raw/schwarz-pax.csv`; set +`PULSECAST_RAW_CSV` to use another file. `data/raw/` and `data/cache/` are +git-ignored. The committed `artifacts/` (JSON + joblib + compact weather frames) +let the API and Docker image run without the raw file or network. ## Artifacts @@ -76,14 +77,15 @@ committed so the API and Docker image work without the raw file or network. | `artifacts/metrics.json` | Versions, coverage, headline metrics, ablation, per-sensor, small summary, limits | | `artifacts/small_metrics.json` | Busy classifier: metrics, confusion matrix, per-sensor, thresholds | | `artifacts/context.json` | Current + 7-day weather, BW holidays | -| `artifacts/data_audit.json` | Row/sensor checks against `mission/data.md` | +| `artifacts/data_audit.json` | Row/device checks for the raw CSV | | `artifacts/model.joblib` | Baked HistGB forecast bundle | | `artifacts/small_model.joblib` | Baked busy classifier | | `artifacts/weather_*.parquet` | Baked weather frames for retraining/inference | ## API -Shapes follow `CONTRACT.md`. The service is internal; Go exposes it publicly. +Response shapes match the Go service contract. The service is internal; Go +exposes it publicly. | Method | Path | Notes | |---|---|---| @@ -160,9 +162,11 @@ artifacts baked in, writable paths only under `/tmp`. | Variable | Default | Use | |---|---|---| +| `PULSECAST_ROOT` | first parent with `pyproject.toml` | Project root | | `PULSECAST_RAW_CSV` | challenge path | Raw CSV location | | `PULSECAST_ARTIFACTS` | `./artifacts` | Baked JSON/model output | | `PULSECAST_CACHE` | `./data/cache` | Parquet/weather cache | +| `PULSECAST_WEATHER` | `PULSECAST_ARTIFACTS` | Baked weather frame location | | `PULSECAST_MODELS` | `/tmp/pulsecast/models` | Retrained model persistence | | `PULSECAST_UPLOADS` | `/tmp/pulsecast/uploads` | Multipart upload landing | | `CONTROL_TOKEN` | unset | Required by `/train`; service returns 503 if unset | @@ -182,7 +186,7 @@ src/pulsecast/ retrain.py runtime forecast retraining from upload server.py FastAPI app and training job manager scripts/ train_small.py, plot_metrics.py, e2e_train.py -docs/ metrics.md, small-model.md, generated figures +docs/ metrics.md, small-model.md, train-e2e.md, generated figures tests/ schema, bootstrap, health, prediction, small-training job model_card.md features, split, baseline, ablation, limits, leakage ``` diff --git a/docs/metrics.md b/docs/metrics.md index 5c891ce..60c4be5 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -1,10 +1,10 @@ # Metrics — paper support -All values in this file are generated by `make train` and stored in -`artifacts/metrics.json` (forecast + ablation + per-sensor + small model) and -`artifacts/small_metrics.json` (small-model detail). No number here is typed by -hand: to refresh a table, rerun `make train && make figures` and copy the new -JSON, or quote the JSON directly. +Every value in this file comes from the committed artifacts: +`artifacts/metrics.json` (forecast + ablation + per-sensor + small model), +`artifacts/small_metrics.json` (small-model detail), and +`artifacts/data_audit.json` (raw-input counts). To refresh a table, rerun +`make train && make figures` and quote the new JSON. ## Run provenance @@ -49,9 +49,10 @@ JSON, or quote the JSON directly. - WAPE improvement: 17.2 percentage points. - The mission reference numbers for seasonal naive (5.40 / 44.7% / 0.462) are reproduced exactly. -- The production row is slightly better than the reference 7-day model - (4.37 / 36.2% / 0.672) because `lag_week` uses the previous *observed* same - weekday/hour across sensor gaps instead of recursion. +- The production row also beats the mission reference 7-day no-weather model + (4.37 / 36.2% / 0.672): weather supplies most of the gain, and `lag_week` + uses the previous *observed* same weekday/hour across sensor gaps instead of + recursion. ## Ablation @@ -174,7 +175,7 @@ Per-sensor small-model holdout: - The small classifier as the production forecast. It is a demo artifact with six features and a per-window p75 target. - Tuned hyperparameters. The model configuration is fixed; the 28-day holdout - is the honest signal, not a search result. + is the held-out result, not a search result. ## Reproduce diff --git a/docs/small-model.md b/docs/small-model.md index a57465a..cb0e0f8 100644 --- a/docs/small-model.md +++ b/docs/small-model.md @@ -1,10 +1,8 @@ # Small model — busy-hour classifier -This is the small, fast, explainable model that powers the "train new data" -demo and the "I coded an AI" artifact. It is deliberately separate from the -production 7-day forecaster: six cheap features, a linear model, a time-based -split, and under a second of fitting. It is not the model behind the dashboard -forecast. +A logistic regression that labels an hour `busy` when footfall exceeds that +sensor's p75. It powers the `model=small` retraining demo, not the dashboard +forecast: six cheap features, a time-based split, and under a second to fit. Every number below comes from the baked artifact `artifacts/small_metrics.json` (version `logit-20261114-052723`, trained diff --git a/docs/train-e2e.md b/docs/train-e2e.md index ada7c0c..eb9b198 100644 --- a/docs/train-e2e.md +++ b/docs/train-e2e.md @@ -60,9 +60,9 @@ Input: 36,846 density rows, 8 sensors, 2026-09-08T23:53Z → 2026-11-07T23:51Z. This is a different window from the baked full-window model in `docs/small-model.md`, so the numbers are not directly comparable: the p75 -target and the 80/20 split are recomputed inside the 60-day upload. The -low recall illustrates the honest limit noted in the small-model doc — a short, -recent window with a different threshold is a different classification problem. +target and the 80/20 split are recomputed inside the 60-day upload. The low +recall illustrates the limit noted in the small-model doc: a short, recent +window with a different threshold is a different classification problem. Post-conditions verified: diff --git a/model_card.md b/model_card.md index 97ed55a..dd348f4 100644 --- a/model_card.md +++ b/model_card.md @@ -68,10 +68,11 @@ a headline accuracy claim. The seasonal-naive row reproduces the mission reference exactly (5.40 / 44.7% / 0.462), and the day-ahead weather row matches 3.36 / 27.8%. The -production model is slightly better than the reference because `lag_week` uses -the previous *observed* same weekday/hour across sensor gaps, while the baseline -uses the exact 168-row lag. R² differs slightly from the reference 0.779 -(0.777 here) because of that feature choice. +production 7-day model edges that day-ahead reference on MAE and WAPE +(3.32 / 27.5%) but is slightly lower on R² (0.777 vs 0.779). The two use +different features; `lag_week` uses the previous *observed* same weekday/hour +across sensor gaps, while the seasonal-naive baseline uses the exact 168-row +lag. Per-sensor holdout MAE improves over each sensor's own seasonal naive by 27.7%–43.9%; the full table, figures, and provenance are in diff --git a/src/pulsecast/config.py b/src/pulsecast/config.py index 5ab2a0d..05ee88a 100644 --- a/src/pulsecast/config.py +++ b/src/pulsecast/config.py @@ -1,4 +1,4 @@ -"""Configuration: paths, sensors, holidays, scenarios, contract versions. +"""Configuration: paths, sensors, holidays, features, scenarios. All timestamps are UTC internally; Europe/Berlin is used only for calendar features and display-oriented payloads. @@ -65,8 +65,8 @@ def control_token() -> str: # --------------------------------------------------------------------------- data -# Multi-sensor window documented in mission/data.md. density-01 has older -# history but the other seven locations start on 2026-06-12. +# Multi-sensor window of the published dataset. density-01 has older history, +# but the other seven locations start on 2026-06-12. MULTI_SENSOR_START = "2026-06-12T00:00:00Z" # 28-day time-based holdout, measured back from the last observed hour. @@ -134,8 +134,8 @@ WEATHER_COLUMNS = { "relative_humidity_2m": "humidity_pct", } -# Baden-Württemberg public holidays, 2025-2026 (mission/data.md). Kept explicit -# so the build is deterministic and needs no holiday library. +# Baden-Württemberg public holidays, 2025-2026. Kept explicit so the build is +# deterministic and needs no holiday library. HOLIDAYS: dict[str, str] = { "2025-01-01": "Neujahr", "2025-01-06": "Heilige Drei Könige", @@ -185,7 +185,6 @@ FEATURE_WEATHER = ["temp_c", "precip_mm"] FEATURE_DAY_NO_HOLIDAY = FEATURE_CALENDAR_NO_HOLIDAY + FEATURE_DAY[6:] FEATURE_DAY_NO_WEATHER = FEATURE_DAY FEATURE_DAY_WEATHER = FEATURE_DAY + FEATURE_WEATHER -FEATURE_DAY_HOLIDAY = FEATURE_DAY + ["is_holiday"] # is_holiday already present; kept for clarity FEATURE_WEEK_NO_WEATHER = FEATURE_WEEK FEATURE_WEEK_WEATHER = FEATURE_WEEK + FEATURE_WEATHER diff --git a/src/pulsecast/model.py b/src/pulsecast/model.py index aab6ea6..372871e 100644 --- a/src/pulsecast/model.py +++ b/src/pulsecast/model.py @@ -16,7 +16,7 @@ from . import config from .features import SENSOR_CODES, future_calendar # Fixed, deliberately small model. No tuning beyond a short sanity pass: the -# 28-day holdout is the honest signal, not the training loss. +# 28-day holdout decides, not the training loss. HISTGB_PARAMS = { "loss": "squared_error", "learning_rate": 0.06, @@ -58,7 +58,7 @@ def metrics(y_true, y_pred) -> dict: def residual_quantiles(eval_frame: pd.DataFrame, pred: np.ndarray) -> dict: - """80% prediction band offsets from the honest holdout residuals.""" + """80% prediction band offsets from the holdout residuals.""" residual = np.asarray(eval_frame["y"], dtype=float) - np.asarray(pred, dtype=float) return { "p10": round(float(np.quantile(residual, 0.10)), 4), diff --git a/src/pulsecast/parse.py b/src/pulsecast/parse.py index 56cb962..c56992a 100644 --- a/src/pulsecast/parse.py +++ b/src/pulsecast/parse.py @@ -1,8 +1,9 @@ """Parse and clean the raw PAX CSV. -Verifies the claims in mission/data.md: 192,464 rows, 12 device IDs, 8 usable -logical locations, three excluded test devices, and the density-01 merge. -The raw file is never written to git; a cached parquet copy lives in data/cache/. +The audit checks the counts the pipeline depends on: 192,464 rows, 12 device +IDs, 8 usable logical locations, three excluded test devices, and the +density-01 merge. The raw file is never written to git; a cached parquet copy +lives in data/cache/. """ from __future__ import annotations diff --git a/src/pulsecast/server.py b/src/pulsecast/server.py index e67e4ac..4d6f0ca 100644 --- a/src/pulsecast/server.py +++ b/src/pulsecast/server.py @@ -1,7 +1,7 @@ """FastAPI ML service: /healthz, /readyz, /meta, /bootstrap, /predict, /simulate, /train. -Shapes follow CONTRACT.md. The service is internal: only Go exposes it publicly, -and the training routes require a bearer control token. +The service is internal: only Go exposes it publicly, and the training routes +require a bearer control token. """ from __future__ import annotations @@ -79,10 +79,9 @@ def _load_runtime(runtime: Runtime) -> None: json.loads(meta_path.read_text()) if meta_path.exists() else None ) - # Baked snapshot artifacts served by GET /bootstrap. Go caches this once at - # startup, so loading them at boot keeps the endpoint a pure in-memory read. - # Project to the contract fields; artifacts carry extra audit fields - # (last_seen) that Go does not need. + # Go caches /bootstrap once at startup, so load it at boot and keep the + # endpoint a pure in-memory read. Project to contract fields because the + # artifacts carry extra audit fields (last_seen). sensors_path = artifacts / "sensors.json" if sensors_path.exists(): sensors = json.loads(sensors_path.read_text()) @@ -653,6 +652,8 @@ def create_app() -> FastAPI: state = runtime() model, path = await _parse_train_request(request) + # Serialize check-and-start so two requests cannot both pass the + # "no active job" check. with state.schedule_lock: with state.jobs_lock: if any(job["status"] in ACTIVE_STATUSES for job in state.jobs.values()): diff --git a/src/pulsecast/small.py b/src/pulsecast/small.py index bcf4bbf..fbbf11c 100644 --- a/src/pulsecast/small.py +++ b/src/pulsecast/small.py @@ -93,8 +93,8 @@ def _attach_weather(hourly: pd.DataFrame, archive: pd.DataFrame | None) -> pd.Da frame[column] = frame[column].where(frame[column].notna(), frame[archive_column]) frame = frame.drop(columns=[archive_column]) - # Honest fallback: no weather available for the uploaded window. Constant - # values keep the other features useful; the job metrics show the effect. + # No weather for the uploaded window: constant values keep the other + # features usable, and the job metrics show the effect. frame["temp_c"] = frame["temp_c"].fillna(14.0) frame["precip_mm"] = frame["precip_mm"].fillna(0.0) return frame diff --git a/src/pulsecast/train.py b/src/pulsecast/train.py index 1a45ab2..1c93dbc 100644 --- a/src/pulsecast/train.py +++ b/src/pulsecast/train.py @@ -170,7 +170,7 @@ def train_bundle( } ) - # Production model: refit on all observed rows after the honest holdout. + # Production model: refit on all observed rows after the holdout is scored. production = fit(features, config.FEATURE_WEEK_WEATHER) residual = residual_quantiles(eval_frame, preds["week_weather"]) sensor_mean = hourly.groupby("sensor", observed=True)["v"].mean().round(4).to_dict() diff --git a/src/pulsecast/weather.py b/src/pulsecast/weather.py index 6ba4475..8c8f280 100644 --- a/src/pulsecast/weather.py +++ b/src/pulsecast/weather.py @@ -1,8 +1,8 @@ """Open-Meteo weather: historical archive for training, forecast for scenarios. -No API key. Responses are cached under data/cache/ so `make train` and the -Docker image need no network. The raw CSV and cache stay out of git; the final -context.json is committed. +No API key. Responses are cached under data/cache/ so `make train` needs no +network after the first fetch. The raw CSV and cache stay out of git; compact +frames and context.json are committed under artifacts/. """ from __future__ import annotations