Replace setup-python with setup-uv: the self-hosted Gitea runner lacks the toolcache and lsb_release dependency, so the job failed before any project step. uv installs its own Python 3.12; the checks are unchanged. Align the README and docs with the committed artifacts: quick start serves from baked artifacts, rebuild steps are separate, and the reference-model comparison no longer overstates the result. Drop stale external-doc references and comments that restated the code.
4.1 KiB
Training end-to-end evidence
Reproduce with:
uv run python scripts/e2e_train.py --port 18000 --report /tmp/e2e.json
The script starts a real uvicorn pulsecast.server:app process with
CONTROL_TOKEN, waits for startup, and drives the HTTP API only. It exits
non-zero on the first failed check. Unit/contract tests (make check) are
separate.
Run
| Field | Value |
|---|---|
| Date | 2026-11-14, 05:33 UTC |
| Branch | feat/ml-final |
| Port | 18000 |
| Checks | 38 passed, 0 failed |
| Total wall time | 10.4 s |
| Forecast version before | histgb-2026-11-14 |
| Data version | 2026-11-07T23:51:00Z |
| Context version | 2026-11-14T03:47:09Z |
Snapshot and robustness (20 checks)
/healthz→{"ok":true};/readyz→ 200model=true./metacovers 8 sensors and the baked model beats the seasonal-naive baseline on the holdout./bootstrap→ 8 sensors, all 8 hourly series (23,952 points), 168 forecast weather hours, 24 holidays, versions equal to/meta./predict: 128/128 combinations (8 sensors × 4 horizons × 4 scenarios) returned the contract point counts (15m1,1h4,48h48,7d168), monotonic UTC timestamps, non-negative values, andlo ≤ v ≤ hi./simulate: all 4 scenarios × 8 sensors × 4 horizons returned valid points.- Weather override changes the forecast (
-8 °C / 6 mmvs30 °C / 0 mm). - Unknown sensor → 404; missing token → 401
unauthorized; wrong token → 401; unknown job → 404not_found; invalid model → 400bad_request; missing JSON path → 400.
Small training job (real 60-day CSV subset)
Input: 36,846 density rows, 8 sensors, 2026-09-08T23:53Z → 2026-11-07T23:51Z.
| Field | Value |
|---|---|
| Job ID | 0b35347963974cbbbe584b77ecd4b846 |
| Status | succeeded |
| Model version | logit-20261114-053344 |
| Wall time (submit → terminal) | 0.172 s (contract < 5 s) |
| n_train / n_test | 8,151 / 2,301 |
| Base rate | 0.180 |
| Accuracy | 0.824 |
| Precision | 0.542 |
| Recall | 0.157 |
| F1 | 0.243 |
| ROC-AUC | 0.813 |
This is a different window from the baked full-window model in
docs/small-model.md, so the numbers are not directly comparable: the p75
target and the 80/20 split are recomputed inside the 60-day upload. The low
recall illustrates the limit noted in the small-model doc: a short, recent
window with a different threshold is a different classification problem.
Post-conditions verified:
small_latest.jobliband<job_id>-small.joblibexist underPULSECAST_MODELS./meta.small_model.model_versionequals the job version and its metrics match the job exactly (in-memory hot-swap)./predictstill returns 168 valid points after the swap.
Forecast training job (full challenge CSV) and 409
Input: full schwarz-pax.csv, 19,383,568 bytes.
| Field | Value |
|---|---|
| Job ID | 235209a0d00f449ba8be717eb76386d6 |
| Status | succeeded |
| Model version | histgb-20261114-053349 |
| Wall time | 4.380 s |
| MAE | 3.32 |
| WAPE | 27.5% |
| R² | 0.777 |
| Baseline MAE / WAPE / R² | 5.40 / 44.7% / 0.462 |
| n_eval | 5,350 |
The ablation in the job payload matches artifacts/metrics.json row for row
(3.96, 3.84, 3.36, 4.08, 3.32). While this job was in running, a second
POST /train returned 409 conflict, proving one job at a time.
Post-conditions verified:
forecast_latest.jobliband<job_id>-forecast.joblibexist underPULSECAST_MODELS./meta.model_versionequals the retrained version and/meta.model_sourceisretrained./predictreports the retrained version;/readyzstays 200.data_versionafter the retrain is2026-11-07T23:00:00Z(the hourly input maximum) rather than the raw-file floor23:51:00Z; both describe the same observed window.
Failure isolation
A malformed CSV (not,a,pulsecast,file) was accepted as a job (202), then
failed with a clear error:
CSV needs a time column (time, timestamp, datetime, date, hour) and a value
column (value, density, footfall, count, pax, visitors); got ['not', 'a',
'pulsecast', 'file']
/healthz still returned 200 afterward: a bad upload cannot take down the
service or block /predict.