fix: use exact 168-row lag for the seasonal-naive baseline

The baseline now reproduces mission/data.md exactly: MAE 5.40, WAPE 44.7%,
R2 0.462. The model's lag_week remains the previous observed same weekday/hour,
which handles sensor gaps and is documented in the model card.
This commit is contained in:
Noel Sabia committed 2026-11-14 05:04:41 +01:00
1 parent 939a8db482
commit dc83825de0
9 files changed
+41 -26

No files matched your search

+10 -6
View File
@@ -50,21 +50,24 @@ Time-based 28-day holdout, no shuffling. Seasonal naive is the baseline. WAPE is
| Model | MAE | WAPE | R² |
|---|---:|---:|---:|
| Seasonal naive (same local hour last week) | 4.23 | 35.0% | 0.656 |
| Seasonal naive (exact 168-row lag) | 5.40 | 44.7% | 0.462 |
| Day-ahead, no weather | 3.96 | 32.8% | 0.706 |
| Day-ahead, + holiday flag | 3.84 | 31.8% | 0.732 |
| Day-ahead, + temperature/rain | 3.36 | 27.8% | 0.763 |
| 7-day-safe, no weather | 4.08 | 33.8% | 0.697 |
| **7-day-safe, + temperature/rain (production)** | **3.32** | **27.5%** | **0.777** |
Weather is the useful signal: adding temperature and rain cuts MAE by ~22%
Weather is the useful signal: adding temperature and rain cuts MAE by ~19%
versus the 7-day no-weather model and ~38% versus seasonal naive. The holiday
flag alone changes day-ahead MAE by −0.12 in this run; holidays are context, not
a headline accuracy claim.
The mission's quick-baseline numbers (seasonal naive 5.40 / 44.7% / 0.462) came
from a different window and feature setup. The direction and the weather gain
reproduce; the exact decimals are not copied.
The seasonal-naive row reproduces the mission reference exactly
(5.40 / 44.7% / 0.462), and the day-ahead weather row matches 3.36 / 27.8%. The
production model is slightly better than the reference because `lag_week` uses
the previous *observed* same weekday/hour across sensor gaps, while the baseline
uses the exact 168-row lag. R² differs slightly from the reference 0.779
(0.777 here) because of that feature choice.
## Uncertainty band
@@ -87,7 +90,8 @@ guarantee.
## No-leakage rules
- All splits are time-based; no random shuffling.
- Seasonal lags use the previous *observed* same weekday/hour; future lags are
- The seasonal-naive baseline uses an exact 168-row lag. The model's
`lag_week` uses the previous *observed* same weekday/hour; future lags are
never filled from future targets.
- Rolling means are shifted at least 24 hours back.
- The production 7-day model uses only ≥168-hour-old information, so a 168-hour