fix: use exact 168-row lag for the seasonal-naive baseline
The baseline now reproduces mission/data.md exactly: MAE 5.40, WAPE 44.7%, R2 0.462. The model's lag_week remains the previous observed same weekday/hour, which handles sensor gaps and is documented in the model card.
This commit is contained in:
1 parent
939a8db482
commit
dc83825de0
9 files changed
+41
-26
No files matched your search
+10
-6
@@ -50,21 +50,24 @@ Time-based 28-day holdout, no shuffling. Seasonal naive is the baseline. WAPE is
|
||||
|
||||
| Model | MAE | WAPE | R² |
|
||||
|---|---:|---:|---:|
|
||||
| Seasonal naive (same local hour last week) | 4.23 | 35.0% | 0.656 |
|
||||
| Seasonal naive (exact 168-row lag) | 5.40 | 44.7% | 0.462 |
|
||||
| Day-ahead, no weather | 3.96 | 32.8% | 0.706 |
|
||||
| Day-ahead, + holiday flag | 3.84 | 31.8% | 0.732 |
|
||||
| Day-ahead, + temperature/rain | 3.36 | 27.8% | 0.763 |
|
||||
| 7-day-safe, no weather | 4.08 | 33.8% | 0.697 |
|
||||
| **7-day-safe, + temperature/rain (production)** | **3.32** | **27.5%** | **0.777** |
|
||||
|
||||
Weather is the useful signal: adding temperature and rain cuts MAE by ~22%
|
||||
Weather is the useful signal: adding temperature and rain cuts MAE by ~19%
|
||||
versus the 7-day no-weather model and ~38% versus seasonal naive. The holiday
|
||||
flag alone changes day-ahead MAE by −0.12 in this run; holidays are context, not
|
||||
a headline accuracy claim.
|
||||
|
||||
The mission's quick-baseline numbers (seasonal naive 5.40 / 44.7% / 0.462) came
|
||||
from a different window and feature setup. The direction and the weather gain
|
||||
reproduce; the exact decimals are not copied.
|
||||
The seasonal-naive row reproduces the mission reference exactly
|
||||
(5.40 / 44.7% / 0.462), and the day-ahead weather row matches 3.36 / 27.8%. The
|
||||
production model is slightly better than the reference because `lag_week` uses
|
||||
the previous *observed* same weekday/hour across sensor gaps, while the baseline
|
||||
uses the exact 168-row lag. R² differs slightly from the reference 0.779
|
||||
(0.777 here) because of that feature choice.
|
||||
|
||||
## Uncertainty band
|
||||
|
||||
@@ -87,7 +90,8 @@ guarantee.
|
||||
## No-leakage rules
|
||||
|
||||
- All splits are time-based; no random shuffling.
|
||||
- Seasonal lags use the previous *observed* same weekday/hour; future lags are
|
||||
- The seasonal-naive baseline uses an exact 168-row lag. The model's
|
||||
`lag_week` uses the previous *observed* same weekday/hour; future lags are
|
||||
never filled from future targets.
|
||||
- Rolling means are shifted at least 24 hours back.
|
||||
- The production 7-day model uses only ≥168-hour-old information, so a 168-hour
|
||||
|
||||
Reference in new issue
Block a user