Fill verified metrics, deployment evidence, and criteria mapping
This commit is contained in:
1 parent
c691324227
commit
c1e2c96a63
4 files changed
+76
-65
No files matched your search
@@ -4,8 +4,8 @@ Two versions of the paper, plus the research trail.
|
||||
|
||||
| File | What it is | Status |
|
||||
|---|---|---|
|
||||
| `main.md` | Version A — 4–5 page hackathon technical report | Draft; ~5–6 pages in a compact layout |
|
||||
| `main-extended.md` | Version B — 10–12 page Zenodo paper | Draft; ~9,200 words |
|
||||
| `main.md` | Version A — 4–5 page hackathon technical report | Draft; ~3,600 words, 5 figures, 4 tables |
|
||||
| `main-extended.md` | Version B — 10–12 page Zenodo paper | Draft; ~9,600 words, 37 references |
|
||||
| `references.md` | Canonical reference list and verification checklist | Draft |
|
||||
| `OUTLINE.md` | Writing plan, claims/non-claims, figures/tables | Working document |
|
||||
| `figures/` | Generated figures used by both versions | Draft figures |
|
||||
@@ -29,20 +29,25 @@ paper/figures/fig_forecast.png
|
||||
## Verification status
|
||||
|
||||
- Numbers: checked against `OKF_Vault/mission/data.md`, the generated `metrics.json`, and `paper/research/07-data-quality.md`.
|
||||
- Metrics discrepancy: `metrics.json` reports **3.32 / 27.5% / 0.777** for the production 7-day model and **3.36 / 27.8% / 0.763** for the day-ahead model; the mission brief lists **3.36 / 27.8% / 0.779**. The paper uses the generated metrics and flags the difference. The human must confirm the canonical run.
|
||||
- k6: only a local 3-VU SSE smoke test (p95 656 ms) exists. The 120-client cluster run is still `[TODO]`.
|
||||
- Canonical metrics: production 7-day **3.32 / 27.5% / 0.777**; day-ahead weather **3.36 / 27.8% / 0.763**; seasonal naive **5.40 / 44.7% / 0.462**. The mission brief’s 3.36/27.8/0.779 was the quick baseline; the paper explains the difference once.
|
||||
- Deployment: live app https://pulse.noelsabia.dev; API https://api-pulse.noelsabia.dev.
|
||||
- v1.1 images: api 16.9 MB, web 249 MB, ml 952 MB.
|
||||
- Plan latency: 42.8 ms local v1.1; contract < 2 s.
|
||||
- 120-client Cloudflare k6: 120/120 SSE opened and closed, 0 connection errors, p95 event 1.03 s, p95 scenario 194 ms, presenter POST 82.69 ms, 2,760 events, 4m53s.
|
||||
- Per-sensor MAE: computed from the production 7-day model; `n ≈ 669` per sensor.
|
||||
- Figures: generated from the raw CSV and the trained model; the plan screen is a placeholder.
|
||||
- References: 12 in Version A, 35 in Version B; all are real sources from the research notes. Verify DOIs/URLs before submission.
|
||||
- References: 12 in Version A, 37 in Version B; all are real sources from the research notes. Verify DOIs/URLs before submission.
|
||||
- AI disclosure: included in both versions.
|
||||
|
||||
## Known TODOs
|
||||
|
||||
- [ ] Canonical metrics decision (3.32/27.5/0.777 vs 3.36/27.8/0.779).
|
||||
- [ ] 120-client k6 result and system metrics (image sizes, cold deployment time).
|
||||
- [ ] Live plan-screen screenshot.
|
||||
- [ ] Demo URL and public code repository URL.
|
||||
- [x] Canonical metrics (3.32/27.5/0.777 production; 3.36/27.8/0.763 day-ahead).
|
||||
- [x] 120-client k6 result, image sizes, and plan latency.
|
||||
- [x] Live app/API URLs.
|
||||
- [ ] Live plan-screen screenshot (the current figure is a labelled placeholder).
|
||||
- [ ] Public code repository URL (demo is filled; code is still `[TODO]`).
|
||||
- [ ] Author names and affiliation.
|
||||
- [ ] Cold deployment time.
|
||||
- [ ] Replace internal project-note citations with public artifacts if a venue requires it.
|
||||
- [ ] Final page-count trim for Version A if a strict 4–5 page limit applies.
|
||||
|
||||
|
||||
Reference in new issue
Block a user