Files

138 lines
8.9 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# PulseCast paper — writing plan
Status: **Version A draft written** (`paper/main.md`, ~3,600 words, 9 references, product-first); **Version B draft written** (`paper/main-extended.md`, ~10,000 words, 44 references), with the privacy/legal and fact-check corrections applied. Research notes are in `paper/research/`.
## Title options
1. **The Pulse Is Not the People: A Code-Level Audit of Bluetooth Footfall Sensing and a Privacy-First Architecture for Urban Decisions** — recommended for the extended paper.
2. **From MAC Addresses to Decisions: What a 24-Hour Footfall Copilot Actually Measures** — softer, more descriptive.
3. **PulseCast: A Decision-First Footfall Copilot — Measurement Limits and a Path to Production** — best fit for the 4–5 page hackathon report.
## Version A — hackathon report (4–5 pages, due Sunday)
Keep the structure required by `agents/paper.md` and `OKF_Vault/mission/paper.md`.
| Section | Length | Content |
|---|---:|---|
| Abstract | 150–200 words | Problem, data, method, headline result, deployment |
| 1. Introduction | 0.5 page | Decision gap; why a dashboard is not enough; contributions |
| 2. Data | 0.75 page | PAX counters; 8 sensors; 5 months; 15 min; 580 m; coverage/gaps; **box: what one value actually is** (10 s scan × 0.7, 16-bit truncation); device bias; privacy; Open-Meteo; BW holidays |
| 3. Method | 1 page | Preprocessing; features; global HistGB; horizons; time-based split; staffing heuristic |
| 4. System | 0.75 page | Nuxt SSR + Onyx; Go API/SSE; FastAPI; SKE namespace-only; Cloudflare Tunnel; build-time batch; cached fallback |
| 5. Evaluation | 1 page | Baseline; weather ablation; holiday effect; per-sensor errors; plan latency; k6 load test |
| 6. Discussion & limitations | 0.5–0.75 page | 8 sensors; 5 months; observed weather; devices not people; **the measurement semantics**; no POS/labor data; single replica; no auth beyond control; **what we would do with more resources** (0.5 page) |
| 7. Related work | 0.25 page | PAX/privacy; pedestrian forecasting; gradient boosting; smart-city dashboards |
| 8. Conclusion & future work | 0.25 page | More sensors; POS/event data; multi-store; city operations |
| References | 8–12 | Real, verifiable only |
**Do not** add the full legal analysis or the full sensing-alternatives comparison to Version A. Put them in Version B. The short report can say: “The value is a relative activity index, not a people count; a code-level audit and a privacy-first redesign are in the extended version.”
**Required figures/tables for Version A:** sensor map; hourly/weekday profile; forecast vs actual; architecture; plan screen. Tables: coverage per sensor; model comparison; ablation; system metrics. Fill the k6 numbers from the real run; do not invent.
## Version B — extended paper (10–12 pages, Zenodo)
| Section | Pages | Content |
|---|---:|---|
| Abstract | 0.25 | Problem, audit finding, alternative, transferability, non-claims |
| 1. Introduction | 1 | Decision gap; the “privacy-preserving” claim; contributions |
| 2. The measurement problem | 2.5 | Code audit; formula; data forensics; reliability taxonomy; existing PulseCast metrics reframed |
| 3. The privacy problem | 1.5 | GDPR/ePrivacy/TTDSG; threat model; 16-bit truncation; operator claims vs code; DPIA checklist |
| 4. What “privacy-first” should mean | 2 | PETs; sensing alternatives; calibration; reference sensing stack; pilot protocol |
| 5. From sparse observations to decisions | 2 | Heatmap; forecast; uncertainty; decision layer; evaluation harness |
| 6. How far it generalizes | 1.5 | Applications matrix; case studies; falsifiers; where it stops |
| 7. Evaluation agenda | 1 | Ground truth; calibration; duty cycle; penetration; privacy budget; decision regret; proposed experiments |
| 8. Discussion & limitations | 0.75 | Single site; no ground truth; opinion bias; legal not advice; hackathon context |
| 9. Related work | 1 | PAX/privacy; pedestrian forecasting; spatiotemporal models; MLOps; urban dashboards |
| 10. Conclusion | 0.5 | What to keep; what to build; a checklist |
| References | 1.5 | 25–35 real, verified sources |
## Figures and tables
### Version A
- Fig. 1: sensor map (8 points; label names).
- Fig. 2: hourly/weekday profile.
- Fig. 3: forecast vs actual for one sensor over the 28-day test.
- Fig. 4: architecture.
- Fig. 5: plan screen (or table).
- Table 1: data coverage per sensor.
- Table 2: model comparison (seasonal naive / no weather / weather).
- Table 3: ablation (weather/holiday; per-sensor MAE).
- Table 4: system metrics (plan latency; 120 SSE clients; image sizes; deployment time).
### Version B (adds)
- Fig. 1: the PAX counter cycle (wake → ~10 s scan → `0.7 × U` → LoRaWAN → 900 s sleep; 1.1% duty cycle).
- Fig. 2: value quantization and the `density-01` calibration break.
- Fig. 3: sparse-to-heatmap-to-decision pipeline.
- Fig. 4: privacy-first sensing and data-flow architecture.
- Fig. 5: transferability matrix (or table).
- Table 1: code audit (what the firmware does vs what the claim says).
- Table 2: data quality (coverage, gaps, zero share, reconstructed `U`, quantization).
- Table 3: sensing alternatives (accuracy, cost, privacy, best use).
- Table 4: model comparison (existing metrics).
- Table 5: transferability matrix.
- Table 6: four-tier roadmap (hackathon / pilot / production; triggers; estimates).
## Claims and non-claims (both versions)
**Supported**
- The published value is `0.7 × U`; `U` is a short-window count of randomized BLE addresses.
- The network is a local inner-city cluster; not city-wide.
- The existing model improves on seasonal naive in a 28-day holdout: MAE 3.36 vs 5.40; WAPE 27.8% vs 44.7%; R² 0.779 vs 0.462.
- The sensor counts devices, not people; penetration is unknown.
- MAC-based sensing has a contested legal basis; hashing is pseudonymisation, not anonymisation.
- Thermal/radar/camera/probe/manual modalities have different accuracy, cost, privacy, and coverage trade-offs.
- The stack transfers directly to staffing/ads, conditionally to waste, as a sibling to sensor placement/LoRaWAN planning, and not to sentiment/learning guide.
**Not supported**
- “The sensor counts people.”
- “The value is a 15-minute count or mean.”
- “The 0.7 factor is a validated penetration correction.”
- “The 16-bit hash is anonymous.”
- “Eight sensors support a city-wide heatmap or store-location ranking.”
- “One model solves all six challenges.”
- “The proposed architecture has been deployed and evaluated.”
- “The staffing plan has proven ROI.”
## Reference strategy
- Version A: 8–12 real, verifiable references; no fake papers.
- Version B: 25–35 real, verifiable references. Prefer primary sources: standards, peer-reviewed papers, official docs, court/regulator decisions.
- Build `paper/references.md` from the research notes; verify every DOI/URL before submission.
- Do not cite a source that the notes do not support. If a claim is missing a source, write `[CITATION TODO]`.
## Files to write when approved
- `paper/main.md` — Version A (short report).
- `paper/main-extended.md` — Version B (extended paper), or a single `paper/main.md` with a version switch.
- `paper/references.md` — verified reference list.
- `paper/figures/` — generated figures (sensor map, profile, forecast, architecture, plan screen, cycle diagram, quantization, pipeline).
- `paper/tables/` — CSV/Markdown tables if useful.
- `paper/README.md` — build/render instructions and artifact links.
## Dependencies / open items
- [x] Data-quality note (`07-data-quality.md`) — complete; figures in `paper/research/figures/`.
- [x] k6 120-client result — v1.1: 120/120 opened/closed, 0 errors, p95 event 331 ms, p95 scenario 646 ms.
- [x] Per-sensor MAE table — computed from the production 7-day model.
- [x] Metrics reconciliation — canonical: production 3.32/27.5/0.777; day-ahead 3.36/27.8/0.763. The mission brief’s 3.36/27.8/0.779 was the quick baseline; explained once.
- [x] Live app/API URLs, image sizes, plan latency.
- [x] Author name and ORCID (Noel Sabia); affiliation intentionally blank.
- [x] Code release timing documented (after the hackathon).
- [ ] Plan figure with raw screenshot (current figure is a print reconstruction); cold deployment time documented as not measured.
- [ ] Figure generation — use the real CSV; no synthetic data in the paper.
- [ ] Author names/affiliation for the Zenodo record.
- [ ] Decision: Version A only, Version B only, or both.
- [ ] Decision: run the post-demo experiments (subsampling, leave-one-sensor-out, factor sensitivity).
- [ ] Verify every number against `OKF_Vault/mission/data.md`, `metrics.json`, or a measured result.
## Recommended sequence
1. Finish Version A for the hackathon deadline. Do not let Version B delay the demo.
2. After the demo, write Version B using the same evidence plus the full audit and architecture.
3. Publish Version B on Zenodo with a DOI; link the code, derived aggregates, and demo.
4. If arXiv endorsement works, post there too; otherwise Zenodo is sufficient.