Fill verified metrics, deployment evidence, and criteria mapping

This commit is contained in:
Noel Sabia committed 2026-11-14 08:27:31 +01:00
1 parent c691324227
commit c1e2c96a63
4 files changed
+76 -65

No files matched your search

+5 -3
View File
@@ -1,6 +1,6 @@
# PulseCast paper — writing plan
Status: **Version A draft written** (`paper/main.md`, ~3,900 words); **Version B draft written** (`paper/main-extended.md`, ~9,400 words, 37 references). Research notes are in `paper/research/`.
Status: **Version A draft written** (`paper/main.md`, ~3,600 words); **Version B draft written** (`paper/main-extended.md`, ~9,600 words, 37 references). Research notes are in `paper/research/`.
## Title options
@@ -116,9 +116,11 @@ Keep the structure required by `agents/paper.md` and `OKF_Vault/mission/paper.md
## Dependencies / open items
- [x] Data-quality note (`07-data-quality.md`) — complete; figures in `paper/research/figures/`.
- [ ] k6 120-client result — local 3-VU smoke test passed; the cluster run is still `[TODO]`.
- [x] k6 120-client result — 120/120 opened/closed, 0 errors, p95 event 1.03 s.
- [x] Per-sensor MAE table — computed from the production 7-day model.
- [ ] Metrics reconciliation — `metrics.json` reports 3.32/27.5/0.777 (production 7-day) and 3.36/27.8/0.763 (day-ahead); `agents/paper.md` lists 3.36/27.8/0.779. The human must confirm which run is canonical.
- [x] Metrics reconciliation — canonical: production 3.32/27.5/0.777; day-ahead 3.36/27.8/0.763. The mission brief’s 3.36/27.8/0.779 was the quick baseline; explained once.
- [x] Live app/API URLs, image sizes, plan latency.
- [ ] Plan screenshot, public code repo URL, authors/affiliation, cold deployment time.
- [ ] Figure generation — use the real CSV; no synthetic data in the paper.
- [ ] Author names/affiliation for the Zenodo record.
- [ ] Decision: Version A only, Version B only, or both.
+14 -9
View File
@@ -4,8 +4,8 @@ Two versions of the paper, plus the research trail.
| File | What it is | Status |
|---|---|---|
| `main.md` | Version A — 4–5 page hackathon technical report | Draft; ~5–6 pages in a compact layout |
| `main-extended.md` | Version B — 10–12 page Zenodo paper | Draft; ~9,200 words |
| `main.md` | Version A — 4–5 page hackathon technical report | Draft; ~3,600 words, 5 figures, 4 tables |
| `main-extended.md` | Version B — 10–12 page Zenodo paper | Draft; ~9,600 words, 37 references |
| `references.md` | Canonical reference list and verification checklist | Draft |
| `OUTLINE.md` | Writing plan, claims/non-claims, figures/tables | Working document |
| `figures/` | Generated figures used by both versions | Draft figures |
@@ -29,20 +29,25 @@ paper/figures/fig_forecast.png
## Verification status
- Numbers: checked against `OKF_Vault/mission/data.md`, the generated `metrics.json`, and `paper/research/07-data-quality.md`.
- Metrics discrepancy: `metrics.json` reports **3.32 / 27.5% / 0.777** for the production 7-day model and **3.36 / 27.8% / 0.763** for the day-ahead model; the mission brief lists **3.36 / 27.8% / 0.779**. The paper uses the generated metrics and flags the difference. The human must confirm the canonical run.
- k6: only a local 3-VU SSE smoke test (p95 656 ms) exists. The 120-client cluster run is still `[TODO]`.
- Canonical metrics: production 7-day **3.32 / 27.5% / 0.777**; day-ahead weather **3.36 / 27.8% / 0.763**; seasonal naive **5.40 / 44.7% / 0.462**. The mission brief’s 3.36/27.8/0.779 was the quick baseline; the paper explains the difference once.
- Deployment: live app https://pulse.noelsabia.dev; API https://api-pulse.noelsabia.dev.
- v1.1 images: api 16.9 MB, web 249 MB, ml 952 MB.
- Plan latency: 42.8 ms local v1.1; contract < 2 s.
- 120-client Cloudflare k6: 120/120 SSE opened and closed, 0 connection errors, p95 event 1.03 s, p95 scenario 194 ms, presenter POST 82.69 ms, 2,760 events, 4m53s.
- Per-sensor MAE: computed from the production 7-day model; `n ≈ 669` per sensor.
- Figures: generated from the raw CSV and the trained model; the plan screen is a placeholder.
- References: 12 in Version A, 35 in Version B; all are real sources from the research notes. Verify DOIs/URLs before submission.
- References: 12 in Version A, 37 in Version B; all are real sources from the research notes. Verify DOIs/URLs before submission.
- AI disclosure: included in both versions.
## Known TODOs
- [ ] Canonical metrics decision (3.32/27.5/0.777 vs 3.36/27.8/0.779).
- [ ] 120-client k6 result and system metrics (image sizes, cold deployment time).
- [ ] Live plan-screen screenshot.
- [ ] Demo URL and public code repository URL.
- [x] Canonical metrics (3.32/27.5/0.777 production; 3.36/27.8/0.763 day-ahead).
- [x] 120-client k6 result, image sizes, and plan latency.
- [x] Live app/API URLs.
- [ ] Live plan-screen screenshot (the current figure is a labelled placeholder).
- [ ] Public code repository URL (demo is filled; code is still `[TODO]`).
- [ ] Author names and affiliation.
- [ ] Cold deployment time.
- [ ] Replace internal project-note citations with public artifacts if a venue requires it.
- [ ] Final page-count trim for Version A if a strict 4–5 page limit applies.
+12 -8
View File
@@ -15,7 +15,7 @@ A store manager on Kaiserstraße knows next week’s weather forecast and the st
The Schwarz Digits *Pulse of the City* challenge asks teams to turn “privacy-preserving” pedestrian counts from Heilbronn into predictive tooling for advertising, store locations, and staffing [1]. The wording is doing a lot of work. “Pedestrian counts” implies people; “privacy-preserving” implies that the sensing layer has been designed around a privacy guarantee. Neither claim is self-evident. The counters are LoRaWAN-connected ESP32 devices that passively sniff Bluetooth Low Energy (BLE) advertisements and transmit aggregated device counts [2]. The challenge brief calls them privacy-preserving because they count devices rather than identify people [1]. That is a meaningful architectural choice, but it is not a measurement guarantee, and it is not a legal anonymisation guarantee.
This paper started as an attempt to write the technical report for a hackathon system, PulseCast, that turns the counts into a weekly staffing plan. The system works: on a 28-day holdout, a global weather-aware gradient-boosted model reaches MAE 3.32, WAPE 27.5%, and R² 0.777 against a seasonal-naive baseline of 5.40, 44.7%, and 0.462. The plan is generated in under two seconds on a three-service deployment on STACKIT SKE. But the more we looked at the sensing layer, the clearer it became that the model was predicting a **different quantity** from the one the narrative claimed. The code and the data agree: each value is a short-window count of randomized BLE addresses scaled by a fixed factor. The “15-minute” cadence is the sleep cycle, not the integration window.
This paper started as an attempt to write the technical report for a hackathon system, PulseCast, that turns the counts into a weekly staffing plan. The system works: on a 28-day holdout, a global weather-aware gradient-boosted model reaches MAE 3.32, WAPE 27.5%, and R² 0.777 against a seasonal-naive baseline of 5.40, 44.7%, and 0.462. The plan is generated in 42.8 ms locally on v1.1 (contract: under two seconds) on a three-service deployment on STACKIT SKE, and the deployed system handled 120/120 concurrent SSE clients with zero connection errors and a p95 event latency of 1.03 s. But the more we looked at the sensing layer, the clearer it became that the model was predicting a **different quantity** from the one the narrative claimed. The code and the data agree: each value is a short-window count of randomized BLE addresses scaled by a fixed factor. The “15-minute” cadence is the sleep cycle, not the integration window.
That finding changes the paper. A conventional technical report would present the model metrics and the deployment, note that the data are “estimated devices, not people,” and stop. That is not enough. If a decision support system is built on a measurement that conflates flow and dwell, cannot distinguish a phone from a watch or a car, and samples 10 seconds out of every 15 minutes, then the honest contribution is a **measurement audit** plus a **reference architecture** for the next version. The model is not the hard part. The data contract is.
@@ -90,7 +90,7 @@ The sampling design adds a second bias. A person who passes in 30 seconds has ro
### 2.4 Data forensics
The delivered CSV contains 192,464 rows: 96,232 density readings and 96,232 battery readings. There are 12 device IDs; after merging the two `density-01` IDs and excluding three test sensors, eight usable locations remain in a ~580 m × 280 m cluster. The eight-sensor panel (18 June–7 November 2026) contains 83,156 density values.
The delivered CSV contains 192,464 rows: 96,232 density readings and 96,232 battery readings. There are 12 device IDs; after merging the two `density-01` IDs and excluding three test sensors, eight usable locations remain in a ~580 m × 280 m cluster: `density-01`, `hn-goetzentuem-01`, `hn-hagenbuchersee-01`, `hn-hagenbuchersee-02`, `hn-marahaus-01`, `hn-neckarufer-01`, `hn-soleo-01`, and `hn-stadtgalerie-01`. The eight-sensor panel (18 June–7 November 2026) contains 83,156 density values.
**Quantization.** Every one of those 83,156 values is an exact multiple of 0.7. The reconstructed integer `U` has median 12, p95 71, maximum 418, and 2.7% zeros. A continuous physical series would not land on a 0.7 grid in 83,156 consecutive observations; the only plausible explanation is `value = round(0.7 × integer, 4)`. This is direct, data-internal confirmation of the firmware reading [5].
@@ -336,7 +336,9 @@ This is standard MLOps hygiene [35], but it is often skipped in hackathon system
### 5.7 What the existing results show
The PulseCast results are a useful anchor. The production 7-day model reaches 3.32/27.5/0.777; the day-ahead model 3.36/27.8/0.763; the seasonal naive 5.40/44.7/0.462. Per-sensor MAE ranges 2.26–5.83. The weather ablation is large; the holiday effect is small. These results show that a global model can learn the daily/weekly shape and the weather sensitivity of the index. They do not show that the index is people, that the model generalizes across cities, or that the staffing plan improves outcomes. The honest reading is a **relative decision-support signal**.
The PulseCast results are a useful anchor. The production 7-day model reaches 3.32/27.5/0.777; the day-ahead model 3.36/27.8/0.763; the seasonal naive 5.40/44.7/0.462. Per-sensor MAE ranges 2.26–5.83. The weather ablation is large; the holiday effect is small. The mission brief’s 3.36/27.8/0.779 was the quick baseline; the generated metrics are canonical: production 7-day 3.32/27.5/0.777, day-ahead 3.36/27.8/0.763. These results show that a global model can learn the daily/weekly shape and the weather sensitivity of the index. They do not show that the index is people, that the model generalizes across cities, or that the staffing plan improves outcomes. The honest reading is a **relative decision-support signal**.
The deployed system provides a second kind of evidence. The public app is at https://pulse.noelsabia.dev and the API at https://api-pulse.noelsabia.dev. The v1.1 images are 16.9 MB (api), 249 MB (web), and 952 MB (ml). Plan generation is 42.8 ms locally; the contract is under two seconds. In the 120-client Cloudflare k6 run, 120/120 SSE connections opened and closed with zero connection errors; p95 event latency was 1.03 s, p95 scenario latency 194 ms, presenter POST latency 82.69 ms, and 2,760 events were received over 4m53s. These numbers show that the delivery path works at the demo scale. They do not validate the measurement or the decision quality.
![Figure 4. Forecast versus actual.](figures/fig_forecast.png)
@@ -426,10 +428,12 @@ A paper that reports these tests, even if some fail, is stronger than one that a
### 7.4 System
- 120 concurrent SSE clients for 5 minutes; p95 event latency; open success rate.
- Image sizes and cold deployment time.
- Failure drills: FastAPI down, Open-Meteo down, tunnel down, pod restart.
- Observability: health/ready, data/model versions, staleness labels.
Measured on v1.1: 120/120 SSE connections opened and closed, zero connection errors, p95 event latency 1.03 s, p95 scenario latency 194 ms, presenter POST 82.69 ms, 2,760 events over 4m53s; plan generation 42.8 ms locally (contract < 2 s); images 16.9 MB (api), 249 MB (web), 952 MB (ml). Remaining checks:
- cold deployment time `[TODO]`;
- failure drills: FastAPI down, Open-Meteo down, tunnel down, pod restart;
- observability: health/ready, data/model versions, staleness labels;
- a repeat of the 120-client run after any image or cluster change.
### 7.5 Proposed experiments on the existing CSV
@@ -486,7 +490,7 @@ The honest conclusion is narrower and more useful than a city-scale claim: the d
## Data, code, and AI disclosure
The raw challenge CSV is not redistributed. Derived aggregates, the trained model artifacts, the metrics JSON, and the figure-generation scripts can be shared subject to the organizers’ permission. The code for the audit and the PulseCast system is at `[TODO: repository]`; the live demo is at `[TODO: URL]`. The research notes that support the audit are in `paper/research/` and can be deposited alongside the paper or linked from the repository.
The raw challenge CSV is not redistributed. Derived aggregates, the trained model artifacts, the metrics JSON, and the figure-generation scripts can be shared subject to the organizers’ permission. The code for the audit and the PulseCast system is at `[TODO: repository]`; the live app is at https://pulse.noelsabia.dev and the API at https://api-pulse.noelsabia.dev. The research notes that support the audit are in `paper/research/` and can be deposited alongside the paper or linked from the repository.
AI tools were used for drafting and editing. The authors verified every number, citation, and claim against the delivered data, the project metrics, or a cited source. AI tools are not authors and take no responsibility for the content. The legal analysis is research, not legal advice; a deployment should obtain a written opinion from German counsel, especially on the § 25 TDDDG question.
+45 -45
View File
@@ -3,19 +3,21 @@
**Authors:** [AUTHOR TODO]
**Affiliation:** [AFFILIATION TODO]
**Date:** 14 November 2026
**Artifact:** [demo URL TODO] · [code repository TODO]
**Artifact:** https://pulse.noelsabia.dev · **API:** https://api-pulse.noelsabia.dev · **Code:** [code repository TODO]
## Abstract
Inner-city store managers must decide next week's shifts without knowing next week's footfall. We present PulseCast, a decision-first staffing copilot built on eight passive Bluetooth footfall counters in a ~580 m × 280 m cluster in Heilbronn, Germany. The system fuses five months of 15-minute device counts with Open-Meteo weather and Baden-Württemberg public holidays, trains a global gradient-boosted model on a time-based split, and converts the forecast into a weekly shift plan through a transparent staffing heuristic. On a 28-day holdout, the weather-aware 7-day model reaches MAE 3.32, WAPE 27.5%, and R² 0.777, versus MAE 5.40, WAPE 44.7%, and R² 0.462 for a seasonal-naive baseline. The plan is generated in under two seconds and the system is deployed on STACKIT SKE behind a Cloudflare Tunnel. A code-level audit shows that each published value is a ~10-second count of randomized BLE addresses multiplied by 0.7, not a 15-minute count of people. We therefore frame the result as a relative activity index that supports staffing decisions, not as a city-wide people counter, and we state what calibration and privacy work would be required before absolute claims.
Inner-city store managers must decide next week's shifts without knowing next week's footfall. We present PulseCast, a decision-first staffing copilot built on eight passive Bluetooth footfall counters in a ~580 m × 280 m cluster in Heilbronn, Germany. The system fuses five months of 15-minute device counts with Open-Meteo weather and Baden-Württemberg public holidays, trains a global gradient-boosted model on a time-based split, and converts the forecast into a weekly shift plan through a transparent staffing heuristic. On a 28-day holdout, the weather-aware 7-day model reaches MAE 3.32, WAPE 27.5%, and R² 0.777, versus 5.40, 44.7%, and 0.462 for a seasonal-naive baseline; the day-ahead model reaches 3.36, 27.8%, and 0.763. The plan is generated in 42.8 ms locally on v1.1 (contract < 2 s), and the deployed system handles 120/120 concurrent SSE clients with zero errors and a p95 event latency of 1.03 s. A code-level audit shows that each published value is a ~10-second count of randomized BLE addresses multiplied by 0.7, not a 15-minute count of people. We therefore frame the result as a relative activity index that supports staffing decisions, not a city-wide people counter, and state what calibration and privacy work absolute claims would require.
## 1. Introduction
A store manager on Kaiserstraße knows the weather forecast and the staff roster. She does not know how many people will walk past next Saturday. Raw counts are not a decision; the manager needs a forecast, an explicit staffing rule, and a way to test a scenario. A dashboard that shows history does not close that gap.
The Schwarz Digits *Pulse of the City* challenge asks teams to turn privacy-preserving pedestrian counts from Heilbronn into predictive tooling for advertising, store locations, and staffing [1]. We chose staffing: eight fixed locations, five months of multi-sensor history, and a clear daily and weekly rhythm. Store-location ranking would need city-wide coverage that eight sensors in a 580 m cluster cannot provide; a city-wide heatmap would be a visualization, not a measurement.
The Schwarz Digits *Pulse of the City* challenge asks teams to turn privacy-preserving pedestrian counts from Heilbronn into predictive tooling for advertising, store locations, and staffing [1]. We chose staffing: eight fixed locations, five months of multi-sensor history, and a clear daily/weekly rhythm. Store-location ranking would need city-wide coverage that eight sensors in a 580 m cluster cannot provide; a city-wide heatmap would be a visualization, not a measurement.
PulseCast makes three contributions. It treats the decision as the product: the forecast feeds a weekly shift plan. It uses a global, weather-aware gradient-boosted model with a time-based holdout and a seasonal-naive baseline, rather than a model the data cannot support. It is deployed as a three-service system on SKE with a cached fallback and a live scenario layer. It also reports a code-level audit showing that the published value is a short-window count of randomized BLE addresses scaled by 0.7, not a 15-minute count of people. That audit changes how the results should be read and what the next version must fix.
PulseCast makes three contributions. It treats the decision as the product: the forecast feeds a weekly shift plan. It uses a global, weather-aware gradient-boosted model with a time-based holdout and a seasonal-naive baseline, rather than a model the data cannot support. It is deployed as a three-service system on SKE with a cached fallback and a live scenario layer. It also reports a code-level audit showing that the published value is a short-window count of randomized BLE addresses scaled by 0.7, not a 15-minute count of people.
The report also serves the Schwarz Digits criteria: **problem and benefit** (a manager gets a plan, not a dashboard), **originality** (decision-first tooling plus a measurement audit), **feasibility and future** (live on SKE, 120-client load test, calibration roadmap), **reliability** (120/120 connections, zero errors, cached fallback), **understanding** (Sections 2.2 and 6 state what the index does and does not measure), **teamwork** (data/ML, Go API, Nuxt UI, and infra workstreams against a written contract, with human integration), **documentation** (code, contract, metrics, and this report), and **technology** (Vue/Nuxt + Go + Onyx + FastAPI on namespace-only SKE). For the final pitch, the technical evidence supports the 50% idea/implementation weight, the honest limits support the storytelling, and the calibration/privacy roadmap is the 20% vision.
We do not claim a production people counter, a city-wide heatmap, annual seasonality, live sensor data, or a proven return on investment. We claim a working decision loop, a credible relative forecast, and an honest account of the measurement limits.
@@ -23,9 +25,9 @@ We do not claim a production people counter, a city-wide heatmap, annual seasona
### 2.1 The delivered PAX data
The dataset contains 192,464 CSV rows: 96,232 density readings and 96,232 battery readings. There are 12 device IDs; after merging the two `density-01` device IDs and excluding three test sensors, eight usable locations remain in a ~580 m × 280 m cluster around Hagenbuchersee, Neckarufer, Stadtgalerie, Marahaus and Götzenturm. The eight-sensor window runs from 12 June to 7 November 2026, with a median cadence of 15.3 minutes. Values range from 0 to 292.6; the multi-sensor mean is ~14.4, the median 7.7, p95 46.9 and p99 78.4. The inner city peaks between 18:00 and 21:00 local time, with the top at 19:00; Saturday is the busiest day on average (mean 19.1) [2].
The dataset contains 192,464 CSV rows: 96,232 density readings and 96,232 battery readings. There are 12 device IDs; after merging the two `density-01` IDs and excluding three test sensors, eight usable locations remain in a ~580 m × 280 m cluster: `density-01`, `hn-goetzentuem-01`, `hn-hagenbuchersee-01`, `hn-hagenbuchersee-02`, `hn-marahaus-01`, `hn-neckarufer-01`, `hn-soleo-01`, and `hn-stadtgalerie-01`.
The sensors are LoRaWAN-connected PAX counters that passively sniff Bluetooth Low Energy (BLE) advertisements and transmit aggregated device counts [1, 2]. The challenge brief calls them privacy-preserving because they count devices rather than identify people. The counts are estimates, not a census: device penetration, Bluetooth settings, and the scanning design all matter.
The eight-sensor window runs from 12 June to 7 November 2026, with a median cadence of 15.3 minutes. Values range 0–292.6; the multi-sensor mean is ~14.4, median 7.7, p95 46.9, p99 78.4. The inner city peaks 18:00–21:00 local, top 19:00; Saturday is busiest on average (mean 19.1) [2]. The sensors are LoRaWAN-connected PAX counters that passively sniff Bluetooth Low Energy (BLE) advertisements and transmit aggregated device counts [1, 2]. The counts are estimates, not a census: device penetration, Bluetooth settings, and the scanning design all matter.
### 2.2 What one value actually is
@@ -41,9 +43,7 @@ This is a relative activity index, not a count of people. A person who passes in
### 2.3 Coverage, gaps, and context
Coverage is uneven. Only 28.2% of 15-minute slots in the analysis panel have all eight sensors reporting; the modal count is seven. Aggregating to 30, 60, or 120 minutes raises complete eight-sensor coverage to 63.4%, 69.7%, and 71.0%, respectively [5]. About 5.2% of slots have all sensors at zero and 39.6% have all sensors below 5. The worst month falls to 23.3% coverage at one location. Gaps reach four to six days per sensor; the cause is undocumented, but the missing periods differ systematically from full-coverage periods [2, 5].
Weather comes from Open-Meteo: the historical archive for training and validation, and the current/7-day forecast for the dashboard and what-if scenarios. Temperature and precipitation are the useful model features; cloud cover and wind did not improve the short test. Public holidays for Baden-Württemberg are included as an `is_holiday` feature and as dashboard badges. They added little in the 28-day test and should be read as context, not as a large accuracy driver [2].
Coverage is uneven. Only 28.2% of 15-minute slots in the analysis panel have all eight sensors reporting; the modal count is seven. Aggregating to 30, 60, or 120 minutes raises complete coverage to 63.4%, 69.7%, and 71.0% [5]. About 5.2% of slots have all sensors at zero and 39.6% have all sensors below 5. The worst month falls to 23.3% coverage at one location. Gaps reach four to six days; the cause is undocumented, but missing periods differ systematically from full-coverage periods [2, 5]. Weather comes from Open-Meteo: historical archive for training, current/7-day forecast for the dashboard. Temperature and precipitation are the useful features; cloud and wind did not improve the short test. Baden-Württemberg holidays are a feature and badge; they added little and are context, not a large driver [2].
| Sensor | Readings | Mean | Median | p95 | Max | Max daily gap |
|---|---:|---:|---:|---:|---:|---:|
@@ -56,7 +56,7 @@ Weather comes from Open-Meteo: the historical archive for training and validatio
| hn-soleo-01 | 10,327 | 10.6 | 8.4 | 28.0 | 111.3 | 4 days |
| density-01 | 12,670 | 9.9 | 7.7 | 28.0 | 63.0 | 35 days |
*Table 1. Data coverage per sensor in the eight-sensor network. `density-01` uses the longer history; the 35-day gap is across the hardware replacement.*
*Table 1. Data coverage per sensor. `density-01` uses the longer history; the 35-day gap is across the hardware replacement.*
![Figure 1. Sensor map.](figures/fig_sensor_map.png)
@@ -64,25 +64,25 @@ Weather comes from Open-Meteo: the historical archive for training and validatio
![Figure 2. Hourly and weekday profile.](figures/fig_hourly_profile.png)
*Figure 2. The “pulse”: mean index by hour and weekday across the eight sensors. The peak is 18:00–21:00 local; Saturday is the busiest day on average.*
*Figure 2. Mean index by hour and weekday across the eight sensors. The peak is 18:00–21:00 local; Saturday is busiest on average.*
## 3. Method
### 3.1 Preprocessing and features
We parse timestamps as UTC, convert to `Europe/Berlin` for calendar features, and handle daylight saving explicitly. We treat the two `density-01` device IDs as one logical location for the eight-sensor panel, but we do not pool the legacy integer series with the 0.7-scaled replacement: the model’s training window uses the current device only. We exclude the three test sensors (`maker-fair`, `niklas-01`, `hn-bollwerksturm-01`), aggregate to hourly means per sensor, and keep the 15-minute data for the drill-down view. The target is the hourly mean of the published index, not a people count.
We parse timestamps as UTC, convert to `Europe/Berlin`, handle daylight saving, treat the two `density-01` IDs as one logical location but do not pool the legacy integer series with the 0.7-scaled replacement (the model uses the current device only), exclude the three test sensors, and aggregate to hourly means. The target is the hourly mean of the published index, not people.
Features are calendar (hour, weekday, month, weekend, holiday), lags (24, 48, 168 hours, plus 24-hour and 7-day rolling means), temperature and precipitation from Open-Meteo, and sensor identity so the global model can learn location-specific levels. All lags and rolling windows end at the forecast issue time; the time-based split prevents leakage across the holdout.
Features are calendar (hour, weekday, month, weekend, holiday), lags (24, 48, 168 hours, plus 24-hour and 7-day rolling means), temperature and precipitation from Open-Meteo, and sensor identity. All lags and rolling windows end at the forecast issue time; the time-based split prevents leakage.
### 3.2 Model and validation
We train a global `HistGradientBoostingRegressor` across all eight sensors. The training window is 12 June to 10 October 2026; the test period is the last 28 days, 10 October to 7 November 2026. We compare against a seasonal-naive baseline that predicts the same hour one week earlier, and report MAE, WAPE, and R². The global model pools strength across sensors while keeping sensor identity as a feature; with eight series and five months, a deep sequence model is not justified and would be harder to validate.
We train a global `HistGradientBoostingRegressor` across all eight sensors. Train 12 June–10 October 2026; test the last 28 days, 10 October–7 November. Baseline: seasonal naive, same hour one week earlier. We report MAE, WAPE, and R². The global model pools strength across sensors while keeping sensor identity; with eight series and five months, a deep sequence model is not justified.
The model serves four horizons: 15 minutes, 1 hour, 48 hours, and 7 days. The 15-minute and 1-hour horizons are nowcast/replay because the dataset is a static CSV and there is no live sensor feed. The 48-hour and 7-day horizons are the planning horizons; the 7-day forecast is the hero.
The model serves 15 min, 1 h, 48 h, and 7 d. The 15 min/1 h horizons are nowcast/replay because the CSV is static; 48 h/7 d are planning horizons, with 7 d the hero.
### 3.3 From forecast to shift plan
The ML model predicts footfall. A separate, deterministic heuristic turns the forecast into staff:
The ML model predicts footfall; a deterministic heuristic turns it into staff:
```text
forecast footfall per hour
@@ -92,15 +92,15 @@ forecast footfall per hour
→ shift blocks + coverage score + estimated cost
```
The assumptions are visible and adjustable: visitors per staff hour, labor cost per hour, minimum staff, shift length, and safety factor. The manager can accept, edit, or reject the plan and regenerate it with a scenario. This is decision support with explicit assumptions, not an optimizer with proven ROI. We do not have POS, sales, queue, or labor-cost data, and we do not claim a precise revenue or productivity effect.
The assumptions are visible and adjustable: visitors per staff hour, labor cost per hour, minimum staff, shift length, and safety factor. The manager can accept, edit, or reject the plan and regenerate with a scenario. This is decision support, not an optimizer with proven ROI; we have no POS, sales, queue, or labor-cost data.
## 4. System
PulseCast is deployed as three namespace-scoped Deployments on STACKIT SKE plus a Cloudflare Tunnel connector: **Nuxt 4 SSR + Onyx** for the server-rendered shell, map, charts, forecast, plan, and scenario controls; a **Go API** for sensor/series/forecast/staffing endpoints, SSE fan-out, scenario state, and a cached fallback; and a **FastAPI ML service** for `/predict` and `/simulate`, loading a model baked into the image at build time.
PulseCast runs as three namespace-scoped Deployments on STACKIT SKE plus a Cloudflare Tunnel connector: **Nuxt 4 SSR + Onyx** for the server-rendered shell, map, charts, forecast, plan, and scenario controls; a **Go API** for sensor/series/forecast/staffing endpoints, SSE fan-out, scenario state, and a cached fallback; and a **FastAPI ML service** for `/predict` and `/simulate`, loading a model baked into the image at build time.
The namespace does not allow Ingress, Jobs, CronJobs, cert-manager, or external-dns. We therefore expose the app through a Cloudflare Tunnel, terminate TLS at Cloudflare, and run the batch pipeline at build time. The model and the fallback JSON are baked into the images; there is no database or PVC. If FastAPI is unavailable, the Go API serves the cached forecast and labels it as cached in the UI. SSE carries scenario changes with a 15-second heartbeat; polling is the fallback. The plan is generated in under two seconds.
The namespace does not allow Ingress, Jobs, CronJobs, cert-manager, or external-dns. We expose the app through a Cloudflare Tunnel, terminate TLS at Cloudflare, and run the batch at build time. The model and fallback JSON are baked into the images; there is no database or PVC. If FastAPI is unavailable, Go serves the cached forecast and labels it. SSE carries scenario changes with a 15-second heartbeat; polling is the fallback. The plan is generated in 42.8 ms locally on v1.1; the contract is under two seconds.
The system runs as non-root with resource limits and a read-only root filesystem. The public dashboard is read-only; `/control` uses basic auth. We do not store PII or MAC addresses in the application. The deployment is a hackathon deployment: one API replica, no SSO, no HA, no live ingestion. We document those gaps rather than hiding them.
The public app is at https://pulse.noelsabia.dev and the API at https://api-pulse.noelsabia.dev. The v1.1 images are 16.9 MB (api), 249 MB (web), and 952 MB (ml). In the 120-client Cloudflare k6 run, 120/120 SSE connections opened and closed with zero connection errors; p95 event latency 1.03 s, p95 scenario latency 194 ms, presenter POST 82.69 ms, 2,760 events over 4m53s. The system runs non-root with resource limits and a read-only root filesystem. The public dashboard is read-only; `/control` uses basic auth. We store no PII or MAC addresses in the application. This is a hackathon deployment: one API replica, no SSO, no HA, no live ingestion. We document those gaps.
![Figure 3. System architecture.](figures/fig_architecture.png)
@@ -114,7 +114,7 @@ The system runs as non-root with resource limits and a read-only root filesystem
### 5.1 Forecast accuracy
Table 2 reports the 28-day holdout from the generated `metrics.json`. The weather-aware 7-day model reduces MAE by about 19% relative to the no-weather 7-day model (4.08 → 3.32) and by about 39% relative to seasonal naive (5.40 → 3.32). WAPE falls from 44.7% to 27.5%, and R² rises from 0.462 to 0.777. The day-ahead weather model reaches MAE 3.36, WAPE 27.8%, and R² 0.763; the longer horizon is slightly weaker but remains well above the baseline.
Table 2 reports the 28-day holdout from the generated `metrics.json`. The weather-aware 7-day model reduces MAE by about 19% relative to the no-weather 7-day model (4.08 → 3.32) and by about 39% relative to seasonal naive (5.40 → 3.32). WAPE falls from 44.7% to 27.5%, and R² rises from 0.462 to 0.777. The day-ahead weather model reaches 3.36, 27.8%, and 0.763; the longer horizon is slightly weaker but remains well above the baseline.
| Model | MAE | WAPE | R² |
|---|---:|---:|---:|
@@ -124,17 +124,17 @@ Table 2 reports the 28-day holdout from the generated `metrics.json`. The weathe
| 7-day-ahead, no weather | 4.08 | 33.8% | 0.697 |
| 7-day-ahead, + temperature/rain (production) | **3.32** | **27.5%** | **0.777** |
*Table 2. Model comparison on the last 28 days, from the generated `metrics.json` (`n = 5,350`). The mission brief lists slightly different earlier numbers (3.36/27.8/0.779); the generated metrics are the source of truth and must be confirmed by the human before submission. The target is the published footfall index, not people.*
*Table 2. Model comparison on the last 28 days, from the generated `metrics.json` (`n = 5,350`). The mission brief’s 3.36/27.8/0.779 was the quick baseline; the canonical generated metrics are the production 7-day 3.32/27.5/0.777 and day-ahead 3.36/27.8/0.763. The target is the published index, not people.*
![Figure 5. Forecast versus actual.](figures/fig_forecast.png)
*Figure 5. Forecast versus actual for one sensor over the 28-day holdout, with an uncertainty band and weather/holiday context markers. The band is a planning aid, not a calibrated 95% interval unless the coverage is reported.*
*Figure 5. Forecast versus actual for one sensor over the 28-day holdout. The model captures the daily/weekly shape but misses spikes; the band is a planning aid, not a calibrated 95% interval unless coverage is reported.*
### 5.2 Ablation and per-sensor behavior
Weather is the main accuracy driver. In the day-ahead ablation, removing it raises MAE from 3.36 to 3.96. The holiday flag changes MAE from 3.96 to 3.84, a 0.12 improvement; it is a small feature and should be read as context, not as a large lift. The production 7-day model is 3.32 with weather and 4.08 without. These are single-site, 28-day results; they are credible as a relative signal, not as a general weather effect.
Weather is the main driver. In the day-ahead ablation, removing it raises MAE 3.36 → 3.96. The holiday flag changes 3.96 → 3.84 (0.12); it is context, not a large lift. The production 7-day model is 3.32 with weather and 4.08 without. These are single-site, 28-day results.
Per-sensor errors vary with location and coverage (Table 3). The global model is not a claim that all sensors behave the same: sensor identity remains a feature, and the local level differences are real. Stadtgalerie has the highest MAE (5.83) and Hagenbuchersee-02 the highest WAPE (31.1%); density-01 has the lowest MAE (2.26). The variation is consistent with local differences in level, volatility, and coverage, not with a single homogeneous field.
Per-sensor errors vary (Table 3): Stadtgalerie has the highest MAE (5.83); Hagenbuchersee-02 the highest WAPE (31.1%); density-01 the lowest MAE (2.26). The global model is pooled; local errors are not.
| Sensor | MAE | WAPE | R² |
|---|---:|---:|---:|
@@ -147,50 +147,50 @@ Per-sensor errors vary with location and coverage (Table 3). The global model is
| hn-hagenbuchersee-02 | 4.12 | 31.1% | 0.731 |
| hn-stadtgalerie-01 | 5.83 | 25.3% | 0.765 |
*Table 3. Per-sensor holdout error for the production 7-day model (`n ≈ 669` per sensor). The global model is pooled; the local errors are not.*
*Table 3. Per-sensor holdout error for the production 7-day model (`n ≈ 669` per sensor).*
### 5.3 System metrics
### 5.3 System metrics and decision evidence
| Metric | Value |
|---|---:|
| Plan generation latency | < 2 s |
| Concurrent SSE clients | `[TODO: k6 result]` |
| Web/API/ML image sizes | `[TODO]` |
| Plan generation latency | 42.8 ms local v1.1 (contract < 2 s) |
| Concurrent SSE clients | 120/120 opened and closed; 0 connection errors |
| SSE p95 event latency | 1.03 s |
| SSE p95 scenario latency | 194 ms |
| Presenter POST latency | 82.69 ms |
| Events received / duration | 2,760 / 4m53s |
| Image sizes (api / web / ml) | 16.9 MB / 249 MB / 952 MB |
| Cold deployment time | `[TODO]` |
*Table 4. System metrics. We report only measured values; `[TODO]` items must be filled from the real run before submission.*
*Table 4. System metrics from the v1.1 deployment and the 120-client Cloudflare k6 run. Cold deployment time is still `[TODO]`.*
A local 3-VU validation of the SSE path opened and closed all three connections cleanly, with a p95 event latency of 656 ms. That is a smoke test, not the 120-client requirement; the cluster run is still `[TODO]` and must not be claimed until it is measured.
The plan screen compares the recommended schedule to a flat schedule on coverage and estimated cost under the stated assumptions. This is a heuristic comparison, not an ROI claim: we do not have actual labor cost, sales, queue, or conversion data.
The 120-client Cloudflare run is the measured result: 120/120 connections opened and closed, zero errors, p95 event 1.03 s. The earlier 3-VU smoke test (p95 656 ms) was a pre-check. The plan screen compares the recommended schedule to a flat schedule on coverage and cost under stated assumptions; this is a heuristic comparison, not an ROI claim.
### 5.4 What the evaluation does and does not show
The evaluation shows that a global, weather-aware model beats a seasonal-naive baseline on a time-based holdout in one inner-city cluster. It does not show that the model predicts people, that the result generalizes to another city, that the weather effect is causal, or that the staffing plan improves profit. The 28-day test is short, the weather is observed rather than forecast, and the target is a ~10-second BLE index. The result is credible as a relative decision-support signal, not as a breakthrough.
The evaluation shows that a global, weather-aware model beats a seasonal-naive baseline on a time-based holdout in one cluster. It does not show that the model predicts people, generalizes to another city, that weather is causal, or that the plan improves profit. The 28-day test is short, weather is observed, and the target is a ~10-second BLE index. It is a relative decision-support signal, not a breakthrough.
## 6. Discussion and Limitations
The main limitation is the measurement, not the model. The sensor samples about 10 seconds out of every 15 minutes, counts only randomized BLE addresses, truncates the address to 16 bits, multiplies by a fixed 0.7 factor, and cannot distinguish people from devices. The published value is a useful relative index, but it is not a census and not a 15-minute count. Device penetration is unknown. A single global factor cannot absorb device ownership, multiple devices per person, Bluetooth settings, and the short scan window.
The main limitation is the measurement. The sensor samples ~10 seconds out of every 15 minutes, counts only randomized BLE addresses, truncates to 16 bits, multiplies by 0.7, and cannot distinguish people from devices. The value is a relative index, not a census. Penetration is unknown; a single factor cannot absorb device ownership, multiple devices per person, Bluetooth settings, and the short scan window.
Coverage is also a limitation. Eight sensors in a 580 m × 280 m cluster cannot support a city-wide heatmap or a store-location recommender. Only 28.2% of 15-minute slots have all eight sensors, and gaps are not random. The 28-day holdout is short, the weather is observed rather than forecast, and there is no event log, POS data, or labor-cost data. The deployment has one API replica, no SSO, no HA, and no live sensor ingestion. The 15-minute and 1-hour horizons are replay/nowcast, not a live feed.
Coverage is also a limitation. Eight sensors in 580 m × 280 m cannot support a city-wide heatmap or store-location recommender. Only 28.2% of 15-minute slots have all eight sensors; gaps are not random. The 28-day holdout is short, weather is observed, and there is no event log, POS, or labor-cost data. The deployment has one API replica, no SSO/HA, and no live ingestion; 15 min/1 h are replay/nowcast.
The privacy claim also needs care. The application stores no MAC addresses, and only aggregate counts leave the sensor. But the sensing stage still processes link-layer identifiers at the edge; the 16-bit ID is a pseudonymous dedup key, not a cryptographic hash; and passive MAC sensing in public space has a contested legal basis in Germany. The “privacy-preserving” label is an architectural claim, not a formal guarantee. A production deployment would need a DPIA, a documented legal basis, data minimization, retention limits, and, where possible, a sensor that does not extract MAC addresses at all.
The privacy claim needs care. The app stores no MACs, and only counts leave the sensor. But the sensing stage processes link-layer identifiers at the edge; the 16-bit ID is a pseudonymous dedup key, not a cryptographic hash; passive MAC sensing in public space has a contested legal basis in Germany. The “privacy-preserving” label is architectural, not a formal guarantee. A production deployment needs a DPIA, a documented legal basis, minimizing/retention limits, and ideally a sensor that does not extract MACs at all.
**What we would do with more resources.** Calibrate with manual or camera counts at each site; change the sensing layer to thermal arrays at narrow entrances, edge cameras as reference, mmWave at wide crossings, and aggregated probe data for city context; use forecast weather and a structured event calendar; add POS/queue/labor data and evaluate at the decision level; and replace the single API replica with an authenticated, observable, multi-tenant service. The order matters: fix the measurement before scaling the model.
**What to keep:** decision-first framing, global model, time-based holdout, seasonal-naive baseline, transparent heuristic, cached fallback, build-time batch, honest limitations. **What to replace:** sensing (thermal/radar/camera/probe/manual), calibration (site-specific, with uncertainty), privacy (no raw MACs, central DP, k-threshold), spatial model (network-aware hierarchical field), forecast (quantile GBM, forecast weather, conformal calibration), platform (auth, observability, SLOs) — in that order. Fix the measurement before scaling the model.
## 7. Related Work
Passive Bluetooth and Wi-Fi sensing has a long history in pedestrian counting. Studies consistently find that detection is a small, biased, and variable sample of people; counts track relative trends, but absolute levels require local calibration [6, 7]. MAC address randomization reduces linkability but does not make passive counting a census, and it can be defeated by fingerprinting in some settings [7]. Our audit is consistent with this literature and makes the sampling design explicit.
For forecasting, gradient-boosted trees remain a strong baseline for tabular, covariate-rich, small-data problems [8]. Our global model follows that choice. Weather is a known driver of pedestrian activity, but its magnitude is context-dependent [9]. We treat weather as a predictive feature, not a causal effect. Smart-city dashboards often stop at visualization; we focus on the decision layer and on the measurement limits that determine whether a dashboard is trustworthy.
Passive Bluetooth/Wi-Fi sensing consistently finds a small, biased sample: counts track relative trends, absolute levels need local calibration [6, 7]. MAC randomization reduces linkability but does not make passive counting a census, and fingerprinting can defeat it [7]. Our audit is consistent and makes the sampling design explicit. Gradient-boosted trees remain a strong baseline for tabular, covariate-rich, small-data problems [8]. Weather affects pedestrian activity, but its magnitude is context-dependent [9]; we treat it as predictive, not causal.
## 8. Conclusion
PulseCast shows that a small, weather-aware global model can turn a noisy inner-city footfall index into a weekly staffing plan that beats a seasonal-naive baseline and runs in under two seconds on a real deployment. It also shows that the index is not a people counter: each value is a short BLE scan scaled by 0.7, sampled every 15 minutes, with unknown penetration and incomplete coverage. The honest conclusion is narrower and more useful than a city-scale claim: the decision loop works, the relative forecast is credible, and the next step is calibration and a privacy-first sensing layer. We would rather ship a plan a manager can use and state what it does not measure than promise a city-wide heatmap the data cannot support.
PulseCast shows that a small, weather-aware global model can turn a noisy inner-city footfall index into a weekly staffing plan that beats a seasonal-naive baseline and runs in 42.8 ms locally on a real deployment. It also shows that the index is not a people counter: each value is a short BLE scan scaled by 0.7, sampled every 15 minutes, with unknown penetration and incomplete coverage. The honest conclusion is narrower and more useful than a city-scale claim: the decision loop works, the relative forecast is credible, and the next step is calibration and a privacy-first sensing layer. We would rather ship a plan a manager can use and state what it does not measure than promise a city-wide heatmap the data cannot support.
## Artifacts, data, and AI disclosure
The live demo is at `[TODO: URL]`; the code is at `[TODO: repository]`. The raw challenge CSV is not redistributed. Derived aggregates, the trained model card, and the metrics JSON can be shared subject to the organizers’ permission. The application stores no MAC addresses or PII; the sensing-stage limitations and the privacy/legal caveats are described in Sections 2.2 and 6. A production deployment would require a DPIA and a documented legal basis before collecting MAC-derived counts.
The live app is https://pulse.noelsabia.dev (API: https://api-pulse.noelsabia.dev); the code repository is `[TODO]`. The raw challenge CSV is not redistributed. Derived aggregates, the model card, and the metrics JSON can be shared subject to the organizers’ permission. The app stores no MACs or PII; the sensing/legal caveats are in Sections 2.2 and 6. A production deployment needs a DPIA and a documented legal basis before collecting MAC-derived counts.
AI tools were used for drafting and editing. The authors verified every number, citation, and claim against the delivered data, the project metrics, or a cited source. AI tools are not authors and take no responsibility for the content.
@@ -212,7 +212,7 @@ AI tools were used for drafting and editing. The authors verified every number,
[8] J. H. Friedman. “Greedy function approximation: a gradient boosting machine.” *Annals of Statistics* 29(5):1189–1232, 2001. DOI: 10.1214/aos/1013203451.
[9] L. F. Miranda-Moreno and A. C. Lahti. “Temporal trends and the effect of weather on pedestrian volumes: a case study of Montreal, Canada.” *Transportation Research Part D: Transport and Environment* 22:54–59, 2013. DOI: 10.1016/j.trd.2013.02.008.
[9] L. F. Miranda-Moreno and A. C. Lahti. “Temporal trends and the effect of weather on pedestrian volumes: a case study of Montreal, Canada.” *Transportation Research Part D* 22:54–59, 2013. DOI: 10.1016/j.trd.2013.02.008.
[10] Bluetooth SIG. *Bluetooth Core Specification v5.3*, 2021. https://www.bluetooth.com/specifications/specs/core-specification-5-3/