Apply fact-check corrections: v1.1 metrics, data populations, citations
This commit is contained in:
1 parent
83b73367d8
commit
ae45c1031f
4 files changed
+59
-56
No files matched your search
@@ -7,7 +7,7 @@
|
||||
|
||||
## Abstract
|
||||
|
||||
Inner-city store managers must decide next week's shifts without knowing next week's footfall. We present PulseCast, a decision-first staffing copilot built on eight passive Bluetooth footfall counters in a ~580 m × 280 m cluster in Heilbronn, Germany. The system fuses five months of 15-minute device counts with Open-Meteo weather and Baden-Württemberg public holidays, trains a global gradient-boosted model on a time-based split, and converts the forecast into a weekly shift plan through a transparent staffing heuristic. On a 28-day holdout, the weather-aware 7-day model reaches MAE 3.32, WAPE 27.5%, and R² 0.777, versus 5.40, 44.7%, and 0.462 for a seasonal-naive baseline; the day-ahead model reaches 3.36, 27.8%, and 0.763. The plan is generated in 42.8 ms locally on v1.1 (contract < 2 s), and the deployed system handles 120/120 concurrent SSE clients with zero errors and a p95 event latency of 1.03 s. A code-level audit shows that each published value is a ~10-second count of randomized BLE addresses multiplied by 0.7, not a 15-minute count of people. We therefore frame the result as a relative activity index that supports staffing decisions, not a city-wide people counter, and state what calibration and privacy work absolute claims would require.
|
||||
Inner-city store managers must decide next week's shifts without knowing next week's footfall. We present PulseCast, a decision-first staffing copilot built on eight passive Bluetooth footfall counters in a ~580 m × 280 m cluster in Heilbronn, Germany. The system fuses five months of 15-minute device counts with Open-Meteo weather and Baden-Württemberg public holidays, trains a global gradient-boosted model on a time-based split, and converts the forecast into a weekly shift plan through a transparent staffing heuristic. On a 28-day holdout, the weather-aware 7-day model reaches MAE 3.32, WAPE 27.5%, and R² 0.777, versus 5.40, 44.7%, and 0.462 for a seasonal-naive baseline; the day-ahead model reaches 3.36, 27.8%, and 0.763. The plan is generated in 19 ms cold on v1.1 (contract < 2 s), and the deployed system handles 120/120 concurrent SSE clients with zero errors and a p95 event latency of 331 ms. A code-level audit of the public 2026 firmware commit e8be8bf shows a ~10-second scan, a 16-bit dedup key, and a 0.7 multiplier; the deployed 2025 binary may differ, and the data confirm the 0.7 scaling and per-wake export, not the scan duration or truncation. We therefore frame the result as a relative activity index that supports staffing decisions, not a city-wide people counter, and state what calibration and privacy work absolute claims would require.
|
||||
|
||||
## 1. Introduction
|
||||
|
||||
@@ -27,23 +27,21 @@ We do not claim a production people counter, a city-wide heatmap, annual seasona
|
||||
|
||||
The dataset contains 192,464 CSV rows: 96,232 density readings and 96,232 battery readings. There are 12 device IDs; after merging the two `density-01` IDs and excluding three test sensors, eight usable locations remain in a ~580 m × 280 m cluster: `density-01`, `hn-goetzentuem-01`, `hn-hagenbuchersee-01`, `hn-hagenbuchersee-02`, `hn-marahaus-01`, `hn-neckarufer-01`, `hn-soleo-01`, and `hn-stadtgalerie-01`.
|
||||
|
||||
The eight-sensor window runs from 12 June to 7 November 2026, with a median cadence of 15.3 minutes. Values range 0–292.6; the multi-sensor mean is ~14.4, median 7.7, p95 46.9, p99 78.4. The inner city peaks 18:00–21:00 local, top 19:00; Saturday is busiest on average (mean 19.1) [2]. The sensors are LoRaWAN-connected PAX counters that passively sniff Bluetooth Low Energy (BLE) advertisements and transmit aggregated device counts [1, 2]. The counts are estimates, not a census: device penetration, Bluetooth settings, and the scanning design all matter.
|
||||
The 12 June model window (12 June–7 November 2026) has a median cadence of 15.3 minutes; values range 0–292.6, with median 8.4, p95 49.7, p99 79.8. Across all 96,232 density rows, the median is 7.7, p95 46.9, p99 78.4 and the multi-sensor mean is ~14.4. In the 12 June model window, the inner city peaks 18:00–21:00 local, top 19:00; Saturday is busiest on average (mean 19.1) [2]. The sensors are LoRaWAN-connected PAX counters that passively sniff Bluetooth Low Energy (BLE) advertisements and transmit aggregated device counts [1, 2]. The counts are estimates, not a census: device penetration, Bluetooth settings, and the scanning design all matter.
|
||||
|
||||
### 2.2 What one value actually is
|
||||
|
||||
A code-level audit of the deployed Code for Heilbronn `lorax` firmware changes the interpretation of every number in this paper [3, 4]. The node wakes from deep sleep, scans BLE passively for about 10 seconds, counts distinct locally administered (“randomized”) addresses, truncates the last two address bytes into a 16-bit ID, multiplies the integer count by a per-sensor factor (default 0.7), sends one LoRaWAN payload, and sleeps for 900 seconds [3, 4].
|
||||
A code-level audit of the public 2026 Code for Heilbronn `lorax` firmware (commit e8be8bf) changes the interpretation of the numbers [3, 4]. The node scans BLE passively for about 10 seconds, counts distinct locally administered addresses, truncates the last two address bytes into a 16-bit ID, multiplies by 0.7, and sleeps for 900 seconds. The deployed 2025 binary may differ; the data confirm the 0.7 scaling and per-wake export, not the scan duration or truncation.
|
||||
|
||||
**The published value is therefore `0.7 × U`, where `U` is the number of distinct randomized BLE addresses seen in a ~10-second scan.** The 15-minute cadence is the sleep cycle, not the integration window. The effective duty cycle is about 1.1%. The firmware sets a 30-second scan duration, but the released code stops on the first 10-second report callback; the 30-second value is not enforced by the scanner [4].
|
||||
In the public 2026 firmware, the published value is `0.7 × U`, where `U` is the number of distinct randomized BLE addresses seen in a ~10-second scan. The 15-minute cadence is the sleep cycle, not the integration window; the effective duty cycle is ~1.1% in that code. Every one of the 83,156 values in the **12 June model window** is a multiple of 0.7; the reconstructed count has median 12, p95 71, max 418, and 2.70% of those values are zero. A hardware replacement creates a scale break in `density-01`; the legacy and current series are not pooled [5].
|
||||
|
||||
The audit is of the public 2026 firmware; the deployed 2025 binary may differ, but the data are consistent with the 0.7 scaling. Every one of the 83,156 density values in the eight-sensor panel is an exact multiple of 0.7. The reconstructed integer count has median 12, p95 71, and a maximum of 418; 2.7% of the 83,156 values reconstruct to `U = 0`. The legacy `density-01` device reports integers on a 1.0 grid until March 2026, then a replacement device reports on the 0.7 grid from April 2026, with a 35.2-day gap between the two series. The two `density-01` series are not on the same scale and must not be pooled [5].
|
||||
This is a relative activity index, not a count of people. A 30-second passer-by has a ~4.4% chance of overlapping a scan; a one-hour lingerer can be counted up to four times. The index conflates flow and dwell, cannot distinguish phones from watches/beacons/cars, and the people-per-device ratio is unknown. The 0.7 factor is a provisioning default, not a validated penetration correction.
|
||||
|
||||
This is a relative activity index, not a count of people. A person who passes in 30 seconds has roughly a 4.4% chance of overlapping a ~10-second scan; a person who lingers for an hour can be counted in up to four consecutive bins. The index conflates flow and dwell, and it cannot distinguish a phone from a smartwatch, earbud case, beacon, or car head-unit. Device penetration and the people-per-device ratio are unknown. The 0.7 factor is a provisioning default, not a validated penetration correction.
|
||||
|
||||
> **Practical consequence.** The model predicts the published index well enough to compare relative peaks and to drive a staffing rule. It does not predict people, and it does not support absolute footfall or revenue claims without a calibration study.
|
||||
> **Practical consequence.** The model predicts the index well enough to compare relative peaks and drive a staffing rule. It does not predict people, and it does not support absolute footfall or revenue claims without calibration.
|
||||
|
||||
### 2.3 Coverage, gaps, and context
|
||||
|
||||
Coverage is uneven. Only 28.2% of 15-minute slots in the analysis panel have all eight sensors reporting; the modal count is seven. Aggregating to 30, 60, or 120 minutes raises complete coverage to 63.4%, 69.7%, and 71.0% [5]. Overall, 5.18% of individual density readings are zero and 39.6% are below 5. The worst month falls to 23.3% coverage at one location. Gaps reach four to six days; the cause is undocumented, but missing periods differ systematically from full-coverage periods [2, 5]. Weather comes from Open-Meteo: historical archive for training, current/7-day forecast for the dashboard. Temperature and precipitation are the useful features; cloud and wind did not improve the short test. Baden-Württemberg holidays are a feature and badge; they added little and are context, not a large driver [2].
|
||||
Coverage is uneven. In the **18 June coverage panel (82,170 readings)**, only 28.2% of 15-minute slots have all eight sensors; the modal count is seven (4,117 slots). Aggregating to 30, 60, or 120 minutes raises complete coverage to 63.4%, 69.7%, and 71.0% [5]. Across all **96,232 density rows**, 5.18% are zero, 10.36% are <1, and 39.57% are <5; in the **12 June model window** the shares are 2.70%, 8.32%, and 35.86%. The worst month falls to 23.3% coverage at one location in the 18 June panel. Gaps reach four to six days; the cause is undocumented, but missing periods differ systematically from full-coverage periods [2, 5]. Weather comes from Open-Meteo: historical archive for training, current/7-day forecast for the dashboard. Temperature and precipitation are the useful features; cloud and wind did not improve the short test. Baden-Württemberg holidays are a feature and badge; they added little and are context, not a large driver [2].
|
||||
|
||||
| Sensor | Readings | Mean | Median | p95 | Max | Max daily gap |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
@@ -56,7 +54,7 @@ Coverage is uneven. Only 28.2% of 15-minute slots in the analysis panel have all
|
||||
| hn-soleo-01 | 10,327 | 10.6 | 8.4 | 28.0 | 111.3 | 4 days |
|
||||
| density-01 | 12,670 | 9.9 | 7.7 | 28.0 | 63.0 | 35 days |
|
||||
|
||||
*Table 1. Data coverage per sensor. `density-01` uses the longer history; the 35-day gap is across the hardware replacement.*
|
||||
*Table 1. Per-sensor coverage and value statistics. Except for the `density-01` max-gap entry, statistics use the 12 June–7 November 2026 model window; the `density-01` 35-day gap spans the longer history and hardware replacement.*
|
||||
|
||||

|
||||
|
||||
@@ -82,25 +80,26 @@ The model serves 15 min, 1 h, 48 h, and 7 d. The 15 min/1 h horizons are nowcast
|
||||
|
||||
### 3.3 From forecast to shift plan
|
||||
|
||||
The ML model predicts footfall; a deterministic heuristic turns it into staff:
|
||||
The ML model predicts the published index; a deterministic heuristic turns it into staff:
|
||||
|
||||
```text
|
||||
forecast footfall per hour
|
||||
forecast index per hour
|
||||
→ calibration factor k (unvalidated)
|
||||
→ visitors per staff per hour (service standard)
|
||||
→ required staff = ceil(footfall / standard × safety factor)
|
||||
→ required staff = ceil(index × k / standard × safety factor)
|
||||
→ minimum staff, shift length, opening hours
|
||||
→ shift blocks + coverage score + estimated cost
|
||||
```
|
||||
|
||||
The assumptions are visible and adjustable: visitors per staff hour, labor cost per hour, minimum staff, shift length, and safety factor. The manager can accept, edit, or reject the plan and regenerate with a scenario. This is decision support, not an optimizer with proven ROI; we have no POS, sales, queue, or labor-cost data.
|
||||
The assumptions are visible and adjustable: calibration factor `k`, visitors per staff hour, labor cost per hour, minimum staff, shift length, and safety factor. `k` converts index units to visitors and is **unvalidated**; setting `k = 1` would treat one index unit as one visitor, which the audit says is not justified. The manager can edit assumptions and regenerate the plan with a scenario. This is decision support, not an optimizer with proven ROI; we have no POS, sales, queue, or labor-cost data.
|
||||
|
||||
## 4. System
|
||||
|
||||
PulseCast runs as three namespace-scoped Deployments on STACKIT SKE plus a Cloudflare Tunnel connector: **Nuxt 4 SSR + Onyx** for the server-rendered shell, map, charts, forecast, plan, and scenario controls; a **Go API** for sensor/series/forecast/staffing endpoints, SSE fan-out, scenario state, and a cached fallback; and a **FastAPI ML service** for `/predict` and `/simulate`, loading a model baked into the image at build time.
|
||||
|
||||
The namespace does not allow Ingress, Jobs, CronJobs, cert-manager, or external-dns. We expose the app through a Cloudflare Tunnel, terminate TLS at Cloudflare, and run the batch at build time. The model and fallback JSON are baked into the images; there is no database or PVC. If FastAPI is unavailable, Go serves the cached forecast and labels it. SSE carries scenario changes with a 15-second heartbeat; polling is the fallback. The plan is generated in 42.8 ms locally on v1.1; the contract is under two seconds.
|
||||
The namespace does not allow Ingress, Jobs, CronJobs, cert-manager, or external-dns. We expose the app through a Cloudflare Tunnel, terminate TLS at Cloudflare, and run the batch at build time. The model and fallback JSON are baked into the images; there is no database or PVC. If FastAPI is unavailable, Go serves the cached forecast and labels it. SSE carries scenario changes with a 15-second heartbeat; polling is the fallback. The plan is generated in 19 ms cold on v1.1; the contract is under two seconds.
|
||||
|
||||
The public app is at https://pulse.noelsabia.dev and the API at https://api-pulse.noelsabia.dev. The v1.1 images are 16.9 MB (api), 249 MB (web), and 952 MB (ml). In the 120-client Cloudflare k6 run, 120/120 SSE connections opened and closed with zero connection errors; p95 event latency 1.03 s, p95 scenario latency 194 ms, presenter POST 82.69 ms, 2,760 events over 4m53s. The system runs non-root with resource limits and a read-only root filesystem. The public dashboard is read-only; `/control` uses basic auth. We store no PII or MAC addresses in the application. This is a hackathon deployment: one API replica, no SSO, no HA, no live ingestion. We document those gaps.
|
||||
The public app is at https://pulse.noelsabia.dev and the API at https://api-pulse.noelsabia.dev. The v1.1 images are 16.9 MB (api), 249 MB (web), and 952 MB (ml). In the v1.1 120-client Cloudflare k6 run, 120/120 SSE connections opened and closed with zero connection errors; p95 event latency 331 ms, p95 scenario latency 646 ms, presenter POST 49.60 ms, over 4m57.9s. The system runs non-root with resource limits and a read-only root filesystem. The public dashboard is read-only; `/control` uses basic auth. We store no PII or MAC addresses in the application. This is a hackathon deployment: one API replica, no SSO, no HA, no live ingestion. We document those gaps.
|
||||
|
||||

|
||||
|
||||
@@ -114,7 +113,7 @@ The public app is at https://pulse.noelsabia.dev and the API at https://api-puls
|
||||
|
||||
### 5.1 Forecast accuracy
|
||||
|
||||
Table 2 reports the 28-day holdout from the generated `metrics.json`. The weather-aware 7-day model reduces MAE by about 19% relative to the no-weather 7-day model (4.08 → 3.32) and by about 39% relative to seasonal naive (5.40 → 3.32). WAPE falls from 44.7% to 27.5%, and R² rises from 0.462 to 0.777. The day-ahead weather model reaches 3.36, 27.8%, and 0.763; the longer horizon is slightly weaker but remains well above the baseline.
|
||||
Table 2 reports the 28-day holdout from the generated `metrics.json`. The weather-aware 7-day model reduces MAE by about 19% relative to the no-weather 7-day model (4.08 → 3.32) and by about 39% relative to seasonal naive (5.40 → 3.32). WAPE falls from 44.7% to 27.5%, and R² rises from 0.462 to 0.777. The day-ahead weather model reaches 3.36, 27.8%, and 0.763; the longer horizon is slightly stronger (3.32 vs 3.36 MAE) and remains well above the baseline.
|
||||
|
||||
| Model | MAE | WAPE | R² |
|
||||
|---|---:|---:|---:|
|
||||
@@ -134,7 +133,7 @@ Table 2 reports the 28-day holdout from the generated `metrics.json`. The weathe
|
||||
|
||||
Weather is the main driver. In the day-ahead ablation, removing it raises MAE 3.36 → 3.96. The holiday flag changes 3.96 → 3.84 (0.12); it is context, not a large lift. The production 7-day model is 3.32 with weather and 4.08 without. These are single-site, 28-day results.
|
||||
|
||||
Per-sensor errors vary (Table 3): Stadtgalerie has the highest MAE (5.83); Hagenbuchersee-02 the highest WAPE (31.1%); density-01 the lowest MAE (2.26). The global model is pooled; local errors are not.
|
||||
Per-sensor errors vary (Table 3): Stadtgalerie has the highest MAE (5.83); Hagenbuchersee-01 the highest WAPE (38.6%); density-01 the lowest MAE (2.26). The global model is pooled; local errors are not.
|
||||
|
||||
| Sensor | MAE | WAPE | R² |
|
||||
|---|---:|---:|---:|
|
||||
@@ -153,18 +152,18 @@ Per-sensor errors vary (Table 3): Stadtgalerie has the highest MAE (5.83); Hagen
|
||||
|
||||
| Metric | Value |
|
||||
|---|---:|
|
||||
| Plan generation latency | 42.8 ms local v1.1 (contract < 2 s) |
|
||||
| Plan generation latency | 19 ms cold on v1.1 (contract < 2 s) |
|
||||
| Concurrent SSE clients | 120/120 opened and closed; 0 connection errors |
|
||||
| SSE p95 event latency | 1.03 s |
|
||||
| SSE p95 scenario latency | 194 ms |
|
||||
| Presenter POST latency | 82.69 ms |
|
||||
| Events received / duration | 2,760 / 4m53s |
|
||||
| SSE p95 event latency | 331 ms |
|
||||
| SSE p95 scenario latency | 646 ms |
|
||||
| Presenter POST latency | 49.60 ms |
|
||||
| Duration | 4m57.9s |
|
||||
| Image sizes (api / web / ml) | 16.9 MB / 249 MB / 952 MB |
|
||||
| Cold deployment time | `[TODO]` |
|
||||
|
||||
*Table 4. System metrics from the v1.1 deployment and the 120-client Cloudflare k6 run. Cold deployment time is still `[TODO]`.*
|
||||
*Table 4. System metrics from the v1.1 deployment and the v1.1 120-client Cloudflare k6 run. Cold deployment time is still `[TODO]`.*
|
||||
|
||||
The 120-client Cloudflare run is the measured result: 120/120 connections opened and closed, zero errors, p95 event 1.03 s. The earlier 3-VU smoke test (p95 656 ms) was a pre-check. The plan screen compares the recommended schedule to a flat schedule on coverage and cost under stated assumptions; this is a heuristic comparison, not an ROI claim.
|
||||
The v1.1 120-client Cloudflare run is the measured result: 120/120 connections opened and closed, zero errors, p95 event 331 ms. The plan screen compares the recommended schedule to a flat schedule on coverage and cost under stated assumptions; this is a heuristic comparison, not an ROI claim.
|
||||
|
||||
### 5.4 What the evaluation does and does not show
|
||||
|
||||
@@ -186,7 +185,7 @@ Passive Bluetooth/Wi-Fi sensing consistently finds a small, biased sample: count
|
||||
|
||||
## 8. Conclusion
|
||||
|
||||
PulseCast shows that a small, weather-aware global model can turn a noisy inner-city footfall index into a weekly staffing plan that beats a seasonal-naive baseline and runs in 42.8 ms locally on a real deployment. It also shows that the index is not a people counter: each value is a short BLE scan scaled by 0.7, sampled every 15 minutes, with unknown penetration and incomplete coverage. The honest conclusion is narrower and more useful than a city-scale claim: the decision loop works, the relative forecast is credible, and the next step is calibration and a privacy-first sensing layer. We would rather ship a plan a manager can use and state what it does not measure than promise a city-wide heatmap the data cannot support.
|
||||
PulseCast shows that a small, weather-aware global model can turn a noisy inner-city footfall index into a weekly staffing plan that beats a seasonal-naive baseline and runs in 19 ms cold on a real deployment. It also shows that the index is not a people counter: each value is a short BLE scan scaled by 0.7, sampled every 15 minutes, with unknown penetration and incomplete coverage. The honest conclusion is narrower and more useful than a city-scale claim: the decision loop works, the relative forecast is credible, and the next step is calibration and a privacy-first sensing layer. We would rather ship a plan a manager can use and state what it does not measure than promise a city-wide heatmap the data cannot support.
|
||||
|
||||
## Artifacts, data, and AI disclosure
|
||||
|
||||
|
||||
Reference in new issue
Block a user