Add PulseCast paper drafts, figures, references, and research notes

This commit is contained in:
Noel Sabia committed 2026-11-14 06:32:23 +01:00
commit c691324227
69 files changed
+23137

No files matched your search

+133
View File
@@ -0,0 +1,133 @@
# PulseCast paper — writing plan
Status: **Version A draft written** (`paper/main.md`, ~3,900 words); **Version B draft written** (`paper/main-extended.md`, ~9,400 words, 37 references). Research notes are in `paper/research/`.
## Title options
1. **The Pulse Is Not the People: A Code-Level Audit of Bluetooth Footfall Sensing and a Privacy-First Architecture for Urban Decisions** — recommended for the extended paper.
2. **From MAC Addresses to Decisions: What a 24-Hour Footfall Copilot Actually Measures** — softer, more descriptive.
3. **PulseCast: A Decision-First Footfall Copilot — Measurement Limits and a Path to Production** — best fit for the 4–5 page hackathon report.
## Version A — hackathon report (4–5 pages, due Sunday)
Keep the structure required by `agents/paper.md` and `OKF_Vault/mission/paper.md`.
| Section | Length | Content |
|---|---:|---|
| Abstract | 150–200 words | Problem, data, method, headline result, deployment |
| 1. Introduction | 0.5 page | Decision gap; why a dashboard is not enough; contributions |
| 2. Data | 0.75 page | PAX counters; 8 sensors; 5 months; 15 min; 580 m; coverage/gaps; **box: what one value actually is** (10 s scan × 0.7, 16-bit truncation); device bias; privacy; Open-Meteo; BW holidays |
| 3. Method | 1 page | Preprocessing; features; global HistGB; horizons; time-based split; staffing heuristic |
| 4. System | 0.75 page | Nuxt SSR + Onyx; Go API/SSE; FastAPI; SKE namespace-only; Cloudflare Tunnel; build-time batch; cached fallback |
| 5. Evaluation | 1 page | Baseline; weather ablation; holiday effect; per-sensor errors; plan latency; k6 load test |
| 6. Discussion & limitations | 0.5–0.75 page | 8 sensors; 5 months; observed weather; devices not people; **the measurement semantics**; no POS/labor data; single replica; no auth beyond control; **what we would do with more resources** (0.5 page) |
| 7. Related work | 0.25 page | PAX/privacy; pedestrian forecasting; gradient boosting; smart-city dashboards |
| 8. Conclusion & future work | 0.25 page | More sensors; POS/event data; multi-store; city operations |
| References | 8–12 | Real, verifiable only |
**Do not** add the full legal analysis or the full sensing-alternatives comparison to Version A. Put them in Version B. The short report can say: “The value is a relative activity index, not a people count; a code-level audit and a privacy-first redesign are in the extended version.”
**Required figures/tables for Version A:** sensor map; hourly/weekday profile; forecast vs actual; architecture; plan screen. Tables: coverage per sensor; model comparison; ablation; system metrics. Fill the k6 numbers from the real run; do not invent.
## Version B — extended paper (10–12 pages, Zenodo)
| Section | Pages | Content |
|---|---:|---|
| Abstract | 0.25 | Problem, audit finding, alternative, transferability, non-claims |
| 1. Introduction | 1 | Decision gap; the “privacy-preserving” claim; contributions |
| 2. The measurement problem | 2.5 | Code audit; formula; data forensics; reliability taxonomy; existing PulseCast metrics reframed |
| 3. The privacy problem | 1.5 | GDPR/ePrivacy/TTDSG; threat model; 16-bit truncation; operator claims vs code; DPIA checklist |
| 4. What “privacy-first” should mean | 2 | PETs; sensing alternatives; calibration; reference sensing stack; pilot protocol |
| 5. From sparse observations to decisions | 2 | Heatmap; forecast; uncertainty; decision layer; evaluation harness |
| 6. How far it generalizes | 1.5 | Applications matrix; case studies; falsifiers; where it stops |
| 7. Evaluation agenda | 1 | Ground truth; calibration; duty cycle; penetration; privacy budget; decision regret; proposed experiments |
| 8. Discussion & limitations | 0.75 | Single site; no ground truth; opinion bias; legal not advice; hackathon context |
| 9. Related work | 1 | PAX/privacy; pedestrian forecasting; spatiotemporal models; MLOps; urban dashboards |
| 10. Conclusion | 0.5 | What to keep; what to build; a checklist |
| References | 1.5 | 25–35 real, verified sources |
## Figures and tables
### Version A
- Fig. 1: sensor map (8 points; label names).
- Fig. 2: hourly/weekday profile.
- Fig. 3: forecast vs actual for one sensor over the 28-day test.
- Fig. 4: architecture.
- Fig. 5: plan screen (or table).
- Table 1: data coverage per sensor.
- Table 2: model comparison (seasonal naive / no weather / weather).
- Table 3: ablation (weather/holiday; per-sensor MAE).
- Table 4: system metrics (plan latency; 120 SSE clients; image sizes; deployment time).
### Version B (adds)
- Fig. 1: the PAX counter cycle (wake → ~10 s scan → `0.7 × U` → LoRaWAN → 900 s sleep; 1.1% duty cycle).
- Fig. 2: value quantization and the `density-01` calibration break.
- Fig. 3: sparse-to-heatmap-to-decision pipeline.
- Fig. 4: privacy-first sensing and data-flow architecture.
- Fig. 5: transferability matrix (or table).
- Table 1: code audit (what the firmware does vs what the claim says).
- Table 2: data quality (coverage, gaps, zero share, reconstructed `U`, quantization).
- Table 3: sensing alternatives (accuracy, cost, privacy, best use).
- Table 4: model comparison (existing metrics).
- Table 5: transferability matrix.
- Table 6: four-tier roadmap (hackathon / pilot / production; triggers; estimates).
## Claims and non-claims (both versions)
**Supported**
- The published value is `0.7 × U`; `U` is a short-window count of randomized BLE addresses.
- The network is a local inner-city cluster; not city-wide.
- The existing model improves on seasonal naive in a 28-day holdout: MAE 3.36 vs 5.40; WAPE 27.8% vs 44.7%; R² 0.779 vs 0.462.
- The sensor counts devices, not people; penetration is unknown.
- MAC-based sensing has a contested legal basis; hashing is pseudonymisation, not anonymisation.
- Thermal/radar/camera/probe/manual modalities have different accuracy, cost, privacy, and coverage trade-offs.
- The stack transfers directly to staffing/ads, conditionally to waste, as a sibling to sensor placement/LoRaWAN planning, and not to sentiment/learning guide.
**Not supported**
- “The sensor counts people.”
- “The value is a 15-minute count or mean.”
- “The 0.7 factor is a validated penetration correction.”
- “The 16-bit hash is anonymous.”
- “Eight sensors support a city-wide heatmap or store-location ranking.”
- “One model solves all six challenges.”
- “The proposed architecture has been deployed and evaluated.”
- “The staffing plan has proven ROI.”
## Reference strategy
- Version A: 8–12 real, verifiable references; no fake papers.
- Version B: 25–35 real, verifiable references. Prefer primary sources: standards, peer-reviewed papers, official docs, court/regulator decisions.
- Build `paper/references.md` from the research notes; verify every DOI/URL before submission.
- Do not cite a source that the notes do not support. If a claim is missing a source, write `[CITATION TODO]`.
## Files to write when approved
- `paper/main.md` — Version A (short report).
- `paper/main-extended.md` — Version B (extended paper), or a single `paper/main.md` with a version switch.
- `paper/references.md` — verified reference list.
- `paper/figures/` — generated figures (sensor map, profile, forecast, architecture, plan screen, cycle diagram, quantization, pipeline).
- `paper/tables/` — CSV/Markdown tables if useful.
- `paper/README.md` — build/render instructions and artifact links.
## Dependencies / open items
- [x] Data-quality note (`07-data-quality.md`) — complete; figures in `paper/research/figures/`.
- [ ] k6 120-client result — local 3-VU smoke test passed; the cluster run is still `[TODO]`.
- [x] Per-sensor MAE table — computed from the production 7-day model.
- [ ] Metrics reconciliation — `metrics.json` reports 3.32/27.5/0.777 (production 7-day) and 3.36/27.8/0.763 (day-ahead); `agents/paper.md` lists 3.36/27.8/0.779. The human must confirm which run is canonical.
- [ ] Figure generation — use the real CSV; no synthetic data in the paper.
- [ ] Author names/affiliation for the Zenodo record.
- [ ] Decision: Version A only, Version B only, or both.
- [ ] Decision: run the post-demo experiments (subsampling, leave-one-sensor-out, factor sensitivity).
- [ ] Verify every number against `OKF_Vault/mission/data.md`, `metrics.json`, or a measured result.
## Recommended sequence
1. Finish Version A for the hackathon deadline. Do not let Version B delay the demo.
2. After the demo, write Version B using the same evidence plus the full audit and architecture.
3. Publish Version B on Zenodo with a DOI; link the code, derived aggregates, and demo.
4. If arXiv endorsement works, post there too; otherwise Zenodo is sufficient.
+57
View File
@@ -0,0 +1,57 @@
# PulseCast paper
Two versions of the paper, plus the research trail.
| File | What it is | Status |
|---|---|---|
| `main.md` | Version A — 4–5 page hackathon technical report | Draft; ~5–6 pages in a compact layout |
| `main-extended.md` | Version B — 10–12 page Zenodo paper | Draft; ~9,200 words |
| `references.md` | Canonical reference list and verification checklist | Draft |
| `OUTLINE.md` | Writing plan, claims/non-claims, figures/tables | Working document |
| `figures/` | Generated figures used by both versions | Draft figures |
| `research/` | Research notes (code audit, privacy law, sensing, heatmap, PETs, applications, data quality) | Complete |
## Build
There is no Pandoc/LaTeX in this workspace. Render with any Markdown-to-PDF tool, or use a two-column template. Figure paths are relative to `paper/`:
```text
paper/figures/fig_sensor_map.png
paper/figures/fig_hourly_profile.png
paper/figures/fig_quantization.png
paper/figures/fig_architecture.png
paper/figures/fig_plan.png
paper/figures/fig_forecast.png
```
`fig_plan.png` is a layout placeholder; replace it with a live screenshot before submission.
## Verification status
- Numbers: checked against `OKF_Vault/mission/data.md`, the generated `metrics.json`, and `paper/research/07-data-quality.md`.
- Metrics discrepancy: `metrics.json` reports **3.32 / 27.5% / 0.777** for the production 7-day model and **3.36 / 27.8% / 0.763** for the day-ahead model; the mission brief lists **3.36 / 27.8% / 0.779**. The paper uses the generated metrics and flags the difference. The human must confirm the canonical run.
- k6: only a local 3-VU SSE smoke test (p95 656 ms) exists. The 120-client cluster run is still `[TODO]`.
- Per-sensor MAE: computed from the production 7-day model; `n ≈ 669` per sensor.
- Figures: generated from the raw CSV and the trained model; the plan screen is a placeholder.
- References: 12 in Version A, 35 in Version B; all are real sources from the research notes. Verify DOIs/URLs before submission.
- AI disclosure: included in both versions.
## Known TODOs
- [ ] Canonical metrics decision (3.32/27.5/0.777 vs 3.36/27.8/0.779).
- [ ] 120-client k6 result and system metrics (image sizes, cold deployment time).
- [ ] Live plan-screen screenshot.
- [ ] Demo URL and public code repository URL.
- [ ] Author names and affiliation.
- [ ] Replace internal project-note citations with public artifacts if a venue requires it.
- [ ] Final page-count trim for Version A if a strict 4–5 page limit applies.
## Source-of-truth rules
- Never invent a number, citation, dataset, result, or quote.
- Every number must match `OKF_Vault/mission/data.md`, the generated `metrics.json`, or a measured result.
- Weather is observed in training; forecast/current weather is used in the dashboard and scenarios.
- Values are estimated devices, not people; device penetration is unknown.
- The 8-sensor / 5-month / 580 m limitation is explicit.
- No city-wide heatmap, no annual seasonality, no live sensor feed, no proven ROI.
- The deployment description matches the real system: Nuxt SSR + Onyx, Go API/SSE, FastAPI, SKE namespace-only, Cloudflare Tunnel, build-time batch, cached fallback.
Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 116 KiB

+11
View File
@@ -0,0 +1,11 @@
{
"sensor": "hn-stadtgalerie-01",
"n": 668,
"residual_sd": 12.387558079399437,
"metrics": {
"mae": 5.832423641662326,
"wape": 25.32882655042187,
"r2": 0.764685954382945,
"n": 668
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 249 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 63 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

+567
View File
@@ -0,0 +1,567 @@
# The Pulse Is Not the People: A Code-Level Audit of Bluetooth Footfall Sensing and a Privacy-First Architecture for Urban Decisions
**Authors:** [AUTHOR TODO]
**Affiliation:** [AFFILIATION TODO]
**Date:** 14 November 2026
**Companion artifact:** `paper/main.md` (4–5 page hackathon report) · [code repository TODO] · [demo URL TODO]
## Abstract
Low-cost Bluetooth footfall counters are increasingly used to justify staffing, advertising, and urban-planning decisions. Their central claim is that they measure pedestrian density while preserving privacy. We audit that claim on the eight-sensor PAX counter network deployed in Heilbronn, Germany. A code-level review of the public firmware shows that each published “15-minute” value is a **~10-second** passive BLE scan: the node counts distinct locally administered addresses, truncates the last two MAC bytes into a 16-bit ID, multiplies by a fixed factor (default 0.7), and sleeps for 900 seconds. The effective duty cycle is about 1.1%. The data confirm the semantics: all 83,156 density values in the eight-sensor panel are exact multiples of 0.7; reconstructed counts have median 12 and maximum 418; a hardware replacement produces a 35.2-day gap and a scale break from a 1.0 integer grid to a 0.7 grid. Coverage is sparse: only 28.2% of 15-minute slots contain all eight sensors, 5.2% are all-zero, and cross-sensor correlation falls from 0.58 to 0.38 after removing hour-of-week structure. Legally, MAC addresses are very likely personal data in this context; truncation is pseudonymisation, not anonymisation; and passive sniffing under § 25 TDDDG is contested. We then propose a privacy-first architecture: thermal/radar/camera/probe/manual sensing with calibration, a network-aware hierarchical spatial model, a quantile forecast with conformal calibration, and a decision layer evaluated by regret rather than MAE alone. We show how the same decision-grade contract transfers directly to staffing and advertising, conditionally to waste routing, as a sibling to sensor placement and LoRaWAN planning, and not at all to sentiment or learning-guide tasks. The contribution is not a better people counter; it is an honest measurement model and a staged path to one.
## 1. Introduction
A store manager on Kaiserstraße knows next week’s weather forecast and the staff roster. She does not know how many people will walk past next Saturday. The available data are counts from eight passive Bluetooth sensors; the decision is a shift plan. A dashboard that shows history does not close that gap. The manager needs a forecast, an explicit staffing rule, and a way to test a scenario before committing.
The Schwarz Digits *Pulse of the City* challenge asks teams to turn “privacy-preserving” pedestrian counts from Heilbronn into predictive tooling for advertising, store locations, and staffing [1]. The wording is doing a lot of work. “Pedestrian counts” implies people; “privacy-preserving” implies that the sensing layer has been designed around a privacy guarantee. Neither claim is self-evident. The counters are LoRaWAN-connected ESP32 devices that passively sniff Bluetooth Low Energy (BLE) advertisements and transmit aggregated device counts [2]. The challenge brief calls them privacy-preserving because they count devices rather than identify people [1]. That is a meaningful architectural choice, but it is not a measurement guarantee, and it is not a legal anonymisation guarantee.
This paper started as an attempt to write the technical report for a hackathon system, PulseCast, that turns the counts into a weekly staffing plan. The system works: on a 28-day holdout, a global weather-aware gradient-boosted model reaches MAE 3.32, WAPE 27.5%, and R² 0.777 against a seasonal-naive baseline of 5.40, 44.7%, and 0.462. The plan is generated in under two seconds on a three-service deployment on STACKIT SKE. But the more we looked at the sensing layer, the clearer it became that the model was predicting a **different quantity** from the one the narrative claimed. The code and the data agree: each value is a short-window count of randomized BLE addresses scaled by a fixed factor. The “15-minute” cadence is the sleep cycle, not the integration window.
That finding changes the paper. A conventional technical report would present the model metrics and the deployment, note that the data are “estimated devices, not people,” and stop. That is not enough. If a decision support system is built on a measurement that conflates flow and dwell, cannot distinguish a phone from a watch or a car, and samples 10 seconds out of every 15 minutes, then the honest contribution is a **measurement audit** plus a **reference architecture** for the next version. The model is not the hard part. The data contract is.
We make five contributions:
1. **A code-level audit** of the deployed Code for Heilbronn `lorax` firmware, with the exact measurement chain: passive BLE scan, locally-administered-address filter, 16-bit truncation, per-sensor factor 0.7, 900-second sleep, RSSI threshold −80 dBm.
2. **A forensic data analysis** of 192,464 rows, confirming the semantics in 83,156 values (100% multiples of 0.7), identifying the `density-01` calibration break, and quantifying coverage, zero-inflation, correlation, and missingness.
3. **A privacy and legal analysis**: why MAC addresses are likely personal data, why truncation is pseudonymisation rather than anonymisation, why § 25 TDDDG / Art. 5(3) ePrivacy is the decisive unresolved question, and what a DPIA-compliant deployment would require.
4. **A privacy-first reference architecture**: a tiered sensing stack (thermal, radar, edge cameras, probe data, manual counts), a calibration program, a central-DP aggregation layer, a network-aware hierarchical spatial model, a quantile forecast with conformal calibration, and a decision layer evaluated by regret.
5. **A transferability matrix** across the six hackathon challenges, with explicit boundary conditions: direct for staffing and advertising, conditional for waste routing, sibling for sensor placement and LoRaWAN planning, and not applicable to cable-car sentiment or the learning guide.
We are deliberately conservative. We do not claim that the current network measures people, that the 0.7 factor is a validated penetration correction, that eight sensors support a city-wide heatmap, or that the staffing plan has proven ROI. We claim that the current system is a useful **relative activity index** for a local cluster, that its measurement semantics can be reconstructed from code and data, and that a privacy-first, calibrated version is feasible with a staged program.
The paper is organized as follows. Section 2 audits the measurement. Section 3 analyzes the privacy and legal problem. Section 4 proposes a privacy-first sensing stack. Section 5 develops the sparse-to-heatmap-to-decision pipeline. Section 6 tests how far the architecture generalizes across the six challenges. Section 7 sets out an evaluation agenda. Section 8 discusses limitations and what to keep from the hackathon build. Section 9 positions the work in related literature. Section 10 concludes.
## 2. The measurement problem
### 2.1 What the challenge claims
The challenge brief describes the data as “numbers of people measured in a given area,” collected by “privacy-preserving sensors” that “estimate crowd density by passively sniffing Bluetooth signals (MAC address advertisements) from nearby smartphones and smartwatches” and transmit “aggregated device counts” over LoRaWAN [1]. It also warns that PAX counters provide estimates because not everyone has a smartphone with Bluetooth turned on [1]. The brief is more careful than the headline: it distinguishes counts from people and asks teams to normalize. But the downstream narrative—store traffic, advertising exposure, staffing demand—treats the index as a footfall measure. That is the gap this section closes.
### 2.2 The code-level audit
The deployed system is maintained by Code for Heilbronn. The public repository `cfhn/lorax` (mirror name `lora-frequenzmessung`) contains the `sensor-pax` firmware, a LoRaWAN/TTN aggregator, a Go backend, TimescaleDB, and a Grafana dashboard [3]. The firmware is derived from the open-source ESP32-Paxcounter / `libpax` lineage [4]. The audit below is of the public 2026 commit; the 2025 binary may differ, but the data are consistent with the 0.7 scaling.
One cycle is:
```text
wake from deep sleep
→ BLE passive scan (window 80 ms, interval 80 ms)
→ count distinct locally administered addresses in ~10 s
→ truncate the last two address bytes to a 16-bit ID
→ deduplicate within that scan only
→ multiply the integer count by factor (default 0.7)
→ send one LoRaWAN payload: sensor_id, 0, value, 1, battery
→ deep sleep 900 s
```
The key code facts are:
- `configuration.blecounter = 1`, `configuration.wificounter = 0` — BLE only [3].
- `configuration.blescantime = 30`, but the vendored `start_BLE_scan()` does not use `blescantime`; the main loop stops on the first 10-second report callback. The effective scan is about **10 seconds**, not 30 [3, 4].
- `libpax_counter_init(log, &count, 10, 1)` — 10-second report timer, cumulative count within the wake [3].
- `mac_add()` accepts only addresses with the **locally administered bit** set; it ignores universal/static addresses [3, 4].
- `id = (uint16_t *)(paddr + 4)` — the “hash” is a **16-bit truncation** of the last two bytes, not a cryptographic hash [3].
- `paxCount * cfg.factor` with `factor` default **0.7**; `sleep_time_sec` default **900 s** [3].
- RSSI threshold **−80 dBm**; BLE advertising queue size **30**; queue drops are logged, not counted in the payload [3].
- The backend stores the parsed value directly; the CSV is a raw export, not a 15-minute mean [3].
The practical consequence is the formula:
```text
published_value ≈ 0.7 × U
U = distinct 16-bit IDs from locally administered BLE addresses seen in a ~10 s scan
sampling interval = ~900 s
duty cycle ≈ 10 / 910 ≈ 1.1%
```
This is not a 15-minute count, a 15-minute mean, or a count of people. It is a short-window count of a particular address regime, scaled by a provisioning constant.
### 2.3 A measurement model
Let `P` be the number of people present in the sensor’s catchment during the scan window, `d` the probability that a person carries a detectable advertising device, `m` the mean number of distinct advertising devices per carrier, and `q` the probability that at least one device advertises within the scan. The expected published value is:
```text
E[y] = f · d · m · q · P
```
with `f = 0.7`. The factor can only be a correct people-per-device conversion if `d · m · q ≈ 1.43`. That is possible if `d` is high and `m ≈ 1.4`, but it is not measured. If `d = 0.5` and `m = 1.4`, `y/P ≈ 0.49` — a 2× undercount. If `d = 0.9` and `m = 1.2`, `y/P ≈ 0.76` — a 24% undercount. The scale is arbitrary without ground truth, and a single global factor cannot track seasonal, demographic, or device-mix changes.
The sampling design adds a second bias. A person who passes in 30 seconds has roughly a 4.4% chance of overlapping a 10-second scan every 900 seconds. A person who lingers for an hour can be counted in up to four consecutive bins. The index therefore conflates **flow** (people passing a point) with **dwell** (people staying in the area). It is not a screenline crossing count and not an occupancy count.
### 2.4 Data forensics
The delivered CSV contains 192,464 rows: 96,232 density readings and 96,232 battery readings. There are 12 device IDs; after merging the two `density-01` IDs and excluding three test sensors, eight usable locations remain in a ~580 m × 280 m cluster. The eight-sensor panel (18 June–7 November 2026) contains 83,156 density values.
**Quantization.** Every one of those 83,156 values is an exact multiple of 0.7. The reconstructed integer `U` has median 12, p95 71, maximum 418, and 2.7% zeros. A continuous physical series would not land on a 0.7 grid in 83,156 consecutive observations; the only plausible explanation is `value = round(0.7 × integer, 4)`. This is direct, data-internal confirmation of the firmware reading [5].
**The calibration break.** The legacy `density-01` device (`863f75b0`, 16 October 2025–12 March 2026) is 100% integer-valued on a 1.0 grid: 0–28, mean 3.008, median 2.0, zero share 29.8%. The replacement (`f8c19730`, 16 April–7 November 2026) is 100% on the 0.7 grid: 0–63, mean 9.741, median 7.7, zero share 5.9%. The gap between the last legacy reading and the first new reading is **35.2065 days**, with no overlap, so a conversion factor cannot be estimated empirically. The level shift is ×3.24 in the mean and ×3.85 in the median, which is not explained by 0.7 alone; season, installation, and sensitivity are confounded. The safe conclusion is that the two series are not on the same scale and must not be pooled [5].
**Cadence.** The nominal 15-minute grid is not a 15:00 wall-clock grid. Current devices report every 15.27–15.37 minutes (median); the legacy device every 15.18 minutes with ~18 s jitter; the new `density-01` every 16.14 minutes with ~2.4 s jitter, producing ~89 readings/day. Flooring to `HH:00/15/30/45` is an approximation; two devices matched to the same slot can be up to ~15 minutes apart [5].
**Coverage.** Only **28.2%** of 15-minute slots contain all eight sensors; the modal count is seven (4,117 slots). Aggregating to 30, 60, or 120 minutes raises complete eight-sensor coverage to **63.4%, 69.7%, and 71.0%**, respectively. Median monthly coverage is 86.1%, but August 2026 falls to 54.2% on average and 23.3% at the worst location. Six of eight locations show a simultaneous multi-week degradation; `density-01` and `hagenbuchersee-01` do not. Coverage is strongly confounded with battery across devices (r = 0.84, n = 8), but the cause cannot be isolated [5].
**Distribution and correlation.** Overall, 5.18% of readings are zero, 10.36% are `<1`, and 39.57% are `<5`. There are no negative or NaN values and no duplicate timestamps; 113 intervals are shorter than 60 seconds and look like retries. Raw cross-location Pearson correlation averages 0.58 (range 0.13–0.90), but drops to 0.38 after removing location-specific hour-of-week means and 0.48 at the daily level. Correlation is not evidence of shared unique visitors [5].
**What the aggregates cannot tell us.** The CSV contains no MAC addresses, address type, RSSI, unique IDs, or ground truth. MAC-randomization rate, penetration rate, unique-device count, dwell time, repeat visits, detection radius, and false-positive/false-negative rates cannot be measured from these aggregates. The 0.7 grid is a proxy for firmware processing, not a direct measurement of randomization or penetration. Any claim about those quantities must come from a calibration study.
**Figures.** Figure 1 shows the eight sensor locations. Figure 2 shows the hourly/weekday pulse. Figure 3 shows the per-device value grids and the `density-01` scale break.
![Figure 1. Sensor map.](figures/fig_sensor_map.png)
*Figure 1. The eight sensor locations in the ~580 m × 280 m inner-city cluster. Measured points only; no city-wide surface.*
![Figure 2. Hourly and weekday profile.](figures/fig_hourly_profile.png)
*Figure 2. Mean index by hour and weekday across the eight sensors. The peak is 18:00–21:00 local; Saturday is the busiest day on average.*
![Figure 3. Value grids and the density-01 scale break.](figures/fig_quantization.png)
*Figure 3. Left: per-device share of readings on the 0.7 grid, the integer grid, both, or neither. Right: the reported value ladders for the two `density-01` devices, showing the 1.0 versus 0.7 spacing.*
### 2.5 Reliability taxonomy
The audit and the data suggest a taxonomy of failure modes, ordered by how much they matter for a decision:
| Failure | Mechanism | Consequence |
|---|---|---|
| Short snapshot | ~10 s scan every ~15 min | Misses peaks between scans; target is a sparse sample, not integrated flow |
| Flow/dwell conflation | Same person counted in consecutive bins; short passer-by often missed | A busy “passing” street and a “lingering” square can produce the same index |
| Randomized-only filter | Only locally administered addresses counted | Static/public-address devices invisible; non-phone random advertisers counted |
| 16-bit truncation | Last two bytes used as ID | Collisions grow with density; undercount at busy times |
| No cross-scan dedup | Bitmap reset every wake | No dwell, repeat-visit, or cross-sensor dedup |
| Device ≠ person | Phones, watches, earbuds, cars, beacons | Unknown multiplier; one person can be several devices |
| Penetration unknown | Bluetooth on/off, discoverability, demographics | Absolute counts not identifiable without calibration |
| RSSI gate | −80 dBm threshold, body shielding, orientation | Detection zone is not a stable catchment |
| Queue drops | 30-slot advertising queue | Undercount under density; not exposed in data |
| Gaps | LoRaWAN joins, power, maintenance | 4–6 day gaps; missing not at random |
| Calibration factor | Default 0.7, undocumented | Single scalar cannot absorb penetration × devices/person × scan probability |
| Calibration break | Hardware replacement; 1.0 → 0.7 grid | Pooling the series fits the instrument, not the city |
The first four are intrinsic to the design; the rest are deployment and calibration issues. A production system must address all of them, but the first four determine whether the metric can be interpreted at all.
### 2.6 What the existing model actually predicts
The PulseCast model is a global `HistGradientBoostingRegressor` trained on hourly means of the published index. On the 28-day holdout (n = 5,350), the production 7-day model reaches MAE 3.32, WAPE 27.5%, and R² 0.777. The day-ahead weather model reaches 3.36, 27.8%, and 0.763. The seasonal-naive baseline is 5.40, 44.7%, and 0.462. Per-sensor MAE ranges from 2.26 (`density-01`) to 5.83 (`hn-stadtgalerie-01`). Weather is the main driver: removing it raises day-ahead MAE from 3.36 to 3.96; the holiday flag changes 3.96 to 3.84, a small effect.
These are credible relative results. They are also results **on the index**, not on people. The model is good at predicting a short-window count of randomized BLE addresses scaled by 0.7; that is a different achievement from predicting footfall. The MAE is in index units, and the index units are not people. A staffing rule that consumes the index inherits the scale ambiguity.
### 2.7 Why the audit matters
If the only use of the data were a relative “busier than usual” signal, the audit would be a caveat. But the PulseCast use case is staffing: the index is converted into required staff through a visitors-per-staff-hour assumption. That conversion is an absolute claim. If the index is 2× low because penetration is 0.5, the plan is 2× low. If the index conflates a passing street with a lingering square, the plan over-staffs the wrong hours. If the 0.7 factor is wrong, the plan is wrong by the same factor. The model’s MAE says nothing about these errors.
The right response is not to discard the data. It is to use them as a **relative activity index** for a local cluster, to calibrate them against ground truth before making absolute claims, and to state the measurement model explicitly. That is the bridge to the architecture in Sections 4–5.
## 3. The privacy problem
### 3.1 MAC addresses are likely personal data
The GDPR defines personal data as any information relating to an identified or identifiable natural person, including an online identifier, and asks whether identification is “reasonably likely” using all means available to the controller or another person (Art. 4(1), Recital 26) [13]. The CJEU’s *Breyer* judgment makes the test relative but broad: a dynamic IP address can be personal data where the controller has the legal means to obtain additional identifying information, and identification need not be in one person’s hands [14]. A MAC address is a device identifier; combined with sensor location, time, and deduplication, it can single out a device and its bearer, even if the controller does not know a name. The EDPB’s 2026 draft anonymisation guidelines use a hardware identifier such as a MAC address as an example of data that can identify a device and, in turn, an individual [17].
The practical conclusion is that a public-space footfall system should treat MAC addresses as personal data unless a documented anonymisation assessment proves otherwise. The burden is on the controller. “We hash, therefore it is anonymous” is not sufficient.
### 3.2 Hashing is pseudonymisation
Hashing or truncating a MAC address reduces direct identifiability, but it does not automatically anonymise. An unsalted hash of a 48-bit MAC is enumerable; a short or truncated hash has a small space and frequent collisions; a keyed hash remains reversible by whoever holds the key. The EDPB’s pseudonymisation guidelines state that pseudonymised data that can be attributed to a person by additional information remain personal data, even if the pseudonymised data and the additional information are not in the same hands [16]. The French Conseil d’État’s *JCDecaux* decision is the clearest warning: truncating the MAC, adding a salt, and applying keyed hashing still left the controller able to identify and link records, and the purpose—measuring routes and repeat passages—was itself incompatible with anonymisation [18].
The Heilbronn firmware’s 16-bit truncation is weaker than a cryptographic hash. It is a dedup key with a 65,536-value space. That is a privacy-by-design choice in the sense that raw MACs are not persisted, but it is not an anonymisation guarantee. The operator’s blog says addresses are “hashed and stored for at most one hour” [2]; the public 2026 firmware resets the bitmap at each wake and has no one-hour store [3]. The two accounts disagree. The paper should report both and not treat either as a legal conclusion.
### 3.3 ePrivacy and § 25 TDDDG
The ePrivacy Directive’s Art. 5(3) requires prior consent for storing information in, or gaining access to information already stored in, terminal equipment, except for transmission or a service explicitly requested by the user [13]. Germany implements this in § 25 TDDDG, which applies regardless of whether the data are personal [20]. The decisive question is whether passive reception of a broadcast MAC is “access” to stored information. The better textual reading is that it is not: the sensor does not store anything in the phone, does not instruct the phone to disclose stored information, and does not retrieve data from the phone’s memory. German legislative materials use “Abruf” and “Auslesen” (retrieval/readout), which imply active access. But the EDPB’s functional approach—MAC addresses are information stored in the terminal equipment, and the receiving entity need not be the one instructing the transmission—creates material risk the other way [15]. The DSK’s digital-services guidance treats reading MAC addresses as a relevant access [21, 22]. There is no published EU or German decision on passive BLE/Wi-Fi sniffing under Art. 5(3)/§ 25.
If § 25 applies, the project cannot rely on legitimate interest, signage, or an opt-out. It needs prior consent meeting the GDPR standard, which is practically impossible at city scale: walking past a sensor, leaving Wi-Fi on, or seeing a sign is not consent. The viable routes are a statutory legal basis or a sensor that does not extract MAC addresses. That is why the architecture in Section 4 prefers thermal/radar/camera/probe sensing.
### 3.4 DPIA and lawful basis
A DPIA is almost certainly mandatory. Continuous sensing of a publicly accessible inner-city area is the paradigm of “systematic monitoring of a publicly accessible area on a large scale” under Art. 35(3)(c) [13]. The Baden-Württemberg private-sector “Muss-Liste” explicitly lists offline tracking of customer movements in department stores/shopping centres and the non-intended use of device sensors or radio signals to determine location or movement over a substantial period (Nos. 2 and 10) [21]. There is no public-sector positive list in Baden-Württemberg, so for the city the statutory criterion applies directly.
A public body cannot rely on Art. 6(1)(f); it needs Art. 6(1)(e) read with Art. 6(3) and a legal basis in Union or Member State law, or consent. A private association or company can use Art. 6(1)(f), but the balancing test is strict for covert public-space tracking, and German and Dutch DPAs have repeatedly found that the data subject’s interests prevail. The DPIA must assess necessity and alternatives: manual counts, light barriers, thermal/radar, on-device video counting, and sample-based surveys. If a less intrusive sensor can meet the accuracy requirement, the MAC-based design is hard to justify.
### 3.5 Enforcement is real but fact-specific
German enforcement has been administrative rather than fine-based for this use case. The ULD Schleswig-Holstein investigated two city-marketing operators using passive probe requests and concluded there was no legal basis; both stopped [22]. The Bavarian authority inspected a fashion chain’s Wi-Fi tracking test after a complaint; the company stopped. The Hamburg authority’s 2014/2015 report treats WLAN/Bluetooth indoor tracking as highly intrusive and personal-data processing requiring a legal basis. There is no published German fine specifically for passive Wi-Fi/BLE footfall tracking; that is not evidence of legality.
The Dutch Enschede case is instructive. The Dutch DPA fined the municipality €600,000 for Wi-Fi tracking in the city centre (pseudonymised MAC + location, used to count unique visitors). The fine was annulled by the Rechtbank Overijssel on 2 February 2024 because the DPA had not proved that the municipality processed personal data; the Raad van State upheld that outcome on 29 July 2026, largely on a procedural ground [19, 36, 37]. The case is not a safe harbour. It shows that the legal status is fact-specific and that the regulator must prove identifiability. The French *JCDecaux* case went the other way and is the stronger warning against relying on hashing/salting [18].
### 3.6 The operator’s claim, examined
The Code for Heilbronn blog says the sensor “remembers the device addresses,” hashes them, stores them for at most one hour, and transmits only the number of counted addresses [2]. The public 2026 firmware does not literally implement a one-hour store: it truncates the last two bytes into a 16-bit bitmap and resets the bitmap at each wake [3]. Both designs are materially better than storing raw MACs. Neither is a formal anonymisation guarantee:
- the raw MAC is observable at the sensor before truncation;
- the 16-bit ID has a small space and frequent collisions;
- the one-hour cache (if present) creates a one-hour linkability window per sensor;
- a keyed or unkeyed hash can be reversed or enumerated by an adversary with the key or the 48-bit space;
- the published per-sensor 15-minute counts can reveal a single device in low-traffic cells.
The honest formulation is: **the architecture is data-minimising at the edge, but the sensing stage still processes personal data, and the published aggregate may become personal data in small cells.** That is a privacy-by-design claim, not an anonymity claim.
### 3.7 What a defensible privacy claim requires
A defensible claim needs a threat model, a privacy unit, a formal or architectural guarantee, a retention policy, a DPIA, and a legal basis. The PET stack in Section 4.4 makes this concrete. The minimum for a pilot is:
- passive only; no probe requests, association requests, or OS APIs;
- no raw MAC retention; memory only; delete immediately after dedup;
- per-sensor, per-window keyed hashing, not a 16-bit truncation;
- no cross-sensor or cross-day linkage;
- aggregate to zones/time bins with a k-threshold and suppression;
- central differential privacy at a trusted aggregator, with a device-day privacy unit and contribution bounds;
- a published budget ledger and explicit guarantee language;
- on-site signage, a real opt-out, and Art. 28 contracts or an Art. 26 arrangement.
If the project cannot document why § 25 does not apply, it should either obtain consent (impractical at city scale) or switch to a sensor that does not extract MAC addresses. That is the strongest privacy argument, and it is also the most reliable measurement argument: the alternatives do not have the same address-regime bias.
## 4. What privacy-first should mean
### 4.1 Sensing alternatives
The good news is that the MAC-based design is not the only option. The alternatives have different accuracy, cost, privacy, and deployment profiles. Table 1 summarizes the evidence.
| Modality | Reported performance | Cost (2026 planning) | Privacy risk | Best use |
|---|---|---|---|---|
| BLE MAC sniffing (baseline) | 5–12% of devices discoverable; 97%/92% count accuracy only when penetration is assumed 100% [10]; MAC randomization defeated for ~96% of Android phones [7] | €100–500 | Very high | None for public footfall; at most a short-lived calibrated index under a DPIA |
| Thermal/IR array | 89–95% doorway accuracy; degrades >30 °C, >7.5 m, groups [30] | €150–400/node | Low | Default at store entrances and narrow passages |
| mmWave radar | 96% presence, 69% count outdoor; −2.5% aggregate bias [28]; 3-month F1 84%, flow r 60–85% [29] | €300–700 | Low | Wide, weather-exposed crossings; presence/flow |
| Edge AI camera | 13.8% day / 29.8% night MAPE outdoors; 2–14% in official review [26] | €500–2,000; commercial €1,100–7,400 | High (DPIA) | 2–4 reference sites; pedestrian/cyclist classification |
| LiDAR | >97% disaggregate count; 1.6–4.3% 15-min error [27] | €1,000–8,000 | Low–moderate | 1–2 high-accuracy reference sites |
| Aggregated probe data | City-wide movement; coverage/bias unknown | Licence | High/commercial | City context, O/D, trends — not street-level truth |
| Manual counts | ~11% clicker error; 2-hour observer fatigue | €25–40/hour | Low | Quarterly calibration audits |
| Sensor fusion | Hierarchical model; no single accuracy figure | €20k–60k development | Depends on inputs | The core of a defensible stack |
*Table 1. Sensing alternatives. Costs are planning estimates, not quotations; accuracy figures are from the cited studies and are context-dependent.*
The table supports a clear ranking for Heilbronn. Thermal arrays are the best privacy/cost/accuracy trade-off at narrow gateways. Edge cameras are the accuracy benchmark but carry the heaviest DPIA and public-acceptance burden. mmWave and LiDAR are gap-fillers: good at presence/flow, weaker at exact counts, and best used at a few wide or high-value sites. Aggregated probe data are useful for city-wide context but are a non-probability sample; they cannot replace local ground truth. Manual counts are the only credible ground truth and should be a recurring program, not a one-off.
### 4.2 A recommended Heilbronn stack
A defensible pilot would use:
- **8–12 thermal nodes** at store entrances, the passages around Stadtgalerie, Marahaus and Götzenturm, and transit shelter entrances;
- **2–4 edge cameras** at wide plazas or complex intersections as the accuracy benchmark, with on-device inference and no frame retention;
- **1–2 mmWave/LiDAR nodes** at wide, weather-exposed crossings (Hagenbuchersee, Neckarufer);
- **aggregated probe data** for city-wide context and event flows, not street-level truth;
- **twice-yearly manual/video ground truth** at each site, stratified by time and weather;
- **transit validators** as a covariate where available.
This stack does not need MAC addresses. It also avoids the address-regime bias: thermal and radar count bodies, cameras count people, and manual counts calibrate the rest.
### 4.3 Calibration
Uncalibrated sensors are indexes, not measurements. The calibration program should estimate a site-specific detection probability and expansion factor, with uncertainty, and propagate that uncertainty into the decision. Options:
- **Ratio/expansion estimator:** `P_hat = k · y`, where `k = 1/r` and `r` is the detection rate measured against manual counts. This is standard sampling theory and is used in Bluetooth transport studies [10, 11].
- **Capture–recapture:** use two independent sensors or a sensor plus manual counts to estimate the unobserved population. This requires careful independence assumptions and is fragile for pedestrians.
- **Regression calibration:** regress ground truth on sensor counts plus covariates (time, weather, site), with a time-based holdout. This is the most flexible but can hide bias if the calibration sample is not representative.
- **Hierarchical Bayesian calibration:** model sensor-specific detection probabilities with partial pooling across sites. This is the right default when there are many sites and few counts per site; it produces 80% credible intervals and borrows strength.
The pilot should publish the calibration dates, the reference method, the estimated factor, and the interval. A single global 0.7 is not a calibration.
### 4.4 The privacy stack
The PET analysis supports a layered design:
- **No raw MACs.** If a MAC-based sensor is used at all, hash at the edge with a per-sensor, per-window key, rotate the key, and destroy the old key. Do not use a plain hash, a static salt, or a 16-bit ID.
- **Edge aggregation.** Transmit counts only; no identifiers in the uplink.
- **No cross-sensor/cross-day linkage.** No trajectories, dwell, or repeat-visit metrics.
- **Central DP at a trusted city aggregator.** Use a device-day privacy unit and contribution bounds (e.g. at most four sensor-bin pairs per device per day). Publish fixed daily/weekly coarse releases with a budget ledger.
- **Do not use local DP for sparse 15-min bins.** Local DP error scales as `1/(ε√n)` versus central `1/(εn)`, so it destroys utility at small-city scale. At ε = 1, a 14-device cell carries ±3 noise; a live 15-min public heatmap is not viable with a meaningful annual budget [23].
- **k-anonymity as a secondary check.** Four spatio-temporal points identify ~95% of people in a 1.5M-person mobility dataset; generalizing to k = 20 erases the local footfall signal [25]. Use k as a suppression rule, not the primary guarantee.
- **No federated learning.** The pedestrian’s phone does not run our code, so FL is not applicable to passive BLE sniffing.
- **No non-DP synthetic data as an anonymity claim.** Non-DP synthetic data has no formal guarantee; use DP synthesis plus membership-inference/Anonymeter evaluation for public release, not for the live dashboard.
The guarantee language matters. Say “privacy-enhanced,” “aggregated with a k-threshold,” or “central DP with ε = …”; do not say “anonymous” unless a documented anonymisation assessment supports it. Publish what cannot be guaranteed: fine-grained public 15-min heatmaps, individual trajectories, repeat-visit metrics.
### 4.5 Pilot protocol
A 12-week pilot at six sites would be enough to estimate detection rates and calibrate the stack. The go/no-go criteria should be explicit:
- **MAPE ≤ 20%** against manual/video ground truth;
- **|bias| ≤ 10%** at the site level;
- **80% interval coverage ≥ 75%**;
- **no raw video or point-cloud retention**;
- **DPIA signed off**; legal basis documented;
- **cost within the pilot budget**.
A planning estimate is **€81k–161k** for year 1 and **€30k–50k** recurring, excluding backend and staff time. These are estimates, not quotations; the point is that the privacy-first stack is not free, but it is comparable to the legal and reputational cost of a contested MAC deployment.
## 5. From sparse observations to decisions
### 5.1 Four estimands
Most overclaiming in urban sensing comes from letting one model silently answer four different questions:
1. the expected count at each sensor;
2. the latent spatial intensity field (heatmap);
3. the forecast of either;
4. the decision (staffing, routing, operations).
The heatmap is a state estimate; the staffing plan is a decision under uncertainty. They need different validation. The paper should name the estimand before choosing a model.
### 5.2 Spatial support
Eight sensors over a 580 m × 280 m bounding box give an equivalent uniform spacing of about **142 m** if they fill the area, or about **72 m** along a single street if they lie along one axis. A 100–150 m cell is defensible; a 25–50 m raster is a visualization, not an inference. The heatmap should be on the **walkable network**, not an isotropic Gaussian over building footprints. Every map should show the distance to the nearest sensor and a prediction-standard-deviation mask.
### 5.3 Spatial model
Footfall is a non-negative, overdispersed, zero-inflated count. A hierarchical negative-binomial model with a low-rank spatial field (Matérn GP/SPDE or a small set of spatial basis functions) is the statistically correct default [31, 32]. Ordinary kriging is a baseline; IDW is a visualization baseline. With eight sensors, a free CAR/BYM2 field on a fine grid is not identifiable—the prior would do almost all the work. Use a parametric Matérn covariance with a hierarchical prior, or fix the smoothness and estimate only range and nugget. Pool over time, but allow weekday/weekend and day/night non-stationarity. Report leave-one-sensor-out error and prediction intervals.
### 5.4 Forecast
The forecast core should be a global (panel) gradient-boosted model with quantile outputs and conformal calibration [33, 34]. Features: calendar, weather forecast, holidays, events (only if announced before issue time), POI/transit, lags, and sensor identity as a pooled effect. This is the strongest small-data baseline and is likely to beat an LSTM/Transformer/GNN on 8 series × 5 months. Deep spatiotemporal models and time-series foundation models should be benchmarks and ablations, not the default. The largest leakage risk is weather: training on observed weather and serving with forecast weather is optimistic. Evaluate with archived forecast weather, or a calibrated forecast-error simulation, and report both the observed-weather oracle and the deployable number.
### 5.5 Decision layer
The decision layer should be explicit about the objective and constraints. For staffing, the current heuristic is a reasonable first step:
```text
required staff = ceil(forecast / visitors-per-staff-hour × safety factor)
subject to minimum staff, shift length, opening hours, cost
```
A production system could use a stochastic optimization or a robust optimization, but only when the utility and constraints are measured. The important move is to separate the predictor from the decision rule, propagate uncertainty, and evaluate **decision regret**: what would the plan have cost if the true footfall had been known? Without POS/queue/labor data, the paper can only compare to a flat schedule under stated assumptions. That is a heuristic comparison, not ROI.
### 5.6 Evaluation harness
A credible evaluation harness includes:
- **time-based holdouts** and rolling-origin evaluation;
- **leave-one-sensor-out** to test spatial generalization;
- **multiple seeds** for stochastic models;
- **Diebold–Mariano** tests for forecast differences;
- **calibration** (interval coverage, CRPS, quantile loss);
- **cost curves** and decision regret;
- **data provenance** (source, version, transformations, missingness);
- **model cards** and a metrics JSON.
This is standard MLOps hygiene [35], but it is often skipped in hackathon systems. The existing PulseCast metrics are a start: they include a seasonal-naive baseline, a time-based holdout, and a weather ablation. They do not include decision regret, spatial cross-validation, or calibration of the uncertainty band.
### 5.7 What the existing results show
The PulseCast results are a useful anchor. The production 7-day model reaches 3.32/27.5/0.777; the day-ahead model 3.36/27.8/0.763; the seasonal naive 5.40/44.7/0.462. Per-sensor MAE ranges 2.26–5.83. The weather ablation is large; the holiday effect is small. These results show that a global model can learn the daily/weekly shape and the weather sensitivity of the index. They do not show that the index is people, that the model generalizes across cities, or that the staffing plan improves outcomes. The honest reading is a **relative decision-support signal**.
![Figure 4. Forecast versus actual.](figures/fig_forecast.png)
*Figure 4. Forecast versus actual for one sensor over the 28-day holdout. The model captures the daily/weekly shape but misses spikes; the band is a planning aid, not a calibrated 95% interval unless coverage is reported.*
## 6. How far it generalizes
### 6.1 The six challenges
The OKF_Vault bundle contains six challenges: Pulse of the City (footfall), WasteRouting.AI (glass fill levels), Measure Where It Matters (heat-sensor placement), LoRaWAN network planning, Riding the Sentiment (cable-car debate), and the Learning Guide (family offers). They are not six versions of the same problem. They differ in what is observed, whether a spatial field exists, whether the decision is forward-looking, and whether the decision changes the observation process.
### 6.2 The shared contract
The shared abstraction is a **decision-grade sensing contract**, not a universal model:
```text
observation: y_t = H_t x_t + e_t
field: x_hat_t = I(y_<=t, H_t, c_t, G)
forecast: x_hat_t+h = F(x_hat_<=t, c_t+h; theta)
decision: a* = argmax_a E[U(a, x_t+h)] subject to constraints
evaluation: L(y, x_hat) + R(a*, x_oracle) + privacy leakage
```
`x_t` is a latent state over spatial units; `H_t` is the observation operator; `c_t` is context available at decision time; `G` is geometry or graph structure; `I` is spatial inference; `F` is temporal forecasting; `U` is the decision utility. The key distinction is **state estimation** versus **design**: in staffing, the pipeline estimates a state and acts on it; in sensor placement, the decision changes what is observed next. Calling both “forecasting” hides the inversion.
### 6.3 Transfer matrix
| Challenge | Class | What transfers | What must change | Evidence |
|---|---|---|---|---|
| Pulse / staffing | Direct | Observation handling, local field, weather-aware forecast, staffing heuristic, deployment | Store-specific assumptions; calibration; outcome data | MAE 3.32 vs 5.40 naive |
| Pulse / advertising | Direct (local) | Same field and forecast; allocation objective replaces staffing | Ad inventory, cost, reach, dwell, conversion; city-wide sites for a real campaign | No result; analytical extension |
| Waste routing | Conditional | Temporal forecast, clustering, configurable VRP, holiday/season context | Coordinates or zone map, capacity/type, fleet/depot/shift, overflow ground truth | No result; dataset documented |
| Sensor placement | Sibling | Observation model, spatial inference, error-vs-cost evaluation | Decision changes `H_t`; active learning/optimal design; no temporal forecast | Protocol only |
| LoRaWAN planning | Sibling | Link/coverage prediction, candidate-site ranking, address-weighted coverage | Packet-loss estimate, buildings/terrain, longer window, GPS calibration | No result; dataset documented |
| Cable-car sentiment | Non-application | At most an event/attention signal | No physical field, no forecast target, no allocation decision; different NLP stack | Descriptive task |
| Learning guide | Non-application | At most a location/context consumer | No sensing, no field, no forecast; matching and child-privacy regime | Product task |
*Table 2. Transferability matrix. The stack generalizes as a contract across a subset; two challenges are siblings and two are out of scope.*
### 6.4 Case studies
**Staffing (empirical).** The only end-to-end evidence-backed case. The path is: PAX index → weather-aware global forecast → staffing heuristic → plan. The next step is calibration and decision-level evaluation.
**Advertising (analytical extension).** The same field and forecast can rank the eight locations and time windows by expected exposure. This is a decision-layer substitution, not a new sensing study. It cannot support city-wide ad planning or effectiveness claims without more sites and ad-economics data.
**Waste routing (conditional).** The temporal-forecast and decision-rule pattern transfers, but the spatial object becomes a container graph, not a metric field. The dataset has no coordinates, capacity, fleet, depot, or shift data; a route-saving claim would be driven by assumed geometry. The honest output is a method and a data-acquisition plan, not a result.
**Sensor placement and LoRaWAN (siblings).** The spatial-inference layer transfers, but the decision is a design decision: where to place sensors or gateways. The evaluation is an error-versus-cost curve, not a temporal forecast. LoRaWAN has real data and explicit candidate sites, but the missing packet-loss baseline is a serious limitation. Both are separate “spatial design” contributions, not applications of the forecast core.
### 6.5 Falsifiers
The generalization claim is only credible if it can fail. The paper should state the tests:
1. Waste forecast fails a naive baseline after accounting for irregular sampling and `overfull` encoding.
2. LoRaWAN link model cannot beat a distance-only baseline on held-out gateways.
3. Advertising allocation is unstable under plausible weather scenarios.
4. Store-location ranking with eight points is no better than random at city scale.
5. Decision regret is large despite low MAE.
6. Aggregate counts can be re-identified, or the privacy claim fails the threat model.
A paper that reports these tests, even if some fail, is stronger than one that asserts six-way generalization.
## 7. Evaluation agenda
### 7.1 Measurement
- Ground-truth counts at each site, stratified by time, weekday, and weather; manual counts are the reference, with inter-rater reliability.
- Penetration and device-multiplicity study for any MAC-based sensor, with a documented sampling design.
- Duty-cycle and detection-radius study; report the probability that a person with a given dwell time is detected.
- False-positive/false-negative audit using co-located thermal/camera/manual counts.
### 7.2 Forecast
- Rolling-origin evaluation, not a single 28-day holdout.
- Leave-one-sensor-out and leave-one-site-out.
- Calibration: interval coverage, CRPS, quantile loss.
- Decision regret: compare the plan to an oracle plan and a flat schedule under explicit assumptions.
- Weather ablation with **archived forecast weather**, not only observed weather.
### 7.3 Privacy
- DPIA before deployment; DPO sign-off; Art. 36 consultation if residual high risk.
- Threat model: singling out, linkability, inference, reconstruction, sensor compromise.
- Formal guarantee: central DP with a stated ε/δ, privacy unit, contribution bound, and budget ledger; k-threshold as a secondary check.
- Re-identification test on the published aggregates; small-cell suppression audit.
- Legal basis and § 25 memo; signage and opt-out usability test.
### 7.4 System
- 120 concurrent SSE clients for 5 minutes; p95 event latency; open success rate.
- Image sizes and cold deployment time.
- Failure drills: FastAPI down, Open-Meteo down, tunnel down, pod restart.
- Observability: health/ready, data/model versions, staleness labels.
### 7.5 Proposed experiments on the existing CSV
These are real analyses that do not require new sensors:
1. **Quantization audit** — confirm 100% multiples of 0.7; reconstruct `U`; compare `density-01` regimes. (Done.)
2. **Coverage/zero-inflation** — per-sensor missingness, zero runs, correlation with battery. (Done.)
3. **Cross-sensor correlation** — raw, detrended, daily; effective number of independent sensors. (Done.)
4. **Subsampling** — train on 1-in-k rows (30/60/120 min) and measure MAE/WAPE degradation. This quantifies the value of temporal sampling.
5. **Leave-one-sensor-out** — train on seven, test on the held-out sensor. This tests shared structure.
6. **Factor sensitivity** — rescale counts by 0.5/0.7/1.0/1.4 and compare staffing plans. This shows why calibration matters for decisions.
7. **Decision-level evaluation** — compare the plan to a flat schedule on coverage and cost under stated assumptions.
Subsampling and factor sensitivity are the most informative for the paper’s thesis: they show that the model’s accuracy is not the same as decision quality, and that a single calibration constant can change the plan.
## 8. Discussion and limitations
The main limitation is the measurement, not the model. The sensor samples about 10 seconds out of every 15 minutes, counts only randomized BLE addresses, truncates the address to 16 bits, multiplies by a fixed 0.7 factor, and cannot distinguish people from devices. The published value is a useful relative index, but it is not a census and not a 15-minute count. Device penetration is unknown. A single global factor cannot absorb device ownership, multiple devices per person, Bluetooth settings, and the short scan window.
Coverage is also a limitation. Eight sensors in a 580 m × 280 m cluster cannot support a city-wide heatmap or a store-location recommender. Only 28.2% of 15-minute slots have all eight sensors, and gaps are not random. The 28-day holdout is short, the weather is observed rather than forecast, and there is no event log, POS data, or labor-cost data. The deployment has one API replica, no SSO, no HA, and no live sensor ingestion. The 15-minute and 1-hour horizons are replay/nowcast, not a live feed.
The privacy claim also needs care. The application stores no MAC addresses, and only aggregate counts leave the sensor. But the sensing stage still processes link-layer identifiers at the edge; the 16-bit ID is a pseudonymous dedup key, not a cryptographic hash; and passive MAC sensing in public space has a contested legal basis in Germany. The “privacy-preserving” label is an architectural claim, not a formal guarantee. A production deployment would need a DPIA, a documented legal basis, data minimization, retention limits, and, where possible, a sensor that does not extract MAC addresses at all.
The paper is also an opinionated design proposal, not a controlled trial. The alternative architecture has not been deployed at Heilbronn scale; the cost figures are planning estimates; the legal analysis is not legal advice. The tone should be constructive: Code for Heilbronn is a volunteer civic-tech group that built a low-cost, privacy-conscious prototype under real constraints. The critique is of the claim and the measurement design, not the volunteers’ intent.
**What to keep from the hackathon build.** The decision-first framing, the global model, the time-based holdout, the seasonal-naive baseline, the transparent staffing heuristic, the cached fallback, the build-time batch, and the honest limitations are all worth keeping. The plan screen is the product; the map and charts support it.
**What to replace.** The sensing layer (thermal/radar/camera/probe/manual), the calibration (site-specific, with uncertainty), the privacy stack (no raw MACs, central DP, k-threshold), the spatial model (network-aware hierarchical field), the forecast (quantile GBM with conformal calibration and forecast weather), and the platform (auth, observability, SLOs, multi-tenancy) in that order. Do not scale the model before fixing the measurement.
## 9. Related work
**Passive counting.** Bluetooth and Wi-Fi sensing has a long history in pedestrian counting. Classic Bluetooth inquiry detected 2–7% of pedestrians in early studies [11]; static Bluetooth sensors produced usable trends but required site-specific correction factors [12]; Wi-Fi/Bluetooth sensors can estimate densities and wait times when penetration is known [10]. MAC randomization reduces linkability but is not a census, and fingerprinting can defeat it [7, 8, 9]. Our audit is consistent with this literature and makes the sampling design explicit.
**Forecasting.** Gradient-boosted trees remain a strong baseline for tabular, covariate-rich, small-data problems [33]. Weather is a known driver of pedestrian activity, but its magnitude is context-dependent; we treat it as predictive, not causal. The heatmap/forecast literature offers a menu of spatial and temporal models; at eight sensors, the statistically principled choice is a hierarchical spatial field plus a global quantile forecast, not a deep model [31, 32, 34].
**Privacy and law.** The legal analysis draws on the GDPR, *Breyer*, the EDPB pseudonymisation and anonymisation guidelines, *JCDecaux*, the Dutch Enschede case, and German DPAs [13–22]. The technical privacy analysis draws on differential privacy, k-anonymity, membership inference, and mobility uniqueness [23–25]. The conclusion is that “privacy-preserving” is an architectural claim that must be specified, not a label.
**Urban sensing and decision support.** Google Maps-style traffic is aggregated, map-matched probe data, not per-user triangulation; its public privacy story is stronger on data minimisation than on formal guarantees for the traffic layer. The lesson for a city is to make the parameters explicit: consent, aggregation, DP, retention, and audit. The sensing alternatives literature provides the accuracy/cost/privacy trade-offs [26–30]. The MLOps literature provides the evaluation hygiene [35]. The contribution here is to connect the measurement audit to the decision layer and to the legal/privacy constraints in one place.
## 10. Conclusion
The PulseCast case is a useful cautionary tale. A small, weather-aware model can turn a noisy inner-city index into a weekly staffing plan that beats a seasonal-naive baseline and runs in under two seconds on a real deployment. But the index is not people. Each value is a short BLE scan scaled by 0.7, sampled every 15 minutes, with unknown penetration, incomplete coverage, and a hardware calibration break. The “privacy-preserving” label is an architectural claim, not a formal guarantee; the 16-bit truncation is not a hash; and passive MAC sensing in public space has a contested legal basis.
The constructive path is clear:
1. **Calibrate** the index against manual/video ground truth, with site-specific factors and uncertainty.
2. **Change the sensing layer** to thermal/radar/camera/probe/manual, with a DPIA and no raw MACs.
3. **Model the field** as a network-aware hierarchical count process, at a 100–150 m support, with uncertainty.
4. **Forecast** with a global quantile GBM, forecast weather, and conformal calibration.
5. **Evaluate the decision**, not just MAE: decision regret, cost curves, and a pilot with go/no-go criteria.
6. **Publish the limits**: what the data do not measure, what the privacy guarantee is, and what would falsify the claim.
The honest conclusion is narrower and more useful than a city-scale claim: the decision loop works, the relative forecast is credible, and the next step is calibration and a privacy-first sensing layer. We would rather ship a plan a manager can use and state what it does not measure than promise a city-wide heatmap the data cannot support.
## Data, code, and AI disclosure
The raw challenge CSV is not redistributed. Derived aggregates, the trained model artifacts, the metrics JSON, and the figure-generation scripts can be shared subject to the organizers’ permission. The code for the audit and the PulseCast system is at `[TODO: repository]`; the live demo is at `[TODO: URL]`. The research notes that support the audit are in `paper/research/` and can be deposited alongside the paper or linked from the repository.
AI tools were used for drafting and editing. The authors verified every number, citation, and claim against the delivered data, the project metrics, or a cited source. AI tools are not authors and take no responsibility for the content. The legal analysis is research, not legal advice; a deployment should obtain a written opinion from German counsel, especially on the § 25 TDDDG question.
## References
[1] Schwarz Digits / Code for Heilbronn. *The Pulse of the City*, challenge brief, 2026. Local: `OKF_Vault/Schwarz Digits/FCH Challenge The _Pulse of the City_.pdf`.
[2] Code for Heilbronn. “Passantenzahlen messen mit LORA,” 18 September 2025. https://codeforheilbronn.de/blog/2025/09/passantenzahlen-messen-mit-lora/
[3] Code for Heilbronn. `lorax` / `sensor-pax` firmware and backend, 2026. https://codeberg.org/cfhn/lorax
[4] `ESP32-Paxcounter` v3.8.0 and `libpax`. https://github.com/cyberman54/ESP32-Paxcounter; https://github.com/dbinfrago/libpax
[5] PulseCast data-quality analysis. `paper/research/07-data-quality.md`, 2026.
[6] Bluetooth SIG. “Enhancing device privacy and energy efficiency with Bluetooth Randomized RPA Updates,” 15 May 2025. https://www.bluetooth.com/blog/enhancing-device-privacy-and-energy-efficiency-with-bluetooth-randomized-rpa-updates/
[7] J. Martin, T. Mayberry, C. Donahue, L. Foppe, L. Brown, C. Riggins, E. C. Rye, and D. Brown. “A study of MAC address randomization in mobile devices and when it fails.” *Proceedings on Privacy Enhancing Technologies* 2017(4):365–383. DOI: 10.1515/popets-2017-0054.
[8] J. K. Becker, D. Li, and D. Starobinski. “Tracking anonymized Bluetooth devices.” *Proceedings on Privacy Enhancing Technologies* 2019(3):50–65. DOI: 10.2478/popets-2019-0036.
[9] M. Vanhoef, C. Matte, M. Cunche, L. S. Cardoso, and F. Piessens. “Why MAC address randomization is not enough: An analysis of Wi-Fi network discovery mechanisms.” *AsiaCCS 2016*, 413–424. DOI: 10.1145/2897845.2897860.
[10] A. Kurkcu and K. Ozbay. “Estimating pedestrian densities, wait times, and flows with Wi-Fi and Bluetooth sensors.” *Transportation Research Record* 2644(1):72–82, 2017. DOI: 10.3141/2644-09.
[11] T. Nicolai and H. Kenn. “About the relationship between people and discoverable Bluetooth devices in urban environments.” *Mobility ’07*, 72–78. DOI: 10.1145/1378063.1378076.
[12] Y. Malinovskiy, N. Saunier, and Y. Wang. “Analysis of pedestrian travel with static Bluetooth sensors.” *Transportation Research Record* 2299:137–149, 2012. DOI: 10.3141/2299-15.
[13] Regulation (EU) 2016/679 (GDPR), https://eur-lex.europa.eu/eli/reg/2016/679/oj; Directive 2002/58/EC (ePrivacy), Art. 5(3), https://eur-lex.europa.eu/eli/dir/2002/58/oj.
[14] CJEU. Case C-582/14, *Patrick Breyer v Bundesrepublik Deutschland*, 19 October 2016. https://curia.europa.eu/juris/liste.jsf?num=C-582/14
[15] EDPB. *Guidelines 2/2023 on Technical Scope of Art. 5(3) of ePrivacy Directive*, v2.0, 7 October 2024. https://www.edpb.europa.eu/system/files/2024-10/edpb_guidelines_202302_technical_scope_art_53_eprivacydirective_v2_en_0.pdf
[16] EDPB. *Guidelines 01/2025 on Pseudonymisation*, 16 January 2025. https://www.edpb.europa.eu/system/files/2025-01/edpb_guidelines_202501_pseudonymisation_en.pdf
[17] EDPB. *Guidelines 02/2026 on Anonymisation*, v1.0, 7 July 2026 (public consultation). https://www.edpb.europa.eu/system/files_en?file=2026-09%2Fedpb_guidelines_202602_anonymisation_v1_en_0.pdf
[18] Conseil d’État. *JCDecaux France*, no. 393714, 8 February 2017. https://www.conseil-etat.fr/fr/arianeweb/CE/decision/2017-02-08/393714
[19] Autoriteit Persoonsgegevens. *Wifi and bluetooth*; *Legal bases for WiFi tracking and Bluetooth tracking*. https://www.autoriteitpersoonsgegevens.nl/en/themes/internet-and-smart-devices/wifi-and-bluetooth
[36] Rechtbank Overijssel. *Enschede / Autoriteit Persoonsgegevens*, ECLI:NL:RBOVE:2024:594, 2 February 2024.
[37] Afdeling bestuursrechtspraak van de Raad van State. *Autoriteit Persoonsgegevens / College van B&W Enschede*, ECLI:NL:RVS:2026:4403, 29 July 2026.
[20] Telekommunikation-Digitale-Dienste-Datenschutz-Gesetz (TDDDG), § 25. https://www.gesetze-im-internet.de/ttdsg/__25.html
[21] LfDI Baden-Württemberg. *Muss-Liste* nach Art. 35 Abs. 4 DS-GVO, Nos. 2 and 10. https://www.baden-wuerttemberg.datenschutz.de/
[22] ULD Schleswig-Holstein. *Tätigkeitsbericht 2019*, §5.4.8. https://www.datenschutzzentrum.de/tb/tb37/uld-37-taetigkeitsbericht-2019.pdf
[23] C. Dwork and A. Roth. “The algorithmic foundations of differential privacy.” *Foundations and Trends in Theoretical Computer Science* 9(3–4):211–407, 2014. DOI: 10.1561/0400000042.
[24] A. Pyrgelis, C. Troncoso, and E. De Cristofaro. “Knock knock, who’s there? Membership inference on aggregate location data.” *NDSS 2018*. https://www.ndss-symposium.org/wp-content/uploads/2018/02/ndss2018_05B-5_Pyrgelis_paper.pdf
[25] Y.-A. de Montjoye, C. A. Hidalgo, M. Verleysen, and V. D. Blondel. “Unique in the crowd: The privacy bounds of human mobility.” *Scientific Reports* 3:1376, 2013. DOI: 10.1038/srep01376.
[26] B. Cyganek, P. Janyst, M. Knapik, Ł. Przebinda, M. Ptaszkiewicz, and T. Tajmajer. “Evaluation of an embedded computer vision system for outdoor media audience analysis.” *PP-RAI 2022*. https://wydawnictwo.umg.edu.pl/pp-rai2022/pdfs/07_pp-rai-2022-101.pdf
[27] A. Lesani, E. Nateghinia, and L. F. Miranda-Moreno. “Development and evaluation of a real-time pedestrian counting system for high-volume conditions based on 2D LiDAR.” *Transportation Research Part C* 114:20–35, 2020. DOI: 10.1016/j.trc.2020.01.018.
[28] G. Vaidya and M. A. Zúñiga. “Exploiting mmWave and deep-learning models to estimate people count in urban scenarios.” *IoTDI 2024*, 73–84. DOI: 10.1109/IoTDI61053.2024.00011.
[29] W. Wang, G. Vaidya, A. K. Bhattacharjee, F. Fioranelli, and M. A. Zúñiga. “A long-term study of mmWave sensing in an outdoor urban scenario.” *DCOSS-IoT 2023*, 240–247. DOI: 10.1109/DCOSS-IoT58021.2023.00049.
[30] C. Perra, A. Kumar, M. Losito, P. Pirino, M. Moradpour, and G. Gatto. “Monitoring indoor people presence in buildings using low-cost infrared sensor array in doorways.” *Sensors* 21(12):4062, 2021. DOI: 10.3390/s21124062.
[31] N. A. C. Cressie. *Statistics for Spatial Data* (rev. ed.). Wiley, 1993.
[32] F. Lindgren, H. Rue, and J. Lindström. “An explicit link between Gaussian fields and Gaussian Markov random fields: The stochastic partial differential equation approach.” *Journal of the Royal Statistical Society: Series B* 73(4):423–498, 2011. DOI: 10.1111/j.1467-9868.2011.00777.x.
[33] J. H. Friedman. “Greedy function approximation: A gradient boosting machine.” *Annals of Statistics* 29(5):1189–1232, 2001. DOI: 10.1214/aos/1013203451.
[34] V. Vovk, A. Gammerman, and G. Shafer. *Algorithmic Learning in a Random World*. Springer, 2005.
[35] D. Sculley, G. Holt, D. Golovin, E. Davydov, T. Phillips, D. Ebner, V. Chaudhary, M. Young, J.-F. Crespo, and D. Dennison. “Hidden technical debt in machine learning systems.” *NeurIPS 2015*, 2503–2511.
+221
View File
@@ -0,0 +1,221 @@
# PulseCast: Turning a Noisy Footfall Index into a Weekly Staffing Plan
**Authors:** [AUTHOR TODO]
**Affiliation:** [AFFILIATION TODO]
**Date:** 14 November 2026
**Artifact:** [demo URL TODO] · [code repository TODO]
## Abstract
Inner-city store managers must decide next week's shifts without knowing next week's footfall. We present PulseCast, a decision-first staffing copilot built on eight passive Bluetooth footfall counters in a ~580 m × 280 m cluster in Heilbronn, Germany. The system fuses five months of 15-minute device counts with Open-Meteo weather and Baden-Württemberg public holidays, trains a global gradient-boosted model on a time-based split, and converts the forecast into a weekly shift plan through a transparent staffing heuristic. On a 28-day holdout, the weather-aware 7-day model reaches MAE 3.32, WAPE 27.5%, and R² 0.777, versus MAE 5.40, WAPE 44.7%, and R² 0.462 for a seasonal-naive baseline. The plan is generated in under two seconds and the system is deployed on STACKIT SKE behind a Cloudflare Tunnel. A code-level audit shows that each published value is a ~10-second count of randomized BLE addresses multiplied by 0.7, not a 15-minute count of people. We therefore frame the result as a relative activity index that supports staffing decisions, not as a city-wide people counter, and we state what calibration and privacy work would be required before absolute claims.
## 1. Introduction
A store manager on Kaiserstraße knows the weather forecast and the staff roster. She does not know how many people will walk past next Saturday. Raw counts are not a decision; the manager needs a forecast, an explicit staffing rule, and a way to test a scenario. A dashboard that shows history does not close that gap.
The Schwarz Digits *Pulse of the City* challenge asks teams to turn privacy-preserving pedestrian counts from Heilbronn into predictive tooling for advertising, store locations, and staffing [1]. We chose staffing: eight fixed locations, five months of multi-sensor history, and a clear daily and weekly rhythm. Store-location ranking would need city-wide coverage that eight sensors in a 580 m cluster cannot provide; a city-wide heatmap would be a visualization, not a measurement.
PulseCast makes three contributions. It treats the decision as the product: the forecast feeds a weekly shift plan. It uses a global, weather-aware gradient-boosted model with a time-based holdout and a seasonal-naive baseline, rather than a model the data cannot support. It is deployed as a three-service system on SKE with a cached fallback and a live scenario layer. It also reports a code-level audit showing that the published value is a short-window count of randomized BLE addresses scaled by 0.7, not a 15-minute count of people. That audit changes how the results should be read and what the next version must fix.
We do not claim a production people counter, a city-wide heatmap, annual seasonality, live sensor data, or a proven return on investment. We claim a working decision loop, a credible relative forecast, and an honest account of the measurement limits.
## 2. Data
### 2.1 The delivered PAX data
The dataset contains 192,464 CSV rows: 96,232 density readings and 96,232 battery readings. There are 12 device IDs; after merging the two `density-01` device IDs and excluding three test sensors, eight usable locations remain in a ~580 m × 280 m cluster around Hagenbuchersee, Neckarufer, Stadtgalerie, Marahaus and Götzenturm. The eight-sensor window runs from 12 June to 7 November 2026, with a median cadence of 15.3 minutes. Values range from 0 to 292.6; the multi-sensor mean is ~14.4, the median 7.7, p95 46.9 and p99 78.4. The inner city peaks between 18:00 and 21:00 local time, with the top at 19:00; Saturday is the busiest day on average (mean 19.1) [2].
The sensors are LoRaWAN-connected PAX counters that passively sniff Bluetooth Low Energy (BLE) advertisements and transmit aggregated device counts [1, 2]. The challenge brief calls them privacy-preserving because they count devices rather than identify people. The counts are estimates, not a census: device penetration, Bluetooth settings, and the scanning design all matter.
### 2.2 What one value actually is
A code-level audit of the deployed Code for Heilbronn `lorax` firmware changes the interpretation of every number in this paper [3, 4]. The node wakes from deep sleep, scans BLE passively for about 10 seconds, counts distinct locally administered (“randomized”) addresses, truncates the last two address bytes into a 16-bit ID, multiplies the integer count by a per-sensor factor (default 0.7), sends one LoRaWAN payload, and sleeps for 900 seconds [3, 4].
**The published value is therefore `0.7 × U`, where `U` is the number of distinct randomized BLE addresses seen in a ~10-second scan.** The 15-minute cadence is the sleep cycle, not the integration window. The effective duty cycle is about 1.1%. The firmware sets a 30-second scan duration, but the released code stops on the first 10-second report callback; the 30-second value is not enforced by the scanner [4].
The audit is of the public 2026 firmware; the deployed 2025 binary may differ, but the data are consistent with the 0.7 scaling. Every one of the 83,156 density values in the eight-sensor panel is an exact multiple of 0.7. The reconstructed integer count has median 12, p95 71, and a maximum of 418; 2.7% of readings are zero. The legacy `density-01` device reports integers on a 1.0 grid until March 2026, then a replacement device reports on the 0.7 grid from April 2026, with a 35.2-day gap between the two series. The two `density-01` series are not on the same scale and must not be pooled [5].
This is a relative activity index, not a count of people. A person who passes in 30 seconds has roughly a 4.4% chance of overlapping a ~10-second scan; a person who lingers for an hour can be counted in up to four consecutive bins. The index conflates flow and dwell, and it cannot distinguish a phone from a smartwatch, earbud case, beacon, or car head-unit. Device penetration and the people-per-device ratio are unknown. The 0.7 factor is a provisioning default, not a validated penetration correction.
> **Practical consequence.** The model predicts the published index well enough to compare relative peaks and to drive a staffing rule. It does not predict people, and it does not support absolute footfall or revenue claims without a calibration study.
### 2.3 Coverage, gaps, and context
Coverage is uneven. Only 28.2% of 15-minute slots in the analysis panel have all eight sensors reporting; the modal count is seven. Aggregating to 30, 60, or 120 minutes raises complete eight-sensor coverage to 63.4%, 69.7%, and 71.0%, respectively [5]. About 5.2% of slots have all sensors at zero and 39.6% have all sensors below 5. The worst month falls to 23.3% coverage at one location. Gaps reach four to six days per sensor; the cause is undocumented, but the missing periods differ systematically from full-coverage periods [2, 5].
Weather comes from Open-Meteo: the historical archive for training and validation, and the current/7-day forecast for the dashboard and what-if scenarios. Temperature and precipitation are the useful model features; cloud cover and wind did not improve the short test. Public holidays for Baden-Württemberg are included as an `is_holiday` feature and as dashboard badges. They added little in the 28-day test and should be read as context, not as a large accuracy driver [2].
| Sensor | Readings | Mean | Median | p95 | Max | Max daily gap |
|---|---:|---:|---:|---:|---:|---:|
| hn-stadtgalerie-01 | 8,982 | 21.1 | 8.4 | 67.2 | 137.2 | 6 days |
| hn-neckarufer-01 | 9,527 | 19.2 | 11.9 | 61.6 | 255.5 | 5 days |
| hn-hagenbuchersee-02 | 10,099 | 19.3 | 11.2 | 64.4 | 200.9 | 4 days |
| hn-marahaus-01 | 9,254 | 13.7 | 9.1 | 38.5 | 194.6 | 6 days |
| hn-hagenbuchersee-01 | 12,345 | 13.0 | 6.3 | 49.7 | 292.6 | 4 days |
| hn-goetzentuem-01 | 9,952 | 11.1 | 8.4 | 29.4 | 210.7 | 5 days |
| hn-soleo-01 | 10,327 | 10.6 | 8.4 | 28.0 | 111.3 | 4 days |
| density-01 | 12,670 | 9.9 | 7.7 | 28.0 | 63.0 | 35 days |
*Table 1. Data coverage per sensor in the eight-sensor network. `density-01` uses the longer history; the 35-day gap is across the hardware replacement.*
![Figure 1. Sensor map.](figures/fig_sensor_map.png)
*Figure 1. The eight sensor locations in the ~580 m × 280 m inner-city cluster. Measured points only; no city-wide surface.*
![Figure 2. Hourly and weekday profile.](figures/fig_hourly_profile.png)
*Figure 2. The “pulse”: mean index by hour and weekday across the eight sensors. The peak is 18:00–21:00 local; Saturday is the busiest day on average.*
## 3. Method
### 3.1 Preprocessing and features
We parse timestamps as UTC, convert to `Europe/Berlin` for calendar features, and handle daylight saving explicitly. We treat the two `density-01` device IDs as one logical location for the eight-sensor panel, but we do not pool the legacy integer series with the 0.7-scaled replacement: the model’s training window uses the current device only. We exclude the three test sensors (`maker-fair`, `niklas-01`, `hn-bollwerksturm-01`), aggregate to hourly means per sensor, and keep the 15-minute data for the drill-down view. The target is the hourly mean of the published index, not a people count.
Features are calendar (hour, weekday, month, weekend, holiday), lags (24, 48, 168 hours, plus 24-hour and 7-day rolling means), temperature and precipitation from Open-Meteo, and sensor identity so the global model can learn location-specific levels. All lags and rolling windows end at the forecast issue time; the time-based split prevents leakage across the holdout.
### 3.2 Model and validation
We train a global `HistGradientBoostingRegressor` across all eight sensors. The training window is 12 June to 10 October 2026; the test period is the last 28 days, 10 October to 7 November 2026. We compare against a seasonal-naive baseline that predicts the same hour one week earlier, and report MAE, WAPE, and R². The global model pools strength across sensors while keeping sensor identity as a feature; with eight series and five months, a deep sequence model is not justified and would be harder to validate.
The model serves four horizons: 15 minutes, 1 hour, 48 hours, and 7 days. The 15-minute and 1-hour horizons are nowcast/replay because the dataset is a static CSV and there is no live sensor feed. The 48-hour and 7-day horizons are the planning horizons; the 7-day forecast is the hero.
### 3.3 From forecast to shift plan
The ML model predicts footfall. A separate, deterministic heuristic turns the forecast into staff:
```text
forecast footfall per hour
→ visitors per staff per hour (service standard)
→ required staff = ceil(footfall / standard × safety factor)
→ minimum staff, shift length, opening hours
→ shift blocks + coverage score + estimated cost
```
The assumptions are visible and adjustable: visitors per staff hour, labor cost per hour, minimum staff, shift length, and safety factor. The manager can accept, edit, or reject the plan and regenerate it with a scenario. This is decision support with explicit assumptions, not an optimizer with proven ROI. We do not have POS, sales, queue, or labor-cost data, and we do not claim a precise revenue or productivity effect.
## 4. System
PulseCast is deployed as three namespace-scoped Deployments on STACKIT SKE plus a Cloudflare Tunnel connector: **Nuxt 4 SSR + Onyx** for the server-rendered shell, map, charts, forecast, plan, and scenario controls; a **Go API** for sensor/series/forecast/staffing endpoints, SSE fan-out, scenario state, and a cached fallback; and a **FastAPI ML service** for `/predict` and `/simulate`, loading a model baked into the image at build time.
The namespace does not allow Ingress, Jobs, CronJobs, cert-manager, or external-dns. We therefore expose the app through a Cloudflare Tunnel, terminate TLS at Cloudflare, and run the batch pipeline at build time. The model and the fallback JSON are baked into the images; there is no database or PVC. If FastAPI is unavailable, the Go API serves the cached forecast and labels it as cached in the UI. SSE carries scenario changes with a 15-second heartbeat; polling is the fallback. The plan is generated in under two seconds.
The system runs as non-root with resource limits and a read-only root filesystem. The public dashboard is read-only; `/control` uses basic auth. We do not store PII or MAC addresses in the application. The deployment is a hackathon deployment: one API replica, no SSO, no HA, no live ingestion. We document those gaps rather than hiding them.
![Figure 3. System architecture.](figures/fig_architecture.png)
*Figure 3. Three services plus `cloudflared`: Nuxt SSR, Go API/SSE and cache, FastAPI ML. Build-time batch; cached fallback; no Ingress, cert-manager, or Jobs.*
![Figure 4. Plan screen.](figures/fig_plan.png)
*Figure 4. The plan screen: recommended shift blocks, coverage, estimated cost, and the “why this plan” context card. The plan is the product; the map and charts support it.*
## 5. Evaluation
### 5.1 Forecast accuracy
Table 2 reports the 28-day holdout from the generated `metrics.json`. The weather-aware 7-day model reduces MAE by about 19% relative to the no-weather 7-day model (4.08 → 3.32) and by about 39% relative to seasonal naive (5.40 → 3.32). WAPE falls from 44.7% to 27.5%, and R² rises from 0.462 to 0.777. The day-ahead weather model reaches MAE 3.36, WAPE 27.8%, and R² 0.763; the longer horizon is slightly weaker but remains well above the baseline.
| Model | MAE | WAPE | R² |
|---|---:|---:|---:|
| Seasonal naive (same hour last week) | 5.40 | 44.7% | 0.462 |
| Day-ahead, no weather | 3.96 | 32.8% | 0.706 |
| Day-ahead, + temperature/rain | 3.36 | 27.8% | 0.763 |
| 7-day-ahead, no weather | 4.08 | 33.8% | 0.697 |
| 7-day-ahead, + temperature/rain (production) | **3.32** | **27.5%** | **0.777** |
*Table 2. Model comparison on the last 28 days, from the generated `metrics.json` (`n = 5,350`). The mission brief lists slightly different earlier numbers (3.36/27.8/0.779); the generated metrics are the source of truth and must be confirmed by the human before submission. The target is the published footfall index, not people.*
![Figure 5. Forecast versus actual.](figures/fig_forecast.png)
*Figure 5. Forecast versus actual for one sensor over the 28-day holdout, with an uncertainty band and weather/holiday context markers. The band is a planning aid, not a calibrated 95% interval unless the coverage is reported.*
### 5.2 Ablation and per-sensor behavior
Weather is the main accuracy driver. In the day-ahead ablation, removing it raises MAE from 3.36 to 3.96. The holiday flag changes MAE from 3.96 to 3.84, a 0.12 improvement; it is a small feature and should be read as context, not as a large lift. The production 7-day model is 3.32 with weather and 4.08 without. These are single-site, 28-day results; they are credible as a relative signal, not as a general weather effect.
Per-sensor errors vary with location and coverage (Table 3). The global model is not a claim that all sensors behave the same: sensor identity remains a feature, and the local level differences are real. Stadtgalerie has the highest MAE (5.83) and Hagenbuchersee-02 the highest WAPE (31.1%); density-01 has the lowest MAE (2.26). The variation is consistent with local differences in level, volatility, and coverage, not with a single homogeneous field.
| Sensor | MAE | WAPE | R² |
|---|---:|---:|---:|
| density-01 | 2.26 | 30.8% | 0.722 |
| hn-goetzentuem-01 | 2.40 | 26.9% | 0.718 |
| hn-hagenbuchersee-01 | 2.47 | 38.6% | 0.602 |
| hn-marahaus-01 | 2.67 | 29.8% | 0.671 |
| hn-soleo-01 | 3.19 | 20.2% | 0.548 |
| hn-neckarufer-01 | 3.63 | 28.2% | 0.695 |
| hn-hagenbuchersee-02 | 4.12 | 31.1% | 0.731 |
| hn-stadtgalerie-01 | 5.83 | 25.3% | 0.765 |
*Table 3. Per-sensor holdout error for the production 7-day model (`n ≈ 669` per sensor). The global model is pooled; the local errors are not.*
### 5.3 System metrics
| Metric | Value |
|---|---:|
| Plan generation latency | < 2 s |
| Concurrent SSE clients | `[TODO: k6 result]` |
| Web/API/ML image sizes | `[TODO]` |
| Cold deployment time | `[TODO]` |
*Table 4. System metrics. We report only measured values; `[TODO]` items must be filled from the real run before submission.*
A local 3-VU validation of the SSE path opened and closed all three connections cleanly, with a p95 event latency of 656 ms. That is a smoke test, not the 120-client requirement; the cluster run is still `[TODO]` and must not be claimed until it is measured.
The plan screen compares the recommended schedule to a flat schedule on coverage and estimated cost under the stated assumptions. This is a heuristic comparison, not an ROI claim: we do not have actual labor cost, sales, queue, or conversion data.
### 5.4 What the evaluation does and does not show
The evaluation shows that a global, weather-aware model beats a seasonal-naive baseline on a time-based holdout in one inner-city cluster. It does not show that the model predicts people, that the result generalizes to another city, that the weather effect is causal, or that the staffing plan improves profit. The 28-day test is short, the weather is observed rather than forecast, and the target is a ~10-second BLE index. The result is credible as a relative decision-support signal, not as a breakthrough.
## 6. Discussion and Limitations
The main limitation is the measurement, not the model. The sensor samples about 10 seconds out of every 15 minutes, counts only randomized BLE addresses, truncates the address to 16 bits, multiplies by a fixed 0.7 factor, and cannot distinguish people from devices. The published value is a useful relative index, but it is not a census and not a 15-minute count. Device penetration is unknown. A single global factor cannot absorb device ownership, multiple devices per person, Bluetooth settings, and the short scan window.
Coverage is also a limitation. Eight sensors in a 580 m × 280 m cluster cannot support a city-wide heatmap or a store-location recommender. Only 28.2% of 15-minute slots have all eight sensors, and gaps are not random. The 28-day holdout is short, the weather is observed rather than forecast, and there is no event log, POS data, or labor-cost data. The deployment has one API replica, no SSO, no HA, and no live sensor ingestion. The 15-minute and 1-hour horizons are replay/nowcast, not a live feed.
The privacy claim also needs care. The application stores no MAC addresses, and only aggregate counts leave the sensor. But the sensing stage still processes link-layer identifiers at the edge; the 16-bit ID is a pseudonymous dedup key, not a cryptographic hash; and passive MAC sensing in public space has a contested legal basis in Germany. The “privacy-preserving” label is an architectural claim, not a formal guarantee. A production deployment would need a DPIA, a documented legal basis, data minimization, retention limits, and, where possible, a sensor that does not extract MAC addresses at all.
**What we would do with more resources.** Calibrate with manual or camera counts at each site; change the sensing layer to thermal arrays at narrow entrances, edge cameras as reference, mmWave at wide crossings, and aggregated probe data for city context; use forecast weather and a structured event calendar; add POS/queue/labor data and evaluate at the decision level; and replace the single API replica with an authenticated, observable, multi-tenant service. The order matters: fix the measurement before scaling the model.
## 7. Related Work
Passive Bluetooth and Wi-Fi sensing has a long history in pedestrian counting. Studies consistently find that detection is a small, biased, and variable sample of people; counts track relative trends, but absolute levels require local calibration [6, 7]. MAC address randomization reduces linkability but does not make passive counting a census, and it can be defeated by fingerprinting in some settings [7]. Our audit is consistent with this literature and makes the sampling design explicit.
For forecasting, gradient-boosted trees remain a strong baseline for tabular, covariate-rich, small-data problems [8]. Our global model follows that choice. Weather is a known driver of pedestrian activity, but its magnitude is context-dependent [9]. We treat weather as a predictive feature, not a causal effect. Smart-city dashboards often stop at visualization; we focus on the decision layer and on the measurement limits that determine whether a dashboard is trustworthy.
## 8. Conclusion
PulseCast shows that a small, weather-aware global model can turn a noisy inner-city footfall index into a weekly staffing plan that beats a seasonal-naive baseline and runs in under two seconds on a real deployment. It also shows that the index is not a people counter: each value is a short BLE scan scaled by 0.7, sampled every 15 minutes, with unknown penetration and incomplete coverage. The honest conclusion is narrower and more useful than a city-scale claim: the decision loop works, the relative forecast is credible, and the next step is calibration and a privacy-first sensing layer. We would rather ship a plan a manager can use and state what it does not measure than promise a city-wide heatmap the data cannot support.
## Artifacts, data, and AI disclosure
The live demo is at `[TODO: URL]`; the code is at `[TODO: repository]`. The raw challenge CSV is not redistributed. Derived aggregates, the trained model card, and the metrics JSON can be shared subject to the organizers’ permission. The application stores no MAC addresses or PII; the sensing-stage limitations and the privacy/legal caveats are described in Sections 2.2 and 6. A production deployment would require a DPIA and a documented legal basis before collecting MAC-derived counts.
AI tools were used for drafting and editing. The authors verified every number, citation, and claim against the delivered data, the project metrics, or a cited source. AI tools are not authors and take no responsibility for the content.
## References
[1] Schwarz Digits / Code for Heilbronn. *The Pulse of the City*, challenge brief, 2026. Local: `OKF_Vault/Schwarz Digits/FCH Challenge The _Pulse of the City_.pdf`.
[2] PulseCast project. `OKF_Vault/mission/data.md`, `mission/mvp.md`, `mission/architecture.md`, 2026.
[3] Code for Heilbronn. “Passantenzahlen messen mit LORA,” 18 September 2025. https://codeforheilbronn.de/blog/2025/09/passantenzahlen-messen-mit-lora/
[4] Code for Heilbronn. `lorax` / `sensor-pax` firmware and backend, 2026. https://codeberg.org/cfhn/lorax; upstream `ESP32-Paxcounter` and `libpax`, https://github.com/cyberman54/ESP32-Paxcounter and https://github.com/dbinfrago/libpax.
[5] PulseCast data-quality analysis, `paper/research/07-data-quality.md`, 2026.
[6] A. Kurkcu and K. Ozbay. “Estimating pedestrian densities, wait times, and flows with Wi-Fi and Bluetooth sensors.” *Transportation Research Record* 2644(1):72–82, 2017. DOI: 10.3141/2644-09.
[7] J. Martin, T. Mayberry, C. Donahue, L. Foppe, L. Brown, C. Riggins, E. C. Rye, and D. Brown. “A study of MAC address randomization in mobile devices and when it fails.” *Proceedings on Privacy Enhancing Technologies* 2017(4):365–383. DOI: 10.1515/popets-2017-0054.
[8] J. H. Friedman. “Greedy function approximation: a gradient boosting machine.” *Annals of Statistics* 29(5):1189–1232, 2001. DOI: 10.1214/aos/1013203451.
[9] L. F. Miranda-Moreno and A. C. Lahti. “Temporal trends and the effect of weather on pedestrian volumes: a case study of Montreal, Canada.” *Transportation Research Part D: Transport and Environment* 22:54–59, 2013. DOI: 10.1016/j.trd.2013.02.008.
[10] Bluetooth SIG. *Bluetooth Core Specification v5.3*, 2021. https://www.bluetooth.com/specifications/specs/core-specification-5-3/
[11] Court of Justice of the European Union. Case C-582/14, *Patrick Breyer v Bundesrepublik Deutschland*, 19 October 2016. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:62014CJ0582
[12] European Data Protection Board. *Guidelines 02/2023 on the Technical Scope of Article 5(3) of the ePrivacy Directive*, 2023. https://www.edpb.europa.eu/documents/guideline/guidelines-22023-on-technical-scope-of-art-53-of-eprivacy-directive_en
+55
View File
@@ -0,0 +1,55 @@
# References — PulseCast paper
Canonical reference list for the **short report** `paper/main.md` (12 entries). The **extended paper** `paper/main-extended.md` has its own 37-entry reference list at the end of that file; keep the two in sync when editing. Verify every DOI/URL before submission. Do not add a source that is not actually cited.
1. Schwarz Digits / Code for Heilbronn. *The Pulse of the City*, challenge brief, 2026. Local: `OKF_Vault/Schwarz Digits/FCH Challenge The _Pulse of the City_.pdf`.
2. PulseCast project. `OKF_Vault/mission/data.md`, `mission/mvp.md`, `mission/architecture.md`, 2026. **Internal project documentation; replace with a public dataset citation if the organizers require it.**
3. Code for Heilbronn. “Passantenzahlen messen mit LORA,” 18 September 2025. https://codeforheilbronn.de/blog/2025/09/passantenzahlen-messen-mit-lora/
4. Code for Heilbronn. `lorax` / `sensor-pax` firmware and backend, 2026. https://codeberg.org/cfhn/lorax. Upstream: `ESP32-Paxcounter`, https://github.com/cyberman54/ESP32-Paxcounter; `libpax`, https://github.com/dbinfrago/libpax.
5. PulseCast data-quality analysis. `paper/research/07-data-quality.md`, 2026. **Internal technical note; cite the delivered CSV instead if a formal reference is required.**
6. A. Kurkcu and K. Ozbay. “Estimating pedestrian densities, wait times, and flows with Wi-Fi and Bluetooth sensors.” *Transportation Research Record* 2644(1):72–82, 2017. DOI: 10.3141/2644-09.
7. J. Martin, T. Mayberry, C. Donahue, L. Foppe, L. Brown, C. Riggins, E. C. Rye, and D. Brown. “A study of MAC address randomization in mobile devices and when it fails.” *Proceedings on Privacy Enhancing Technologies* 2017(4):365–383. DOI: 10.1515/popets-2017-0054.
8. J. H. Friedman. “Greedy function approximation: a gradient boosting machine.” *Annals of Statistics* 29(5):1189–1232, 2001. DOI: 10.1214/aos/1013203451.
9. L. F. Miranda-Moreno and A. C. Lahti. “Temporal trends and the effect of weather on pedestrian volumes: a case study of Montreal, Canada.” *Transportation Research Part D: Transport and Environment* 22:54–59, 2013. DOI: 10.1016/j.trd.2013.02.008.
10. Bluetooth SIG. *Bluetooth Core Specification v5.3*, 2021. https://www.bluetooth.com/specifications/specs/core-specification-5-3/
11. Court of Justice of the European Union. Case C-582/14, *Patrick Breyer v Bundesrepublik Deutschland*, 19 October 2016. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:62014CJ0582
12. European Data Protection Board. *Guidelines 02/2023 on the Technical Scope of Article 5(3) of the ePrivacy Directive*, 2023. https://www.edpb.europa.eu/documents/guideline/guidelines-22023-on-technical-scope-of-art-53-of-eprivacy-directive_en
## Extended-paper additions (verify before use)
The extended paper can add primary sources from `paper/research/`:
- Bluetooth SIG, “Enhancing device privacy and energy efficiency with Bluetooth Randomized RPA Updates,” 2025.
- Apple Platform Security, “Bluetooth security” and “Wi-Fi privacy.”
- Android Open Source Project, “MAC randomization.”
- EDPB, *Guidelines 01/2025 on Pseudonymisation*; *Guidelines 02/2026 on Anonymisation* (draft).
- CJEU, *Planet49* (C-673/17); *M5A*; *KNLTB*; *EDPS v SRB* (C-413/23 P).
- Conseil d’État, 8 February 2017, no. 393714 (JCDecaux).
- Dutch DPA, Wi-Fi/Bluetooth tracking guidance; Rechtbank Overijssel, ECLI:NL:RBOVE:2024:594; Raad van State, ECLI:NL:RVS:2026:4403.
- ULD Schleswig-Holstein, TB 2019, §5.4.8; BayLDA TB 2015/16; HmbBfDI TB 2014/2015.
- Google Maps 101; Google navigation-data help; Google ETA GNN paper; Google Busy Areas help.
- NIST SP 800-226; Dwork & Roth, *The Algorithmic Foundations of Differential Privacy*; OpenDP.
- Pyrgelis et al., membership inference on aggregate location data; de Montjoye et al., mobility uniqueness.
- Sensing alternatives: Cyganek et al. 2022; Lesani et al. 2020; Vaidya & Zúñiga 2024; Wang et al. 2023; Perra et al. 2021; Mohammadmoradi et al. 2017.
- Heatmap/forecast: Cressie 1993; Diggle & Ribeiro 2007; Banerjee et al. 2014; Lindgren et al. 2011; Rasmussen & Williams 2006; Webster & Oliver 2007; Friedman 2001; Vovk et al. conformal prediction; N-BEATS; TFT; Chronos/TimesFM/Moirai evaluations.
## Verification checklist
- [ ] Every reference is real, relevant, and verifiable.
- [ ] Every DOI resolves.
- [ ] Every URL returns 200.
- [ ] Every claim in the paper cites a source that actually supports it.
- [ ] No fake papers, no invented results, no invented quotes.
- [ ] Internal project notes are replaced with public sources where required.
- [ ] The reference count is 8–12 for the short report and 25–35 for the extended paper.
+293
View File
@@ -0,0 +1,293 @@
---
title: "Synthesis: the paper we can defend"
status: research-synthesis
date: 2026-11-14
scope: "PulseCast / Future City Hackathon 2026; extended paper, not the 4–5 page hackathon report"
---
# Synthesis: the paper we can defend
## 0. One-sentence thesis
**The PulseCast build proves a decision loop, but its sensing layer is not a people counter: each “15-minute” value is a ~10-second count of randomized BLE addresses, truncated to 16 bits and multiplied by a fixed 0.7 factor; the defensible paper is therefore a code-level audit of that claim plus a privacy-first, decision-grade architecture for the next version — not a claim that eight sensors measure a city.**
## 1. What we now know
### 1.1 The measurement chain (code-level, high confidence)
The deployed Code for Heilbronn `lorax` firmware (`sensor-pax`) does this:
```text
wake from deep sleep
→ BLE passive scan (window 80 ms, interval 80 ms)
→ count distinct locally-administered BLE addresses in a ~10 s window
→ truncate the last 2 address bytes to a 16-bit ID
→ deduplicate within that scan only
→ multiply the integer count by factor (default 0.7)
→ send one LoRaWAN payload: sensor_id, 0, value, 1, battery
→ deep sleep 900 s
```
Key code facts:
- `configuration.blecounter = 1`, `configuration.wificounter = 0` — BLE only.
- `configuration.blescantime = 30`, but the vendored `start_BLE_scan()` does not use `blescantime`; the main loop stops on the first 10 s report callback. Effective scan ≈ **10 s**, not 30 s.
- `libpax_counter_init(log, &count, 10, 1)` — 10 s report timer, cumulative count within the wake.
- `mac_add()` accepts only addresses with the **locally administered bit** set; it ignores universal/static addresses.
- `id = (uint16_t *)(paddr + 4)` — the “hash” is a **16-bit truncation** of the last two bytes, not a cryptographic hash.
- `paxCount * cfg.factor` with `factor` default **0.7**; the value is formatted with four decimals.
- `sleep_time_sec` default **900 s**.
- RSSI threshold **−80 dBm**; BLE advertising queue size **30**; queue drops are logged, not counted in the payload.
- The backend stores the parsed value directly; the CSV is a raw export, not a 15-minute mean.
**Formula:** `published_value ≈ 0.7 × U`, where `U` is the number of distinct 16-bit IDs from randomized BLE addresses seen in a ~10 s scan. The sampling interval is ~15 min. The duty cycle is **~1.1%** (10 s / ~910 s), not 3.3%.
**Sampling consequence:** a passer-by with a 30 s dwell has roughly a **4.4% chance** of overlapping a scan; a one-hour lingerer can be counted up to four times. The metric conflates **flow** (people passing) with **dwell** (people staying), and the dedup bitmap resets every wake, so neither is measured cleanly.
**Randomization nuance:** within a 10 s scan, BLE address rotation is rare (RPA rotation is typically 900 s, measured 15–45 min). MAC randomization therefore does **not** mainly inflate the within-scan count; it destroys cross-scan/cross-sensor deduplication and re-identification. The locally-administered-bit filter is also a heuristic: it excludes public/static-address devices and includes non-phone random advertisers (TVs, beacons, earbuds, car head-units).
### 1.2 Forensic confirmation (data-level, high confidence)
The 8-sensor production window (2026-06-12 → 2026-11-07) contains **83,156 density values**. Every one is an exact multiple of 0.7. The reconstructed integer `U` has median **12**, p95 **71**, max **418**, and **2.7% zeros** (from the reliability note; verify independently before submission).
The legacy `density-01` device `863f75b0` (2025-10-16 → 2026-03-12) is integer-valued; its replacement `f8c19730` (2026-04-16 → 2026-11-07) is 0.7-scaled. This is a **firmware/calibration break**, not a footfall change. Any model trained across the break fits the instrument, not the city.
Additional data-quality findings from the empirical analysis (to verify and cite):
| Finding | Value | Implication |
|---|---:|---|
| Panel slots with all 8 sensors reporting | **28.2%** of 13,728 15-min slots; modal count is 7 locations (4,117 slots) | The “8-sensor panel” is sparse; a random heatmap often has fewer than 8 points |
| All-8 coverage when relaxed to 30 / 60 / 120 min | 63.4% / 69.7% / 71.0% | Temporal aggregation hides the sparsity; it does not create measurements |
| Worst monthly coverage | 23.3% at one location; August 2026 averages 54.2% | Coverage is non-stationary; six of eight locations degrade together |
| All-sensor zero / <1 / <5 share | 5.2% / 10.4% / 39.6% | Zero-inflation and low-count dominance; log/negative-binomial models needed |
| Cross-sensor Pearson correlation | mean 0.58, range 0.13–0.90 | Sensors are correlated but not redundant; the field is not 8 independent measurements |
| Cross-sensor CV at the same timestamp | median 0.66, p90 1.05 | Large spatial variability; a single network-wide index hides local differences |
| Hourly shape correlation across sensors | mean r = 0.79 | Relative daily shape is consistent; absolute levels are not |
| Peak hour by sensor | 16, 19, 20, 21 | “Peak at 19:00” is not universal |
| Weekend/weekday ratio by sensor | 0.81–1.24 | “Saturday busiest” is not universal |
| Missingness bias ratio | median 0.69, range 0.38–1.49 | Gaps are not random; observed counts differ systematically from full-coverage periods |
| Battery median / share <20% | 17% / 51.3% | Power is a plausible cause of dropouts; battery health is part of data quality |
| Longest flatline run | 20 readings / 5 h | Stuck-sensor or repeated-value risk |
| Near-duplicate intervals <60 s | 113 | Duplicate/reboot artifacts; needs dedup rules |
These are empirical findings from the delivered CSV; they do not require new sensors. They make the “local pulse map, not city-wide surface” claim concrete.
**What the aggregates cannot tell us:** MAC-randomization rate, penetration rate, unique-device count, dwell time, repeat visits, detection radius, or false-positive/false-negative rates. The 0.7 grid is a proxy for firmware/vendor processing, not a direct measurement of randomization or penetration. The paper must not infer these from the CSV.
### 1.3 Reliability failure modes
| Failure | Mechanism | Consequence |
|---|---|---|
| Short snapshot | ~10 s scan every 15 min | Misses peaks between scans; target is a sparse sample, not integrated flow |
| Randomized-only filter | Only locally administered addresses counted | Static/public-address devices invisible; non-phone gadgets with random addresses counted |
| MAC randomization | RPA rotation ~15 min; rare within a 10 s scan | Does **not** mainly inflate within-scan counts; it prevents cross-scan/cross-sensor dedup and re-identification |
| 16-bit truncation | Last two bytes used as ID | Collisions grow with density; undercount at busy times |
| No cross-scan dedup | Bitmap reset every wake | Same person counted again in every 15-min bin; dwell not measurable |
| Device ≠ person | Phones, watches, earbuds, cars, beacons | Unknown multiplier; one person can be several devices |
| Penetration unknown | Bluetooth on/off, discoverability, demographics | Absolute counts not identifiable without calibration |
| RSSI gate | −80 dBm threshold, body shielding, orientation | Detection zone is not a stable catchment |
| Queue drops | 30-slot advertising queue | Undercount under density; not exposed in data |
| Gaps | LoRaWAN joins, power, maintenance | 4–6 day gaps; missing not at random |
| Factor | Default 0.7, undocumented | Single scalar cannot absorb penetration × devices/person × scan probability |
### 1.4 Privacy and law
The architecture is better than raw MAC retention: raw addresses stay in RAM, only counts leave the sensor. But:
- The 16-bit truncation is **not** a cryptographic hash; it is a dedup key with a small ID space.
- The operator’s blog says MACs are “hashed and stored for at most one hour”; the public 2026 firmware resets the bitmap every wake and has no one-hour store. The two accounts disagree; report both.
- MAC addresses are very likely personal data in this context: a device identifier plus sensor location, time, and deduplication can single out a device and its bearer. `[CJEU]` *Breyer*, C-582/14, paras 42–46; GDPR Art. 4(1), Recital 26; EDPB Guidelines 02/2026 (draft), para. 25.
- Hashing or truncating a MAC is **pseudonymisation, not anonymisation**. The Conseil d'État's JCDecaux decision held that truncation + salt + keyed hashing still left the controller able to identify and link records; the Dutch Enschede Wi-Fi tracking fine was annulled on evidential grounds, not because the law was settled. `[Enforcement]` CE, 8 Feb 2017, no. 393714; ECLI:NL:RBOVE:2024:594; ECLI:NL:RVS:2026:4403.
- Passive sniffing under ePrivacy Art. 5(3) / § 25 TDDDG is **contested**: the better textual reading is that passive reception is not “access” to terminal equipment, but the EDPB's functional approach creates material risk; if § 25 applies, prior consent is required and legitimate interest/signage are not enough. `[Law]` ePrivacy Art. 5(3); TDDDG § 25; `[Guidance]` EDPB Guidelines 02/2023, paras 39, 42; DSK OH Digitale Dienste (Nov 2024).
- A **DPIA is almost certainly mandatory** for continuous sensing of a publicly accessible inner-city area (Art. 35(3)(c)); the Baden-Württemberg private-sector “Muss-Liste” explicitly covers offline tracking of customer movements and radio-signal location over a substantial period. `[Law]` GDPR Art. 35(3)(c); `[Guidance]` LfDI BW Muss-Liste, Nos. 2 and 10.
- German enforcement has been administrative rather than fine-based for this use case: the ULD Schleswig-Holstein investigated two city-marketing operators using passive probe requests and concluded there was no legal basis; both stopped. The Bavarian authority inspected a fashion chain's Wi-Fi tracking test; the company stopped. `[DPA]` ULD TB 2019, §5.4.8; BayLDA TB 2015/16, §§3.10, 22.10; HmbBfDI TB 2014/2015, §V.9.
- “Privacy-preserving” is an **architectural claim**, not a formal guarantee. A defensible claim needs a threat model, a privacy unit, a k-threshold or DP budget, a retention policy, a DPIA, and a legal basis. If § 25 cannot be excluded, the viable route is a sensor that does not extract MAC addresses at all.
**Recommended privacy stack (from the PET note):**
- **No raw MACs**; edge counting with a rotating, destroyed salt; no cross-sensor or cross-day linkage.
- **Central DP at a trusted city aggregator** is the realistic pilot: device-day privacy unit, contribution bounds (e.g. at most four sensor-bin pairs per device per day), fixed daily/weekly coarse releases, and a published budget ledger.
- **Do not use local DP for sparse 15-min bins**: its error scales as `1/(ε√n)` versus central `1/(εn)`, so it destroys utility at small-city scale. At ε = 1, a 14-device cell carries ±3 noise; a live 15-min public heatmap is not viable with a meaningful annual budget.
- **k-anonymity is a secondary check**, not the primary guarantee: four spatio-temporal points identify ~95% of people in a 1.5M-person mobility dataset; generalizing to k = 20 erases the local footfall signal.
- **Secure aggregation/MPC/HE/TEE complement DP** but protect data in computation, not the released statistic; a single city-owned aggregator does not need them in the pilot.
- **Federated learning is not applicable** to passive BLE sniffing: the pedestrian's phone does not run our code.
- **Non-DP synthetic data has no formal guarantee**; use DP synthesis plus membership-inference/Anonymeter evaluation for public release, not for the live dashboard.
- **Publish the guarantee and the limits**: state the privacy unit, ε/δ, contribution bound, release granularity, and what cannot be guaranteed (fine-grained public 15-min heatmaps, individual trajectories, repeat-visit metrics).
### 1.5 What actually measures footfall better
The alternatives note gives measured ranges. The short version:
| Modality | Strength | Weakness | Role |
|---|---|---|---|
| Thermal/IR array | 89–93% doorway occupancy; low privacy risk; €150–400/node | Narrow passages; degrades >30 °C, >7.5 m, groups | Default at store entrances and narrow gateways |
| mmWave radar | 96% presence, 69% count outdoor; low privacy risk; €300–700 | Groups, rain, ~8 m range; count error too high without calibration | Wide, exposed crossings |
| Edge AI camera | 13.8% day / 29.8% night MAPE outdoors; 2–14% in official review | High privacy cost; lighting/occlusion; DPIA | 2–4 reference/validation sites |
| LiDAR | >97% disaggregate count; 1.6–4.3% 15-min error | €1k–8k; installation; eye safety | 1–2 high-value reference sites |
| Aggregated probe data | City-wide movement, Google/HERE/TomTom scale | Non-probability sample; penetration/bias unknown; privacy-enhanced, not anonymous | Context and city-wide trends, not street-level truth |
| Manual counts | Ground truth; ~11% clicker error | Labour; sampling; not continuous | Quarterly calibration audits |
| Sensor fusion | Bayesian hierarchical model combines modalities and propagates uncertainty | Engineering; identifiability; drift | The core of the “right way” |
**Recommended stack for Heilbronn:** 8–12 thermal nodes at narrow entries; 1–2 mmWave/LiDAR nodes at wide/open locations; 2–4 edge cameras as reference/validation; aggregated probe data for city-wide context; twice-yearly manual/video ground truth; transit validators as covariates. No MAC sniffing, no face recognition.
**Pilot protocol:** 12 weeks at six sites; go/no-go at **MAPE ≤ 20%**, **|bias| ≤ 10%**, and **80% interval coverage ≥ 75%**. Year-1 cost estimate **€81k–161k**, recurring **€30k–50k** `[ESTIMATE]`; DPIA under Art. 35(3)(c) GDPR and BDSG §4; no raw video or point clouds retained.
### 1.6 Heatmap and forecast
- Eight sensors over ~580 m × 280 m give ~142 m equivalent spacing in 2D, or ~72 m along a single street. A **100–150 m** support is defensible; a 25–50 m raster is visualization, not inference.
- Use a **negative-binomial hierarchical model with a low-rank spatial field** (Matérn GP/SPDE or spatial basis functions), on the **walkable network**, not isotropic Euclidean smoothing over buildings.
- Forecast with a **global quantile gradient-boosted model** (calendar, weather forecast, holidays, events, lags, sensor identity) and **conformal calibration**. Deep spatiotemporal models and time-series foundation models are benchmarks/ablations at this data scale, not the default.
- The largest leakage risk is **weather**: training on observed weather and serving with forecast weather is optimistic. Report both the observed-weather oracle and the forecast-weather number.
- Evaluate at the **decision** level (staffing cost/coverage, overflow risk, ad exposure stability), not only MAE/WAPE/R². A point forecast without an interval is not decision-grade.
### 1.7 Generalization across the OKF_Vault challenges
The applications matrix supports a **conditional** claim:
| Class | Challenges | What transfers |
|---|---|---|
| **Direct** | Staffing, advertising | Same sensing/field/forecast; new decision layer |
| **Conditional** | Waste routing; store location | Temporal/decision pattern transfers; waste needs a graph field and geometry; store location needs city-wide data (not supported by 8 sensors) |
| **Sibling** | Sensor placement (AI TRAQC); LoRaWAN planning (HNVG) | Spatial inference and design evaluation transfer; the decision changes the observation operator; no temporal forecast |
| **Non-application** | Cable-car sentiment; learning guide | No physical field, no forecast target, no resource-allocation decision |
This is stronger than “one model for six challenges” because it states the boundary conditions.
## 2. The paper’s contributions
1. **A code-level audit** of the deployed PAX counter firmware and backend, with the exact measurement chain (10 s scan, 0.7 factor, 16-bit truncation, randomized-only filter, 1.1% duty cycle).
2. **Forensic data evidence**: 100% of 83,156 values are multiples of 0.7; the `density-01` calibration break; the heterogeneous gaps and zero-inflation.
3. **A privacy/legal analysis**: why “privacy-preserving” is architectural, not formal; the 16-bit truncation is not a hash; the legal basis and DPIA requirements.
4. **A privacy-first reference architecture**: edge aggregation, local DP/secure aggregation, k-threshold, no raw trajectories, retention limits; a sensing stack that uses thermal/radar/camera/probe/manual data instead of MAC sniffing as the default.
5. **A sparse-to-heatmap-to-decision pipeline**: network-aware Bayesian field, quantile GBM forecast, conformal calibration, decision-level evaluation.
6. **A transferability matrix** across the six challenges, with explicit non-applications and falsifiers.
7. **A staged roadmap**: what to keep from the hackathon build, what to replace, what to measure before claiming ROI.
## 3. Proposed structure (extended paper, ~10–12 pages)
| Section | Pages | Content |
|---|---:|---|
| Abstract | 0.25 | Problem, audit finding, alternative, transferability, non-claims |
| 1. Introduction | 1 | Decision gap; the “privacy-preserving” claim; contributions |
| 2. The measurement problem | 2.5 | Code audit; formula; data forensics; reliability taxonomy; existing PulseCast metrics reframed |
| 3. The privacy problem | 1.5 | GDPR/ePrivacy/TTDSG; threat model; 16-bit truncation; operator claims vs code; DPIA checklist |
| 4. What “privacy-first” should mean | 2 | PETs; sensing alternatives; calibration; reference sensing stack |
| 5. From sparse observations to decisions | 2 | Heatmap; forecast; uncertainty; decision layer; evaluation harness |
| 6. How far it generalizes | 1.5 | Applications matrix; case studies; falsifiers; where it stops |
| 7. Evaluation agenda | 1 | Ground truth; calibration; duty cycle; penetration; privacy budget; decision regret; proposed experiments |
| 8. Discussion & limitations | 0.75 | Single site; no ground truth; opinion bias; legal not advice; hackathon context |
| 9. Related work | 1 | PAX/privacy; pedestrian forecasting; spatiotemporal models; MLOps; urban dashboards |
| 10. Conclusion | 0.5 | What to keep; what to build; a checklist |
| References | 1.5 | 25–35 real, verified sources |
For the **4–5 page hackathon report**, keep the existing structure and add a 0.5–1 page “what we would do with more resources” subsection. Do not try to fit the extended audit into 5 pages.
## 4. Claims and non-claims
**Supported by evidence:**
- The published value is `0.7 × U`, where `U` is a short-window count of randomized BLE addresses. `[code + data]`
- The production 7-day model reaches MAE 3.32, WAPE 27.5%, R² 0.777 on the 28-day holdout; the day-ahead model reaches 3.36/27.8/0.763. Per-sensor MAE ranges from 2.26 (density-01) to 5.83 (Stadtgalerie), so the pooled model is not a homogeneous field. `[measured; metrics.json]`
- The 8-sensor network is a local inner-city cluster, not a city-wide sensor. `[data]`
- The existing model improves on seasonal naive in a 28-day holdout: MAE 3.36 vs 5.40; WAPE 27.8% vs 44.7%; R² 0.779 vs 0.462. `[measured]`
- The sensor counts devices, not people; penetration is unknown. `[data + literature]`
- MAC-based sensing has a contested legal basis and is not a formal anonymization guarantee. `[law + PET]`
- Thermal/radar/camera/probe/manual modalities have different accuracy, cost, privacy, and coverage trade-offs. `[literature]`
- The stack transfers directly to staffing/ads, conditionally to waste, as a sibling to sensor placement/LoRaWAN planning, and not at all to sentiment/learning guide. `[applications matrix]`
**Not supported:**
- “The sensor counts people.”
- “The value is a 15-minute count or mean.”
- “The 0.7 factor is a validated penetration correction.”
- “The 16-bit hash is anonymous.”
- “Eight sensors support a city-wide heatmap or store-location ranking.”
- “One model solves all six challenges.”
- “The proposed architecture has been deployed and evaluated.” (It is a design/research agenda.)
- “The staffing plan has proven ROI.” (No POS/labor/outcome data.)
## 5. Figures and tables
**Figures**
1. The PAX counter cycle: wake → ~10 s scan → `0.7 × U` → LoRaWAN → 900 s sleep. Include the 1.1% duty cycle.
2. Value quantization and the `density-01` calibration break: integer series vs 0.7-scaled series.
3. Sensor map with coverage/gaps and the 580 m × 280 m cluster.
4. Privacy-first sensing and data-flow architecture (edge aggregation, DP/secure aggregation, k-threshold, no raw trajectories).
5. Sparse-to-heatmap-to-decision pipeline (network-aware Bayesian field → quantile forecast → decision layer → evaluation).
**Tables**
1. Code audit: what the firmware does vs what the challenge claims.
2. Data quality: per-sensor coverage, gaps, zero share, reconstructed `U`, quantization.
3. Sensing alternatives: accuracy, cost, privacy, deployment constraints, best use.
4. Model comparison: seasonal naive / no weather / weather; existing PulseCast metrics.
5. Transferability matrix across the six challenges.
6. Roadmap: demo / pilot / production; what to build, what to buy, triggers, estimates.
## 6. Proposed experiments (post-demo, optional)
These are real analyses we can run on the existing CSV; none require new sensors.
1. **Quantization audit**: confirm 100% multiples of 0.7 in the 8-sensor window; reconstruct `U`; compare `density-01` device regimes. `[data agent already doing this]`
2. **Gap/zero-inflation analysis**: per-sensor missingness, zero share, runs of zeros, correlation with battery/LoRaWAN.
3. **Cross-sensor correlation**: at matched timestamps; pairwise correlation; effective number of independent sensors; coefficient of variation. This tests whether the “8-sensor field” is really 8 independent measurements.
4. **Subsampling experiment**: train the existing global model on 1-in-k rows (simulate 30 min, 1 h, 2 h cadence) and measure MAE/WAPE degradation. This quantifies the value of temporal sampling and supports the “increase duty cycle or use continuous probes” recommendation.
5. **Leave-one-sensor-out**: train on 7 sensors, test on the held-out sensor. If the global model fails, the “one model for all sensors” claim is weak; if it succeeds, it supports shared structure.
6. **Factor sensitivity**: rescale the counts by 0.5/0.7/1.0/1.4 and compare the staffing plan (coverage/cost) under each. This shows why calibration matters for decisions, even if relative patterns are stable.
7. **Decision-level evaluation**: compare the staffing plan to a flat schedule on coverage and cost under explicit assumptions; report it as a heuristic comparison, not ROI.
These experiments can be done in `paper/research/` with the existing venv. They are optional for the hackathon report but strengthen the extended paper.
## 7. What to keep from the hackathon build
- Decision-first framing: the plan, not the dashboard.
- Global model across sensors; time-based holdout; seasonal-naive baseline.
- Transparent staffing heuristic with visible assumptions.
- Cached fallback and staleness labels.
- Build-time batch for a 24-hour demo.
- Honest limitations and non-claims.
## 8. What to replace or add for the “right way”
| Layer | Hackathon | Right way | Trigger |
|---|---|---|---|
| Sensing | 8 BLE PAX counters | Thermal at doorways; mmWave at wide sites; 2–4 cameras as reference; probe data for city context; manual counts for calibration | Before any absolute-count or ROI claim |
| Calibration | Fixed 0.7 factor | Quarterly manual counts; capture-recapture/penetration study; Bayesian calibration with uncertainty | Before staffing cost claims |
| Privacy | Aggregate counts; no raw MACs in app | Edge aggregation; local DP/secure aggregation; k-threshold; no raw trajectories; DPIA; legal basis; retention limits | Before public deployment |
| Spatial | Local pulse map | Network-aware negative-binomial hierarchical field at 100–150 m; uncertainty/reliability mask | Before heatmap-based decisions |
| Forecast | Global HistGB; point + band | Quantile GBM; conformal calibration; forecast-weather evaluation; decision-level metrics | Before operational use |
| Platform | 3 pods; tunnel; cached fallback | Auth/SSO; observability; SLOs; multi-tenant; IaC; cost controls | When external users/uptime matter |
| Evaluation | MAE/WAPE/R²; plan latency | Ground truth; calibration; decision regret; privacy audit; A/B or pilot | Before claiming ROI or generalization |
## 8.5 Tone and positioning
The paper should be **constructive, not adversarial**. Code for Heilbronn is a volunteer civic-tech group that built a low-cost, privacy-conscious prototype under real constraints. The critique is of the **claim** (“privacy-preserving,” “footfall,” “15-minute counts”) and of the **measurement design**, not of the volunteers’ intent. Acknowledge:
- the genuine data-minimisation choices (no raw MAC storage, counts only over LoRaWAN, locally administered filter);
- the hardware/power constraints that motivate a 10 s scan and a 15 min sleep;
- the value of the network as a **relative activity index** for local decisions;
- the fact that the challenge brief, not the firmware alone, frames the data as “people.”
The paper’s constructive contribution is the measurement model, the calibration program, and the privacy-first alternative. Avoid “the sensor is fake” or “the data are worthless.” The defensible claim is: **the data are useful for relative patterns in a local cluster, but not for absolute footfall, city-wide heatmaps, or ROI claims without calibration.**
## 9. Open decisions for the human
1. **Paper shape:** extended 10–12 page Zenodo paper (recommended) vs 4–5 page hackathon report vs both.
2. **New experiments:** run the subsampling/LOSO/factor-sensitivity analyses? (Recommended: yes, post-demo, label as analysis.)
3. **Prototype:** design/architecture paper only, or a small new prototype? (Recommended: design + evaluation plan; no new build before the demo.)
4. **Legal scope:** Germany/EU focus with a short generalization note? (Recommended: yes.)
5. **Publication:** Zenodo DOI first, arXiv if endorsement? (Recommended: yes.)
6. **Reference budget:** 25–35 refs for the extended paper; 8–12 for the hackathon report. Confirm the deviation from `agents/paper.md` is acceptable.
## 10. Source notes
- `01-bluetooth-pax-reliability.md` — BLE/PAX reliability, code trace, forensic quantization, literature.
- `02-privacy-law-mac.md` — pending; legal classification, ePrivacy/TTDSG, DPIA, enforcement.
- `03-google-maps-probe-data.md` — Google Maps traffic/ETA, terminology, privacy model, alternatives.
- `04-sensing-alternatives.md` — camera/thermal/radar/LiDAR/CSI/telco/GPS/transit/manual/fusion.
- `05-heatmap-forecasting.md` — sparse-to-heatmap, forecasting, covariates, transfer, evaluation.
- `06-applications-matrix.md` — six challenges, shared abstraction, transferability, case studies.
- `07-data-quality.md` — pending; empirical data-quality analysis and figures.
- `08-privacy-enhancing-technologies.md` — DP, k-anonymity, secure aggregation, synthetic data, threat models.
- `09-paxcounter-code-audit.md` — our source-code audit of the deployed `lorax` firmware and upstream reference.
+640
View File
@@ -0,0 +1,640 @@
# BLE PAX counters: how they count, how they fail, and what can be validated
**Research note 01 — PulseCast / Future City Hackathon 2026**
Prepared: 14 November 2026
Scope: passive BLE scanning for pedestrian counting; the 8-sensor Heilbronn network; implications for a privacy-first alternative.
**Confidence labels** used throughout:
| Label | Meaning |
|---|---|
| **[Established]** | Replicated peer-reviewed finding or protocol fact. |
| **[Primary-code]** | Verified directly in the public Code for Heilbronn firmware / commit. |
| **[Forensic]** | Derived from the delivered CSV; reproducible with the code in Appendix A. |
| **[Peer-reviewed]** | Published in a peer-reviewed venue; single study unless stated. |
| **[Gray]** | Practitioner, government, vendor or thesis source; useful but not peer-reviewed. |
| **[Inference]** | Reasoned from evidence; not directly measured. |
| **[Uncertain]** | Conflicting or missing evidence. |
> **The single most important correction.** The PulseCast series is **not** a 15-minute count of people, and **not** a 15-minute aggregate of device counts. In the public Code for Heilbronn firmware, each sensor wakes every 900 s, scans BLE for about **10 s**, counts distinct locally-administered advertising addresses in that window, multiplies the integer by **0.7**, and transmits **one value**. The 15-minute cadence is the *sampling interval*, not the integration window. Every downstream claim about "people per 15 minutes" inherits this.
>
> **Caveat on firmware versions.** The 2026 production window is exactly consistent with the public 2026 firmware: 100% of its 83,156 published values are exact multiples of 0.7. The older `density-01` hardware produced integer values from October 2025 to March 2026, then switched to multiples of 0.7 in April 2026 — a firmware/calibration break. The exact scan duration of the 2025 build is not public. The direction of every argument below is unchanged whether the scan was 10 s or the configured 30 s.
---
## Executive summary
1. **The unit is a short snapshot, not a 15-minute count.** The public firmware scans for ≈10 s every 900 s — a temporal duty cycle of ≈1.1% (at most 3.3% if the configured 30 s were honoured). A passer-by is detected only if their dwell overlaps the scan window; a person who lingers is counted again in every 15-minute bin. The metric therefore conflates **flow** (people passing) with **dwell** (people staying), and is not a screenline crossing count. **[Primary-code]**
2. **The published value is `0.7 × U`, where `U` is an integer count of distinct locally-administered BLE addresses in one scan.** In the 8-sensor production window (2026-06-12 → 2026-11-07), **100% of 83,156 values are exact multiples of 0.7**; the reconstructed `U` ranges 0–418, median 12, p95 71, p99 114, with 2.7% zeros. The older `density-01` series is integer-valued until 2026-03, then switches to 0.7 — evidence of a firmware/calibration change, not a change in footfall. **[Forensic]**
3. **A single scalar cannot correct three different biases.** The factor 0.7 would have to absorb (i) the share of people carrying any detectable advertising device (`d < 1`), (ii) the mean number of distinct advertising devices per carrier (`m > 1`), and (iii) the probability that a device advertises within the short scan (`q < 1`). The true relationship is `y/P = 0.7 · d · m · q`. No calibration data are published, and the default is a provisioning fallback, not a validated penetration correction. **[Primary-code] [Inference]**
4. **MAC randomization does not inflate the count *within* a 10-s scan, but it destroys everything across scans.** RPA rotation is 900 s by default (range 1–3600 s), and measured address lifetimes are ~15–45 min. Within a 10-s window, rotation is rare; across 15-minute bins, the sensor's dedup bitmap is reset at every wake, so the same person is counted again. Randomization mainly prevents **cross-sensor deduplication, dwell estimation and re-identification**, not within-scan counting. **[Established] [Primary-code]**
5. **The counter silently excludes public-address devices and silently includes non-phone gadgets.** The firmware counts only addresses with the locally-administered bit set. A phone or wearable using a public/static address is invisible; a smart TV, beacon, earbud case or car head-unit with a random address is counted. Vendor OUI cannot be used to separate phones from infrastructure because the counted addresses are random. **[Primary-code] [Inference]**
6. **The literature does not support absolute counts from BLE alone.** Classic Bluetooth discoverability was 2–7% of pedestrians; modern BLE representativity in transit settings was 14–49%, with strong age bias. Absolute indoor count errors range from MAE 1.4–5.5 people (MAPE 24–91%, worst in low occupancy), and a BLE-only ridership model can have **negative R²** until fused with other data. The consistent finding is: **relative trends yes, absolute levels no — unless calibrated against ground truth.** **[Peer-reviewed] [Gray]**
7. **Environmental failure modes are large relative to the signal.** Outdoor BLE range was 20–25 m for two phones in a vineyard test near Heilbronn; a human body adds 10–20 dB path loss (10 dB back-shadowing, 3 dB side at 3 m); Wi-Fi coexistence cut BLE throughput by ~41% in a shielded-chamber experiment; and LoRaWAN adds 1% duty-cycle constraints and real packet loss. Crowding adds collisions and queue drops. **[Peer-reviewed] [Gray]**
8. **The 15-minute series is heterogeneous and incomplete.** Seven sensors have a median gap of 15.3–15.4 min; `density-01` runs at 16.1 min. For the seven named sensors, 2.6–16.2% of intervals exceed 16 min; `density-01` is at a 16.1-min median, so 99.8% of its intervals exceed 16 min. Outages reach ~2.5 h. Missingness is caused by LoRaWAN joins, transmission failures and power, and is not a random sample of footfall. **[Forensic]**
9. **The privacy claim is a design claim, not a property of the data.** The current firmware reduces addresses to a 16-bit ID (the last two bytes; the blog calls this "hashing"), resets every wake, and transmits only counts — genuine data minimisation. But the sensing is still involuntary, the sensor is capable of raw capture, and the legal status of MAC-based counting in Germany is contested: one state DPA concluded that WLAN tracking for visitor counting was unlawful, and the DSK treats reading MAC addresses as touching terminal-equipment privacy. "Privacy-preserving" must be argued from architecture and governance, not asserted. **[Gray] [Uncertain]**
10. **Implication for PulseCast.** Keep the forecaster, change the claim. Treat the series as a **relative activity index in device-address units**, validate it against manual counts, publish the unit and the factor, propagate calibration uncertainty into staffing decisions, and never present device counts as people. The privacy-first alternative must beat *this* sampling design — not a strawman — on cost, coverage, accuracy and privacy.
---
## 1. How BLE PAX counting works
### 1.1 Radio basics
BLE operates in the 2.4 GHz ISM band, divided into 40 channels of 2 MHz. Three channels are reserved for advertising: **37 (2402 MHz), 38 (2426 MHz), 39 (2480 MHz)**. Legacy advertising transmits the same payload on up to three of these channels, one at a time, with a random delay of 0–10 ms per advertising event; the advertising interval is configurable from 20 ms to 10.24 s. A passive scanner tunes to the advertising channels, reports the advertiser's 48-bit address and an RSSI value, and never needs to connect. [1][3]
Two consequences matter for counting:
- **The address is self-asserted.** Before a connection is authenticated, the advertising address is whatever the device chooses. It can be public, random static, resolvable private (RPA) or non-resolvable private (NRPA). A listener cannot tell whether two addresses belong to the same physical device, or whether one device is impersonating another. [1][4]
- **RSSI is a noisy proxy for proximity, not distance.** It has 1 dB resolution and is affected by orientation, body shadowing, multipath and channel. Using it as a hard range gate creates a detection zone that is neither circular nor stable. [20][21]
### 1.2 The PulseCast / Code for Heilbronn implementation
The Heilbronn network is maintained by Code for Heilbronn (CFHN). Their public blog and repository identify the hardware and firmware; the Fraunhofer IAO/KODIS project page documents the wider Heilbronn and Schwäbisch Hall deployment. [8]
- **Hardware:** Heltec Wireless Paper / Heltec WiFi LoRa 32 V3 — ESP32-S3, SX1262 LoRa transceiver, BLE 4.2. [6][7]
- **Firmware:** PlatformIO project `sensor-pax`, built on a vendored `paxlib` derived from the Deutsche Bahn `libpax` / ESP32-Paxcounter lineage. BLE only; Wi-Fi counting is compiled out. [7]
- **Counting rule:** `mac_add()` accepts an advertising report only if the **locally administered bit** (`paddr[0] & 0b10`) is set; it then uses the **last two bytes** of the address as a 16-bit ID in a bitmap. New bit → increment; existing bit → ignore. The bitmap is cleared on every boot. [7] The blog calls this "hashing"; the code does **not** hash — it truncates the address to its low 16 bits. That is a dedup key, not a cryptographic hash, and it leaks the low bits of the address.
- **Scan parameters:** passive scan; `BLESCANWINDOW = 80 ms`, `BLESCANINTERVAL = 80 ms` → 100% duty cycle *during the scan*; RSSI threshold `-80 dBm`; report interval **10 s**; counter mode **1 (cumulative)**. [7]
- **Wake cycle:** `sleep_sec` default **900 s**; the node wakes, scans, waits for the first 10-s report callback, stops the scan, sends one LoRaWAN uplink, and deep-sleeps. [7]
- **Payload:** `paxCount * factor`, with `factor` default **0.7** in `provision.h`; the value is formatted with four decimals. [7]
A code-level trace of one cycle:
```
wake (deep-sleep reset)
└─ paxSetup(): blescantime=30, window=80ms, interval=80ms, RSSI≥−80 dBm
libpax_counter_init(callback, count, 10 s, mode=1)
└─ start_BLE_scan(): passive; duplicate filter disabled
└─ first 10-s timer callback → new_data_available = true
└─ paxStop(); paxCount = count_from_libpax.pax
└─ payload = sprintf("%s,0,%.4f,1,%.4f", id, paxCount * cfg.factor, battery)
└─ LoRaWAN TX; deep sleep 900 s
```
**Why ≈10 s, not 30 s.** `paxSetup()` sets `configuration.blescantime = 30`, but the vendored `start_BLE_scan()` accepts `blescantime` and never uses it: it only programs scan window and interval, then starts scanning. The only stop condition is `main.cpp`'s wait for `new_data_available`, which is set by the 10-s report callback. The effective scan is therefore **≈10 s** (plus up to 100 ms of loop delay). If a deployed build implemented `blescantime` as a hard stop, the scan would be 30 s and the duty cycle 3.3%; the argument below is unchanged in direction. **[Primary-code, high confidence for the public commit; moderate confidence for the deployed 2025 binary]**
**Why the bitmap reset matters.** With `counter_mode = 1`, `report()` does *not* reset the bitmap. But each wake is a cold boot from deep sleep, so the bitmap starts empty anyway. The dedup window is therefore **one scan**, not one hour and not one day. The 2025 blog says addresses are "hashed and stored for at most one hour" — that may describe an earlier firmware or a backend design; the public 2026 firmware does not implement a one-hour store. This is an open question (Section 7). [6][7]
### 1.3 What one published value means
Let:
- `U` = number of distinct 16-bit IDs derived from locally-administered advertising addresses observed during one ≈10 s scan;
- `f` = configured multiplier (default 0.7);
- `y` = published "density" value.
Then:
```
y = f · U
```
`U` is **not** the number of people in the area, the number of devices in the area, the number of devices passing in 15 minutes, or the number of 15-minute unique visitors. It is a **short-window count of distinct randomized advertising addresses**, scaled by a constant.
If a person carries `m` advertising devices, is detectable with probability `d` (device ownership × Bluetooth state × advertising state), and their devices advertise within the scan with probability `q`, then the expected relationship to the true number of people present `P` is:
```
E[y] = f · d · m · q · P
```
The factor `f = 0.7` can only be a correct people-per-device conversion if `d · m · q = 1/0.7 ≈ 1.43`. That is possible if `d` is high and `m ≈ 1.4`, but it is not measured. Assuming `q ≈ 1` (devices in range for the full scan), if `d = 0.5` and `m = 1.4`, `y/P ≈ 0.49` — a 2× undercount. If `d = 0.9` and `m = 1.2`, `y/P ≈ 0.76` — a 24% undercount. The scale is therefore **arbitrary without ground truth**, and a single global factor cannot track seasonal, demographic or device-mix changes. **[Inference]**
### 1.4 Forensic evidence from the delivered CSV
The delivered file `data/raw/schwarz-pax.csv` contains 192,464 rows; 96,232 are `densityData`. The 8-sensor production window (2026-06-12 → 2026-11-07) contains **83,156 density rows** across eight sensor names. Every one of those values is an exact multiple of 0.7:
| Sensor (production window) | Rows | Values exact multiples of 0.7 | Reconstructed `U`: median | p95 | max | `U = 0` |
|---|---:|---:|---:|---:|---:|---:|
| hn-stadtgalerie-01 | 8,982 | 100% | 12 | 96 | 196 | 0.0% |
| hn-neckarufer-01 | 9,527 | 100% | 17 | 88 | 365 | 0.1% |
| hn-marahaus-01 | 9,254 | 100% | 13 | 55 | 278 | 0.0% |
| hn-hagenbuchersee-02 | 10,099 | 100% | 16 | 92 | 287 | 0.7% |
| hn-hagenbuchersee-01 | 12,345 | 100% | 9 | 71 | 418 | 5.6% |
| hn-soleo-01 | 10,327 | 100% | 12 | 40 | 159 | 4.8% |
| hn-goetzentuem-01 | 9,952 | 100% | 12 | 42 | 301 | 3.0% |
| density-01 (f8c19730) | 12,670 | 100% | 11 | 40 | 90 | 5.5% |
| **All eight** | **83,156** | **100%** | **12** | **71** | **418** | **2.7%** |
**Interpretation.** A continuous-valued physical series would not land on exact multiples of 0.7 in 83,156 consecutive observations. The only plausible explanation is that the published value is `round(0.7 × integer, 4)`. The reconstructed `U` is an integer count of distinct randomized addresses in a scan. This is a **direct, data-internal confirmation** of the firmware reading. **[Forensic]**
**The calibration break.** The older `density-01` hardware (`863f75b0`, 2025-10-16 → 2026-03-12) is **100% integer-valued** and has a mean of 1.7–5.5 depending on month. The replacement hardware (`f8c19730`, 2026-04-16 → 2026-11-07) is **100% multiples of 0.7** and has a mean of 8.3–11.6 across full months. The series therefore splices two instrument regimes: an integer-count firmware and a 0.7-scaled firmware. Any model that uses the 13-month `density-01` history across the break is fitting a calibration change, not footfall. **[Forensic]**
**What this does not prove.** The data cannot tell us the deployed scan duration, the exact `factor` value per sensor (only that 0.7 appears throughout the delivered 2026 production data), whether the 2025 firmware used a one-hour dedup window, or whether the sensors were re-provisioned. Those are open questions (Section 7).
### 1.5 Dedup, dwell time and the 15-minute cadence
A PAX counter *can* estimate dwell by tracking a persistent ID across consecutive scans and recording first/last seen. This one does not:
- **Within a scan**, the bitmap deduplicates by a 16-bit ID taken from the last two address bytes. Two devices sharing those bytes collide (expected collisions ≈ `U(U−1)/(2·65536)`; negligible at `U = 12`, ~1.3 at `U = 418`). A device that changes only its low bytes is counted as new.
- **Across scans**, the bitmap is gone. There is no first/last-seen timestamp in the payload. The only data that leave the sensor are `U`, `U × 0.7` and battery voltage.
- **Dwell resolution** would be at best 15 min even if IDs persisted, and RPA rotation (~15 min default) caps observable track length. Any dwell estimate beyond ~15 min is structurally impossible with this design.
What repeated counts *can* proxy is **aggregate persistence**: a person who stays for an hour is counted in up to four bins, while a passer-by is counted with probability ≈ `(scan + dwell)/900`. But this is confounded with arrival rate and with the number of devices per person, and cannot be inverted without strong assumptions. **Do not claim dwell from this data.** **[Primary-code] [Inference]**
### 1.6 Transport, aggregation and missingness
- **One row per uplink.** The evidence is consistent with one row per uplink; the 15-minute series is a sequence of point samples, not an average or sum over the interval.
- **Cadence is sensor-specific.** Seven sensors have a median gap of 15.3–15.4 min; `density-01` has 16.1 min. For the seven named sensors, 2.6–16.2% of intervals exceed 16 min; `density-01` runs at a 16.1-min median, so 99.8% of its intervals exceed 16 min. Long outages reach ~2.5 h. The pooled share of >16-min gaps is 23.7%, inflated by `density-01`'s cadence.
- **Missingness is not random.** A failed LoRaWAN join, a failed TX, a battery brownout or a sensor reboot removes a row. The application sends once and sleeps; if the uplink fails, the count is not queued for a later wake. Missing rows are therefore **missing observations**, not zeros, and may correlate with weather, power and site conditions. **[Primary-code] [Forensic]**
- **LoRaWAN constraints.** EU868 imposes a 1% duty cycle on the default sub-band; a 51-byte payload at SF7–SF12 has a time-on-air from tens of milliseconds to >2 s. A 15-minute uplink is well within the duty cycle, but join storms, downlinks and retransmissions can still fail. In a Brno urban measurement, the cumulative packet delivery ratio was 83%, and 50% of nodes within 1–2 km of a gateway experienced at least one packet loss. [23] **[Peer-reviewed]**
---
## 2. MAC address randomization
### 2.1 Address taxonomy
| Type | Top two bits | Lifetime | Who can resolve it | Relevance to counting |
|---|---|---|---|---|
| Public | — (OUI-assigned) | Permanent | Anyone; OUI identifies vendor | **Excluded** by the CFHN LAA filter; would allow vendor/type filtering if counted |
| Random static | `11` | Until power cycle | Anyone; no resolution | Counted if the LAA bit is set; stable within a session |
| Resolvable private (RPA) | `10` | 900 s default; 1–3600 s configurable | Only holders of the IRK | Counted; rotates; can be tracked via payload side channels |
| Non-resolvable private (NRPA) | `00` | Implementation-defined | No one | Counted; rotates; used by some Apple advertising |
The locally-administered bit is set for all random types. That is why the CFHN filter counts them and excludes public addresses. [1][4]
### 2.2 Rotation intervals
| Source | Claim | Confidence |
|---|---|---|
| Bluetooth Core Spec / SIG | RPA timeout default **900 s**; configurable 1–3600 s; new "Randomized RPA Updates" default range 480–900 s | **[Established]** [2] |
| Apple Platform Security | iOS/iPadOS use address randomization; no interval stated | **[Established]** [4] |
| Android AOSP | Controller/host support for RPA generation and resolution; no interval stated | **[Established]** [5] |
| Becker et al. 2019 (measurement) | Address lifetime: Windows 10 ~16 min; macOS/iOS ~20 min; Android 15–45 min; Fitbit Charge static (never rotates) | **[Peer-reviewed]** [17] |
Wi-Fi private-address rotation (per-network, days-to-weeks) is a **separate mechanism**; do not transfer its intervals to BLE. The 900 s RPA default and the 900 s wake cycle are the same order of magnitude, so a device can rotate its BLE address between consecutive bins while the person stays in place. Because the counter resets at every wake, this does **not** double-count within a bin; it mainly makes cross-bin and cross-sensor linkage impossible.
### 2.3 Effect on unique counts, dwell and re-identification
| Quantity | Effect of randomization | Why |
|---|---|---|
| Within-scan unique count | **Little inflation** | Rotation interval (15–45 min) ≫ scan (10 s); most devices keep one address per scan |
| Across-bin count | **No dedup possible** | Bitmap resets each wake; a lingerer is counted again; rotation is irrelevant |
| Dwell time | **Not measurable** | No first/last-seen timestamp; rotation caps track length at ~15 min |
| Cross-sensor re-identification | **Prevented at the address level** | Different sensors see different addresses; no shared ID |
| Privacy | **Improved, not solved** | RPA can be tracked via payload carry-over (Becker: Apple 53 min, Windows 11.2 h); static-address accessories are permanent [17] |
**The counterintuitive result:** MAC randomization is often blamed for *inflating* PAX counts. In this design, it does not inflate the within-scan count; the dominant overcount is **repeated counting of the same lingering person across 15-minute bins**, and the dominant undercount is **short dwell inside a 10-s scan**. Randomization mainly destroys the ability to *correct* either bias. This is a more precise and more damaging critique than "random MACs inflate counts." **[Primary-code] [Inference]**
### 2.4 Re-identification beyond rotation
Randomization is not anonymity. Becker, Li and Starobinski showed that BLE advertising payloads contain identifying tokens that change asynchronously with the address; an "address-carryover" algorithm tracked:
- Windows 10 devices for up to **11.2 h** across 12 address randomizations;
- Apple devices for up to **53 min** across five randomizations;
- a Fitbit Charge indefinitely, because its address never changed.
They also found an iOS side channel that reveals user activity, and a Microsoft Surface pen that leaks an identifier. Android devices in their sample did not advertise suitable tokens, but that is an implementation detail, not a guarantee. [17] **[Peer-reviewed]**
For PulseCast, the implication is architectural: even if the current firmware transmits only counts, the **sensor is capable of capturing raw addresses**, and BLE advertising in general has been used for re-identification research. A privacy claim must therefore cover firmware integrity, update channels, physical access and retention — not only the payload.
### 2.5 Spoofing, cloning, static MACs and vendor OUI
- **Spoofing is trivial.** An ESP32 or a phone can advertise arbitrary addresses at arbitrary intervals. A single attacker can generate hundreds of distinct 16-bit IDs in a 10-s window, inflating a sensor's count without any real footfall. There is no authentication before connection. [1] **[Established]**
- **Cloning is trivial.** Copying a known address produces a duplicate; the 16-bit ID makes collisions easy to construct deliberately.
- **Static-address devices are invisible.** The LAA filter excludes public addresses and any random-static device whose address happens to have the LAA bit clear. A Fitbit with a static random address is counted (LAA set), but a public-address phone is not. The LAA bit is **not** the same as the Bluetooth random-address type bits: the spec fixes the two most significant bits, not the LAA bit. If an OS does not set LAA=1 for its random addresses, the filter drops them. This is an unquantified undercount and a candidate explanation for the low raw counts. **[Inference] [Uncertain]**
- **Vendor OUI is unusable here.** OUI is only meaningful for public addresses. Because the counter discards public addresses, it cannot distinguish a phone from a TV, a beacon, a car head-unit or a smart meter. The count is "locally-administered BLE advertisers," not "people."
- **Non-device sources.** BLE is used by earbuds, watches, fitness trackers, smart home devices, beacons, cars, scooters and point-of-sale terminals. A shopping street at 19:00 contains many such devices, and their density is not proportional to pedestrian flow.
### 2.6 The privacy posture of the current design
The CFHN design is genuinely data-minimising:
- Addresses are reduced to a 16-bit ID at the edge; only that ID is kept, and only for one scan.
- Only counts leave the sensor; no raw addresses are transmitted.
- The blog explicitly says addresses are hashed and stored for at most one hour. [6] The public 2026 code truncates rather than hashes, and keeps the ID only for one scan; the one-hour claim should be treated as unverified for the deployed firmware.
But:
- The sensing is **passive and involuntary**. A bystander cannot consent, cannot see the sensor, and cannot opt out except by disabling Bluetooth.
- The sensor is **capable of raw capture**; the privacy guarantee depends on firmware behaviour that a third party cannot verify from the data.
- The 16-bit ID is **not a cryptographic anonymity guarantee**. It is a truncation used as a dedup key: it leaks the low 16 bits of the address, makes collisions easy to construct, and provides no cryptographic protection.
- German data protection authorities have treated MAC-based visitor counting sceptically. The Schleswig-Holstein DPA's 2019 activity report concluded that WLAN tracking for visitor counting was unlawful; the DSK's 2022 guidance says that reading MAC addresses can fall under §25 TDDDG's consent requirement for terminal-equipment access. The legal position is contested and fact-specific, not settled. [33][34] **[Gray] [Uncertain]**
The honest formulation for the paper: **"The current firmware minimises data, but the modality is privacy-invasive by design: it observes identifiers that people did not choose to broadcast to the operator, and its privacy properties depend on unverifiable edge behaviour."** That is defensible; "the sensors are privacy-preserving" is not.
---
## 3. Failure modes
The table separates mechanisms that **under-count** from those that **over-count** and those that **change the composition** of the count. Directions are for the current design; some are context-dependent.
| # | Mechanism | Direction | Magnitude / evidence | Confidence |
|---|---|---|---|---|
| 1 | **Temporal sampling**: ≈10 s scan every 900 s (1.1% duty cycle) | Under-count of passers-by; over-count of lingerers across bins | A person with 30 s dwell has ≈(10+30)/900 ≈ 4.4% chance of overlapping a scan; a 1 h lingerer is counted up to 4 times | **[Primary-code] [Inference]** |
| 2 | **Construct mismatch**: circular presence zone, no direction | Unknown; conflates flow and dwell | A PAX scan counts devices present during the window, not screenline crossings; NCHRP's guidance is built around screenline counts | **[Established]** [24] |
| 3 | **Detection range / RSSI gate** | Under-count | 20–25 m outdoors for two phones in a vineyard test near Heilbronn; RSSI threshold −80 dBm; range varies with antenna, height, enclosure | **[Gray]** [9] |
| 4 | **Body shadowing / orientation** | Under-count, composition bias | +10 dB path loss for one body, +15 dB for two (indoor); 10 dB back vs 3 dB side at 3 m | **[Peer-reviewed]** [20][21] |
| 5 | **Crowding / collisions / queue drops** | Under-count at high density | HCI advertising reports can be dropped when the 30-deep queue fills; dense advertising increases collisions; no drop counter in the vendored build | **[Primary-code] [Inference]** |
| 6 | **Wi-Fi coexistence** | Under-count | BLE throughput −41% with Wi-Fi channel 6 active; PER 2–9% (up to 12%) in a shielded chamber | **[Gray]** [22] |
| 7 | **Only locally-administered addresses** | Under-count of public-address and LAA-clear devices; over-count of non-phone random devices | Public-address phones/wearables invisible; TVs, beacons, earbuds counted; LAA bit is not the random-address type bits | **[Primary-code] [Inference]** |
| 8 | **16-bit ID collisions** | Under-count | ≈`U(U−1)/(2·65536)`; negligible at median, ~1.3 collisions at U=418 | **[Primary-code] [Inference]** |
| 9 | **Device multiplicity** | Over-count | One person may carry phone + watch + earbuds; wearables can outnumber discoverable phones; exact ratio unknown for Heilbronn | **[Peer-reviewed]** [15] **[Uncertain]** |
| 10 | **Demographic / behavioural bias** | Composition bias | Smartphone use 86% of 16+ in Germany but 53% of 65+; BLE representativity fell when elderly dominated; locked iPhones were not detected in the KODIS test | **[Official statistics]** [31][32] **[Gray]** [9][15] |
| 11 | **LoRaWAN loss / join failure** | Missing rows (not zeros) | 2.6–16.2% of intervals >16 min for the seven named sensors; `density-01` at 16.1 min; outages up to ~2.5 h; Brno urban PDR 83% | **[Forensic] [Peer-reviewed]** [23] |
| 12 | **Clock / time sync** | Binning / phase error | Payload has no device timestamp; server receive time is used. Sleep scheduling and network latency shift the sampling phase; clock drift changes it over time | **[Primary-code] [Uncertain]** |
| 13 | **Placement / enclosure / weather** | Unknown, site-specific | Height, orientation, glass, metal, condensation, snow; no placement metadata in the CSV | **[Inference]** |
| 14 | **Firmware / calibration change** | Level shift | `density-01` switches from integer counts to 0.7-scaled values in April 2026; mean jumps from ~5.5 to ~9.5 | **[Forensic]** |
| 15 | **Spoofing / adversarial inflation** | Over-count | Trivial to advertise hundreds to thousands of addresses; no auth | **[Established]** [1] |
| 16 | **Cross-sensor double counting** | Over-count | No shared ID across sensors; overlapping coverage would count the same device twice; 580 m cluster makes overlap limited but not zero | **[Primary-code] [Inference]** |
**The dominant three.** For a paper, do not present a flat list. Argue that three mechanisms dominate and are not fixable by calibration:
1. **Temporal sampling (1.1%)** — the sensor sees a tiny, phase-random slice of each 15-minute interval. This is a design property, not noise.
2. **Construct mismatch (presence vs flow)** — the count is proportional to `flow × dwell`, so a street with cafés and a street with commuters can produce the same count at different footfall.
3. **Unknown penetration × multiplicity** — `d` and `m` move in opposite directions and are not measured. A single constant cannot track them across seasons, weather, events and demographics.
---
## 4. Evidence table
Peer-reviewed and primary sources first; gray literature is marked.
| Study / source | Setting | Method | Key finding | Citation |
|---|---|---|---|---|
| Nicolai & Kenn (2007) | Bremen (DE) and San Francisco (US) | Manual gate counts + Bluetooth inquiry | 2.3% (Bremen) and 6.2% (SF) of people had discoverable Bluetooth; R² 0.85–0.91 in Bremen; Bath study (O'Neill et al., as reported) ~7%, R² 0.89 | [10] |
| Malinovskiy, Saunier & Wang (2012) | Montreal and Seattle | Static Bluetooth sensors + manual counts | 5% (Montreal) and 2.25% (Seattle) sample rates; trends and dwell (>60 s) usable; low-volume scenarios insufficient | [11] |
| Weppner & Lukowicz (2013) | Kaiserslautern public viewing | Phone-based Bluetooth scans | >75% accuracy on 7 crowd-density classes using relative features; only 0.2% of people carried a scanning phone | [12] |
| Kurkcu & Ozbay (2017) | NYU transit terminal | 6 Wi-Fi/Bluetooth sensors, 2 months | If penetration is known, counts/flows/wait times can be accurate; reported Bluetooth detection as low as 2%; expansion depends on known penetration | [13] |
| Angel et al. (2024) | Tel Aviv, 65 detectors | Calibration + speed classification | 89% pedestrian detection accuracy; 60k unique MACs/day; accuracy is record classification, not volume validation | [14] |
| Minea (2022) | Bucharest subway | Manual counts vs BLE/BT scans | 14–49% representativity; 31.6% average at one station; age bias; wearables outnumber discoverable phones | [15] |
| Matsuda et al. (2023) | 4 indoor spaces, Japan | BLE scanners + camera/manual labels | MAE 1.4–5.5 people; MAPE 24–91%; performance poor below 10 people | [16] |
| Becker, Li & Starobinski (2019) | Lab/field BLE capture | Address + payload tracking | Address life ~16 min (Windows), ~20 min (Apple), 15–45 min (Android); carry-over tracking 53 min (Apple), 11.2 h (Windows); Fitbit static | [17] |
| Martin et al. (2017) | 2.6M Wi-Fi MAC corpus | Wi-Fi randomization study | iOS randomized; Android adoption low in 2015–16; 96% of Android defeatable | [18] |
| Fenske et al. (2021) | 160 phone models | Lab Wi-Fi randomization test | Newer phones are better; many older phones still leak; randomization is device/OS-dependent | [19] |
| Januszkiewicz (2018) | Indoor 2.4 GHz | FDTD + measurements | +10 dB path loss for one body, +15 dB for two; 10–20 dB near body | [20] |
| Deng, Fu & Wang (2018) | Indoor BLE | RSSI measurements | 10 dB back-shadowing, 3 dB side at 3 m; 6–8 dB typical BSIE | [21] |
| Mahmud et al. (2023), INL | Anechoic chamber | BLE + Wi-Fi coexistence | BLE throughput −41% under Wi-Fi; PER 2–9% (up to 12%) | [22] |
| Masek et al. (2021) | Brno, 311 locations | LoRaWAN urban measurement | 83% cumulative PDR; 50% of nodes within 1–2 km had packet loss | [23] |
| Ryus et al. (2014), NCHRP 797 | US | Standards/methods review | Correction factors from validation counts; annual revalidation; accuracy vs consistency; screenline vs presence | [24] |
| FHWA TMG Ch. 4 | US | Federal guidance | Manual counts as ground truth; automated counters need calibration; short counts are highly variable | [25] |
| Ohlms et al. (2018), VTRC 19-R4 | Virginia | State program review | Bluetooth alone not feasible as of 2016; reported Wi-Fi detection ~26.4%; validation produces site-specific correction factors | [26] |
| Hook & Regal (1995, 2000) | Epidemiology | Capture-recapture theory | Four assumptions; positive dependence biases estimates downward; empirical underestimation 16–25% | [27][28] |
| Code for Heilbronn (2025–26) | Heilbronn | Blog + firmware repo | Heltec ESP32 + SX1262; BLE-only; LAA filter; 16-bit ID (truncation); 0.7 factor; 900 s sleep; one uplink per wake | [6][7] |
| Fraunhofer IAO/KODIS (2023) | Heilbronn vineyards | 3 LoPy4 + 2 phones | BLE range 20–25 m; locked iPhone not detected; simultaneous Wi-Fi+BLE causes errors; trend use only | [9] |
| Destatis (2024); Bitkom (2026) | Germany | Household/person surveys | 90.8% of households have a smartphone; 86% of 16+ use one; 53% of 65+ | [31][32] |
**What the table establishes.** Across four decades and three radio generations (classic Bluetooth inquiry, Wi-Fi probe requests, BLE advertising), the same pattern recurs: **detection is a small, biased, variable sample of people; counts track trends; absolute levels require external calibration.** The Heilbronn deployment is not an outlier; it is a particularly sparse instance of a known design.
**What the table does not establish.** No study validates a 10-s BLE scan every 15 min against manual counts in a German shopping street. No study measures the device-multiplicity distribution in Heilbronn. No study quantifies the effect of the 0.7 factor. These are gaps, not evidence of accuracy.
---
## 5. Calibration and validation
### 5.1 What counts as ground truth?
| Method | Strength | Weakness | Use for BLE |
|---|---|---|---|
| Manual screenline counts | Direct, transparent, cheap | Labour-intensive; 2 h fatigue; observer bias; no direction at high flow | **Best short-term reference**; stratify by daypart and day type |
| Video with manual annotation | Auditable; can rewind; direction | Privacy; storage; occlusion in crowds | **Best for dense sites**; annotate a sample, not 24/7 |
| Passive infrared / laser beam | Continuous; cheap | Occlusion; group counting; no identity | **Independent continuous reference**, but has its own bias |
| Turnstiles / gate counts | Exact at controlled entries | Only for enclosed sites | Good for a shop entrance, not a street |
| Wi-Fi probe counts | Higher detection than BLE in some studies | Randomization; OS differences; also privacy-invasive | **Not ground truth**; a second biased sensor |
The NCHRP guidance is explicit: accuracy is the magnitude of difference from a ground-truth count; consistency is the residual variability after correcting expected bias; correction factors should be developed locally and revalidated at least annually. [24] The FHWA TMG likewise treats manual counts as the reference and warns that short non-motorized counts are highly variable. [25]
### 5.2 Ratio / expansion estimation
The simplest calibration is a ratio estimator:
```
P_hat = y / r, r = mean(y / P) over validation samples
```
Equivalently, `P_hat = k · y` with `k = 1/r`. This is the "penetration rate" or "expansion factor" approach used in Bluetooth transport studies; the ratio estimator itself is standard sampling theory. [11][13][30]
**What it requires:** a stable `r` across the conditions where it is applied. **What breaks it:**
- `r` varies with device mix (iOS/Android/wearables), age, weather, time of day, event type and sensor placement.
- `r` is a ratio of two noisy quantities; its variance is dominated by small `P` bins. At `P = 10`, a ±3 count error is ±30%.
- A single global `k` applied to all sensors hides site-specific detection zones.
- If `r` is estimated from a period with different firmware/factor, it is not transferable.
NCHRP's guidance is explicit that a correction factor is only valid for the site and conditions where it was developed; local factors and auxiliary counts are preferred over a single global constant. [24] The same logic applies to BLE: a global 0.7 cannot absorb site, season and device-mix variation. The lesson is not "Bluetooth is useless"; it is **"a single expansion factor is not enough; you need auxiliary counts and a model."**
### 5.3 Capture-recapture
With two or more independent observations of the same population, the Lincoln–Petersen estimator is:
```
N_hat = (n1 · n2) / m
```
where `n1`, `n2` are the counts in each sample and `m` is the number matched. Four assumptions must hold: closed population, correct matching, independence of samples, and homogeneous capture probability. [27]
**Why this is hard for BLE:**
- **Matching** requires a persistent identifier. The current firmware has none; even with persistent IDs, RPA rotation breaks matching after ~15 min.
- **Independence** is violated if both sensors see the same bodies/obstacles or if a person carrying more devices is more likely to be captured by both.
- **Heterogeneity** is severe: a person with a phone + watch has multiple chances; a person with a locked iPhone may have none. Heterogeneous capture probabilities can bias `N_hat` downward; positive dependence also biases it downward. Hook & Regal found 16–25% underestimation in empirical capture-recapture studies with positive dependence. [28]
**Verdict:** capture-recapture is a useful **diagnostic** (e.g., two adjacent sensors with overlapping coverage, or manual + sensor), but naive Lincoln–Petersen on BLE counts will not give a defensible population estimate. If used, report the assumptions and a sensitivity range, not a point estimate.
### 5.4 Regression calibration
Regress ground truth on sensor counts:
```
P_t = α + β · y_t + γ · X_t + ε_t
```
where `X_t` includes hour, day type, weather, sensor ID and event flags. Use a time-based holdout and report out-of-sample MAE/MAPE, not in-sample R².
**Why it can help:** it allows the detection rate to vary with covariates; it can absorb site-specific factors; it gives a principled way to propagate uncertainty.
**Why it can mislead:**
- Classical measurement error in `y` attenuates `β` toward zero; the naive model looks less responsive than it is. Regression calibration replaces `y` with `E[P|y]` and can correct this, but only if the calibration model is correctly specified and validated on independent data. [29]
- If the sensor's detection rate changes after the calibration period (firmware, device mix, season), the calibration is stale.
- Extrapolating beyond the observed range (e.g., event peaks) is unsupported.
- A high R² on a holdout does not prove the level is right; it proves the *variation* is tracked.
For PulseCast, the honest use is: **calibrate the index to people for a defined site and period, report the calibration window and covariates, and revalidate.** Do not apply a June calibration to November without evidence.
### 5.5 What can and cannot be inferred
| Question | Can the current data answer it? | Why |
|---|---|---|
| Is today busier than yesterday? | **Yes, as a relative index** | Same sensor, same unit, short horizon; detection conditions similar |
| What is the hourly/daily profile? | **Yes, descriptively** | The series has a coherent daily/weekly shape; high lag-1 autocorrelation (0.84–0.95 at 15 min) |
| How many people passed in the last 15 min? | **No** | `y = 0.7 × U`, where `U` is a 10-s snapshot count; not an integrated flow |
| How many people are in the area now? | **No, not without calibration** | `U` is devices, not people; `d`, `m`, `q` unknown |
| How long do people stay? | **No** | No first/last-seen; dedup resets; RPA rotation |
| Which sensor has more footfall? | **Only with equal detection zones and factors** | Placement, range and factor differ; the 0.7 factor is global but the environment is not |
| Did an event increase footfall? | **Suggestive** | Relative change is visible; magnitude is not identified |
| Are older people under-represented? | **Yes, likely** | 53% smartphone use among 65+ vs 86% overall; BLE representativity falls with elderly share [15][31][32] |
| Can we forecast next week? | **Yes, in device units** | The series is predictable; the forecast inherits the unit error |
| Can we claim privacy-preservation? | **Only with architecture + governance evidence** | Payload is minimised; sensing is involuntary; firmware is unverifiable |
### 5.6 A minimum validation protocol for PulseCast
If the paper wants to make a calibration claim, this is the smallest defensible protocol:
1. **Sites:** at least 3 sensors spanning the range of environments (shopping street, riverfront, mall entrance).
2. **Reference:** manual screenline counts by two observers, 15-min bins, 2 h per session; video for dense bins.
3. **Design:** 2 weeks; 4 dayparts (morning, lunch, afternoon, evening); 2 weekdays + 1 weekend day per week; at least 200 people per bin for stable ratios.
4. **Synchronisation:** align sensor timestamps to the manual bin; discard bins with a gap > 1 min.
5. **Metrics:** ratio `y/P`, bias, MAE, MAPE, Bland–Altman limits of agreement; report by site and daypart, not only pooled.
6. **Model:** calibration with hour, day type, weather and sensor ID; time-based holdout; report out-of-sample error.
7. **Uncertainty:** bootstrap the ratio; propagate the calibration interval into the staffing recommendation.
8. **Revalidation:** repeat at least seasonally; re-check after firmware or device-mix changes.
9. **Report:** publish the raw `U`, the factor, the calibration window and the residual diagnostics. Do not publish only the calibrated people number.
This is more work than a hackathon usually allows, but even a **one-day, three-site pilot** would be enough to replace "we assume 0.7" with "we measured a ratio of X (95% CI Y) on one day in November." That is a much stronger paper.
---
## 6. Implications for PulseCast
### 6.1 Reframe the target
The current target is a 15-minute "density" value. The defensible target is a **relative activity index**: the number of distinct locally-administered BLE advertising addresses seen in a short scan, scaled by a documented constant. Forecast that index, and separately report a calibrated people estimate only where validation exists.
Suggested wording for the paper:
> "The sensor reports `y = 0.7 · U`, where `U` is the number of distinct locally-administered BLE advertising addresses observed during a single ~10-second scan. `U` is a device-address count, not a person count. We treat `y` as a relative activity index and evaluate forecasts in index units. A people-level interpretation would require external calibration, which this dataset does not contain."
This is honest, precise and still supports the forecasting contribution.
### 6.2 Keep the forecasting, change the claim
The series is predictable: daily and weekly cycles, weather sensitivity, holiday effects. A global gradient-boosted model with time-based holdout is a reasonable method. The forecast skill is real **in index units**. The current model's holdout metrics (MAE 3.36, WAPE 27.8%, R² 0.779; 7-day MAE 4.37, WAPE 36.2%, R² 0.672) are computed on `y`, so they measure index forecast skill; a people-level error cannot be derived from them without a calibration study. The paper can claim:
- "We forecast the sensor activity index with MAE X, WAPE Y%, R² Z, against a seasonal-naive baseline."
- "The weather ablation shows temperature/rain add information beyond the time features."
- "The staffing heuristic translates index units into staff hours under explicit assumptions."
It cannot claim:
- "We forecast footfall."
- "The model estimates visitors."
- "The staffing plan is calibrated to people."
unless a validation study is added.
### 6.3 Sensitivity analysis for the 0.7 factor
Because `f = 0.7` is a default, not a measurement, the paper should show how conclusions change under plausible alternatives. A simple approach:
- Recompute the staffing plan for `f ∈ {0.4, 0.7, 1.0, 1.4}` (or a range implied by the literature).
- Report the range of staff hours, not a single number.
- State that the ranking of hours is invariant to a global multiplicative factor, but the **level** and the **cost** are not.
This turns an unidentifiable constant into a transparent assumption. It also protects against the reviewer objection "why 0.7?"
### 6.4 Do not compare levels across sensors
The eight sensors differ in placement, range, enclosure, cadence (`density-01` at 16.1 min vs 15.3 min), and possibly factor. A ranking of "busiest sensor" is only valid if detection zones and factors are equal. The paper can compare **temporal shapes** (when each site peaks) more defensibly than **levels** (how many people each site has).
If a heat map is required, label it "activity index, not footfall" and avoid summing across sensors (sums double-count if coverage overlaps and mix units).
### 6.5 What a privacy-first alternative must demonstrate
The paper argues for a privacy-first alternative. To be credible, the alternative should be evaluated on the same axes as the PAX counter:
| Axis | PAX counter (current) | Requirement for alternative |
|---|---|---|
| Unit | Device-address count × 0.7 | People or clearly defined physical quantity |
| Sampling | 10 s / 900 s (1.1%) | Continuous or documented sampling; no phase bias |
| Direction | None | Direction or screenline crossing |
| Dwell | Not measurable | Measurable or explicitly out of scope |
| Identity | 16-bit address ID at edge | No device identifiers at all |
| Consent | Impossible | Notice + opt-out, or genuinely anonymous sensing |
| Accuracy | Unvalidated | Validated against manual/video ground truth |
| Failure modes | Range, body, crowding, weather, LoRaWAN | Documented and quantified |
| Cost | Low | Comparable total cost (hardware + install + maintenance) |
| Governance | Firmware-dependent | Auditable, open, DPIA-compatible |
Candidate modalities that can meet these requirements better: thermal array counting, radar/LiDAR, stereo vision with on-device counting (no images stored), passive infrared beams at entrances, or manual/video sampling with a statistically designed schedule. Each has its own failure modes; the paper should compare, not assert.
### 6.6 What the paper should say — and not say
**Say:**
- "The deployed sensors count locally-administered BLE advertising addresses in short scans; the published value is 0.7 × that count."
- "This is a relative index; absolute people counts are not identified without external calibration."
- "The 15-minute series is a sequence of point samples, not an aggregate."
- "MAC randomization and the absence of cross-scan dedup prevent dwell estimation and cross-sensor deduplication."
- "The privacy claim depends on edge firmware behaviour and on a legal basis that is contested."
- "Our forecast is evaluated in index units; the staffing plan is a decision aid under explicit assumptions."
**Do not say:**
- "The sensors count people."
- "The data is 15-minute footfall."
- "0.7 converts devices to people." (It is a default, not a validated conversion.)
- "The sensors are privacy-preserving." (Say "data-minimising" and explain the residual risks.)
- "MAC randomization inflates the counts." (In this design, the dominant bias is temporal sampling and repeated counting of lingerers, not within-scan rotation.)
- "The model generalises to other cities." (It is a single 580 m cluster, 5 months, 8 sensors.)
---
## 7. Open questions
1. **Which firmware ran in 2025?** The 2025 blog describes a one-hour hashed store; the 2026 code resets every scan. Resolve by obtaining the deployed binary or the git history/tags for `sensor-pax`. **Why it matters:** the 2025–Mar 2026 `density-01` segment may have a different dedup window and factor.
2. **What are the deployed factors?** The default is 0.7; provisioning requires a value. Are all sensors 0.7? Were any calibrated? **Why it matters:** a per-sensor factor changes cross-sensor comparisons.
3. **What is the actual scan duration in the field?** The code suggests ≈10 s; the configured value is 30 s. Measure with a BLE advertiser of known rate plus a current/logic trace, or instrument the firmware. **Why it matters:** the duty cycle (1.1% vs 3.3%) changes the undercount factor by 3×.
4. **What is the device-multiplicity distribution in Heilbronn?** Phone + watch + earbuds vs phone only; locals vs tourists; age groups. **Why it matters:** `m` is the main overcount; `d` is the main undercount; their product is unknown.
5. **What is the share of locked/standby phones that do not advertise?** The KODIS test found a locked iPhone SE was not detected; is that reproducible across models? **Why it matters:** it sets the effective `d`.
6. **How much do sensors overlap?** With 20–25 m range and a 580 m cluster, overlap is likely limited, but adjacent sensors may see the same device. **Why it matters:** cross-sensor double counting.
7. **Is missingness random?** Compare missing intervals with weather, battery voltage and gateway logs. **Why it matters:** if outages correlate with heat/storm, the series is biased.
8. **What is the legal basis in Baden-Württemberg?** A DPIA, a documented legitimate-interest assessment, and signage/notice. **Why it matters:** the paper's privacy argument needs a jurisdiction-specific anchor, not a generic GDPR gesture.
9. **Can a privacy-first sensor match the cost?** Thermal/radar/video vs ESP32 + LoRaWAN. **Why it matters:** the alternative must be deployable at 8+ sites, not only in a lab.
10. **What would falsify the "relative index" claim?** A manual validation showing that the index's ratio to people varies systematically by daypart or weather. **Why it matters:** it defines the paper's own success criterion.
---
## References
[1] Bluetooth SIG. *Bluetooth Core Specification v5.3.* 2021. https://www.bluetooth.com/specifications/specs/core-specification-5-3/ — advertising channels, intervals, address types, scan parameters.
[2] Bluetooth SIG. "Enhancing device privacy and energy efficiency with Bluetooth Randomized RPA Updates." 15 May 2025. https://www.bluetooth.com/blog/enhancing-device-privacy-and-energy-efficiency-with-bluetooth-randomized-rpa-updates/ — RPA timeout default 900 s, range 1–3600 s; randomized update range 480–900 s.
[3] Bluetooth SIG. *Understanding Reliability in Bluetooth Technology.* 2020. https://www.bluetooth.com/wp-content/uploads/2020/10/EN-Understanding_Reliability.pdf — three primary advertising channels; random 0–10 ms delay; collision avoidance.
[4] Apple. "Bluetooth security." *Apple Platform Security.* https://support.apple.com/guide/security/bluetooth-security-sec82597d97e/web — BLE address randomization; no interval stated.
[5] Android Open Source Project. "HCI requirements." https://source.android.com/docs/core/connect/bluetooth/hci_requirements — RPA generation/resolution support; address type fields.
[6] Code for Heilbronn. "Passantenzahlen messen mit LORA." 18 September 2025. https://codeforheilbronn.de/blog/2025/09/passantenzahlen-messen-mit-lora/ — hardware, claimed hashing, one-hour retention claim, LoRaWAN.
[7] Code for Heilbronn. *lorax / lora-frequenzmessung*, branch `develop`. https://codeberg.org/cfhn/lorax — firmware: `sensor-pax/src/main.cpp`, `pax.h`, `provision.h`; vendored `sensor-pax/lib/paxlib/libpax.cpp`, `blescan.cpp`, `libpax_api.cpp`. Accessed 14 November 2026.
[8] Fraunhofer IAO / KODIS. "Frequenzmessung im innerstädtischen Bereich." https://www.kodis.iao.fraunhofer.de/de/projekte/frequenzmessung.html — ESP32 Paxcounter deployment in Heilbronn and Schwäbisch Hall.
[9] Straßburg, S., Buhl, D., Ködel, L., Göbels, V.P., & Schaefer, K. "Passenger Frequency Measurement — Part I." Research Center KODIS, 11 December 2023. https://medium.com/research-center-kodis/passenger-frequency-measurement-part-i-6a75d2aa63fd — 20–25 m BLE range; locked iPhone not detected; Wi-Fi/BLE interference; trend use only.
[10] Nicolai, T., & Kenn, H. (2007). "About the relationship between people and discoverable Bluetooth devices in urban environments." *Mobility '07*, 72–78. DOI: 10.1145/1378063.1378076.
[11] Malinovskiy, Y., Saunier, N., & Wang, Y. (2012). "Analysis of pedestrian travel with static Bluetooth sensors." *Transportation Research Record* 2299:137–149. DOI: 10.3141/2299-15.
[12] Weppner, J., & Lukowicz, P. (2013). "Bluetooth based collaborative crowd density estimation with mobile phones." *IEEE PerCom 2013*, 193–200. DOI: 10.1109/PerCom.2013.6526732.
[13] Kurkcu, A., & Ozbay, K. (2017). "Estimating pedestrian densities, wait times, and flows with Wi-Fi and Bluetooth sensors." *Transportation Research Record* 2644(1):72–82. DOI: 10.3141/2644-09.
[14] Angel, A., Cohen, A., Dalyot, S., & Plaut, P. (2024). "Estimating pedestrian traffic with Bluetooth sensor technology." *Geo-spatial Information Science* 27(5):1391–1404. DOI: 10.1080/10095020.2023.2247446.
[15] Minea, M. (2022). "An experimental assessment of people's location efficiency using low-energy communications-based movement tracking." *Sensors* 22(22):9025. DOI: 10.3390/s22229025.
[16] Matsuda, Y., Ueda, K., Taya, E., Suwa, H., & Yasumoto, K. (2023). "BLECE: BLE-based crowdedness estimation method for restaurants and public facilities." *ICMU 2023*, 1–6. DOI: 10.23919/ICMU58504.2023.10412158.
[17] Becker, J.K., Li, D., & Starobinski, D. (2019). "Tracking anonymized Bluetooth devices." *Proceedings on Privacy Enhancing Technologies* 2019(3):50–65. DOI: 10.2478/popets-2019-0036.
[18] Martin, J., Mayberry, T., Donahue, C., Foppe, L., Brown, L., Riggins, C., Rye, E.C., & Brown, D. (2017). "A study of MAC address randomization in mobile devices and when it fails." *Proceedings on Privacy Enhancing Technologies* 2017(4):365–383. DOI: 10.1515/popets-2017-0054.
[19] Fenske, E., Brown, D., Martin, J., Mayberry, T., Ryan, P., & Rye, E. (2021). "Three years later: A study of MAC address randomization in mobile devices and when it succeeds." *Proceedings on Privacy Enhancing Technologies* 2021(3):164–181. DOI: 10.2478/popets-2021-0042.
[20] Januszkiewicz, Ł. (2018). "Analysis of human body shadowing effect on wireless sensor networks operating in the 2.4 GHz band." *Sensors* 18(10):3412. DOI: 10.3390/s18103412.
[21] Deng, Z., Fu, X., & Wang, H. (2018). "An IMU-aided body-shadowing error compensation method for indoor Bluetooth positioning." *Sensors* 18(1):304. DOI: 10.3390/s18010304.
[22] Mahmud, S.A., Kasera, S., Ji, M., & Agarwal, V. (2023). *Experimental Evaluation of Interference in 2.4 GHz Wireless Network.* INL/RPT-23-74719, Idaho National Laboratory. https://www.osti.gov/servlets/purl/2242485.
[23] Masek, P., Stusek, M., Svertoka, E., Pospisil, J., Burget, R., Lohan, E.S., Marghescu, I., Hosek, J., & Ometov, A. (2021). "Measurements of LoRaWAN technology in urban scenarios: A data descriptor." *Data* 6(6):62. DOI: 10.3390/data6060062.
[24] Ryus, P., Ferguson, E., Laustsen, K.M., Schneider, R.J., Proulx, F.R., Hull, T., & Miranda-Moreno, L. (2014). *Guidebook on Pedestrian and Bicycle Volume Data Collection.* NCHRP Report 797, Transportation Research Board. https://escholarship.org/content/qt11q5p33w/qt11q5p33w.pdf.
[25] Federal Highway Administration. *Traffic Monitoring Guide*, Chapter 4: "Traffic Monitoring for Non-Motorized Traffic." 2013. https://www.fhwa.dot.gov/policyinformation/tmguide/tmg_2013/traffic-monitoring-for-non-motorized.cfm.
[26] Ohlms, P.B., Dougald, L.E., & MacKnight, H.E. (2018). *Assessing the Feasibility of a Pedestrian and Bicycle Count Program in Virginia.* VTRC 19-R4, Virginia Transportation Research Council. https://vtrc.virginia.gov/media/vtrc/vtrc-pdf/vtrc-pdf/19-R4.pdf.
[27] Hook, E.B., & Regal, R.R. (1995). "Capture-recapture methods in epidemiology: Methods and limitations." *Epidemiologic Reviews* 17(2):243–264. DOI: 10.1093/oxfordjournals.epirev.a036192.
[28] Hook, E.B., & Regal, R.R. (2000). "Accuracy of alternative approaches to capture-recapture estimates of disease frequency: Internal validity analysis of data from five sources." *American Journal of Epidemiology* 152(8):771–778. DOI: 10.1093/aje/152.8.771.
[29] Carroll, R.J., Ruppert, D., Stefanski, L.A., & Crainiceanu, C.M. (2006). *Measurement Error in Nonlinear Models: A Modern Perspective*, 2nd ed. Chapman & Hall/CRC. ISBN 978-1-58488-633-4.
[30] Cochran, W.G. (1977). *Sampling Techniques*, 3rd ed. Wiley. ISBN 978-0-471-16240-7.
[31] Statistisches Bundesamt (Destatis). "Ausstattung privater Haushalte mit Informations- und Kommunikationstechnik." 2024. https://www.destatis.de/DE/Themen/Gesellschaft-Umwelt/Einkommen-Konsum-Lebensbedingungen/Ausstattung-Gebrauchsgueter/Tabellen/liste-gebietsstaende-Infotechnik.html.
[32] Bitkom Research. "Smartphone-Markt 2026." 2026. https://bitkom-research.de/studien/smartphone-markt-2026 — 86% of 16+ use a smartphone; 53% of 65+.
[33] Unabhängiges Landeszentrum für Datenschutz Schleswig-Holstein. *37. Tätigkeitsbericht 2019*, pp. 97–99. https://www.datenschutzzentrum.de/tb/tb37/uld-37-taetigkeitsbericht-2019.pdf — WLAN tracking for visitor counting considered unlawful.
[34] Datenschutzkonferenz (DSK). *Orientierungshilfe für Anbieter digitaler Dienste.* 2022, para. 22. https://www.datenschutzkonferenz-online.de/media/oh/OH_Digitale_Dienste.pdf — reading MAC addresses can require consent under §25 TDDDG.
[35] Yung, C. (2024). *A machine learning framework for public transport ridership estimation using multi-source data fusion with low-cost Bluetooth data.* Master's thesis, Aalto University. https://urn.fi/URN:NBN:fi:aalto-202501241673 — BLE-only model R² = −0.49; fused model R² = 0.86. **[Gray/thesis]**
---
## Appendix A — Reproduction
The forensic claims are reproducible from the delivered CSV with standard Python. No external data are needed.
```python
import csv, datetime as dt, statistics, collections
rows = [r for r in csv.DictReader(open("data/raw/schwarz-pax.csv"))
if r["type"] == "densityData"]
start = dt.datetime.fromisoformat("2026-06-12T00:00:00+00:00")
end = dt.datetime.fromisoformat("2026-11-08T00:00:00+00:00")
prod = [r for r in rows
if start <= dt.datetime.fromisoformat(r["time"]) < end
and r["name"] not in ("maker-fair", "niklas-01", "hn-bollwerksturm-01")]
vals = [float(r["value"]) for r in prod]
raw = [round(v / 0.7) for v in vals]
# 1. All production values are exact multiples of 0.7
assert all(abs(v / 0.7 - round(v / 0.7)) < 1e-6 for v in vals)
# 2. Reconstructed integer counts
print("rows:", len(vals))
print("median U:", statistics.median(raw))
print("p95 U:", sorted(raw)[int(0.95 * len(raw))])
print("max U:", max(raw))
print("zero share:", raw.count(0) / len(raw))
# 3. Per-sensor multiples-of-0.7 check (production window only)
by = collections.defaultdict(list)
for r in prod:
by[r["name"]].append(float(r["value"]))
for name, v in by.items():
share = sum(abs(x / 0.7 - round(x / 0.7)) < 1e-6 for x in v) / len(v)
print(f"{name:26s} n={len(v):6d} multiples0.7={share:.1%}")
# 4. The old density-01 hardware is integer-valued, not 0.7-scaled
old = [r for r in rows if r["name"] == "density-01"
and dt.datetime.fromisoformat(r["time"]) < dt.datetime.fromisoformat("2026-04-01T00:00:00+00:00")]
old_vals = [float(r["value"]) for r in old]
print("old density-01 integer share:",
sum(abs(v - round(v)) < 1e-6 for v in old_vals) / len(old_vals))
```
Expected output for the production window: `rows: 83156`, `median U: 12`, `p95 U: 71`, `max U: 418`, `zero share: ~0.027`, and `multiples0.7=100.0%` for all eight production sensors. The older `density-01` hardware (`863f75b0`, 2025-10 → 2026-03) is 100% integer-valued; the replacement (`f8c19730`, 2026-04 → 2026-11) is 100% multiples of 0.7.
---
## Appendix B — Source map for the strongest claims
| Claim | Source |
|---|---|
| 10-s scan every 900 s; 16-bit ID (truncation); LAA filter; 0.7 factor | `sensor-pax/src/pax.h`, `main.cpp`, `provision.h`; `lib/paxlib/libpax.cpp`, `blescan.cpp`, `libpax_api.cpp` [7] |
| 100% of production values are multiples of 0.7 | Forensic, Appendix A |
| `density-01` integer-to-0.7 regime break | Forensic, Appendix A; monthly means in Section 1.4 |
| RPA default 900 s, range 1–3600 s | Bluetooth SIG 2025 [2] |
| Address lifetimes 15–45 min | Becker et al. 2019 [17] |
| 20–25 m range; locked iPhone not detected | KODIS blog [9] |
| 10–20 dB body loss | Januszkiewicz 2018 [20]; Deng et al. 2018 [21] |
| BLE throughput −41% under Wi-Fi | INL report [22] |
| 2–7% classic BT discoverability | Nicolai & Kenn 2007 [10]; Malinovskiy et al. 2012 [11] |
| 14–49% BLE representativity; age bias | Minea 2022 [15] |
| MAE 1.4–5.5 people; MAPE 24–91% | Matsuda et al. 2023 [16] |
| BLE-only R² = −0.49 | Yung 2024 [35] |
| 83% LoRaWAN PDR in Brno | Masek et al. 2021 [23] |
| Capture-recapture assumptions and downward bias | Hook & Regal 1995/2000 [27][28] |
| NCHRP correction-factor methodology | Ryus et al. 2014 [24] |
| German legal uncertainty | ULD 2019 [33]; DSK 2022 [34] |
+732
View File
@@ -0,0 +1,732 @@
---
title: "MAC-address-based footfall sensing in the EU/Germany: legal and privacy research note"
status: research-note
date: 2026-11-14
scope: "Passive BLE/Wi-Fi MAC sensing in public spaces, with a focus on Heilbronn and the Code for Heilbronn PAX counter design"
---
# MAC-address-based footfall sensing in the EU/Germany
**Purpose.** This note examines the legal and privacy status of passive BLE/Wi-Fi sensing of MAC addresses in public spaces in Germany, and what that means for a Heilbronn-style footfall deployment that aggregates device counts. It is written for a technical paper, not as a legal opinion.
**Disclaimer.** This is a research note, not legal advice. The decisive German question — whether passive reception of broadcast MAC addresses is an "access" within the meaning of § 25 TDDDG — is not settled by published case law or supervisory guidance. A deployment should obtain a written opinion from German counsel before going live, especially if it is operated by or for a public body.
**Evidence labels used below.** `[Law]` binding EU/German law; `[CJEU]` binding interpretation by the Court of Justice; `[Guidance]` EDPB, DSK, CNIL, AP or ICO guidance (persuasive, not binding); `[DPA]` a national supervisory authority's published position or annual report; `[Enforcement]` an enforcement decision or court judgment; `[Technical]` code, standards, or peer-reviewed measurement literature. Full references are in §8.
**Why the Heilbronn case is not generic.** The deployed design described by Code for Heilbronn is a passive BLE scanner that hashes device addresses and keeps them for at most one hour, transmits only counts over LoRaWAN, and publishes 15-minute counts. The upstream reference implementation (`ESP32-Paxcounter`/`libpax`) uses a 16-bit value derived from the last two bytes of the MAC address in a 60-second bitmap. These details matter: they determine whether the system is processing personal data, whether it is pseudonymising or anonymising, and what a DPIA must actually assess. See §2.2 and §5.2.
---
## 1. Bottom line
1. **In this context, MAC addresses should be treated as personal data unless a documented anonymisation assessment proves otherwise.** A MAC address is a device identifier; combined with sensor location and time it singles out a device and can be linked to a person by the controller or by third parties. The CJEU's test is relative but broad: "all the means likely reasonably to be used" by the controller or another person, including means accessible through third parties. `[CJEU]` *Breyer*, paras 42–46; `[Law]` GDPR Art. 4(1), Recital 26. The EDPB's 2026 draft anonymisation guidelines expressly use a MAC address as an example of a hardware identifier that can identify a device and, in turn, an individual. `[Guidance]` EDPB Guidelines 02/2026, para. 25.
2. **Hashing a MAC address is pseudonymisation, not anonymisation.** The operator's statement that "no conclusions can be drawn about devices or persons" is not supported by hashing alone. An unsalted hash of a 48-bit MAC is enumerable; a short or truncated hash (the upstream reference uses 16 bits) has a small space and frequent collisions; a keyed hash can be re-identified by whoever holds the key, because the MAC space is small enough to enumerate. `[Guidance]` EDPB Guidelines 01/2025 (consultation version), para. 22; `[Enforcement]` Conseil d'État, 8 Feb 2017, no. 393714, para. 8; `[DPA]` Autoriteit Persoonsgegevens, "Wifi and bluetooth". Only the aggregated output can become anonymous, and only if the legal test is met.
3. **Edge processing does not remove the need for a legal basis.** The sensor necessarily processes the raw MAC in memory to deduplicate. That is processing of personal data. Anonymising the output later does not retroactively make the collection stage lawful. The legal basis must cover the collection and the anonymisation step, not just the published counts. `[Guidance]` EDPB Guidelines 01/2025, para. 23; `[Guidance]` CNIL, 24 Feb 2020 (anonymisation within minutes still requires a legal basis, information and opt-out).
4. **ePrivacy Art. 5(3) and § 25 TDDDG may apply independently of the GDPR, and independently of whether the data are personal.** Both provisions require prior consent for storing information in, or gaining access to information already stored in, terminal equipment. § 25 TDDDG expressly applies regardless of personal data. `[Law]` ePrivacy Directive Art. 5(3); `[Law]` TDDDG § 25(1), (2); `[Guidance]` DSK, OH Digitale Dienste (Nov 2024), para. 26.
5. **Whether passive sniffing of broadcast MACs is an "access" under Art. 5(3)/§ 25 is the central unresolved question.** The better textual reading is arguably that it is not: the sensor does not store anything in the phone, does not instruct the phone to disclose stored information, and does not retrieve data from the phone's memory; it merely receives a frame the phone broadcasts autonomously. German legislative materials speak of "Abruf" and "Auslesen" (retrieval/readout), which imply an active access. `[Law]` BT-Drs. 19/27441, p. 38. But the EDPB's broad functional approach — MAC addresses are "information" stored in the terminal equipment, and a receiving entity need not be the one instructing the transmission — creates material risk the other way. `[Guidance]` EDPB Guidelines 2/2023, paras 39, 42. There is no published EU or German decision on passive Wi-Fi/BLE sniffing under Art. 5(3)/§ 25.
6. **If § 25 applies, the project cannot rely on legitimate interest, signage or an opt-out.** § 25 requires prior consent meeting GDPR Art. 4(11) and Art. 7; the only exceptions are transmission and strict necessity for a digital service explicitly requested by the user. `[Law]` TDDDG § 25(2). Consent at city scale is practically impossible: walking past a sensor, leaving Wi-Fi on, or seeing a sign is not consent. `[CJEU]` *Planet49*, paras 52–65; `[Law]` GDPR Recital 32. If § 25 applies, the viable routes are a statutory legal basis or a sensor that does not extract MAC addresses. Separately, if the data are personal, Art. 14 information and Art. 21(4) objection notice are required; if the data are truly anonymous, there is no GDPR notification duty, but transparency remains good practice.
7. **For a public body, Art. 6(1)(f) is unavailable.** A municipality acting in the performance of its tasks must rely on Art. 6(1)(e) read with Art. 6(3) and a legal basis in Union or Member State law, or on consent. `[Law]` GDPR Art. 6(1) last sentence, Art. 6(3). For a private association or company, Art. 6(1)(f) is available but the balancing test is strict for covert public-space tracking; German and Dutch DPAs have repeatedly found that the data subject's interests prevail. `[DPA]` ULD Schleswig-Holstein, TB 2019, pp. 97–99; `[DPA]` Autoriteit Persoonsgegevens, "Legal bases for WiFi tracking and Bluetooth tracking"; `[CJEU]` *M5A*, *KNLTB*.
8. **A DPIA is almost certainly mandatory.** Continuous sensing of a publicly accessible inner-city area is the paradigm of "systematic monitoring of a publicly accessible area on a large scale" under Art. 35(3)(c). The Baden-Württemberg supervisory authority's private-sector "Muss-Liste" explicitly lists offline tracking of customer movements in department stores/shopping centres and the non-intended use of device sensors or radio signals to determine location or movement over a substantial period. `[Law]` GDPR Art. 35(3)(c); `[Guidance]` LfDI BW, Muss-Liste, Nos. 2 and 10. There is no public-sector positive list in Baden-Württemberg, so for the city the statutory criterion applies directly. `[Guidance]` LfDI BW/MJM BW, Schwellenwertanalyse (2024).
9. **German enforcement has been administrative, not fine-based, for this exact use case.** The ULD Schleswig-Holstein investigated two city-marketing operators (a Verein and a GmbH) that passively captured probe requests and concluded there was no legal basis; both stopped. The Bavarian authority inspected a fashion chain's Wi-Fi tracking test after a complaint; the company stopped. The Hamburg authority's 2014/2015 report treats WLAN/Bluetooth indoor tracking as highly intrusive and personal-data processing requiring a legal basis. `[DPA]` ULD TB 2019, §5.4.8; `[DPA]` BayLDA TB 2015/16, §§3.10, 22.10; `[DPA]` HmbBfDI TB 2014/2015, §V.9. No published German fine specifically for passive Wi-Fi/BLE footfall tracking was found; that is not evidence of legality.
10. **The most important EU enforcement case ended in the regulator's loss — on evidence, not on the law.** The Dutch DPA fined the municipality of Enschede €600,000 for Wi-Fi tracking in the city centre (pseudonymised MAC + location, used to count unique visitors). The fine was annulled by the Rechtbank Overijssel on 2 February 2024 because the AP had not proved that the municipality processed personal data; the Raad van State upheld that outcome on 29 July 2026, essentially on the procedural ground that the AP's new "direct identification" argument had not been in the original decision. The case is not a safe harbour: it shows that the legal status is fact-specific and that the regulator must prove identifiability. `[Enforcement]` ECLI:NL:RBOVE:2024:594; ECLI:NL:RVS:2026:4403.
11. **The French JCDecaux case is the strongest warning against relying on hashing/salting.** The Conseil d'État upheld the CNIL's refusal of a Wi-Fi tracking experiment on the La Défense esplanade: truncating the MAC, adding a salt and keyed hashing still left the controller able to identify and link records, and the purpose — measuring routes and repeat passages — was itself incompatible with anonymisation. `[Enforcement]` Conseil d'État, 8 Feb 2017, no. 393714, paras 7–8.
12. **For Heilbronn, the defensible architecture is: no raw MAC retention; passive reception only; per-sensor, per-window keyed hashes; no cross-sensor or cross-day linkage; aggregate to zones/time bins; k-threshold plus suppression and, where useful, differential privacy; a DPIA before deployment; on-site signage; a real opt-out; and Art. 28 contracts.** If the project cannot document why § 25 does not apply, it should either obtain consent (which is impractical at city scale) or switch to a sensor that does not extract MAC addresses. The engineering recommendations in §7 are starting points, not legal safe harbours.
---
## 2. Legal classification
### 2.1 The statutory test
`[Law]` GDPR Art. 4(1) defines personal data as "any information relating to an identified or identifiable natural person"; an identifiable person is one "who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person."
`[Law]` Recital 26 adds the decisive qualifier: account should be taken of "all the means reasonably likely to be used, such as singling out, either by the controller or by another person, to identify the natural person directly or indirectly." The assessment must consider "all objective factors, such as the costs of and the amount of time required for identification, taking into consideration the available technology at the time of the processing and technological developments." Anonymous information — information that does not relate to an identified or identifiable person, or personal data rendered anonymous so that the data subject is no longer identifiable — falls outside the GDPR.
`[Law]` Recital 30 explicitly includes online identifiers: "natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags." It warns that this "may leave traces which, combined with unique identifiers and other information received by the servers, may be used to create profiles of the natural persons and identify them."
`[CJEU]` *Breyer* (C-582/14) is the leading case. A dynamic IP address can be personal data for a website operator where the operator has the legal means to obtain additional identifying information from the ISP; it is not necessary that all the identifying information be in one person's hands. `[CJEU]` *Breyer*, paras 42–46. The "means likely reasonably to be used" test excludes only identification requiring a "disproportionate effort in terms of time, cost and manpower." `[CJEU]` *Breyer*, para. 46. The possibility of combining data must not be "purely hypothetical." `[CJEU]` *Breyer*, para. 49.
`[CJEU]` The Court has repeatedly read the concept broadly. In *Nowak*, it held that "personal data" is not limited to information about private or sensitive matters; exam scripts and examiner comments can be personal data. `[CJEU]` C-434/16, para. 35. In *Vyriausioji tarnybinės etikos komisija*, it confirmed that indirect identification can result from combining data. `[CJEU]` C-184/20. In *Gesamtverband Autoteile-Handel*, it held that a vehicle identification number can be personal data where it can be linked to an identifiable person; the link can arise from the content, purpose or effect of the information. `[CJEU]` C-319/22, paras 45–49. In *IAB Europe*, it held that a TC String — a sequence of characters encoding a user's consent preferences — can be personal data where it can be linked to an identifiable user, even though it does not contain a name. `[CJEU]` C-604/22. In *EDPS v SRB*, it clarified that pseudonymised data are not personal data "in all cases and for every person": the assessment is relative to the recipient's reasonably available means of re-identification. `[CJEU]` C-413/23 P, paras 55–56, 82–85. But the same judgment confirms that pseudonymised data remain personal for the entity that holds the key. `[CJEU]` C-413/23 P, para. 76.
`[Guidance]` The Article 29 Working Party's Opinion 4/2007 remains the canonical explanation: "personal data" covers "any information" relating to a person by reason of its content, purpose or result; identification can be indirect; the assessment is contextual and risk-based. `[Guidance]` WP29 Opinion 4/2007, pp. 9–12.
### 2.2 MAC addresses in the Heilbronn pipeline
A MAC address is a 48-bit identifier assigned to a network interface. For a universally administered address, the first 24 bits are an Organizationally Unique Identifier (OUI) that identifies the vendor; the last 24 bits are assigned by the vendor. Randomised addresses set the locally administered bit and do not carry a real vendor OUI. It is a device identifier, not a person identifier. But in a footfall deployment it is processed together with:
- the receiving sensor's identity and location;
- a timestamp (the Heilbronn data are on a ~15-minute cadence);
- the fact that the same device was seen before (deduplication);
- potentially, the signal strength and channel;
- potentially, the device type inferred from the OUI.
That combination is what makes the data identifying. A single MAC address plus "sensor 7, 14:15" can single out a device and its bearer in a small area; a sequence of such records across sensors can reconstruct a path. The controller may not know the person's name, but it can treat the device bearer differently (count them once, exclude them, measure repeat visits), which is the functional meaning of "identifiable" in the EDPB's 2026 draft. `[Guidance]` EDPB Guidelines 02/2026, paras 19–25.
`[Technical]` The Heilbronn deployment, as described by Code for Heilbronn, uses a Heltec Wireless Paper board with an ESP32 (BLE/Wi-Fi) and an SX1262 LoRa transceiver. The sensor "receives these packets and remembers the device addresses. These MAC addresses are hashed and stored for at most one hour" (author's translation). Only the number of counted addresses is transmitted; hashed addresses do not leave the sensor. `[Technical]` Code for Heilbronn, "Passantenzahlen messen mit LoRa" (18 Sep 2025). The upstream reference implementation (`ESP32-Paxcounter`/`libpax`) uses a passive BLE scan (no scan requests) and, in `mac_add()`, a 16-bit ID built from the last two bytes of the address, stored in a 65,536-bit bitmap that is cleared every 60 seconds. It counts only locally administered (randomised) addresses and ignores universally administered (burned-in) addresses. `[Technical]` `libpax/lib/libpax/libpax.cpp`; `ESP32-Paxcounter` v3.8.0; see also the project's code audit, `09-paxcounter-code-audit.md`. The upstream documentation says the opposite (that personal MAC addresses are not evaluated), so the actual deployed firmware must be verified; the code audit flags this discrepancy. The upstream project's own legal note warns that MAC sniffing may be illegal depending on jurisdiction and links to Dutch/EU guidance, a decision, and a German Bundestag analysis. `[Technical]` Paxcounter legal note.
This design is materially better than storing raw MACs. It is not, by itself, a legal anonymisation guarantee:
- the raw MAC is observable at the sensor before hashing;
- the hash length and algorithm of the deployed firmware are undocumented; the upstream 16-bit ID has a small space and frequent collisions;
- the one-hour cache creates a one-hour linkability window per sensor;
- if the hash is unsalted, the 48-bit MAC space is small enough to enumerate; if it is keyed, the key holder can re-identify;
- the published 15-minute per-sensor counts can reveal a single device in low-traffic cells.
`[Guidance]` The EDPB's 2026 draft says a hardware identifier such as a MAC address "can identify the device and, in turn, the individual using the device." `[Guidance]` EDPB Guidelines 02/2026, para. 25. The Dutch DPA's position is even more direct: MAC addresses combined with sensor location and time are "indirectly identifying personal data" because they can be combined with camera images, payment data, hotspot logins, or RFID records. `[DPA]` Autoriteit Persoonsgegevens, "Wifi and bluetooth".
**Conclusion on classification.** A MAC address is not automatically personal data in every conceivable context, but in a public-space footfall system with location, time and deduplication, it is very likely personal data (or at least pseudonymous personal data). The burden is on the controller to demonstrate that the data are anonymous for every relevant recipient. A claim that "we hash, therefore it is anonymous" is not sufficient. See §5.
### 2.3 Pseudonymous versus anonymous
`[Law]` GDPR Art. 4(5) defines pseudonymisation as processing personal data "in such a manner that the personal data can no longer be attributed to a specific data subject without the use of additional information," provided the additional information is kept separately and protected. Pseudonymised data remain personal data for the controller that holds the additional information, and for any recipient that can reasonably re-identify the person; they may be anonymous for a recipient who cannot. `[CJEU]` C-413/23 P, paras 82–85. `[Guidance]` EDPB Guidelines 01/2025 (consultation version), para. 22: pseudonymised data that can be attributed to a person by additional information are personal data, "even if pseudonymised data and additional information are not in the hands of the same person"; even if the controller erases the additional information, the data become anonymous only if the conditions for anonymity are met.
`[Guidance]` The EDPB's 2026 draft sets out the current analytical framework: anonymity is assessed from the perspective of each relevant entity; the test is whether the person can be distinguished from others using means reasonably likely to be used, in a way that makes it possible to treat them differently. `[Guidance]` EDPB Guidelines 02/2026, executive summary and paras 19–25. The draft's three criteria — **No Record Isolation, No Linkage, No Inference** — are discussed in §5.1.
`[Enforcement]` The Conseil d'État's JCDecaux decision is the clearest judicial statement that technical pseudonymisation is not enough. JCDecaux proposed to truncate the last half-octet of the MAC, add a salt, and apply keyed hashing. The court held that these processes "seek to prevent third parties from accessing the data" but "leave the processing manager able to identify the persons concerned and do not prevent correlating records relating to the same individual or inferring information about him" (author's translation). `[Enforcement]` CE, 8 Feb 2017, no. 393714, para. 8. The court also held that the purpose — measuring the repetition of passages and the routes taken — was incompatible with anonymisation. `[Enforcement]` Ibid. The Dutch DPA makes the same point in its guidance: hashing a MAC address without additional measures is pseudonymisation, not anonymisation; the hashing formula is in the controller's possession; adding a salt does not change that; storing multiple location points per device means the data are not anonymous. `[DPA]` Autoriteit Persoonsgegevens, "Wifi and bluetooth", Quick answers.
`[Technical]` The practical implication for Heilbronn is a staged analysis:
| Stage | What is processed | Legal character | Consequence |
|---|---|---|---|
| Passive radio reception | 802.11/BLE frames; MAC in the frame header | MAC is a device identifier; likely personal data when combined with location/time | Reception is processing; a legal basis is needed if personal data |
| Raw MAC in sensor RAM | Full MAC for milliseconds | Personal data; transient | Must be minimised; no logging, no transmission, no packet capture |
| Dedup identifier | Hash or 16-bit ID; up to 1 hour | Pseudonymous personal data while the key/raw data exist | Legal basis; retention limit; no cross-sensor/cross-day linkage |
| Count per interval | Integer | Aggregate; may be anonymous if no small-cell inference | Still assess; ePrivacy may apply to the collection; DPIA |
| Published CSV/Grafana | Counts per sensor/15 min | Potentially personal data in small cells; not publication-safe | Coarsen, suppress, add noise before release |
The table is the single most important analytical move in this note: it prevents the common error of arguing from the anonymity of the output to the lawfulness of the input.
**Terminology: "frequency measurement" and "Bluetooth tracking."** German practice uses "Passantenfrequenzmessung" or "Besucherstrommessung" for footfall measurement. That is a purpose, not a technology. The legal analysis depends on the sensor: a light barrier, a thermal/radar counter, or an on-device video counter that does not identify devices may avoid the GDPR/ePrivacy issues altogether; the CNIL expressly distinguishes simple footfall counters (a mat that counts steps, a turnstile) from devices that collect personal data. `[Guidance]` CNIL, 24 Feb 2020. "Bluetooth tracking" is likewise a family of techniques: passive BLE scanning of advertising addresses (the Heilbronn design), active BLE scanning, beacon-based proximity, or app-based beacon scanning. The common legal issue is whether a device identifier and location/time are processed; the specific radio technology does not change the analysis. `[Guidance]` EDPB Guidelines 2/2023, paras 57–60 (IoT/BLE examples); `[DPA]` AP, "Wifi and bluetooth".
### 2.4 MAC randomisation and BLE
`[Technical]` Modern devices randomise their Wi-Fi MAC addresses. Apple uses a private Wi-Fi address per SSID on iOS 14/iPadOS 14/macOS 14 and later; Android 10+ randomises by default for new networks; BLE uses resolvable private addresses that rotate, typically every 15 minutes. `[Technical]` Apple Platform Security, "Wi-Fi privacy"; Android Open Source Project, "MAC randomization"; Martin et al., PETS 2017; Becker et al., PETS 2019.
Randomisation reduces linkability but does not eliminate it:
- some devices still use the hardware MAC in certain frames or configurations;
- probe-request randomisation policies vary by OS, device model, and power state;
- BLE advertisers may include identifying tokens in the payload or use static addresses; resolvable private addresses can be resolved by bonded devices, and address changes may be asynchronous with payload changes; `[Technical]` Becker et al., PETS 2019;
- fingerprinting beyond the MAC (sequence numbers, information elements, RSSI patterns) can re-identify a device across randomisation cycles; `[Technical]` Vanhoef et al., AsiaCCS 2016; Martin et al., PETS 2017;
- randomisation introduces measurement bias: the counted population is "devices currently using a locally administered address that the scanner hears," not all devices or people. `[Technical]` Code audit, §2.1.
**Legal implication.** MAC randomisation is a privacy feature outside the controller's control. It is not an anonymisation technique the controller can rely on. It may reduce the risk and the accuracy of the count, but the controller must still assess the data it actually processes, including the raw MAC before randomisation is observed and the hashed ID during the dedup window. `[Guidance]` EDPB Guidelines 02/2026, para. 25; `[DPA]` AP, "Wifi and bluetooth".
---
## 3. ePrivacy Directive and § 25 TDDDG
### 3.1 The EU rule
`[Law]` ePrivacy Directive 2002/58/EC, Art. 5(3), as amended by Directive 2009/136/EC, provides:
> "Member States shall ensure that the storing of information, or the gaining of access to information already stored, in the terminal equipment of a subscriber or user is only allowed on condition that the subscriber or user concerned has given his or her consent, having been provided with clear and comprehensive information, in accordance with Directive 95/46/EC, inter alia, about the purposes of the processing. This shall not prevent any technical storage or access for the sole purpose of carrying out the transmission of a communication over an electronic communications network, or as strictly necessary in order for the provider of an information society service explicitly requested by the subscriber or user to provide the service."
`[Law]` Recital 24 explains that the terminal equipment of users and the information stored in it are part of the private sphere; hidden identifiers and other similar devices can enter terminal equipment without the user's knowledge and should be prohibited. Recital 25 refers to "cookies or similar devices." Recital 66 (added by Directive 2009/136/EC) says the user's consent should be required for storing information in terminal equipment or accessing information already stored. The proposed ePrivacy Regulation has not been adopted; the Directive remains the applicable EU instrument.
`[CJEU]` *Planet49* confirms that Art. 5(3) applies regardless of whether the information is personal data, and that the consent standard is the GDPR standard: active behaviour, specific information, no pre-ticked boxes. `[CJEU]` C-673/17, paras 52–65. The EDPB reads the provision as technology-neutral, covering any information stored or accessed. `[Guidance]` EDPB Guidelines 2/2023, paras 9–12.
`[Guidance]` The EDPB's Guidelines 2/2023 are the most detailed technical interpretation. Key points:
- "Information" includes non-personal data and personal data, regardless of who stored it. `[Guidance]` EDPB Guidelines 2/2023, paras 9–12.
- The notion of "stored information" includes information stored by the user, by a hardware manufacturer (expressly "such as the MAC addresses of network interface controllers"), or by any other party. `[Guidance]` Ibid., para. 39.
- "Gaining access" includes reading a value that the user did not store; the WP29 Opinion 9/2014 example is "requesting the MAC address of a network interface via the OS API." `[Guidance]` Ibid., para. 11, citing WP29 Opinion 9/2014, p. 8.
- Routing identifiers such as the MAC or IP address of the terminal equipment, communicated to remote actors, can trigger Art. 5(3); the fact that the receiving entity is not the entity instructing the sending does not preclude application. `[Guidance]` Ibid., para. 42.
- Local processing entirely inside the terminal equipment, with no information leaving the device, is outside Art. 5(3). `[Guidance]` Ibid., para. 44.
- IP-only tracking can trigger Art. 5(3) where the IP address originates from the terminal equipment; the controller must assess. `[Guidance]` Ibid., paras 54–56.
### 3.2 The German rule
`[Law]` § 25 TDDDG (formerly § 25 TTDSG, in force since 1 December 2021; renamed on 14 May 2024) provides:
> "(1) The storage of information in the terminal equipment of the end user or the access to information already stored in the terminal equipment shall only be permitted if the end user has consented on the basis of clear and comprehensive information. The information and consent shall be provided in accordance with Regulation (EU) 2016/679.
> (2) Consent under paragraph 1 is not required if (1) the sole purpose of the storage or access is the transmission of a communication over a public telecommunications network, or (2) the storage or access is strictly necessary for the provider of a digital service to provide a digital service explicitly requested by the user."
`[Law]` § 2(2) No. 6 TDDDG defines "terminal equipment" broadly: any equipment directly or indirectly connected to the interface of a public telecommunications network for sending, processing or receiving messages; the connection may be wire, optical fibre or electromagnetic. A smartphone with Wi-Fi or BLE enabled is terminal equipment; it is connectable to a public network (directly or through a router).
`[Law]` § 25 applies regardless of whether the information is personal data. `[Guidance]` DSK, OH Digitale Dienste (Nov 2024), para. 26 ("Im Unterschied zu den datenschutzrechtlichen Vorschriften begründet § 25 Abs. 1 TDDDG ein Einwilligungserfordernis ... unabhängig von einem Personenbezug"). The DSK guidance also says that, for mobile devices, "access to hardware device identifiers, advertising identifiers, phone numbers, SIM serial numbers (IMSI), contacts, call lists, Bluetooth beacons or SMS communication" must be considered, and "for all devices, the reading of the unique identifiers of the network hardware (MAC addresses) must be considered." `[Guidance]` DSK, OH Digitale Dienste, para. 22.
`[Law]` Enforcement: § 28(1) No. 13 TDDDG makes it an administrative offence to store or access information contrary to § 25(1); § 28(2) sets a fine of up to €300,000 for private offenders. `[Law]` § 28(4) provides that no fines are imposed on public authorities and other public bodies within the meaning of § 2(1) or (2) BDSG. `[Law]` § 29(2) allocates § 25 supervision over telecom providers and federal public bodies to the BfDI; for a municipal body in Baden-Württemberg, the competent authority is the LfDI Baden-Württemberg.
### 3.3 Does passive sniffing of broadcast MACs trigger consent?
This is the decisive question for Heilbronn. The elements are:
| Element | ePrivacy Art. 5(3) | § 25 TDDDG | Application to passive MAC sniffing |
|---|---|---|---|
| Information | Any information, not only personal data | Any information; expressly no personal data required | A MAC address is information; the EDPB treats NIC MAC addresses as stored information (`[Guidance]` EDPB Guidelines 2/2023, para. 39) |
| Terminal equipment | Equipment of a subscriber/user connected to a public network | § 2(2) No. 6 TDDDG; broad | A smartphone with Wi-Fi/BLE enabled is terminal equipment |
| Storage | Storing in the terminal equipment | Same | The sensor stores nothing in the phone: no |
| Access | Gaining access to information already stored | "Zugriff" on stored information | The sensor receives a broadcast frame; it does not instruct the phone, retrieve from its memory, or use an OS API. Arguably no "access" |
| Consent | Prior, informed, active; narrow exceptions | Same; § 25(2) exceptions | If access: consent required; legitimate interest is not a substitute; signage/opt-out is not consent |
| Personal data required? | No | No (DSK OH para. 26) | Even an anonymous output does not escape § 25 if the collection is an access |
**The case for "no access."** The ordinary meaning of "access" and the German legislative materials point to an active retrieval. The draft TTDSG explanatory memorandum describes the provision as protecting end users from third parties who "store or read out" information on their terminal equipment; it uses "Abruf" (retrieval) and "Auslesen" (readout). `[Law]` BT-Drs. 19/27441, p. 38. The DSK guidance's examples are active: access to hardware identifiers, JavaScript fingerprinting, Bluetooth beacons. `[Guidance]` DSK, OH Digitale Dienste, paras 22–24. The EDPB's own examples of "gaining access" are an OS API request and software-instructed transmission; even where the EDPB says the receiver need not be the entity instructing the transmission (para. 42), the device is still transmitting to a remote actor. In passive sniffing, the sensor is an unintended receiver of a broadcast, not an addressee of a communication. `[Guidance]` EDPB Guidelines 2/2023, paras 11, 42, 44. If every radio receiver "accessed" every device that broadcasts a MAC, ordinary Wi-Fi operation would require consent from every passer-by; that is not a sensible reading of Art. 5(3).
**The case for "access."** The EDPB's functional approach is broad: it says information includes MAC addresses of network interface controllers; routing identifiers such as the MAC address communicated to remote actors can trigger Art. 5(3); and the fact that the receiving entity is not the one instructing the sending does not preclude application. `[Guidance]` EDPB Guidelines 2/2023, paras 39, 42. A passive sensor is a receiving entity; the device sends the MAC; the sensor obtains the information. The purpose of Art. 5(3) is to protect the private sphere of terminal equipment against hidden tracking; passive MAC tracking is exactly the kind of invisible tracking the rule targets. The DSK guidance does not limit "Auslesen" to an OS API; it says "bei allen Geräten ist ... das Auslesen der eindeutigen Kennungen der Netzwerk-Hardware (MAC-Adressen) zu berücksichtigen." `[Guidance]` DSK, OH Digitale Dienste, para. 22. The Dutch DPA's guidance asks whether the sensor "gains access to information in a user's peripheral equipment because of the way the WiFi or Bluetooth sensors read data from them" and says such access requires explicit consent. `[DPA]` AP, "Legal bases for WiFi tracking and Bluetooth tracking". The AP's guidance is nuanced: it says access to information in peripheral equipment requires explicit consent, while also describing a legitimate-interest route for Wi-Fi/Bluetooth tracking. The tension reflects the interaction between the Dutch Telecommunications Act's Art. 11.7a (the national Art. 5(3) implementation) and the GDPR. It is another reason to obtain local advice rather than transplanting the Dutch model.
**Assessment.** There is no published EU or German decision on passive Wi-Fi/BLE sniffing under Art. 5(3)/§ 25. The better textual and purposive reading — in this author's assessment, not a settled ruling — is that passive reception is not "access" because the sensor does not interact with the terminal equipment's storage; the device broadcasts the frame for network discovery, not in response to the sensor. But the EDPB's broad language and the DSK's MAC example create a material risk that a supervisory authority or court would take the opposite view. The risk is higher if the sensor:
- sends probe requests, association requests, or any active query;
- uses an OS API or an app on the device to read the MAC;
- captures payload or other stored information;
- is presented as a "Wi-Fi hotspot" or otherwise induces the device to connect;
- combines the MAC with other data to build profiles.
The Heilbronn design, as described, is at the favourable end: passive BLE scan, no scan requests, no association, hashed addresses kept on the sensor, only counts transmitted. `[Technical]` Code for Heilbronn blog; code audit §1.1. That makes the "no access" argument stronger, but it does not eliminate the uncertainty. A written § 25 memo should:
1. describe the exact frames received and confirm that no active request is sent;
2. confirm that no data are written to the device and no OS API is used;
3. explain why receiving a broadcast frame is not "access to information already stored" (no retrieval from storage; the device initiates the transmission; the frame is not the stored information);
4. address the EDPB's paras 39 and 42 and the DSK's para. 22, and explain why they do not cover passive reception;
5. state the residual risk and the fallback if the authority disagrees.
If the memo cannot be made robust, the project should assume § 25 applies. In that case, consent is required. Consent at city scale is not realistically obtainable: the sensor collects from all passers-by, including those who have not installed an app, have not seen a sign, or have not taken any active step. `[CJEU]` *Planet49*, paras 52–65; `[Law]` GDPR Recital 32. A sign, an opt-out, or leaving Wi-Fi on is not consent. The only remaining routes are a statutory legal basis (for a public body) or a redesign that does not extract MAC addresses.
### 3.4 Notification and transparency
**Short answer.** If Art. 5(3)/§ 25 applies, prior notification is mandatory, but notification is not consent. If Art. 5(3)/§ 25 does not apply, notification is still required under GDPR Art. 14 (and Art. 21(4) for the right to object) whenever personal data are processed; if the data are truly anonymous, there is no GDPR notification duty, but transparency remains good practice and may be required by ePrivacy if consent is needed. The rest of this section explains the detail.
If Art. 5(3)/§ 25 applies, the controller must provide "clear and comprehensive information" before consent. A sign that merely says "Wi-Fi tracking" is not enough; it must identify the controller, the purposes, the data, the retention, the rights, and how to consent or refuse. But information is not consent; the consent must be active and prior.
If Art. 5(3)/§ 25 does not apply, the GDPR transparency regime still applies to personal data:
- Art. 13 if the data are obtained from the data subject; Art. 14 if not. Passive sensing is arguably "not obtained from the data subject" in the ordinary sense, but the Conseil d'État characterized MAC collection as "direct collection" because the data are collected directly from the devices, even though the persons take no active step. `[Enforcement]` CE, 8 Feb 2017, no. 393714, para. 5. Under the GDPR, the safer approach is to provide Art. 13-style information at the point of collection, via on-site signage and a QR link to the full notice.
- Art. 14(5)(b) allows an exception where providing information would involve a disproportionate effort. The exception is interpreted narrowly; for a fixed sensor network in public space, signage is feasible, so the exception is unlikely to apply. `[Law]` GDPR Art. 14(5)(b).
- Art. 21(4) requires the right to object to be brought to the data subject's attention explicitly and separately, at the latest at the time of the first communication, where processing is based on Art. 6(1)(e) or (f). Signage should state the right to object and how to exercise it.
If the output is truly anonymous, the GDPR information duties do not apply to the output. But the collection stage still processes personal data (unless the sensor never extracts a MAC), so information is still needed for the collection. And if § 25 applies, the ePrivacy information duty applies regardless of anonymity.
---
## 4. GDPR obligations
### 4.1 Who is the controller?
`[Law]` Art. 4(7) defines the controller as the natural or legal person, public authority, agency or other body which alone or jointly with others determines the purposes and means of the processing. Art. 4(8) defines the processor as a person processing personal data on behalf of the controller. Art. 26 applies to joint controllers; Art. 28 to processors.
`[CJEU]` Joint controllership is broad. In *Wirtschaftsakademie*, the operator of a Facebook fan page was a joint controller with Facebook for the collection and processing of visitor data, even though it did not have access to the data, because it influenced the purposes and means. `[CJEU]` C-210/16, paras 28–39. In *Fashion ID*, a website operator that embedded a Facebook "Like" button was a joint controller for the collection and transmission of the data, but not for Facebook's subsequent processing. `[CJEU]` C-40/17, paras 65–71. The EDPB's Guidelines 07/2020 emphasize that joint controllership requires a common decision on purposes and means, or a convergence of decisions; it does not require equal responsibility or access. `[Guidance]` EDPB Guidelines 07/2020.
For Heilbronn, the roles are not determined by labels such as "citizen project," "open data," or "city partner." They depend on who decides:
- why the data are collected (traffic planning, cleaning, urban development, research, commercial analytics);
- which sensors are deployed, where, for how long, and at what granularity;
- what is retained, published, and shared;
- who can access the raw or pseudonymous data;
- who benefits from the processing.
If the city determines the purposes and the Verein implements them under instruction, the city is controller and the Verein may be processor. If the Verein decides the purposes and means and the city merely receives the output, the Verein is controller (and needs its own legal basis). If they jointly decide, they are joint controllers and need an Art. 26 arrangement. If the Verein uses the data for its own purposes (e.g., open-data publication, research, community dashboards) in addition to the city's purposes, it is a controller for those purposes, even if it is a processor for the city's purposes. `[Law]` Art. 28(10).
`[Guidance]` The EDPB's Guidelines 07/2020 and the CJEU case law make clear that a written contract cannot disguise the reality: if the "processor" determines purposes or means, it is a controller. `[Guidance]` EDPB Guidelines 07/2020, paras 20–30; `[Law]` Art. 28(10).
### 4.2 Lawful basis
**Public body.** `[Law]` Art. 6(1)(f) does not apply to processing carried out by public authorities in the performance of their tasks. A municipality must rely on Art. 6(1)(e) (public task) read with Art. 6(3), which requires a legal basis in Union or Member State law, or on consent under Art. 6(1)(a). The legal basis must be clear, precise and foreseeable; a general competence to promote urban development or tourism may not be sufficient for continuous, covert MAC tracking. The city should identify the specific statutory task and document why the processing is necessary and proportionate. `[Law]` GDPR Art. 6(1)(e), 6(3); Recital 45. In Baden-Württemberg, the municipal framework is the Gemeindeordnung and the Landesdatenschutzgesetz (LDSG BW); the specific basis for footfall measurement should be confirmed by counsel.
**Private association or company.** `[Law]` Art. 6(1)(f) permits processing that is necessary for the purposes of a legitimate interest pursued by the controller or a third party, unless the data subject's interests or fundamental rights and freedoms override. The test is threefold: (1) a legitimate interest; (2) necessity and proportionality; (3) balancing. `[CJEU]` *M5A*, paras 37–44; `[CJEU]` *KNLTB*, paras 40–49; `[CJEU]` *Meta*, paras 42–47. A commercial interest can be legitimate, but it is not automatically sufficient; the processing must be necessary and the balancing must favour it. `[CJEU]` *KNLTB*, para. 49.
For public-space footfall tracking, the balancing is difficult:
- people do not expect their phone's radio emissions to be collected by a third party;
- they cannot practically avoid the measurement area;
- the processing is invisible;
- the data can reveal movement and presence;
- less intrusive alternatives exist (manual counts, light barriers, thermal/radar sensors, on-device video counting);
- the German and Dutch DPAs have repeatedly found the data subject's interests prevail. `[DPA]` ULD TB 2019, pp. 97–99; `[DPA]` AP, "Legal bases for WiFi tracking and Bluetooth tracking".
Safeguards — immediate anonymisation, no raw retention, per-window hashing, k-threshold, DP noise, spatial/temporal limits, opt-out — can improve the balancing, but they cannot cure a lack of necessity if a non-identifying sensor can meet the purpose. `[CJEU]` *M5A* (necessity/proportionality); `[Guidance]` EDPB Guidelines 01/2025, para. 23 (pseudonymisation can reduce risk and support legitimate interests, but does not create a legal basis).
**Consent.** `[Law]` Consent must be freely given, specific, informed and unambiguous, by a statement or clear affirmative action. `[Law]` Art. 4(11); Art. 7; Recital 32. Silence, pre-ticked boxes, inactivity, and "continuing to browse" are not consent. `[CJEU]` *Planet49*, paras 52–65. In a public-space passive sensing system, consent is practically unworkable: the controller cannot obtain prior consent from every passer-by, and an opt-in app covers only a self-selected subset. A sign is information, not consent. An opt-out is not consent. Leaving Wi-Fi or Bluetooth on is not consent. The Hamburg DPA warns that an enabled interface cannot be treated as consent to tracking. `[DPA]` HmbBfDI TB 2014/2015, p. 142.
**Special categories.** `[Law]` Art. 9 prohibits processing special categories of personal data unless an exception applies. Footfall counts are not inherently sensitive, but location data can reveal sensitive information when a sensor is placed near a hospital, clinic, place of worship, shelter, school, or political event. The DPIA should assess whether the sensor locations or the combination with other data could create Art. 9 data; if so, the processing needs an Art. 9(2) basis or must be redesigned. `[Law]` Art. 9; Recital 51.
### 4.3 Transparency and signage
`[Law]` Art. 12 requires information to be concise, transparent, intelligible and easily accessible, in clear and plain language. Art. 13/14 specify the content: identity and contact details of the controller and DPO; purposes; legal basis; categories of data; recipients; retention; rights; right to object; right to complain to a supervisory authority; whether provision is statutory/contractual; automated decision-making (not applicable).
For a public-space deployment, the practical standard is:
- signs at every entrance to the measurement zone, before the zone, not inside it;
- a short plain-language statement of purpose and data;
- a QR code to the full privacy notice;
- German and English (at least);
- accessible formats (large print, plain language);
- the identity of the controller and the DPO;
- the legal basis (public task or legitimate interest);
- the retention periods;
- the right to object and how to exercise it;
- the opt-out mechanism;
- the complaint authority (LfDI Baden-Württemberg);
- a statement of what is *not* collected (e.g., no names, no payload, no raw MAC retention).
`[Guidance]` The ICO's Wi-Fi location analytics guidance (2016, UK; no longer available at the original ICO URL) recommended a privacy impact assessment, notices at the point of collection and on the website, a defined retention period, and a simple opt-in/opt-out. The current ICO anonymisation and DPIA guidance applies the same principles. `[Guidance]` ICO, "Wi-Fi location analytics" (2016), as summarised in Bird & Bird (2016); ICO, "Anonymisation"; ICO, "Data protection impact assessments". The ICO is not binding in Germany, but its checklist is consistent with GDPR Art. 12–14 and Art. 35.
`[Guidance]` The CNIL recommends a two-level information model: a short first-level notice (purpose, controller, rights, opt-out) and a fuller second-level notice (all Art. 13/14 information). `[Guidance]` CNIL, 24 Feb 2020. This is a useful template for the Heilbronn signage.
**Do not overclaim anonymity.** A sign that says "fully anonymous" while the sensor processes raw MACs and keeps hashed IDs for an hour is misleading. The notice should distinguish the stages: "The sensor reads device addresses in memory, converts them into a temporary pseudonymous identifier, keeps that identifier for at most X minutes for deduplication, and transmits only counts. Raw addresses are not stored or transmitted." If the claim cannot be substantiated, do not make it.
### 4.4 Data minimisation and storage limitation
`[Law]` Art. 5(1)(c) requires personal data to be adequate, relevant and limited to what is necessary; Art. 5(1)(e) requires retention no longer than necessary. The controller must be able to justify each data element and each retention period.
For the Heilbronn pipeline:
| Data | Recommended handling | Legal rationale |
|---|---|---|
| Raw MAC | Memory only; deleted immediately after dedup (milliseconds); never logged, never transmitted, no packet capture | Minimisation; reduces breach impact; supports the argument that the sensor does not "store" information |
| Hashed/pseudonymous ID | Per-sensor, per-window keyed hash; retained only for the dedup window; key destroyed after the window; no cross-sensor or cross-day linkage | Pseudonymous data; retention limitation; prevents movement profiles |
| Dedup window | No longer than necessary for the count; if the output is 15-minute counts, a 15-minute window is the natural default; a 1-hour window needs a documented reason | The current 1-hour cache is longer than the 15-minute reporting interval; it increases linkability |
| Counts | Aggregate at the edge; transmit only counts; no per-device records | Data minimisation; the output may be anonymous |
| Published data | Zone × hour (or coarser); k-threshold; suppression; DP noise; no per-sensor minute data; no repeat-visit metrics | Anonymisation/PETs; see §5 |
| Logs | No MACs; only health/telemetry (battery, uptime, error counts) | Security without creating a new personal-data store |
`[Guidance]` The CNIL's 2020 guidance provides a useful benchmark: if the data are anonymised within minutes, the CNIL recommends a maximum of five minutes before no identifying data are retained; if the data are pseudonymised at collection and anonymised or destroyed by the end of the day (in practice, within 24 hours), the CNIL accepts that regime; in all other cases, consent is required. `[Guidance]` CNIL, 24 Feb 2020. The CNIL also recommends not collecting precise timestamps for repeat-visit measurement, limiting the geographic area, and deleting pseudonymised data at the end of the measurement. `[Guidance]` Ibid. These are French recommendations, not German law, but they are a sensible benchmark for a DPIA.
`[Guidance]` The Hamburg DPA's 2014/2015 report says WLAN/Bluetooth tracking is highly intrusive and requires a legal basis, transparency before entering the area, and consent where applicable; it warns that an enabled interface is not consent. `[DPA]` HmbBfDI TB 2014/2015, pp. 141–143. It does not set a maximum retention period, but the logic supports short retention and no profiles.
`[Guidance]` The ULD's 2019 report is more specific: tracking based on passively captured probe requests for visitor counting in public spaces has no legal basis; a legitimate-interest balancing leads to the data subjects' interests prevailing because the data are collected without their knowledge and they have practically no way to prevent it. `[DPA]` ULD TB 2019, pp. 97–99. The ULD notes that simply disabling probe requests is difficult or impossible on some operating systems; often only airplane mode stops them. `[DPA]` Ibid., p. 98. This is directly relevant to the "opt-out by turning off Wi-Fi" argument: it is not a realistic opt-out.
### 4.5 Data protection impact assessment
`[Law]` Art. 35(1) requires a DPIA where a type of processing is likely to result in a high risk to the rights and freedoms of natural persons. Art. 35(3) lists three cases in particular: (a) a systematic and extensive evaluation of personal aspects relating to natural persons, based on automated processing including profiling, on which decisions with legal or similarly significant effects are based; (b) large-scale processing of special categories of data or criminal-offence data; and (c) systematic monitoring of a publicly accessible area on a large scale. The Heilbronn deployment falls under (c). `[Law]` GDPR Art. 35(3)(c).
Assuming the deployment processes personal data — which it does if it extracts MAC addresses for deduplication — it meets the criterion:
- **systematic:** continuous, planned, automated sensing over months;
- **publicly accessible area:** inner-city streets and squares;
- **large scale:** a large number of passers-by over months, multiple sensors, and a substantial geographic extent; the exact number of data subjects is not in the delivered data, but Recital 91 lists the number of data subjects, the volume of data, the duration, and the geographic extent as relevant factors. `[Law]` Recital 91.
`[Guidance]` The Article 29 Working Party's DPIA guidelines (WP248 rev.01) list nine criteria; two or more indicate a high risk. The relevant ones here are: systematic monitoring; large-scale processing; innovative technology; matching or combining datasets; data concerning vulnerable data subjects; data not obtained from the data subject; invisible processing; and preventing data subjects from exercising a right or using a service. `[Guidance]` WP29, WP248 rev.01, pp. 10–12. A public-space MAC tracker hits at least five.
`[Guidance]` The LfDI Baden-Württemberg's private-sector "Muss-Liste" (2018) explicitly includes:
- No. 2: "Extensive processing of data about the presence of persons" — example: "Offline tracking of customer movements in department stores, shopping centres or similar" — "a company processes GPS and WLAN data of passers-by and customers in order to track routes and purchasing behaviour." `[Guidance]` LfDI BW, Muss-Liste, No. 2.
- No. 10: "Non-intended use of sensors of a mobile device in the possession of the data subject or of radio signals sent by such devices to determine the location or movement of persons over a substantial period" — same example. `[Guidance]` LfDI BW, Muss-Liste, No. 10.
This is directly on point: a private operator in Baden-Württemberg must conduct a DPIA for this processing. For a public body, there is no positive list for the public sector in Baden-Württemberg; the Ministry of Justice's 2024 threshold analysis states: "In Baden-Württemberg there is no positive list for the public sector." `[Guidance]` LfDI BW/MJM BW, Schwellenwertanalyse (29 Nov 2024). But Art. 35(3)(c) applies directly, and the risk assessment under Art. 35(1) may still require a DPIA. A public body should not treat the absence of a list as a negative list.
`[Law]` The DPIA must contain at least: a systematic description of the processing and purposes; an assessment of necessity and proportionality; an assessment of the risks; and the measures envisaged to address the risks. `[Law]` Art. 35(7). The controller must seek the DPO's advice (Art. 35(2)), and may seek the views of data subjects or their representatives (Art. 35(9)). If the residual risk remains high, the controller must consult the supervisory authority before processing (Art. 36). The DPIA must be reviewed when the processing changes (Art. 35(11)).
For Heilbronn, the DPIA should specifically cover:
- the legal basis and the § 25 TDDDG analysis;
- the exact data flow, including the raw MAC, hash length, key management, cache duration, and aggregation rule;
- the re-identification risk, including small-cell counts, differencing, auxiliary data (camera, payment, hotspot, loyalty), and MAC randomisation;
- the necessity and alternatives (manual counts, light barriers, thermal/radar, on-device video counting);
- the transparency and opt-out mechanisms, and their effectiveness;
- the roles of the city, the Verein, the LoRaWAN network operator, and any hosting provider;
- the retention schedule and deletion verification;
- the security measures and breach response;
- the residual risk and whether Art. 36 prior consultation is needed;
- a published non-technical summary, if possible.
`[Guidance]` The BfDI's DPIA page and the ICO's DPIA guidance provide checklists; the BfDI is not the competent authority for Heilbronn, but its methodology is the federal reference. `[Guidance]` BfDI, "Datenschutz-Folgenabschätzungen und Listen von Verarbeitungsvorgängen"; ICO, "Data protection impact assessments".
### 4.6 Security and breach
`[Law]` Art. 5(1)(f) requires integrity and confidentiality; Art. 32 requires technical and organisational measures appropriate to the risk, including pseudonymisation and encryption, resilience, availability, and regular testing. For an edge sensor network, the relevant measures include:
- secure boot, signed firmware, and secure updates;
- no default passwords; unique credentials; network segmentation;
- encryption at rest and in transit (LoRaWAN payload encryption is not a substitute for end-to-end application-layer protection if the data are sensitive);
- physical tamper protection;
- no raw MAC storage; no packet capture; no debug logs containing identifiers;
- access control and audit logging for the backend;
- a vulnerability disclosure and incident response process;
- vendor security attestations (ISO 27001, SOC 2 Type II, or equivalent);
- regular penetration testing and firmware review.
`[Law]` A personal data breach must be notified to the supervisory authority within 72 hours where there is a risk to rights and freedoms (Art. 33), and to data subjects where the risk is high (Art. 34). A processor must notify the controller without undue delay (Art. 33(2)). If the data are truly anonymous, the GDPR breach regime does not apply to the anonymous output; but the collection stage may still involve personal data, and a breach of the sensor or backend could expose pseudonymous identifiers. The incident response plan should cover both.
### 4.7 Data subject rights and opt-out
`[Law]` Data subjects have rights of access, rectification, erasure, restriction, objection, and portability (Art. 15–20), plus the right not to be subject to automated decision-making (Art. 22, not applicable here) and the right to complain to a supervisory authority (Art. 77). Art. 11 provides a limited exception: if the controller can demonstrate that it cannot identify the data subject, it may not be obliged to comply with Arts. 15–20, but it must inform the data subject if possible and must assist if the data subject provides additional information enabling identification.
For a MAC tracking system, the controller may not be able to link a person to a MAC. The practical rights are therefore:
- **Objection (Art. 21):** if the legal basis is Art. 6(1)(e) or (f), the data subject can object; the controller must stop unless it demonstrates compelling legitimate grounds. The right must be brought to the data subject's attention explicitly and separately at the latest at first communication (Art. 21(4)). Signage should state it.
- **Opt-out:** a mechanism to stop counting a device. The CNIL recommends a dedicated Wi-Fi network named `wifi_tracking_optout` or a website/app where the user enters the MAC address; turning off Wi-Fi is not a satisfactory opt-out. `[Guidance]` CNIL, 24 Feb 2020. For BLE, a MAC-based opt-out is technically fragile because of randomisation; a dedicated opt-out SSID only works for Wi-Fi. A practical BLE opt-out may be a web form plus a commitment to delete existing pseudonymous data and suppress future matches; but the controller should be honest about its limits.
- **Access/erasure:** if the controller cannot identify the data subject, Art. 11 may limit the response; but the controller should have a process and explain it in the notice.
- **Complaint:** the LfDI Baden-Württemberg; the notice should give the address.
`[Guidance]` The CNIL's position is that for short-delay anonymisation, the access, rectification and restriction rights are not useful because the controller does not need to identify the persons; the right to object must still be provided. `[Guidance]` CNIL, 24 Feb 2020. This is a sensible model for a privacy-preserving design, but it does not remove the need for a legal basis or the Art. 21 notice.
### 4.8 Processors, joint controllers, and transfers
`[Law]` Art. 28 requires a written contract with a processor, including: processing only on documented instructions; confidentiality; security; sub-processor authorisation; assistance with data subject rights and DPIAs; breach notification under Art. 33(2); deletion or return at the end; audit and inspection. Art. 28(4) applies the same obligations to sub-processors. Art. 28(10) provides that a processor that determines purposes and means becomes a controller for that processing.
`[Law]` Art. 26 requires joint controllers to arrange their responsibilities in a transparent manner, in particular for the exercise of data subject rights and the provision of information; the essence of the arrangement must be made available to data subjects. The arrangement does not affect the fact that data subjects can exercise their rights against either controller.
For Heilbronn, the contracts should cover:
- the city/Verein/vendor roles and the purposes;
- the exact data fields, retention, and aggregation rules;
- a prohibition on secondary use, re-identification, profiling, model training, or sale;
- a prohibition on transferring raw or pseudonymous data outside the EU/EEA without a legal transfer mechanism;
- security requirements, audit rights, and incident notification;
- deletion and return at the end, with certification;
- sub-processor approval and flow-down obligations;
- cooperation with the DPIA and prior consultation;
- liability and indemnity.
`[Law]` International transfers require Art. 44–49. If a vendor or cloud provider is outside the EEA, the controller needs an adequacy decision (e.g., the EU–US Data Privacy Framework, if the recipient is certified), standard contractual clauses plus a transfer impact assessment, or another Art. 46 mechanism. `[CJEU]` *Schrems II*, C-311/18. For a local pilot, EU hosting and EU sub-processors are the simplest route.
---
## 5. Anonymisation and privacy-enhancing technologies
### 5.1 The legal standard
`[Law]` Recital 26 is the starting point: anonymous information is information that does not relate to an identified or identifiable person, or personal data rendered anonymous so that the data subject is not or no longer identifiable. The assessment considers all means reasonably likely to be used by the controller or another person, including costs, time, available technology, and technological developments.
`[CJEU]` *Breyer* makes the test relative and contextual: identification through third-party data can make data personal even if the controller alone cannot identify; only a disproportionate effort excludes identifiability. `[CJEU]` C-582/14, paras 42–46. *EDPS v SRB* adds that the same pseudonymised data can be personal for the sender and non-personal for a recipient who lacks the key, but the sender's own transparency and legal-basis obligations remain. `[CJEU]` C-413/23 P, paras 82–85.
`[Guidance]` The EDPB's 2026 draft Anonymisation Guidelines set out the current framework:
- Anonymity is assessed from the perspective of each relevant entity, typically the party for whom the data are intended to be anonymous. `[Guidance]` EDPB Guidelines 02/2026, executive summary.
- Data can relate to a person by content, purpose or effect; the link need not be readily apparent; aggregate data can still relate to individuals if techniques can reveal information about a single individual. `[Guidance]` Ibid., paras 17–19.
- A hardware identifier such as a MAC address can identify a device and, in turn, the individual using it. `[Guidance]` Ibid., para. 25.
- The three criteria are **No Record Isolation**, **No Linkage**, and **No Inference**. `[Guidance]` Ibid., §3.4.
- Aggregate data will usually satisfy No Record Isolation if the aggregation was done correctly, but they must still pass No Linkage and No Inference. `[Guidance]` Ibid., para. 58.
- Inference can produce record-level data from aggregate data, including through differencing, outlier analysis, and combining multiple aggregate datasets. `[Guidance]` Ibid., paras 76–78. The EDPB gives the example of salary aggregates that reveal an individual's salary by subtraction. `[Guidance]` Ibid., Example 16.
- Re-identification techniques are less likely to be effective against aggregate data, but this depends on the amount of data aggregated and the auxiliary information available. `[Guidance]` Ibid., para. 86.
- The guidelines do not approve any specific technique as a safe harbour; the assessment must be documented and reassessed as technology evolves. `[Guidance]` Ibid., paras 84–87.
`[Guidance]` The Article 29 Working Party's Opinion 05/2014 remains a useful taxonomy: anonymisation techniques include randomisation (noise, permutation, differential privacy) and generalisation (k-anonymity, l-diversity, t-closeness), plus suppression; the three risks are singling out, linkability, and inference. `[Guidance]` WP29 Opinion 05/2014, pp. 5–8. The Opinion also makes clear that hashing is pseudonymisation, not anonymisation, where the original values can be recovered or brute-forced. `[Guidance]` Ibid., pp. 20–21.
`[Guidance]` The ICO's current anonymisation guidance uses a similar risk-based approach, including a "motivated intruder" test, and distinguishes anonymisation from pseudonymisation. `[Guidance]` ICO, "Anonymisation"; ICO, "Pseudonymisation". The ICO is UK guidance, not binding in the EU, but it is consistent with the EDPB framework.
### 5.2 The operator's claim, examined
Code for Heilbronn's blog says: "These MAC addresses are hashed and stored for at most one hour. This means that no conclusions can be drawn about specific devices or persons and no profiles can be created. Moreover, these MAC addresses do not leave the sensors even hashed, because only the number of counted addresses is sent." `[Technical]` Code for Heilbronn, 18 Sep 2025 (author's translation).
The first sentence is a reasonable description of the design intent. The second sentence is too strong as a legal and technical claim:
- **Hashing is pseudonymisation, not anonymisation.** The controller holds the hash function/key; the raw MAC is observable before hashing; the hashed value is a unique identifier during the retention window. `[Guidance]` EDPB Guidelines 01/2025, para. 22; `[DPA]` AP, "Wifi and bluetooth"; `[Enforcement]` CE, 8 Feb 2017, no. 393714, para. 8.
- **The hash space matters.** A full SHA-256 hash of a 48-bit MAC is not a strong anonymisation technique because the input space is small and enumerable; an unsalted hash can be brute-forced. A 16-bit ID (the upstream reference) has only 65,536 values; collisions are frequent, and the ID is not a cryptographic hash. A keyed hash with a secret, rotated key is better, but it is still pseudonymous while the key exists. `[Technical]` Code audit, §§1.2, 2.2; `[Guidance]` WP29 Opinion 05/2014, pp. 20–21.
- **The one-hour cache creates a one-hour linkability window.** If the sensor stores the hashed ID for an hour, it can link multiple detections of the same device within that hour. Whether that creates a "profile" depends on the data structure and the sensor's coverage; a bitmap of IDs per sensor is less revealing than a timestamped trajectory, but it is still linkability. The operator's claim of "no profiles" should be replaced with a precise description of what is retained and why.
- **The output counts can still be personal data in small cells.** A 15-minute count of 1 at a sensor reveals that one device was present. If the sensor covers a small area or a sensitive location, that can be identifying. The count is not automatically anonymous because it is a number. `[Guidance]` EDPB Guidelines 02/2026, paras 17–19, 76–78.
**Recommendation.** Replace the absolute claim with a precise, verifiable statement:
> "The sensor reads BLE advertising addresses in memory. It converts each address into a temporary keyed pseudonymous identifier, retains that identifier only for the deduplication window (maximum X minutes), and transmits only the number of distinct identifiers per interval. Raw addresses are never stored or transmitted. The published counts are aggregated to zones and time bins and are suppressed below a minimum count. This reduces identifiability; it is not a claim that the data are anonymous in every context."
This is defensible, transparent, and consistent with the legal standard.
### 5.3 k-anonymity and cell suppression
`[Technical]` k-anonymity, introduced by Sweeney, requires that each record be indistinguishable from at least k−1 other records with respect to the quasi-identifiers. `[Technical]` Sweeney, 2002. For footfall counts, there are two different uses:
1. **Record-level k-anonymity on the underlying device-location records.** Generalise location (e.g., sensor zone instead of exact coordinates) and time (e.g., hour instead of minute), and suppress combinations with fewer than k devices. This reduces singling out but does not prevent linkage or inference.
2. **Cell suppression on aggregate counts.** Publish a count only if it is at least k; suppress or merge cells below k; use complementary suppression so that suppressed cells cannot be recovered by subtraction from totals. This is the practical analogue for a count product.
`[Guidance]` There is no legal k threshold. The CNIL's 2020 guidance speaks of a "taux de collision élevé" (high collision rate) — many MAC addresses must map to the same identifier after anonymisation — rather than a fixed k. `[Guidance]` CNIL, 24 Feb 2020. The EDPB's 2026 draft does not set a k; it requires the three criteria. `[Guidance]` EDPB Guidelines 02/2026, §3.4. The ICO's guidance discusses k-anonymity as a technique with limitations, not a safe harbour. `[Guidance]` ICO, "Anonymisation".
For a Heilbronn-style product, a defensible starting point is:
- publish zone × hour (or zone × day), not sensor × 15 minutes;
- require k ≥ 10 (or 20 for sensitive locations/times) distinct devices per published cell;
- suppress cells below k; merge them into a larger area/time bin or mark them as "below threshold";
- apply complementary suppression so that a suppressed cell cannot be derived from published totals;
- avoid overlapping zones that allow differencing;
- avoid publishing repeat-visit rates or trajectories; these require linking devices across time and are incompatible with the anonymity objective;
- document the choice of k and the suppression rules in the DPIA and in a public methodology note.
`[Guidance]` The EDPB's 2026 draft expressly warns that aggregate data can fail No Inference through differencing and combination; a k-threshold alone is not enough. `[Guidance]` EDPB Guidelines 02/2026, paras 76–78. The CNIL's requirement to avoid precise timestamps for repeat-visit measurement points in the same direction. `[Guidance]` CNIL, 24 Feb 2020.
### 5.4 Differential privacy
`[Technical]` Differential privacy (DP) provides a formal guarantee that the output of a computation is approximately the same whether or not any single individual's data are included. The standard mechanism adds calibrated noise (Laplace or Gaussian) to a query result; the privacy loss is controlled by ε (epsilon), with smaller ε meaning stronger privacy and more noise. `[Technical]` Dwork & Roth, 2014.
For footfall counts:
- **Sensitivity:** for a count query, adding or removing one person changes the count by 1, so the sensitivity is 1.
- **Mechanism:** Laplace noise with scale 1/ε, or a Gaussian mechanism with the appropriate σ.
- **Budget:** every release consumes privacy budget; repeated queries compose. A dashboard that lets users query arbitrary time/space cells can exhaust the budget and leak information through differencing. Use a fixed set of published tables, a pre-defined query set, or a privacy budget tracker.
- **Local vs central:** central DP at the edge/backend is more accurate; local DP would require adding noise on each device, which is not possible for a passive sensor that does not run code on the phone. Secure aggregation can avoid a trusted curator, but it is more complex.
- **Post-processing:** clamping negative noisy counts to zero and rounding are allowed post-processing, but they can affect the guarantee; document the pipeline.
- **Utility:** for small counts, DP noise can swamp the signal; combine with coarser aggregation and suppression. For large city-scale counts, the relative error is small.
`[Guidance]` DP is a PET, not a legal safe harbour. The EDPB's 2026 draft does not endorse a specific ε or declare DP outputs anonymous; it requires the No Isolation/No Linkage/No Inference assessment. `[Guidance]` EDPB Guidelines 02/2026, §3.4. The ICO's PETs guidance describes differential privacy as a technique that adds noise to produce aggregate statistics, while noting the need to assess the privacy budget and the utility trade-off. `[Guidance]` ICO, "Privacy-enhancing technologies (PETs)". A DP output with a large ε, or with repeated queries, can still be personal data.
**Recommendation.** Use DP as an additional layer, not the only layer. Publish only a fixed set of coarse tables; set a privacy budget per period; document ε and the composition; combine with k-threshold and suppression; do not expose an ad-hoc query interface over the raw counts. A reasonable starting point for a public dashboard is ε ≈ 0.1–1 per release, but the choice must be justified by the DPIA and the expected number of releases. This is an engineering recommendation, not a legal requirement.
### 5.5 MAC-specific PETs and key management
The following measures are the technical core of a defensible design:
1. **Passive only.** Do not send probe requests, association requests, or any active query. Do not impersonate a hotspot. Do not use an OS API or an app to read identifiers. `[Technical]` Code audit §1.1 (passive BLE scan).
2. **No raw retention.** Keep the raw MAC in volatile memory only for the minimum time needed to compute the dedup identifier; never write it to flash, logs, packet captures, or the LoRaWAN payload. `[Technical]` Code audit §3 (upstream design); Code for Heilbronn blog (only counts transmitted).
3. **Keyed hash, not a plain hash.** Use a keyed MAC (HMAC-SHA-256) or a keyed pseudonymisation function with a secret key stored in a secure element. Do not use an unsalted hash; do not use a static global salt; do not truncate to 16 bits. `[Guidance]` ICO, "Pseudonymisation" (long salts/peppers; test computational feasibility); `[Guidance]` WP29 Opinion 05/2014, pp. 20–21.
4. **Per-sensor, per-window keys.** Derive a different key for each sensor and each counting window (e.g., 15 minutes or 1 hour). Destroy the key at the end of the window. This prevents cross-sensor and cross-window linkage. `[DPA]` AP, "Legal bases for WiFi tracking and Bluetooth tracking" (recommends hashing differently for each measuring location); `[Guidance]` CNIL, 24 Feb 2020 (salt renewed regularly).
5. **No cross-sensor linkage.** Do not send the same pseudonymous ID to multiple sensors or to the backend. Aggregate at the sensor; if the backend needs to combine counts, combine only counts, not identifiers.
6. **No trajectories.** Do not store a sequence of sensor IDs per device; do not compute paths, dwell times, or repeat-visit rates. The JCDecaux case shows that a purpose requiring routes/repeat visits is incompatible with anonymisation. `[Enforcement]` CE, 8 Feb 2017, no. 393714, para. 8.
7. **Short dedup window.** Align the window with the reporting interval; if the output is 15-minute counts, a 15-minute window is the natural default. A 1-hour window needs a documented justification. `[Law]` Art. 5(1)(e); `[Guidance]` CNIL, 24 Feb 2020.
8. **Edge aggregation.** Transmit only counts. Do not transmit hashed IDs, even encrypted. This reduces the attack surface and supports the argument that the identifiers do not leave the sensor. `[Technical]` Code for Heilbronn blog; code audit §3.
9. **No fingerprinting.** Do not use sequence numbers, information elements, RSSI patterns, or payload tokens to re-identify devices across MAC randomisation. `[Technical]` Vanhoef et al., 2016; Martin et al., 2017; Becker et al., 2019.
10. **Secure key management.** Store keys in a secure element or trusted execution environment; restrict access; rotate; audit; zeroise on tamper. If the key is compromised, the pseudonymous data become linkable to raw MACs. `[Law]` Art. 32.
### 5.6 When do aggregated counts become anonymous?
There is no single threshold. The legal test is contextual: the counts are anonymous if they do not relate to an identified or identifiable person, considering all means reasonably likely to be used by the controller or another person. `[Law]` Recital 26; `[CJEU]` *Breyer*, paras 42–46. The following questions are a practical test:
1. **Is there any record-level data left?** If the controller retains raw MACs, hashed IDs, per-device timestamps, or trajectories, the data are personal (or pseudonymous) regardless of what is published.
2. **Can a published cell be isolated to one person?** If a cell has a count of 1, or a unique combination of attributes, it can single out an individual. Apply k-threshold and suppression.
3. **Can two published tables be linked to the same individual?** If the controller publishes per-sensor and per-zone tables, or overlapping time bins, differencing can reveal a small cell. Use consistent geography, complementary suppression, and a fixed publication set.
4. **Can auxiliary data identify the person?** Camera footage, payment records, hotspot logins, loyalty data, event attendance, or social media can be combined with a count to identify a person. The EDPB's No Linkage criterion requires assessing these means. `[Guidance]` EDPB Guidelines 02/2026, §3.4.2.
5. **Can the purpose or effect of the data relate to an individual?** Even a count can relate to a person if it is used to treat them differently (e.g., to deny entry, target advertising, or infer sensitive behaviour). `[Guidance]` EDPB Guidelines 02/2026, paras 17–19.
6. **Is the anonymisation irreversible and documented?** If the key is destroyed, the raw data are gone, and no linkage is reasonably possible, the data may be anonymous. Document the assessment, the threat model, and the reassessment schedule. `[Guidance]` EDPB Guidelines 02/2026, paras 84–87; `[Guidance]` ICO, "Anonymisation".
For the Heilbronn data, the current 15-minute per-sensor CSV is best treated as **internal aggregate data that may still be personal data in small cells**, not as a publication-safe anonymous product. Before public release, coarsen to zone × hour/day, apply k-threshold and suppression, consider DP noise, remove repeat-visit/trajectory metrics, and document the assessment. `[Guidance]` EDPB Guidelines 02/2026, paras 76–78; `[Guidance]` CNIL, 24 Feb 2020.
### 5.7 Suggested publication rules
| Rule | Suggested setting | Why |
|---|---|---|
| Spatial resolution | Zone (not individual sensor) | Reduces singling out and differencing |
| Temporal resolution | Hour or day (not 15 minutes) | Reduces presence inference and differencing |
| Minimum cell count | k ≥ 10; k ≥ 20 for sensitive locations/times | Suppresses small cells; no legal k, but a defensible starting point |
| Complementary suppression | Suppress additional cells so totals do not reveal suppressed cells | Prevents subtraction attacks |
| DP noise | Laplace/Gaussian; fixed query set; privacy budget | Formal protection against inference; not a standalone legal basis |
| Repeat visits | Do not publish | Requires cross-time linkage; incompatible with anonymity |
| Trajectories | Do not publish | Requires cross-sensor linkage; JCDecaux shows the problem |
| Sensitive locations | Exclude or coarsen | Art. 9 risk; chilling effects |
| Methodology | Publish a note | Accountability; allows scrutiny |
| Reassessment | Annual or on change | Technology and auxiliary data evolve |
These are engineering and disclosure-control recommendations, not legal advice. The DPIA should justify the chosen parameters.
---
## 6. Enforcement and regulatory practice
### 6.1 Cases and decisions
| Jurisdiction | Matter | Facts | Outcome | Lesson |
|---|---|---|---|---|
| Netherlands | **Enschede Wi-Fi tracking** `[Enforcement]` | Municipality used at least 10 sensors in the city centre (25 May 2018–30 Apr 2020) to capture MAC addresses of Wi-Fi-enabled devices, pseudonymised them, stored a 6-month long-term table, and estimated unique visitors. AP fined €600,000 (decision 11 Mar 2021, announced 29 Apr 2021) for processing personal data without a legal basis. | Rechtbank Overijssel annulled the fine on 2 Feb 2024 because the AP had not proved that the municipality processed personal data; the Raad van State upheld on 29 Jul 2026, essentially because the AP's new "direct identification" argument had not been in the original decision. The fine is off. | The AP's theory (pseudonymised MAC + location = personal data) was not accepted on the evidence. The case is fact-specific and procedural; it is not a ruling that Wi-Fi tracking is lawful or anonymous. The regulator must prove identifiability; the controller should document its own assessment. |
| France | **JCDecaux / La Défense** `[Enforcement]` | Six Wi-Fi counting boxes on public advertising furniture, capturing MAC addresses within 25 m and calculating positions, for a four-week test. JCDecaux proposed truncating the last half-octet, salting, and keyed hashing. The CNIL refused authorisation (deliberation 2015-255, 16 Jul 2015). | Conseil d'État, 8 Feb 2017, no. 393714, upheld the refusal. The court held that hashing/salting left the controller able to identify and link records, and that the purpose (routes and repeat passages) was incompatible with anonymisation. | Pseudonymisation is not anonymisation; the purpose matters; a real opt-out and information are required. This is the strongest EU judicial warning against a "we hash it" defence. |
| Germany (Schleswig-Holstein) | **ULD offline tracking** `[DPA]` | Two city-marketing operators (a Verein and a GmbH) passively captured probe requests to measure visitor flows in two cities. The ULD opened administrative proceedings. | The ULD's legal view: MAC addresses are personal data; there is no legal basis; legitimate interest fails because the data are collected without knowledge and people cannot practically prevent it; no consent. Both operators stopped; the proceedings were closed. | This is the closest German analogue to a citizen-association footfall project. A Verein is not exempt. Passive probe-request tracking can be treated as unlawful even without a fine. |
| Germany (Bavaria) | **BayLDA offline-tracking review** `[DPA]` | 2016 review of 11 retailers; all denied using offline tracking. A complaint led to an on-site inspection of a fashion chain's Wi-Fi tracking test at a store entrance, where a sign had been posted. | The company stopped the test after discussion; the BayLDA could not complete a technical/legal assessment. | Supervisory attention is real; a sign at the entrance did not make the test lawful. The case shows the value of a prior DPIA and a legal basis. |
| Germany (Hamburg) | **HmbBfDI indoor tracking guidance** `[DPA]` | The 2014/2015 activity report discusses WLAN/Bluetooth indoor tracking of customers and passers-by. | The DPA says such tracking massively interferes with rights and is only permitted under very narrow conditions; MAC addresses and movement profiles are personal data; a legal basis is required; transparency before entering the area is required; an enabled interface is not consent. | A German DPA views indoor/outdoor WLAN/Bluetooth tracking as high-risk. The report predates the GDPR but its reasoning is consistent with the current framework. |
| Germany (federal) | **TDDDG fines** `[Law]` | § 28(1) No. 13 TDDDG makes a violation of § 25(1) an administrative offence; § 28(2) sets a fine of up to €300,000 for private offenders. | § 28(4) excludes fines against public bodies. | A private operator (including a Verein, depending on its status) faces a TDDDG fine; a public body faces supervisory measures, complaints, and reputational risk, not a TDDDG fine. |
**No published German fine specifically for passive Wi-Fi/BLE footfall tracking was found.** The German cases were administrative/stop-and-desist. That is not evidence of legality; it may reflect the novelty of the technology, the cooperation of the operators, and the difficulty of proving personal data (as in Enschede).
**Temporal caveat.** The JCDecaux decision (2017) applies the French law implementing Directive 95/46/EC; the ULD (2019), BayLDA (2016) and Hamburg (2014/2015) materials predate the TTDSG/TDDDG and, in some cases, the GDPR. They are cited for their reasoning on MAC addresses, identifiability, legitimate interest and transparency, not as direct applications of the current German statute. The current framework is the GDPR plus § 25 TDDDG; the older decisions show how DPAs approach the technology, but they do not resolve the § 25 passive-sniffing question.
### 6.2 Regulatory guidance
- **CNIL (France), 24 Feb 2020.** Three regimes for audience/footfall measurement in publicly accessible spaces: (1) anonymisation within minutes (CNIL recommends a maximum of five minutes) — legitimate interest, limited information, opt-out; (2) pseudonymisation at collection and anonymisation/destruction by the end of the day (max 24 hours) — legitimate interest with safeguards; (3) all other cases — consent. `[Guidance]` CNIL, 24 Feb 2020. The guidance also recommends deleting raw MACs immediately, hashing with a regularly renewed salt, deleting pseudonymised data at the end, limiting the area and duration, avoiding precise timestamps for repeat-visit measurement, and providing a real opt-out (dedicated SSID or website/app; turning off Wi-Fi is not satisfactory).
- **Autoriteit Persoonsgegevens (Netherlands).** Wi-Fi/Bluetooth tracking data are personal data; hashing alone is pseudonymisation; storing multiple location points per device means the data are not anonymous; municipalities need an independent legal basis, specific periods, and precisely described areas; private parties may rely on legitimate interest only in limited cases (property protection, safety, crowd control), with proportionality, subsidiarity, immediate anonymisation, different hashing per location, spatial/temporal limits, and opt-out. `[DPA]` AP, "Wifi and bluetooth"; "Legal bases for WiFi tracking and Bluetooth tracking".
- **DSK (Germany), Nov 2024.** § 25 TDDDG applies regardless of personal data; reading MAC addresses and Bluetooth beacons must be considered; the guidance follows EDPB Guidelines 2/2023. `[Guidance]` DSK, OH Digitale Dienste, paras 22, 26.
- **LfDI Baden-Württemberg.** The private-sector Muss-Liste includes offline tracking of customer movements and the non-intended use of device sensors/radio signals to determine location or movement over a substantial period; a DPIA is mandatory for private controllers. There is no public-sector positive list in Baden-Württemberg. `[Guidance]` LfDI BW, Muss-Liste, Nos. 2, 10; LfDI BW/MJM BW, Schwellenwertanalyse (2024).
- **ICO (UK), 2016/current.** The 2016 Wi-Fi location analytics guidance (no longer available at the original ICO URL) recommended a privacy impact assessment, notices at the point of collection, a defined retention period, and a simple opt-in/opt-out. The current anonymisation, pseudonymisation, PETs, and DPIA guidance applies the same principles. `[Guidance]` ICO, "Wi-Fi location analytics" (2016; summarised in Bird & Bird, 2016); ICO, "Anonymisation"; "Pseudonymisation"; "Privacy-enhancing technologies"; "Data protection impact assessments".
### 6.3 What the enforcement record means for Heilbronn
1. **The risk is real but the legal question is unsettled.** Enschede shows a €600,000 fine for the exact use case; it also shows the fine can be annulled if the regulator cannot prove personal data. Do not treat either outcome as a safe harbour.
2. **German DPAs are skeptical of passive public-space MAC tracking.** The ULD's 2019 position is the closest to Heilbronn: passively captured probe requests, visitor counting, a Verein and a GmbH, no legal basis, legitimate interest fails. The operators stopped.
3. **A sign is not a legal basis.** The BayLDA inspected a store with a sign; the company stopped. The Hamburg DPA says an enabled interface is not consent. Signage is necessary but not sufficient.
4. **Hashing is not a magic defence.** The French Conseil d'État rejected truncation + salting + keyed hashing because the controller could still identify/link and the purpose required it. The Dutch AP says hashing is pseudonymisation. The German DSK says § 25 applies regardless of personal data.
5. **A public body's main risk is not a TDDDG fine.** § 28(4) TDDDG excludes fines against public bodies; but the city can face supervisory orders, Art. 58 GDPR measures, complaints, political scrutiny, and reputational damage. A private Verein or vendor can face a TDDDG fine up to €300,000 and GDPR fines.
---
## 7. Compliance checklist for a privacy-preserving architecture
This checklist is written for the Heilbronn/Paxcounter design. It is a practical starting point, not a legal opinion. Each item should be evidenced in the DPIA or the processing record.
### A. Legal route (decide before building)
- [ ] **Identify the controller(s).** City, Verein, vendor, LoRaWAN network operator, hosting provider. Document who decides purposes and means. If joint, draft an Art. 26 arrangement; if processor, draft an Art. 28 DPA. `[Law]` Arts. 4(7), 26, 28.
- [ ] **Identify the legal basis.** Public body: Art. 6(1)(e) + Art. 6(3) + the specific legal basis in BW law; Art. 6(1)(f) is unavailable. Private: Art. 6(1)(f) with a documented legitimate-interest assessment; consent only if genuinely obtainable. `[Law]` Art. 6.
- [ ] **Write the § 25 TDDDG memo.** Describe the exact frames, confirm passive-only reception, confirm no OS API/active request, address EDPB Guidelines 2/2023 paras 39/42 and DSK OH para. 22, state the residual risk and the fallback. If the memo is not robust, assume § 25 applies and do not deploy without consent or a statutory basis. `[Law]` § 25 TDDDG; `[Guidance]` EDPB Guidelines 2/2023; DSK OH.
- [ ] **Check whether the purpose is a public task.** If the city uses the data for cleaning, maintenance, urban development, or tourism, identify the statutory task and the necessity/proportionality analysis. `[Law]` Art. 6(1)(e), 6(3).
- [ ] **Do not use "open data" as a legal basis.** Publishing counts is a separate purpose; assess compatibility (Art. 6(4)) or rely on a separate basis. If the output is anonymous, the GDPR does not apply to the output, but the collection still needs a basis.
### B. DPIA and governance
- [ ] **DPIA before deployment.** Art. 35(3)(c); LfDI BW Muss-Liste Nos. 2/10 for private operators; no public-sector list in BW, but Art. 35(3)(c) applies directly. `[Law]` Art. 35; `[Guidance]` WP248; LfDI BW.
- [ ] **Involve the DPO.** Art. 35(2); document the advice. `[Law]` Art. 35(2), 39.
- [ ] **Assess necessity and alternatives.** Compare with manual counts, light barriers, thermal/radar, on-device video counting, and sample-based surveys. Explain why MAC/BLE is necessary and why a less intrusive method is not sufficient. `[CJEU]` *M5A*; `[Law]` Art. 5(1)(c).
- [ ] **Assess re-identification risk.** Small cells, differencing, auxiliary data, MAC randomisation, hash length, cache duration, cross-sensor linkage. Document the threat model and the residual risk. `[Guidance]` EDPB 02/2026; `[Guidance]` WP29 Opinion 05/2014.
- [ ] **Prior consultation if residual high risk.** Art. 36; allow time for the LfDI BW to respond. `[Law]` Art. 36.
- [ ] **Record of processing.** Art. 30; include the purpose, data categories, recipients, retention, security measures, and the § 25 analysis. `[Law]` Art. 30.
- [ ] **Review and audit.** Art. 35(11); annual review or on change; independent audit of the sensor firmware and backend.
### C. Architecture and PETs
- [ ] **Prefer a non-identifying sensor.** If a light barrier, thermal sensor, radar, or on-device video counter can meet the accuracy requirement, use it. A MAC-free design avoids the core legal problem. `[Guidance]` CNIL, 24 Feb 2020 (distinguishes simple footfall counters that do not identify).
- [ ] **Passive only.** No probe requests, no association requests, no hotspot impersonation, no OS API. `[Technical]` Code audit §1.1.
- [ ] **No raw MAC retention.** Memory only; delete immediately after dedup; no logs, no pcap, no flash, no LoRaWAN payload. `[Technical]` Code for Heilbronn blog; code audit §3.
- [ ] **Keyed hash, per sensor, per window.** HMAC-SHA-256 or equivalent; secret key in a secure element; rotate every window; destroy the old key. Do not use a plain hash, a static salt, or a 16-bit ID. `[Guidance]` ICO, "Pseudonymisation"; `[DPA]` AP, "Legal bases".
- [ ] **No cross-sensor/cross-window linkage.** Aggregate at the sensor; transmit counts only; do not send the same pseudonymous ID to the backend. `[DPA]` AP, "Legal bases".
- [ ] **No trajectories or repeat-visit metrics.** Do not store per-device paths, dwell times, or repeat visits. `[Enforcement]` CE, 8 Feb 2017, no. 393714, para. 8.
- [ ] **Short dedup window.** Align with the reporting interval; document why a longer window is necessary. `[Law]` Art. 5(1)(e).
- [ ] **Edge aggregation.** Counts only; no identifiers in the uplink. `[Technical]` Code for Heilbronn blog.
- [ ] **No fingerprinting.** No sequence numbers, information elements, RSSI patterns, or payload tokens. `[Technical]` Vanhoef et al., 2016; Martin et al., 2017; Becker et al., 2019.
- [ ] **Secure key management.** Secure element/TEE; access control; rotation; zeroisation; tamper detection. `[Law]` Art. 32.
- [ ] **Sensor hardening.** Secure boot, signed firmware, secure updates, unique credentials, no default passwords, network segmentation, physical tamper protection. `[Law]` Art. 32.
- [ ] **No secondary use.** Contractually prohibit re-identification, profiling, model training, sale, and sharing with third parties. `[Law]` Art. 28(3); `[Guidance]` EDPB 07/2020.
### D. Transparency and rights
- [ ] **On-site signage before the measurement zone.** Identify the controller, DPO, purposes, legal basis, data categories, retention, rights, opt-out, and complaint authority. `[Law]` Arts. 12–14, 21(4).
- [ ] **QR code to the full notice.** German and English; accessible formats. `[Guidance]` CNIL, 24 Feb 2020 (two-level information).
- [ ] **Do not claim "fully anonymous" unless documented.** Describe the staged processing precisely. `[Guidance]` EDPB 02/2026; `[DPA]` AP, "Wifi and bluetooth".
- [ ] **Right to object.** Art. 21(4) notice; a functioning objection channel. `[Law]` Art. 21.
- [ ] **Opt-out mechanism.** Dedicated Wi-Fi SSID `wifi_tracking_optout` for Wi-Fi; a web/app form for MAC/BLE; delete existing pseudonymous data on opt-out; suppress future matches; do not require turning off Wi-Fi/BLE as the only option. `[Guidance]` CNIL, 24 Feb 2020.
- [ ] **Acknowledge opt-out limits.** MAC randomisation makes a MAC-based opt-out fragile; document the limitation and the alternative (spatial/temporal limits, no trajectories). `[Technical]` Martin et al., 2017.
- [ ] **Data subject request process.** Art. 11/15–21; explain the limits; log requests; respond within one month. `[Law]` Arts. 11–12, 15–21.
### E. Contracts and transfers
- [ ] **Art. 28 DPA with every processor.** Scope, instructions, confidentiality, security, sub-processors, assistance, deletion/return, audit. `[Law]` Art. 28(3).
- [ ] **Art. 26 arrangement if joint controllers.** Allocate transparency, rights, security, and breach responsibilities; publish the essence. `[Law]` Art. 26.
- [ ] **Sub-processor approval.** LoRaWAN network operator, hosting provider, analytics provider; flow-down obligations; list of sub-processors. `[Law]` Art. 28(4).
- [ ] **EU hosting and transfers.** Prefer EU/EEA; if outside, Art. 44–49 mechanism, SCCs + TIA, or adequacy decision; Schrems II. `[CJEU]` C-311/18.
- [ ] **No raw/pseudonymous data to the vendor.** If the vendor needs data for support, provide counts or synthetic data; no re-identification. `[Law]` Art. 28(3); `[Guidance]` EDPB 07/2020.
- [ ] **Deletion and certification at the end.** Return or delete; certify; audit. `[Law]` Art. 28(3)(g).
### F. Operations and publication
- [ ] **Publish only coarse aggregates.** Zone × hour/day; k ≥ 10/20; suppression; complementary suppression; DP noise; no per-sensor minute data; no repeat visits/trajectories. `[Guidance]` EDPB 02/2026; CNIL, 24 Feb 2020.
- [ ] **Document the disclosure-control method.** k, ε, suppression rules, aggregation rule, and the re-identification assessment. `[Guidance]` EDPB 02/2026, paras 84–87.
- [ ] **No ad-hoc query interface over raw counts.** Fixed tables; privacy budget; no differencing. `[Technical]` Dwork & Roth, 2014.
- [ ] **Monitor and reassess.** New auxiliary data, new sensors, new purposes, and new technology can change the anonymity assessment. `[Guidance]` EDPB 02/2026; ICO, "Anonymisation".
- [ ] **Incident response.** Art. 33/34; processor notification; sensor compromise; key compromise; data exfiltration. `[Law]` Arts. 33–34.
- [ ] **Training and accountability.** Staff and volunteers handling data; confidentiality; data minimisation; no shadow copies. `[Law]` Art. 32.
### G. Suggested pilot parameters
| Parameter | Suggested starting point | Note |
|---|---|---|
| Sensor | Passive BLE/Wi-Fi; no active requests; no payload | The most defensible § 25 position |
| Raw MAC | Memory only; ≤ 1 s; no logs/pcap/flash | Minimisation; breach impact |
| Dedup ID | HMAC-SHA-256; per-sensor, per-window key; ≥ 128-bit output | Not a 16-bit ID; not a plain hash |
| Key rotation | Every 15 min or 1 h; destroy old key | Prevent cross-window linkage |
| Dedup window | 15 min (default) or 1 h with justification | Align with reporting interval |
| Cross-sensor linkage | None | No trajectories; no repeat visits |
| Uplink | Counts only | No identifiers leave the sensor |
| Publication | Zone × hour/day; k ≥ 10 (20 for sensitive) | No per-sensor 15-min data |
| DP | Laplace/Gaussian; fixed tables; ε budget | Additional layer, not a safe harbour |
| Retention | Raw 0; pseudonymous ≤ window; aggregates 12–24 months | Document and verify deletion |
| Signage | Every entrance; DE/EN; QR; DPO; opt-out | Before the measurement zone |
| Opt-out | Dedicated SSID + web form; delete existing data | Acknowledge MAC-randomisation limits |
| DPIA | Before pilot; Art. 35(3)(c); DPO; Art. 36 if needed | LfDI BW Muss-Liste Nos. 2/10 for private operators |
| Contracts | Art. 28 DPA; Art. 26 if joint; EU hosting | No secondary use; audit; deletion |
| Legal basis | Public task (Art. 6(1)(e)) or, for a private operator only, legitimate interest (Art. 6(1)(f)) | § 25 memo; no consent at scale |
### H. Red flags
- "We hash the MAC, so it is anonymous." — Hashing is pseudonymisation; a short/unsalted hash is weak.
- "We do not store MACs, so GDPR does not apply." — The sensor processes the MAC in memory; the collection stage needs a legal basis.
- "The sign is the consent." — A sign is information, not consent. `[CJEU]` *Planet49*.
- "People can opt out by turning off Wi-Fi." — The CNIL says this is not satisfactory; the ULD says it is practically impossible on some devices.
- "We use a global salt." — A global salt links all devices across time/locations; use per-sensor, per-window keys.
- "We keep hashed IDs for six months to measure repeat visits." — That is a movement profile; JCDecaux shows the problem; the purpose is incompatible with anonymisation.
- "We share data with the vendor for benchmarking." — That is secondary use; the vendor may become a controller; Art. 28(10).
- "We combine with camera/payment data to validate counts." — That creates linkability and may create Art. 9 data; avoid.
- "We publish per-sensor 15-minute counts." — Small cells can identify individuals; coarsen/suppress.
- "The Enschede fine was annulled, so Wi-Fi tracking is legal." — The fine was annulled on evidence/procedure, not on the merits; the legal question is unsettled.
- "The city is a public body, so no fines." — TDDDG § 28(4) excludes TDDDG fines, but GDPR supervisory measures, complaints, and reputational risk remain.
---
## 8. Citations
All URLs were accessed on 14 November 2026. Guidance and enforcement decisions are persuasive or fact-specific; they are not binding law unless stated.
### 8.1 EU legislation
1. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (General Data Protection Regulation), OJ L 119, 4.5.2016, p. 1–88. <https://eur-lex.europa.eu/eli/reg/2016/679/oj>
2. Directive 2002/58/EC of the European Parliament and of the Council of 12 July 2002 concerning the processing of personal data and the protection of privacy in the electronic communications sector (Directive on privacy and electronic communications), OJ L 201, 31.7.2002, p. 37–47, as amended by Directive 2009/136/EC, OJ L 337, 18.12.2009, p. 11–36. <https://eur-lex.europa.eu/eli/dir/2002/58/oj>
### 8.2 CJEU case law
3. Case C-582/14, *Patrick Breyer v Bundesrepublik Deutschland*, ECLI:EU:C:2016:779, 19 October 2016. <https://curia.europa.eu/juris/liste.jsf?num=C-582/14>
4. Case C-434/16, *Peter Nowak v Data Protection Commissioner*, ECLI:EU:C:2017:994, 20 December 2017.
5. Case C-184/20, *Vyriausioji tarnybinės etikos komisija*, ECLI:EU:C:2022:601, 1 August 2022.
6. Case C-319/22, *Gesamtverband Autoteile-Handel eV v Scania CV AB*, ECLI:EU:C:2023:857, 9 November 2023.
7. Case C-413/23 P, *European Data Protection Supervisor v Single Resolution Board*, ECLI:EU:C:2025:645, 4 September 2025.
8. Case C-673/17, *Bundesverband der Verbraucherzentralen und Verbraucherverbände – Verbraucherzentrale Bundesverband eV v Planet49 GmbH*, ECLI:EU:C:2019:801, 1 October 2019.
9. Case C-604/22, *IAB Europe v Gegevensbeschermingsautoriteit*, ECLI:EU:C:2024:214, 7 March 2024.
10. Case C-40/17, *Fashion ID GmbH & Co. KG v Verbraucherzentrale NRW eV*, ECLI:EU:C:2019:629, 29 July 2019.
11. Case C-210/16, *Unabhängiges Landeszentrum für Datenschutz Schleswig-Holstein v Wirtschaftsakademie Schleswig-Holstein GmbH*, ECLI:EU:C:2018:388, 5 June 2018.
12. Case C-708/18, *Asociaţia de Proprietari bloc M5A Scara A v S.C. ANAF*, ECLI:EU:C:2019:1063, 11 December 2019.
13. Case C-621/22, *Koninklijke Nederlandse Lawn Tennisbond (KNLTB) v Autoriteit Persoonsgegevens*, ECLI:EU:C:2024:857, 4 October 2024.
14. Case C-252/21, *Meta Platforms Inc. v Bundeskartellamt*, ECLI:EU:C:2023:537, 4 July 2023.
15. Case C-311/18, *Data Protection Commissioner v Facebook Ireland Ltd and Maximillian Schrems*, ECLI:EU:C:2020:559, 16 July 2020.
### 8.3 EDPB and Article 29 Working Party
16. EDPB, *Guidelines 2/2023 on Technical Scope of Art. 5(3) of ePrivacy Directive*, Version 2.0, adopted 7 October 2024. <https://www.edpb.europa.eu/system/files/2024-10/edpb_guidelines_202302_technical_scope_art_53_eprivacydirective_v2_en_0.pdf>
17. EDPB, *Guidelines 01/2025 on Pseudonymisation*, adopted 16 January 2025, version for public consultation. <https://www.edpb.europa.eu/system/files/2025-01/edpb_guidelines_202501_pseudonymisation_en.pdf>
18. EDPB, *Guidelines 02/2026 on Anonymisation*, Version 1.0, adopted 7 July 2026 for public consultation. <https://www.edpb.europa.eu/system/files_en?file=2026-09%2Fedpb_guidelines_202602_anonymisation_v1_en_0.pdf>
19. EDPB, *Guidelines 07/2020 on the concepts of controller and processor in the GDPR*, Version 2.0, adopted 7 July 2021. <https://www.edpb.europa.eu/our-work-tools/our-documents/guidelines/guidelines-072020-concepts-controller-and-processor-gdpr_en>
20. Article 29 Data Protection Working Party, *Opinion 4/2007 on the concept of personal data*, WP 136, 20 June 2007. <https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2007/wp136_en.pdf>
21. Article 29 Data Protection Working Party, *Opinion 05/2014 on anonymisation techniques*, WP 216, 10 April 2014. <https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2014/wp216_en.pdf>
22. Article 29 Data Protection Working Party, *Guidelines on Data Protection Impact Assessment (DPIA) and determining whether processing is "likely to result in a high risk"*, WP 248 rev.01, 4 October 2017, endorsed by the EDPB. <https://ec.europa.eu/newsroom/article29/items/611236>
23. Article 29 Data Protection Working Party, *Opinion 9/2014 on the application of Directive 2002/58/EC to device fingerprinting*, WP 219, 25 November 2014 (cited by EDPB Guidelines 2/2023 for the MAC-address/OS-API example). <https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2014/wp219_en.pdf>
### 8.4 German law and guidance
24. Telekommunikation-Digitale-Dienste-Datenschutz-Gesetz (TDDDG), §§ 2(2) No. 6, 25, 28, 29. <https://www.gesetze-im-internet.de/ttdsg/__25.html>; <https://www.gesetze-im-internet.de/ttdsg/__28.html>; <https://www.gesetze-im-internet.de/ttdsg/__29.html>
25. Entwurf eines Gesetzes zur Regelung des Datenschutzes und des Schutzes der Privatsphäre in der Telekommunikation und bei Telemedien (TTDSG), BT-Drs. 19/27441, p. 38 (explanatory memorandum on § 24 TTDSG, now § 25 TDDDG). <https://dserver.bundestag.de/btd/19/274/1927441.pdf>
26. Datenschutzkonferenz (DSK), *Orientierungshilfe der Aufsichtsbehörden für Anbieter:innen von digitalen Diensten (OH Digitale Dienste)*, Version 1.2, November 2024, paras 22, 26. <https://www.datenschutzkonferenz-online.de/media/oh/OH_Digitale_Dienste.pdf>
27. Landesbeauftragter für Datenschutz und die Informationsfreiheit Baden-Württemberg (LfDI BW), *Liste von Verarbeitungsvorgängen nach Art. 35 Abs. 4 DS-GVO für den nicht-öffentlichen Bereich ("Muss-Liste")*, 2018, Nos. 2 and 10. <https://www.baden-wuerttemberg.datenschutz.de/wp-content/uploads/2018/05/Liste-von-Verarbeitungsvorg%C3%A4ngen-nach-Art.-35-Abs.-4-DS-GVO-LfDI-BW.pdf>
28. LfDI BW / Ministerium für Justiz und Migration Baden-Württemberg, *Prüfung der Notwendigkeit einer Datenschutzfolgenabschätzung – Schwellenwertanalyse*, 29 November 2024 (noting that there is no positive list for the public sector in Baden-Württemberg). <https://jum.baden-wuerttemberg.de/fileadmin/redaktion/m-jum/intern/PDF/Migration/Erlasse_und_Anwendungshinweise/2024/29.11.2024_Anlage_01_Schwellwertanalyse.pdf>
29. Unabhängiges Landeszentrum für Datenschutz Schleswig-Holstein (ULD), *Tätigkeitsbericht 2019*, §5.4.8 "Offline-Tracking/Ortung von Mobiltelefonen in Fußgängerzone", pp. 97–99. <https://www.datenschutzzentrum.de/tb/tb37/uld-37-taetigkeitsbericht-2019.pdf>
30. Der Hamburgische Beauftragte für Datenschutz und Informationsfreiheit (HmbBfDI), *25. Tätigkeitsbericht Datenschutz 2014/2015*, §V.9 "Indoor-Ortung von Personen in Geschäften", pp. 141–143. <https://datenschutz-hamburg.de/fileadmin/user_upload/HmbBfDI/Datenschutz/Taetigkeitsberichte_Datenschutz/Taetigkeitsberichte_PDF/25.TB_2014-2015.pdf>
31. Bayerisches Landesamt für Datenschutzaufsicht (BayLDA), *Tätigkeitsbericht 2015/16*, §3.10 "Offline-Tracking", pp. 30–31, and §22.10 "Offline-Tracking", p. 148. <https://www.lda.bayern.de/media/baylda_report_07.pdf>
32. Bundesbeauftragte für den Datenschutz und die Informationsfreiheit (BfDI), *23. Tätigkeitsbericht 2009/2010*, on WLAN scanning of access points (MAC address + SSID + signal strength + geocoordinates as potentially personal data; not a phone-tracking case). <https://www.bfdi.bund.de/SharedDocs/Publikationen/Taetigkeitsberichte/TB_BfDI/23TB_09_10.pdf?__blob=publicationFile&v=6>
33. BfDI, *Datenschutz-Folgenabschätzungen und Listen von Verarbeitungsvorgängen*. <https://www.bfdi.bund.de/DE/Fachthemen/Inhalte/Technik/Datenschutz-Folgenabschaetzungen.html>
34. Bundesministerium für Wohnen, Stadtentwicklung und Bauwesen / Innenstadtprogramm, *Methoden zur Erfassung und Analyse von Passantenfrequenzen* (policy methods report; describes MAC-based counting as "an sich anonym" while acknowledging de-anonymisation — not a legal assessment). <https://www.innenstadtprogramm.bund.de/SharedDocs/downloads/Webs/ZIZ/DE/Erfahrungsbericht_Passantenfrequenz.pdf?__blob=publicationFile&v=2>
### 8.5 Other EU/EEA supervisory authorities
35. Commission Nationale de l'Informatique et des Libertés (CNIL), *Dispositifs de mesure d'audience et de fréquentation dans des espaces accessibles au public: la CNIL rappelle les règles*, 24 February 2020. <https://www.cnil.fr/fr/dispositifs-de-mesure-daudience-et-de-frequentation-dans-des-espaces-accessibles-au-public-la-cnil>
36. Conseil d'État (France), *JCDecaux France*, no. 393714, 8 February 2017, ECLI:FR:CECHR:2017:393714.20170208. <https://www.conseil-etat.fr/fr/arianeweb/CE/decision/2017-02-08/393714>
37. Autoriteit Persoonsgegevens (Netherlands), *Wifi and bluetooth* (theme page, last edited 9 April 2025). <https://www.autoriteitpersoonsgegevens.nl/en/themes/internet-and-smart-devices/wifi-and-bluetooth>
38. Autoriteit Persoonsgegevens, *Legal bases for WiFi tracking and Bluetooth tracking* (last edited 23 May 2025). <https://www.autoriteitpersoonsgegevens.nl/en/themes/internet-and-smart-devices/wifi-and-bluetooth/legal-bases-for-wifi-tracking-and-bluetooth-tracking>
39. Autoriteit Persoonsgegevens, *Boete wifitracking Enschede*, 29 April 2021, updated 29 July 2026. <https://www.autoriteitpersoonsgegevens.nl/documenten/boete-wifitracking-enschede>
40. Rechtbank Overijssel, *Enschede / Autoriteit Persoonsgegevens*, ECLI:NL:RBOVE:2024:594, 2 February 2024. <https://uitspraken.rechtspraak.nl/details?id=ECLI%3ANL%3ARBOVE%3A2024%3A594>
41. Afdeling bestuursrechtspraak van de Raad van State, *Autoriteit Persoonsgegevens / College van B&W Enschede*, ECLI:NL:RVS:2026:4403, 29 July 2026. <https://www.raadvanstate.nl/@159747/202401622-1-a3>
### 8.6 UK Information Commissioner's Office (persuasive, not binding in the EU)
42. ICO, *Anonymisation*. <https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/anonymisation/>
43. ICO, *Pseudonymisation*. <https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/anonymisation/pseudonymisation/>
44. ICO, *Privacy-enhancing technologies (PETs)*. <https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/privacy-enhancing-technologies/>
45. ICO, *Data protection impact assessments (DPIAs)*. <https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/accountability-and-governance/data-protection-impact-assessments-dpias/>
46. ICO, *Wi-Fi location analytics* (2016; no longer available at the original ICO URL). Key recommendations summarised in Bird & Bird, "ICO publishes new guidance on Wi-Fi location analytics" (2016). <https://www.twobirds.com/en/insights/2016/uk/ico-publishes-new-guidance-on-wifi-location-analytics>
### 8.7 Technical and academic sources
47. Code for Heilbronn e.V., *Passantenzahlen messen mit LoRa*, 18 September 2025. <https://codeforheilbronn.de/blog/2025/09/passantenzahlen-messen-mit-lora/>
48. *ESP32-Paxcounter* v3.8.0, open-source repository; see also the project's legal note, <https://paxcounter.org/legalnote/>. <https://github.com/cyberman54/ESP32-Paxcounter>
49. *libpax*, counting library extracted from Paxcounter. <https://github.com/dbinfrago/libpax>
50. PulseCast project, *Source-code audit: how the reference PAX counter actually counts*, research note `09-paxcounter-code-audit.md`, 14 November 2026.
51. L. Sweeney, "k-anonymity: A model for protecting privacy", *International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems* 10(5), 2002, pp. 557–570. DOI: 10.1142/S0218488502001648.
52. C. Dwork and A. Roth, "The Algorithmic Foundations of Differential Privacy", *Foundations and Trends in Theoretical Computer Science* 9(3–4), 2014, pp. 211–407. DOI: 10.1561/0400000042.
53. J. Martin, T. Mayberry, C. Donahue, L. Foppe, L. Brown, C. Riggins, E. C. Rye, and D. Brown, "A Study of MAC Address Randomization in Mobile Devices and When it Fails", *Proceedings on Privacy Enhancing Technologies* 2017(4), pp. 365–383. DOI: 10.1515/popets-2017-0054.
54. J. K. Becker, D. Li, and D. Starobinski, "Tracking Anonymized Bluetooth Devices", *Proceedings on Privacy Enhancing Technologies* 2019(3), pp. 50–65. DOI: 10.2478/popets-2019-0036.
55. M. Vanhoef, C. Matte, M. Cunche, L. S. Cardoso, and F. Piessens, "Why MAC Address Randomization is not Enough: An Analysis of Wi-Fi Network Discovery Mechanisms", *Proceedings of the 11th ACM Asia Conference on Computer and Communications Security (AsiaCCS 2016)*, pp. 413–424. DOI: 10.1145/2897845.2897860.
56. Apple, *Wi-Fi privacy* (Platform Security Guide). <https://support.apple.com/guide/security/secb9cb3140c/web>
57. Android Open Source Project, *MAC randomization*. <https://source.android.com/docs/core/connect/wifi-mac-randomization>
---
*End of note. This is a research note, not legal advice. Before deployment, obtain a written German law opinion on § 25 TDDDG, a DPIA sign-off, and a review of the actual firmware's hash length, key management, and cache behaviour.*
+739
View File
@@ -0,0 +1,739 @@
# Google Maps Traffic/ETA and Mobility Data Products: How They Work, What Is Verified, and Privacy-Preserving Alternatives
**Research memo for the sensing-layer paper**
Prepared: 2026-11-14
Scope: Google Maps traffic and ETA; terminology; privacy claims and limits; commercial and public alternatives; implications for a first-party, privacy-first probe-data architecture.
**Evidence labels used throughout**
| Label | Meaning |
|---|---|
| **Verified** | Stated in a primary source (Google documentation, standard, peer-reviewed paper) and consistent across sources. |
| **Company claim** | Published by the vendor but not independently audited or reproducible from public data. |
| **Not documented** | No public evidence found in the sources reviewed. Absence of evidence, not proof of absence. |
| **Misconception** | Contradicted by the available primary sources, or a category error. |
> **One-sentence thesis.** Google Maps is best understood as an **aggregated probe-data and map-matching system** that fuses device-side positioning with historical baselines and machine-learned ETA prediction; its public documentation does not describe it as a system that triangulates every handset from cell towers, and its public privacy story is stronger on data minimisation and retention than on formal, auditable guarantees for the traffic layer.
**Key findings**
1. **As publicly described, Google Maps traffic is a probe-data product, not a triangulation product.** Live traffic comes from aggregate location data from people navigating; historical patterns and a graph neural network produce ETAs [Google-Maps101; Google-NavData; Google-ETA-GNN].
2. **Positioning is fused on the device.** GPS, Wi-Fi, cell, and sensors are combined; GLA builds crowdsourced Wi-Fi/cell maps with a rotating device ID [Google-LocationData; Google-GeolocationAPI].
3. **Timeline is separate and off by default.** It is now on-device with an optional encrypted backup and a three-month default auto-delete; live traffic does not require it [Google-Timeline2023; Google-TimelinePrivacy; Google-BusyAreas].
4. **The strongest privacy claim is narrow.** Google claims differential privacy for busyness/popular times; no reviewed document claims DP for the traffic layer, and no ε is published [Google-BusyAreas].
5. **"Anonymized" is not a formal guarantee.** Aggregation leaks membership; k-anonymity has well-known limits; membership inference on aggregate location data is feasible [Pyrgelis-2018; Sweeney-2002; deMontjoye-2013].
6. **Alternatives are complementary, not drop-in ground truth.** HERE, TomTom, INRIX, Waze, Apple, telco, Placer/SafeGraph/Unacast, smart cards, and fleet data all have coverage and bias limits [HERE-RTT; TomTom-Prediction; SafeGraph-Bias; Vodafone-Cambridge].
7. **Penetration is the key unknown.** Published work suggests 2–3% probe penetration can be enough for velocity, 3.5% for 5-minute flow, and 0.2% for hourly flow — but penetration is non-uniform and usually unknown to the data user [Herrera-2009; Seo-2015; Luo-2026].
8. **For our architecture:** opt-in, on-device aggregation, local DP, secure aggregation, a k-threshold, no raw trajectories, and bounded retention. Claim "privacy-enhanced", not "anonymous" [Prio-2017; DAP-2026; RAPPOR-2014].
---
## 1. How it works
### 1.1 The stack in one page
```text
Device-side positioning
GPS (satellite ranging) ─┐
Wi-Fi (AP scan / RTT) ─┼─► fused location estimate (OS / GLA)
Cell (cell-ID / timing) ─┤
Sensors (IMU, barometer)─┘
│
▼
Probe sample while navigating
(lat, lon, speed, heading, accuracy, timestamp, route context)
│
▼
Map matching to road segments
(snap noisy points to the road graph; reject outliers)
│
▼
Aggregation over users, segments, time bins, direction
(robust speed/travel-time statistics; sample counts)
│
├──► live traffic layer (green / orange / red)
├──► historical baseline (road × time-of-day × day-of-week)
└──► ML ETA model (live + historical + incidents + road attributes)
```
Google's public description is consistent across its consumer blog, help pages, and developer documentation: live traffic comes from **aggregate location data from people navigating with Google Maps**, and ETA prediction combines that live signal with **historical traffic patterns** and machine learning [Google-Maps101; Google-NavData; Google-RMI]. The exact sampling rate, penetration, map-matching algorithm, aggregation threshold, and per-road coverage are not public.
### 1.2 Positioning: GPS, Wi-Fi, cell, and sensors
Google's location stack does not rely on a single technology. Google states that Android estimates location using **GPS, sensors (accelerometer, gyroscope, magnetometer, barometer), mobile network signals, and Wi-Fi signals**, fused to produce the most accurate estimate available to apps [Google-LocationData]. Google Maps Help gives user-facing accuracy expectations: GPS is described as accurate to about **20 metres**, Wi-Fi helps where GPS is weak (especially indoors), and cell towers can be accurate only to **a few thousand metres** [Google-LocationAccuracy].
The important architectural point is that these are **device-side or OS-side positioning inputs**. In the public model, Maps receives a fused location estimate; there is no public description of Maps reconstructing a handset's position from raw tower angles. The Google Geolocation API makes the fallback explicit: it accepts cell-tower and Wi-Fi access-point observations (including Wi-Fi MAC address and signal strength) and returns latitude, longitude, and an **accuracy radius** [Google-GeolocationAPI]. That is a server-side positioning service for devices without built-in geolocation; it is not evidence that Maps traffic is computed by tower triangulation.
Two distinctions matter for the paper:
- **GPS is trilateration, not triangulation.** A GPS receiver measures signal travel time to several satellites, converts time to range, and solves for position; four satellites are needed to resolve three spatial coordinates plus receiver clock error [GPSgov-Trilateration; NOAA-Geodesy]. The common phrase "GPS triangulation" is technically wrong.
- **Wi-Fi positioning is usually fingerprinting or multilateration, not triangulation.** Ordinary Wi-Fi positioning matches the observed set of access-point identifiers and signal strengths against a database of known access points. Wi-Fi RTT (round-trip time) measures distances to three or more access points and uses **multilateration**, typically accurate to 1–2 metres under supported conditions [Android-RTT]. Neither is angle-based triangulation.
Google also operates **Google Location Accuracy (GLA)**, formerly Google Location Services. When GLA is on, Android periodically collects GPS and information about Wi-Fi access points, mobile networks, and device sensors using a **temporary rotating device identifier** that is not associated with a specific person. Google says it uses this to improve location accuracy and to build **crowdsourced maps of Wi-Fi access points and mobile network towers** [Google-LocationData]. This is one of the clearest public descriptions of the "Google has a database of Wi-Fi and cell locations" claim. It is still not the same as passive sniffing of nearby people's device MAC addresses: the object being mapped is the **infrastructure** (access points, towers), and the identifier used for the device's own reports rotates.
### 1.3 Probe data and map matching
A **probe** is a position/speed report from a participating device or vehicle. In traffic engineering, the same idea is called **floating car data (FCD)**: vehicles act as moving sensors, and their trajectories are map-matched to the road network to estimate link speeds and travel times [TomTom-API; HERE-Probe]. Google's consumer description uses the phrase "aggregate location data" rather than "floating car data", but the logic is the same.
Google's navigation-data help page is unusually specific about what is collected during navigation [Google-NavData]:
- **GPS location**;
- **navigation details**, such as the route taken;
- **sensor data** from the device, such as the barometer.
Collection starts shortly after the user taps **Start** (or begins moving with glanceable directions on) and stops after arrival. Google says it uses this data "by itself or combined with others" for navigation, alternate routes, and real-time updates such as traffic, disruptions, and weather [Google-NavData].
The public description stops short of the pipeline details that matter for reproducibility:
- How often samples are taken (the 2009 launch described anonymous speed updates; current docs do not give a cadence).
- How many probes are needed before a road segment is published.
- How probes are filtered for outliers, stops, mode (driving vs cycling vs transit), and GPS error.
- How map matching handles parallel roads, tunnels, parking garages, and urban canyons.
- Whether probes from Android Auto, iOS, Waze, or third-party Maps Platform customers are pooled, and with what weighting.
These are **not documented** in the sources reviewed. That absence is itself a finding: Google publishes the product behaviour, not the measurement methodology.
### 1.4 Live traffic, historical traffic, and ETA
Google separates three related products:
1. **Live traffic.** Current conditions on road segments, rendered as green ("no traffic delays"), orange ("medium amount of traffic"), red ("traffic delays"), and darker red for slower speeds [Google-TrafficLegend]. Google's 2020 explanation says this is powered by aggregate location data from people navigating [Google-Maps101].
2. **Historical / typical traffic.** Per-road patterns by time of day and day of week. Google says it combines a historical traffic database with live conditions using machine learning, and that since COVID-19 it prioritises patterns from the **last two to four weeks** while deprioritising older patterns [Google-Maps101].
3. **ETA prediction.** The estimated travel time for a route, combining live conditions, historical patterns, road attributes (paved/unpaved, size, directness), and incidents. Google's 2020 blog claims ETA predictions are "consistently accurate for over 97% of trips" and that a DeepMind partnership reduced inaccurate ETAs by up to 50% in some cities [Google-Maps101; DeepMind-2020]. The peer-reviewed CIKM paper describes a graph neural network over "supersegments" (connected road subgraphs) deployed in production, with significant reductions in negative ETA outcomes (40%+ in Sydney) [Google-ETA-GNN].
Google's developer documentation adds a useful rule of thumb: **short routes depend largely on current real-time information; longer routes use AI modelling where nearby segments are predicted from real-time data and more distant segments rely more heavily on historical patterns; roads with limited real-time signals rely more heavily on historical data** [Google-RMI]. That is a sensible engineering statement, and it implies that "live traffic" coverage is not uniform: some roads are data-rich, others are model-filled.
Incidents come from multiple sources: user reports in Maps, Waze reports (Waze says its alerts reach both Waze and Google Maps drivers), and authoritative feeds from road operators [Google-Maps101; Waze-Cities; Google-RMI]. This fusion is why Google Maps can show a crash or closure even when probe speeds alone would be ambiguous.
### 1.5 Timeline, Popular Times, and Busy Areas
Google's **Timeline** (formerly Location History) is a separate, opt-in product. It is **off by default**, and when enabled it saves precise device location regularly — including when Google apps are not in use — to build a personal map of visits and routes [Google-TimelinePrivacy; Google-TimelineManage]. Since the December 2023 announcement, Timeline is designed to be saved **on the device**, with an optional end-to-end encrypted cloud backup; the default auto-delete period was reduced to **three months**, with 18- and 36-month options still available [Google-Timeline2023; Google-TimelineManage].
**Popular Times, wait times, and visit duration** are computed from aggregated and anonymized data from users who have opted in to Timeline. Google says the data is shown only when there are enough visits, and it explicitly claims **differential privacy** for busyness data; it also says it does not calculate busyness for residential places and suppresses busyness when it lacks enough data [Google-BusyAreas]. The same help page describes **Busy Areas** as an aggregate of live busyness trends from places in a predefined area of interest, never the exact location of an individual [Google-BusyAreas].
There is a documented tension here. Google's location-data page says Timeline data can be used to "show community trends, such as popular times and transit crowdedness, based on aggregated location data" [Google-LocationData], while the Timeline help page says Timeline is now saved on the device [Google-TimelinePrivacy; Google-TimelineManage]. Google does not publicly specify how on-device Timeline data and aggregated community trends are reconciled, or what leaves the device and under which settings. That is a legitimate question for the paper, but it should be framed as an **ambiguity in public documentation**, not as proof that raw trajectories are uploaded.
Also note the separation: Google's live-traffic description relies on **navigation data**, not on Timeline. Timeline is used for busyness and personal history; it is not required for the traffic layer [Google-NavData; Google-BusyAreas]. This matters because critics often conflate "Location History" with "traffic data collection". They are different settings with different defaults.
### 1.6 What Google actually does, step by step
A defensible summary for the paper:
1. The device/OS fuses GPS, Wi-Fi, cell, and sensor signals into a location estimate with an accuracy radius [Google-LocationData; Google-GeolocationAPI].
2. While the user navigates, Maps collects GPS, route, and sensor data and stops at arrival [Google-NavData].
3. Google map-matches probe samples to road segments and aggregates speeds/travel times across users and time bins [Google-Maps101; Google-RMI].
4. Live segment speeds feed the traffic layer; historical patterns are learned per road and time period; the two are combined by ML for ETA [Google-Maps101; DeepMind-2020; Google-ETA-GNN].
5. Incidents from users, Waze, and road operators are fused into routing and ETA [Google-Maps101; Waze-Cities; Google-RMI].
6. Separately, opt-in Timeline data supports personal history and, in aggregate, Popular Times/Busy Areas with a DP claim [Google-TimelinePrivacy; Google-BusyAreas].
That is the publicly described model. Everything below the line — sample counts, thresholds, weighting, error models, DP parameters — is either proprietary or unpublished.
---
## 2. Terminology: triangulation, trilateration, multilateration, fingerprinting, probe data
### 2.1 Definitions
| Term | What is measured | How position is obtained | Typical example | Common misuse |
|---|---|---|---|---|
| **Triangulation** | Angles (bearings) from known points | Intersect bearing lines | Surveying with a theodolite; classical geodetic networks | Calling any multi-signal positioning "triangulation" |
| **Trilateration** | Distances (ranges) from known points | Intersect distance circles/spheres | GPS/GNSS; some terrestrial ranging systems | Calling GPS "triangulation" |
| **Multilateration** | Multiple range or range-difference measurements | Solve for the point that best fits all measurements | Wi-Fi RTT with 3+ APs; TDOA systems; some cellular positioning | Confusing it with triangulation or with fingerprinting |
| **Fingerprinting** | A radio "signature" (e.g., set of Wi-Fi BSSIDs and RSSI values) | Match the observed signature to a pre-built database of known locations | Google Geolocation API; Wi-Fi positioning | Treating it as geometric triangulation |
| **Cell-ID positioning** | Serving cell / tower identity | Use the known or estimated location of the cell | Coarse mobile location | Assuming tower-level accuracy is metres |
| **Probe data** | Positions, speeds, headings, timestamps from participating devices/vehicles | Map-match and aggregate; not a positioning method itself | Floating car data; Google Maps navigation data; HERE/TomTom probes | Calling probe data "triangulation" |
| **Map matching** | A noisy point sequence plus a road graph | Snap points to the most plausible road segment/path | Traffic speed estimation; routing | Assuming every probe is correctly matched |
| **Aggregation** | Many probe records | Combine into segment × time-bin statistics | Traffic layer; heatmaps; demand indices | Assuming aggregation is a formal privacy guarantee |
Sources for the geometric definitions: NOAA's geodetic tutorial defines **triangulation** as measuring angles in a chain of triangles, and **trilateration** as measuring the lengths of triangle sides [NOAA-Geodesy]. GPS.gov describes GPS positioning as **trilateration** from satellite ranges [GPSgov-Trilateration]. Android's Wi-Fi RTT documentation describes **multilateration** from distances to three or more access points [Android-RTT]. Google's Geolocation API documentation describes cell-tower and Wi-Fi access-point inputs and an accuracy radius, which is consistent with fingerprinting/weighted positioning rather than angle-based triangulation [Google-GeolocationAPI].
### 2.2 What Google actually does vs the common misconception
| Misconception | What the sources actually support |
|---|---|
| "Google Maps triangulates every phone from cell towers." | Google fuses GPS, Wi-Fi, cell, and sensors on the device, and uses **aggregate location data from people navigating** for traffic [Google-LocationData; Google-Maps101]. No reviewed Google document describes a tower-triangulation pipeline for Maps traffic. |
| "Google Maps traffic requires Location History." | Live traffic is described as coming from navigation data; Timeline is a separate opt-in setting used for personal history and busyness [Google-NavData; Google-BusyAreas]. |
| "Incognito mode stops Google from using you for traffic." | Google's Incognito documentation says it stops saving search/navigation history, updating Timeline, and personalizing Maps; it does **not** state that anonymous navigation/traffic measurement stops [Google-Incognito]. |
| "Google uses Bluetooth MAC sniffing to count cars." | No reviewed Google documentation describes passive Bluetooth or client-MAC sniffing as a Maps traffic source. Google's documented radio positioning uses **Wi-Fi access-point** observations and cell-tower data [Google-GeolocationAPI; Google-LocationData]. That is infrastructure-side fingerprinting, not passive counting of nearby people's device MACs. |
| "Anonymized means differentially private." | Google uses generalization/k-anonymity and DP as distinct techniques in its general anonymization page [Google-Anonymization]. DP is explicitly claimed for busyness data [Google-BusyAreas], but no reviewed document claims DP for the traffic layer. |
| "More probes always mean better traffic." | Probe penetration is spatially and temporally non-uniform; accuracy depends on sample size, map matching, and road coverage, not just volume [Luo-2026; Seo-2015; Mazare-2011]. |
The most important correction for our paper: **as publicly described, Google's traffic product is a probe-data product, not a triangulation product.** The radio-positioning layer exists, but it is an input to the device's location estimate. The traffic layer is built from map-matched movement observations and historical models. If we criticise "triangulation" as a sensing layer, we should be precise about which layer we mean.
### 2.3 Recommended terminology for our paper
- Use **"aggregated probe data with map matching"** for our proposed sensing layer. This describes what we actually collect and how we turn it into road/area statistics.
- Use **"multi-receiver multilateration"** only if we estimate positions from ranges or range differences measured by multiple receivers.
- Use **"fingerprinting"** if we match observed Wi-Fi/Bluetooth signatures to known locations.
- Use **"triangulation"** only when angles are measured. If we use it colloquially, define it explicitly and note that GPS is trilateration.
- Avoid **"anonymous"** unless we have a formal guarantee and an audit. Prefer **"privacy-enhanced"**, **"aggregated with a k-threshold"**, or **"local differential privacy with a stated ε"**.
- Avoid **"counts people"**. Use **"estimated devices"** and state that device penetration is unknown. Multiple devices per person and shared devices are both real.
A useful sentence for the paper:
> Google's public documentation does not describe traffic as computed by triangulating handsets from cell towers; it describes fusing device-side location signals and aggregating map-matched probe trajectories from people who navigate with the app. The relevant privacy question is therefore not "can Google locate a handset?" but "what is retained, at what granularity, for how long, and under which formal guarantee?"
---
## 3. What is verified vs claimed
### 3.1 Claim-by-claim assessment
| # | Claim | Status | Evidence | Caveat |
|---|---|---|---|---|
| 1 | Google Maps live traffic uses aggregate location data from people navigating | **Verified** | Google's 2020 Maps 101 post and navigation-data help page [Google-Maps101; Google-NavData] | Google does not publish sample counts, penetration, or per-road coverage. |
| 2 | Google combines live traffic with historical patterns using ML | **Verified** | Google Maps 101; DeepMind blog; ETA GNN paper [Google-Maps101; DeepMind-2020; Google-ETA-GNN] | The relative weight of live vs historical is not public. |
| 3 | Google uses a graph neural network for ETA | **Verified** | CIKM 2021 paper describes production deployment [Google-ETA-GNN] | The paper reports aggregate improvements, not per-city production metrics. |
| 4 | Google's ETA predictions are accurate for "over 97% of trips" | **Company claim** | Google Maps 101 [Google-Maps101] | "Accurate" is undefined; no independent audit or error distribution is published. |
| 5 | DeepMind reduced inaccurate ETAs by up to 50% in some cities | **Company claim** | Google Maps 101; DeepMind blog [Google-Maps101; DeepMind-2020] | City-specific, not a global guarantee; the baseline and metric are not fully specified. |
| 6 | Google collects GPS, route, and sensor data during navigation | **Verified** | Navigation-data help page [Google-NavData] | Collection starts at Start and stops at arrival; the page does not give a sampling cadence. |
| 7 | Google uses Wi-Fi and cell signals for location | **Verified** | Location-data page; Geolocation API docs [Google-LocationData; Google-GeolocationAPI] | This is positioning, not proof that traffic uses raw Wi-Fi/cell observations. |
| 8 | Google builds crowdsourced maps of Wi-Fi APs and cell towers with a rotating device ID | **Verified** | Location-data page [Google-LocationData] | The object mapped is infrastructure; the device identifier rotates. |
| 9 | Google uses differential privacy for Maps busyness data | **Verified as a Google statement** | Busy Areas help page [Google-BusyAreas] | No ε, sensitivity, composition, or independent audit is published. |
| 10 | Google uses differential privacy for the traffic layer | **Not documented** | No reviewed Google document claims this | Absence of evidence; do not assert either way without a source. |
| 11 | Google describes traffic data as aggregated/anonymized | **Company claim** | Google uses "aggregated and anonymized" language [Google-Maps101; Google-BusyAreas] | "Anonymized" is not a formal privacy model; aggregation can leak membership [Pyrgelis-2018]. |
| 12 | Google deletes navigation data after processing | **Company claim** | Navigation-data help page [Google-NavData] | No timeframe, audit, or backup-retention detail is given for this specific data. |
| 13 | Google permanently deletes trip start/end points | **Historical company claim** | 2009 launch post and contemporaneous coverage [Google-2009; Google-AU-2009] | A 2009 statement; current docs say "after processing, it's deleted" without a timeframe. |
| 14 | Timeline is on-device and off by default | **Verified** | 2023 blog; Timeline help pages [Google-Timeline2023; Google-TimelinePrivacy; Google-TimelineManage] | Cloud backup is optional; other settings (Web & App Activity, ads) can still save location. |
| 15 | Incognito stops traffic contribution | **Not documented** | Incognito help page lists only account-history/personalization effects [Google-Incognito] | Do not claim it does or does not; say Google does not document it. |
| 16 | Google uses Bluetooth MAC sniffing for traffic | **Misconception / not documented** | No reviewed Google source describes it | Do not attribute MAC sniffing to Google Maps. |
| 17 | Google triangulates handsets from cell towers for traffic | **Misconception** | Public model is fused device positioning + aggregate probes [Google-LocationData; Google-Maps101] | Tower data is one positioning input, not the traffic algorithm. |
| 18 | Google Maps traffic coverage is global | **Company claim, underspecified** | Maps is described as covering 220+ countries; traffic-layer coverage is not published [Google-Maps101] | Country coverage of Maps ≠ traffic-layer coverage of every road. |
### 3.2 What Google does not disclose
The following are **not documented** in the public sources reviewed. They are the questions a rigorous paper should ask rather than answer by assumption:
- **Penetration and sample size** per road segment and time bin.
- **Minimum probe threshold** before a segment is published.
- **Map-matching and outlier filters**, including how stops, mode, and GPS jumps are handled.
- **Whether Wi-Fi/cell observations contribute directly to traffic probes**, or only to the device's location estimate.
- **DP parameters** (ε, δ, sensitivity, composition) for busyness, and whether traffic uses any formal DP mechanism.
- **Retention period** for navigation data beyond "after processing".
- **Traffic-layer coverage** by country, road class, and time of day.
- **Independent audits** of accuracy or privacy claims.
This is not a reason to accuse Google of wrongdoing. It is a reason to treat its published claims as **company claims** and to design our own system so that these parameters are explicit.
### 3.3 A note on the 2009 launch
Google's 2009 announcement is the origin of its "crowdsourced traffic" story. It described GPS-enabled phones with **My Location** enabled sending anonymous speed and location data; Google combined thousands of such reports into live traffic. Google also said it found the start and end points of every trip and **permanently deleted** that data so that even Google would not have access to it [Google-2009; Google-AU-2009]. That is a strong early privacy claim, and it is worth citing as historical context. But it is a **2009 company statement**, not a current technical specification. Current documentation uses softer language ("after processing, it's deleted") and does not publish a retention period [Google-NavData].
---
## 4. Privacy model
### 4.1 What Google says it does
Google's main Privacy Policy is the umbrella document; the location-data page provides Maps-specific detail [Google-PrivacyPolicy; Google-LocationData].
| Mechanism | Where Google states it | What it is meant to protect | What it does not establish |
|---|---|---|---|
| **Aggregation** | Traffic and busyness descriptions [Google-Maps101; Google-BusyAreas] | Individual probe is hidden in a group | Membership inference, small-cell leakage, or bias |
| **Anonymization / pseudonymization** | Location-data and anonymization pages [Google-LocationData; Google-Anonymization] | Removes or generalizes identifiers | A formal guarantee; "anonymous" is undefined |
| **Generalization → k-anonymity, l-diversity** | Anonymization page [Google-Anonymization] | Re-identification via quasi-identifiers is harder | Homogeneity, background knowledge, composition attacks |
| **Differential privacy** | Busy Areas help page [Google-BusyAreas] | Membership inference with a mathematical guarantee | Utility loss; no ε published; scope limited to busyness |
| **Rotating identifiers** | GLA description [Google-LocationData] | Limits long-term linkability of device reports | Linkability within a session; other identifiers |
| **On-device Timeline** | 2023 blog; Timeline help [Google-Timeline2023; Google-TimelinePrivacy] | Keeps personal history off Google servers by default | Backups, other settings, device compromise |
| **Retention / auto-delete** | Timeline and retention pages [Google-TimelinePrivacy; Google-Retention] | Limits long-term accumulation | Data already used; backups; derived aggregates |
| **Incognito mode** | Maps help [Google-Incognito] | Stops account history and personalization | Does not document stopping anonymous traffic measurement |
| **Location data collection toggle** | iOS Maps help [Google-LocationToggle] | Allows opting out of anonymous location collection | Platform-level location services may still operate |
One nuance: Google's navigation-data page says it does **not associate map updates with the user's Google Account or device**, but it also lists data that may be account-associated — direction searches, ads shown during navigation, and Gemini activity [Google-NavData]. The privacy boundary is therefore **per data type**, not per user. "Navigation data is not associated with your account" does not mean "no location data is associated with your account."
### 4.2 Aggregation, k-anonymity, and DP are not the same thing
This is the core conceptual distinction for the paper.
- **Aggregation** combines many records. It is a data-processing step, not a privacy guarantee. If a cell has one dominant contributor, or if an adversary can compare releases with and without a target, aggregation can leak membership. Pyrgelis et al. show that membership inference on aggregate location time-series is feasible, and that some defences fail or impose large utility costs [Pyrgelis-2018]. De Montjoye et al. show that four random spatio-temporal points can uniquely characterise 95% of individuals in a coarse mobility dataset [deMontjoye-2013]. Aggregation raises the cost of re-identification; it does not make it impossible.
- **k-anonymity** requires each released record to be indistinguishable from at least k−1 others on the quasi-identifiers [Sweeney-2002]. It protects against direct re-identification only if the quasi-identifier set and the adversary's background knowledge are modelled correctly. It does not protect against attribute disclosure, homogeneity, or differencing attacks. Google's general anonymization page describes k-anonymity and l-diversity as techniques it uses [Google-Anonymization]; Google's search location handling uses a concrete spatial threshold — an area of at least **3 km² or 1,000 people** — to avoid revealing a precise search location [Google-LocationData]. That is a useful precedent for a spatial k-threshold, but it is not a statement about traffic data.
- **Differential privacy (DP)** requires the output distribution to change by at most a bounded factor when one individual's data is added or removed [Dwork-2006; Dwork-Roth-2014]. DP provides a formal membership guarantee and composes across releases, but it requires calibrated noise, a stated ε/δ, and a privacy budget. **Local DP** randomizes on the device before transmission, so the server never sees the raw value [RAPPOR-2014]. **Central DP** assumes a trusted curator. **Secure aggregation** is complementary: it hides individual contributions cryptographically, but does not add noise, so it does not by itself provide DP [Prio-2017; Bonawitz-2017; DAP-2026].
Google's public claims mix these categories. The busyness page says "differential privacy", which is the strongest claim [Google-BusyAreas]. The traffic pages say "aggregated" and "anonymized", which are weaker and undefined [Google-Maps101; Google-NavData]. A paper that treats "Google says it anonymizes traffic" as equivalent to "Google has a DP guarantee for traffic" would overstate the evidence.
### 4.3 Retention: what is actually documented
| Data | Documented retention | Source |
|---|---|---|
| Timeline (Location History) | Auto-delete after **3, 18, or 36 months**; default reduced to **3 months** in the 2023 update | [Google-TimelinePrivacy; Google-TimelineManage; Google-Timeline2023] |
| Web & App Activity location | **30 days** for IP-based and device location | [Google-LocationData; Google-Retention] |
| Navigation data | "After processing, it's deleted" — **no timeframe published** | [Google-NavData] |
| Deleted account data | Deletion generally takes **~2 months**; backups may retain for up to **6 months** | [Google-Retention] |
| Advertising server logs | Partial IP removed after **9 months**; cookie information after **18 months** | [Google-Retention] |
| TfL Oyster journey history (comparison) | **8–9 weeks**, then disassociated from the card | [TfL-Access] |
| TfL contactless journey history (comparison) | **13 months** | [TfL-Access] |
The retention picture is mixed. Google has improved consumer controls for Timeline (on-device, shorter default), but the traffic/navigation pipeline is still described in vague terms. For a privacy-first architecture, "deleted after processing" is not a specification; we should be able to state exactly when raw data is destroyed, what derived aggregates remain, and how long they are retained.
### 4.4 Where the privacy story is weaker than the marketing
1. **"Anonymized" is doing a lot of work.** Google uses the word for traffic and busyness, but does not publish the de-identification method, the quasi-identifier set, the k threshold, or a re-identification test. Under GDPR-style analysis, pseudonymized data remains personal data; anonymization is a high bar [ISO-20889; NIST-800-188].
2. **In the Maps material reviewed, DP is claimed only for busyness.** That is a meaningful claim, but it is narrow. There is no public ε, no sensitivity analysis, and no public independent audit. A DP claim without parameters is not reproducible.
3. **Aggregation is not a guarantee.** The membership-inference literature shows that aggregate location time-series can leak whether a target contributed [Pyrgelis-2018]. Small cells, regular routines, and differencing across time are the attack surfaces.
4. **On-device Timeline is a real improvement, but the boundary is fuzzy.** If Timeline data is on-device, how does it support aggregated community trends? Google's own pages say both things without reconciling them [Google-TimelinePrivacy; Google-LocationData]. The answer may be backups, separate settings, or aggregated on-device computation; Google does not say.
5. **Incognito is not a traffic opt-out.** It stops account history and personalization, not necessarily anonymous measurement [Google-Incognito]. Users may reasonably assume otherwise; the documentation should be clearer.
6. **No public independent audit.** Accuracy and privacy claims are self-reported. That is normal for commercial maps, but it means the paper should label them as claims.
### 4.5 A checklist for distinguishing marketing from verification
When a vendor says... ask:
| Phrase | Question that turns it into evidence |
|---|---|
| "Anonymous" | Which formal model? k-anonymity, DP, or just de-identified? What is k or ε? Has an independent party tested re-identification? |
| "Anonymized" | Is the data irreversibly de-identified, or pseudonymized? Can it be joined back to an identifier? |
| "Differential privacy" | Local or central? What ε/δ? What sensitivity? How is the budget composed across releases? |
| "On-device" | What leaves the device? Are there backups? Are aggregates uploaded? Under which settings? |
| "Opt-in" | Is it default-off? Is consent granular (traffic vs ads vs research)? Can it be withdrawn? |
| "Deleted" | Deleted from which systems, when, and from backups? Are derived aggregates retained? |
| "Aggregated" | What is the spatial/temporal bin? What is the minimum count? Are small cells suppressed? |
| "Real-time" | What is the update cadence, latency, and coverage? Is it live data or a model? |
These questions are also the specification for our own system.
---
## 5. Alternatives: other mobility-data products
### 5.1 Comparison table
| Source / product | What it measures | Primary data source | Coverage / resolution | Accuracy and bias notes | Documented privacy posture |
|---|---|---|---|---|---|
| **Google Maps traffic/ETA** | Segment speeds, travel times, ETAs, incidents | Aggregate location data from people navigating; GPS/Wi-Fi/cell/sensor positioning; historical patterns; DOT/Waze incidents [Google-Maps101; Google-NavData; Google-RMI] | Global consumer Maps; traffic-layer coverage not published; per-road segments | 97% ETA claim is company-reported; penetration unknown; urban/device bias | Aggregated/anonymized; DP claimed for busyness, not traffic; Timeline on-device; retention controls [Google-BusyAreas; Google-Timeline2023] |
| **HERE Real-Time Traffic / Traffic Analytics** | Flow speeds, congestion, incidents, historical probe statistics | Connected-car probes, vehicle sensors, fixed sensors, government feeds; 160+ auto-grade probe sources [HERE-RTT; HERE-OnePager; HERE-Probe] | 75 countries; flow updated every minute; incidents every 2 minutes; +4 h prediction [HERE-RTT; HERE-OnePager] | Multi-source fusion; confidence field distinguishes real-time vs historical vs speed-limit-only data [HERE-RTT]; fleet mix bias | Commercial; aggregated/anonymized probe data; customer contracts govern use |
| **TomTom Traffic / Traffic Stats** | Flow speeds, travel times, incidents, historical traffic stats, O/D | Floating car data from ~600 million probes: PNDs, in-dash systems, apps, fleets [TomTom-Prediction; TomTom-Stats; TomTom-API] | Global; country coverage varies by product; 5-min historical epochs; sample size and percentiles published in Traffic Stats [TomTom-API] | FCD is a non-probability sample; device/app mix bias; TomTom publishes sample counts and percentiles, which is good practice | Commercial; anonymized FCD; aggregated historical products |
| **INRIX AI Traffic** | Real-time speeds, travel times, incidents, historical analytics | Probe data, 20 years of history, "trillions of data points"; all road classes including non-TMC [INRIX] | Claims all roads, including rural and arterial; country coverage varies | "Most accurate" is a company claim; probe mix and penetration not public | Commercial; aggregated; privacy policy applies |
| **Waze for Cities** | Driver-reported incidents, closures, speeds, traffic flow | Crowdsourced GPS + driver reports; ~200 million drivers [Waze-Cities] | Strong in populated areas; weaker where few Wazers; incident-centric | Self-selected drivers; reporting bias; rural weakness noted by DOT practitioners [IowaDOT-2020] | Free data-sharing with cities; "anonymized driver speed and traffic flow data" [Waze-Cities]; Google-owned |
| **Apple Maps** | Traffic, routing, ETAs, incidents | iPhone probes; random identifiers; on-device personalization [Apple-Privacy] | Global consumer Maps; coverage not published | No public penetration or accuracy data; strong consumer privacy posture | Random identifiers that reset; location fuzzing to a less-exact area within 24 h; no history of searches/where you've been [Apple-Privacy] |
| **Telco mobility data** (Orange Flux Vision, Telefónica, Vodafone) | Population presence, flows, O/D matrices, dwell patterns | Mobile network signalling / cell-tower events [Telefonica; Orange-Flux; Vodafone-Cambridge] | National/regional; cell-tower granularity; rural areas coarser; low-density areas may be suppressed | Subscriber market-share bias; cell-level spatial error (hundreds of metres to kilometres) [Google-LocationAccuracy]; background events uneven; low-density suppression [Vodafone-Cambridge] | Aggregated/anonymized; GDPR/CNIL claims; Orange calls its process "irreversible anonymization" [Orange-Flux]; legal basis and thresholds vary |
| **Placer.ai** | Footfall, visits, trade areas, audience behavior | Panel of tens of millions of mobile devices, sourced from vetted app partners [Placer-Data] | US-focused; POI/venue level; daily refresh | Panel/app bias; extrapolation; company claims >90% correlation with ground truth [Placer-Data] | Strips MAIDs; k-anonymity of 50 (at least 50 devices); no user-level display; opt-in/opt-out required of partners [Placer-Data; Placer-Trust] |
| **SafeGraph** | POI visits, foot traffic, spend patterns | Licensed data from third-party aggregators; no SDK; opt-in device data [SafeGraph-Privacy; SafeGraph-Warren] | US visit panel; POI/census-block level | Small fraction of US population; documented spatial/temporal bias [SafeGraph-Bias]; extrapolation error | Aggregated; DP noise claimed in Patterns; sensitive-category suppression; no individual-level sale [SafeGraph-Warren] |
| **Unacast** | Footfall, visitor journeys, trade areas, demographics | Opt-in mobile app data; first-party aggregation; ML model over aggregates and contextual features [Unacast-Method; Unacast-Legal] | US-focused; POI/hexagon level; 48–72 h delay; not real-time [Unacast-Legal] | Panel bias; ML extrapolation; no real-time use | Aggregated; no real-time tracking; sensitive-place suppression (PrivacyCheck); opt-out honored [Unacast-Legal] |
| **Transit smart cards** (TfL Oyster/RODS) | Transit entries/exits, O/D flows, station counts | Fare-gate taps, Oyster/contactless, rolling O/D surveys [TfL-OpenData] | London; station/line level; 5% Oyster journey sample (2009); RODS ongoing [TfL-OpenData] | Fare-paying transit users only; no car traffic; tap-out is not universal, so some journeys are inferred/auto-completed [TfL-Oyster] | Pseudonymized journey data; Oyster history retained 8–9 weeks then disassociated; contactless 13 months [TfL-Access; TfL-Oyster] |
| **GPS fleet telematics** (Geotab/Altitude) | Vehicle movements, speeds, corridor volumes, commercial vehicle patterns | First-party telematics devices in commercial fleets; millions of vehicles [Geotab-FAQ; Geotab-Privacy] | Commercial vehicles; corridor/regional aggregates; primary-road coverage claims ~99.9% in some products [Geotab-FAQ] | Commercial fleet bias (trucks/vans, not passenger cars); route and vocation bias | Aggregated/de-identified; minimum-count thresholds; geospatial obfuscation; salted/hashed VINs; no individual vehicle data [Geotab-FAQ] |
| **First-party probe data (our proposal)** | Segment speeds, travel times, area footfall/dwell | Opt-in app/device probes; on-device binning; local DP; secure aggregation | Our deployment footprint; not city-wide unless calibrated | Penetration unknown; device ≠ person; needs ground-truth calibration | Design-dependent; can be strong if formal DP + k-threshold + no raw trajectories + retention limits |
### 5.2 Notes on coverage, accuracy, and bias
**Coverage is not the same as accuracy.** HERE, TomTom, and INRIX claim broad road coverage; Waze claims a large driver community; Google claims global Maps usage. None of these claims tells us the **probe penetration on a specific road at a specific hour**. Penetration is the key parameter for turning probe samples into population estimates, and it is usually unknown to the data user [Mazare-2011; Luo-2026].
**All commercial probe datasets are non-probability samples.** They come from people who use a particular app, drive a particular vehicle, or carry a particular device. That creates selection bias by age, income, smartphone platform, vehicle type, and geography. SafeGraph's panel has been shown to have systematic spatial and temporal bias [SafeGraph-Bias]. Placer and Unacast try to correct this with ML and ground-truth calibration, but the correction is proprietary and not independently reproducible [Placer-Data; Unacast-Method].
**Telco data is population-scale but spatially coarse.** It is attractive for O/D matrices and regional flows, not for lane-level traffic. Cell-tower granularity, subscriber market share, and low-density suppression limit its use for street-level sensing [Telefonica; Orange-Flux; Vodafone-Cambridge]. Orange's claim of "irreversible anonymization" is a legal/technical position, not an independently verified DP guarantee [Orange-Flux].
**Smart cards are excellent for transit, not a substitute for road traffic.** They provide high-quality O/D data for fare-paying transit users, but they cannot measure road traffic and they exclude non-fare users. TfL's 5% Oyster sample and RODS are good examples of aggregate release with explicit retention limits [TfL-OpenData; TfL-Access].
**Fleet data is high-quality but commercially biased.** Geotab's Altitude product comes from commercial vehicles, so it over-represents trucks, vans, and fleet routes. It is useful for freight and corridor planning, not for general passenger-car traffic [Geotab-FAQ; Geotab-Privacy].
**Waze is incident-rich and coverage-skewed.** It is strong where drivers use Waze and report incidents; it is weaker in rural or low-usage areas. DOT practitioners have noted this limitation [IowaDOT-2020]. It is also Google-owned, so it is not an independent alternative to Google.
**Apple is the closest consumer-privacy comparator.** Apple says Maps data is associated with random identifiers that reset, location is fuzzed within 24 hours, and Apple does not retain a history of searches or where you have been [Apple-Privacy]. Apple does not publish traffic coverage or penetration, so its accuracy claims are not directly comparable. It is a useful example of "privacy by architecture" language, but it is still a company claim.
### 5.3 Which source for which job?
| Job | Best-fit sources | Why |
|---|---|---|
| Live segment speeds / ETA | Google, HERE, TomTom, INRIX, Waze | Broad probe bases and mature map matching; use at least two for cross-checking |
| Historical traffic patterns | HERE, TomTom, INRIX, Google | Multi-year archives; TomTom/HERE publish sample counts/percentiles |
| Incident detection | Waze, Google, HERE, INRIX | Crowd reports + operator feeds; Waze is incident-centric |
| Regional O/D / population flows | Telco (Orange, Telefónica, Vodafone) | Population-scale signalling data; coarse spatial resolution |
| Retail footfall / trade areas | Placer, SafeGraph, Unacast | Panel-based visit estimates; calibrate with store counts |
| Transit O/D | TfL/agency smart-card data | High-quality mode-specific data; not general traffic |
| Freight / commercial vehicle flows | Geotab/Altitude, HERE, TomTom | Fleet telematics; commercial bias |
| Our own venues / corridors | First-party probe + ground counters | Control over consent, aggregation, retention, and calibration |
---
## 6. From aggregates to heatmaps and demand forecasts
### 6.1 Spatial and temporal binning
A probe dataset becomes a heatmap or demand index through four operations:
1. **Map matching** (for road data): snap each point to a road segment and direction.
2. **Spatial binning**: use road segments for traffic, or a hierarchical grid for area data. H3 is a common open-source hexagonal hierarchy; it supports multi-resolution aggregation and stable joins across datasets [H3]. Geohash or administrative boundaries are alternatives.
3. **Temporal binning**: 1–5 minutes for real-time traffic, 15 minutes for operational dashboards, 1 hour for demand forecasts. The bin size must be large enough to satisfy the k-threshold and small enough to be useful.
4. **Statistic choice**: count, median/mean speed, travel time, dwell time, or a normalized index. Use robust statistics (median, trimmed mean) and publish sample counts/uncertainty.
For footfall, the raw output is usually **estimated devices per bin**, not people. A device may be a phone, tablet, or vehicle; one person may carry multiple devices; a device may be shared. The paper should say "estimated devices" at least once and explain why.
### 6.2 Expansion and bias correction
There are three ways to turn a sample into a population estimate:
- **Design-based expansion**: weight each device by the inverse of its selection probability. This requires knowing the panel design and penetration, which commercial providers rarely publish.
- **Calibration**: adjust the sample so that its marginal totals match known ground truth (traffic counts, transit entries, store door counts, population). This is the most defensible approach for a first-party deployment.
- **Model-based correction**: train a model to predict ground truth from sample features (device counts, dwell time, POI attributes, weather, time). Placer, SafeGraph, and Unacast all do some version of this [Placer-Data; SafeGraph-Bias; Unacast-Method].
The trap is that model-based correction can hide bias rather than remove it. If the panel under-represents a neighbourhood, a model trained on the same panel will learn the wrong baseline. Calibration against independent counts is essential.
### 6.3 Forecasting
A demand forecast typically combines:
- **Historical baseline**: same hour/day-of-week/season, robust to outliers.
- **Live signal**: current probes, lagged and smoothed.
- **Context**: weather, holidays, events, school terms, roadworks.
- **Model**: gradient boosting, temporal fusion, graph neural networks, or a simple seasonal model with prediction intervals.
Google's ETA system is a production example of graph-based spatio-temporal forecasting [Google-ETA-GNN; DeepMind-2020]. TomTom and HERE offer predictive traffic products with historical and real-time inputs [TomTom-Prediction; HERE-RTT]. For footfall, Placer and Unacast produce visit estimates and trends, not necessarily forward forecasts [Placer-Data; Unacast-Method].
Two rules for our paper:
1. **Report uncertainty.** A point forecast without an interval is not decision-grade. Use quantile or conformal intervals, and validate coverage.
2. **Separate nowcast from forecast.** A 15-minute "forecast" that is mostly the current observation is a nowcast. Say so, especially if the data feed is replay or batch.
### 6.4 Failure modes
| Failure mode | What happens | Mitigation |
|---|---|---|
| **Penetration bias** | Probe share varies by road, hour, device, and app; counts are not population counts | Estimate penetration where possible; calibrate to counts; publish sample size |
| **Panel / app bias** | Panel over-represents certain ages, incomes, platforms, or geographies | Compare to ground truth; weight; audit by stratum; disclose |
| **Urban canyon** | GPS multipath and blocked sky view degrade accuracy; tall buildings and parking garages make the blue dot wrong [Google-LocationAccuracy] | Fuse Wi-Fi/cell; filter outliers; use map matching; report accuracy |
| **Indoor coverage** | GPS is weak or unavailable indoors; Wi-Fi/cell may be coarse [Google-LocationAccuracy] | Use indoor anchors/Wi-Fi; treat indoor data separately; do not overclaim |
| **Device multiplicity** | One person carries multiple devices, or one device is shared | Report devices, not people; where possible, de-duplicate at the event level without persistent IDs |
| **Road coverage gaps** | Minor roads, rural areas, and new roads have few probes | Fall back to historical/functional-class models; mark model-filled segments |
| **Temporal drift** | App updates, privacy settings, pandemics, construction change penetration | Monitor sample size and bias over time; retrain; change-point detection |
| **Privacy suppression** | k-threshold/DP removes small cells, which are often the most interesting | Report suppression rate; use complementary suppression; do not silently impute zeros |
---
## 7. Implications for our architecture
### 7.1 Design principles
1. **Opt-in, default off, granular.** Separate consent for probe measurement, product analytics, and research. Withdrawal stops future collection and deletes raw data.
2. **Minimise at the source.** The device should map-match and bin locally; raw GPS/Wi-Fi/cell observations should not leave the device unless necessary.
3. **No raw trajectories, no persistent IDs.** Use rotating pseudonyms or no identifier at all. Never store a per-device trajectory that can be joined across days.
4. **Local DP before transmission.** Add calibrated noise on the device, with a stated ε and a privacy budget. Do not rely on the server to anonymize after receiving raw data.
5. **Secure aggregation.** Use a protocol such as Prio or DAP so that the server sees only aggregate sums, not individual contributions [Prio-2017; DAP-2026]. This protects against a curious server and complements DP.
6. **k-threshold and suppression.** Publish a cell only if at least k distinct contributors are present; suppress or coarsen small cells; exclude sensitive sites (residential, medical, places of worship, schools, shelters).
7. **Retention limits.** Raw data on device only; aggregates retained for a defined period; no cross-day join keys; document backups.
8. **Transparency.** Publish the schema, spatial/temporal bins, k, ε, retention, coverage, and bias audits. If we cannot publish a parameter, we should not claim the guarantee.
### 7.2 Reference architecture
```text
┌──────────────────────────── Device ────────────────────────────┐
│ GPS / Wi-Fi / cell / IMU │
│ │ │
│ ▼ │
│ map-match to road segment or H3 cell │
│ │ │
│ ▼ │
│ local bin: 15 min (traffic) / 1 h (footfall) │
│ │ │
│ ▼ │
│ local DP: calibrated noise; clip contribution; budget │
│ │ │
│ ▼ │
│ k-threshold check: only send if ≥ k contributors in the bin │
│ │ │
│ ▼ │
│ secret-share / encrypt (Prio, DAP, or 2-of-3 MPC) │
└────────────────────────────┬───────────────────────────────────┘
│ no raw points, no persistent ID
▼
┌──────────────────── Aggregation servers ───────────────────────┐
│ compute sums / averages over shares; never see individual data │
│ tolerate dropouts; verify client contributions │
└────────────────────────────┬───────────────────────────────────┘
│ aggregate only
▼
┌──────────────────── Analytics / product ───────────────────────┐
│ heatmaps, segment speeds, demand indices, forecasts │
│ publish uncertainty, sample counts, suppression rate │
└────────────────────────────────────────────────────────────────┘
```
This is a **privacy-enhanced** architecture, not a magical one. It still leaks aggregate information; DP and k-threshold bound that leakage. It still has sampling bias; calibration addresses that. It is auditable in a way that "we anonymize it" is not.
### 7.3 Concrete defaults
| Parameter | Suggested default | Rationale |
|---|---|---|
| Consent | Off by default; separate toggles for measurement/research/ads | Matches Google's Timeline default and GDPR expectations; avoids consent bundling |
| Spatial bin (traffic) | Road segment + direction | Traffic is a link-level phenomenon; hexagons blur parallel roads |
| Spatial bin (footfall) | H3 resolution 8 (~0.7 km²) or 9 (~0.1 km²) | H3 gives stable hierarchical aggregation [H3]; choose based on venue density |
| Temporal bin | 15 min traffic; 1 h footfall | Balances k-threshold feasibility and operational usefulness |
| k-threshold | 20–50 distinct contributors per cell/time bin | Placer uses 50 [Placer-Data]; Google's search uses an area of at least 3 km² or 1,000 people for a different purpose [Google-LocationData]. Start at 50 for public release, 20 for internal ops |
| Local DP ε | 1–3 per release, with a daily/weekly budget | A starting range to be justified in the DPIA; must be documented and tested. Do not claim DP without stating ε |
| Secure aggregation | 2-of-3 servers or Prio/DAP | Tolerates one honest server; protects against curious aggregator [Prio-2017; DAP-2026] |
| Raw retention | On device only; delete after upload/binning | Minimises breach impact and re-identification risk |
| Aggregate retention | 90 days hot, 1 year cold; no cross-day keys | Enough for seasonal analysis; limits long-term linkage |
| Sensitive sites | Exclude residential, medical, places of worship, schools, shelters, and other sensitive categories | Extends Google's residential exclusion and Placer/SafeGraph sensitive-place policies [Google-BusyAreas; Placer-Trust; SafeGraph-Warren] |
| Transparency | Publish schema, k, ε, retention, coverage, suppression rate, bias audit | Makes claims verifiable; supports peer review |
### 7.4 Sensing-layer comparison for the paper
| Layer | What it measures | Identifiers | Accuracy for traffic | Coverage | Consent | Privacy risk | Verdict |
|---|---|---|---|---|---|---|---|
| **Passive Bluetooth/Wi-Fi MAC sniffing** | Presence of discoverable devices; dwell time; rough counts | Device MAC (often randomized) | Poor for speed; count bias | Local receiver coverage; pedestrian/vehicle mix | None (passive) | High: identifiers that may be persistent or linkable when randomization is absent; no consent | Avoid as a primary sensing layer |
| **Multi-receiver RSSI / AoA multilateration** | Position from signal strength/angle | Device MAC or ephemeral ID | Can be good indoors; multipath outdoors | Requires dense receiver infrastructure | Usually none or venue notice | High if IDs retained; medium if ephemeral and on-device | Use only with ephemeral IDs and aggregation; not a traffic-speed solution |
| **Optical / thermal counters** | Counts, direction, sometimes dwell | None | N/A (not speed) | Count-line specific | Notice; no personal data if no images | Low if images are not retained | Good ground truth for calibration |
| **First-party app probe** | Speed, travel time, footfall, dwell | Rotating pseudonym or none | Good where penetration is sufficient; map-matched | App users; coverage depends on adoption | Opt-in | Medium; can be strong with local DP + secure aggregation | Best fit for our proposal |
| **Telco mobility** | Population presence/flows/O-D | Subscriber ID (internal), aggregated output | Coarse; not lane-level | Population-scale; rural gaps | Legal basis/consent varies | Medium–high; aggregation essential | Good for regional O-D, not street traffic |
| **Transit smart card** | Transit entries/exits/O-D | Card ID (pseudonymized) | N/A for road traffic | Transit network only | Fare terms/privacy policy | Medium; journey reconstruction risk | Good for transit O-D |
| **Manual counts** | Counts, classification | None | N/A | Sparse, expensive | Notice | Low | Ground truth, not a live layer |
The table makes the paper's argument concrete: **MAC sniffing is typically the weakest option for both reliability and privacy**. It measures presence, not movement; it depends on discoverable, often randomized identifiers; it lacks consent; and it cannot produce a defensible speed estimate without a dense, calibrated receiver network. Aggregated first-party probe data is not perfect, but it measures the quantity we actually care about (movement) and can be designed with formal privacy controls.
### 7.5 What to claim and what not to claim
| Claim | Safe wording | Why |
|---|---|---|
| "We measure traffic" | "We estimate segment speeds/travel times from opt-in probe data, calibrated against ground counts" | Probe data is a sample; "measure" overstates certainty |
| "We count people" | "We estimate devices; penetration is unknown" | Multiple devices per person; shared devices; panel bias |
| "Our data is anonymous" | "Our data is aggregated with a k-threshold and local DP at ε = ..." | "Anonymous" is undefined and often false |
| "We use triangulation" | "We use map-matched probe data" or "multi-receiver multilateration" | Triangulation is angle-based; probe data is not a positioning method |
| "Google triangulates phones" | "Google fuses device-side location and aggregates navigation probes" | The public model does not support the stronger claim |
| "Google uses DP for traffic" | "Google claims DP for busyness; no DP claim for traffic was found" | Distinguishes verified scope |
| "MAC sniffing counts crowds" | "MAC sniffing is a biased presence proxy; randomization breaks uniqueness" | Defensible critique |
| "Our forecast is accurate" | "Our forecast has MAE/WAPE ... on a held-out period, with limitations" | Numbers must match measured results |
### 7.6 Evaluation plan
1. **Ground truth.** Door counters, loop detectors, transit gate counts, or manual counts. Without ground truth, a probe-based estimate is only a proxy.
2. **Accuracy metrics.** MAE, WAPE, MAPE (with care around zeros), R², and prediction-interval coverage. Report per sensor/segment, not just aggregate.
3. **Bias audit.** Compare errors by neighbourhood, income proxy, device type, time of day, and day of week. Report the worst stratum, not only the mean.
4. **Privacy audit.** State k, ε, retention, and suppression rate. Run a membership-inference or re-identification test on the released aggregates. If we cannot run it, say so.
5. **Ablation.** Compare raw vs aggregated vs local-DP vs secure-aggregation pipelines on the same data. Report the utility cost of each privacy layer.
6. **Robustness.** Missing data, low penetration, urban canyon, indoor venues, holidays, and weather.
7. **Operational metrics.** Latency, update cadence, coverage, and failure modes (e.g., fallback to historical baseline).
### 7.7 Threats to validity
- **Device ≠ person.** All device-based counts inherit this.
- **Penetration unknown.** We can estimate it only with calibration; otherwise we report an index, not a population count.
- **Panel skew.** Our opt-in sample may not represent the venue's visitors.
- **Map-matching errors.** Parallel roads, tunnels, and parking lots can misassign probes.
- **Privacy suppression.** k-threshold and DP remove small cells; the remaining map is biased toward busy areas.
- **Company claims.** Google/HERE/TomTom/INRIX/Placer/SafeGraph/Unacast numbers are not independently verified; use them as context, not ground truth.
---
## 8. Key papers and standards
### 8.1 Probe data and traffic estimation
- **Google Maps ETA with graph neural networks.** Derrow-Pinion et al., *ETA Prediction with Graph Neural Networks in Google Maps*, CIKM 2021. Production GNN over road supersegments; strong applied reference [Google-ETA-GNN].
- **DeepMind traffic prediction.** Lange & Perez, *Traffic prediction with advanced Graph Neural Networks*, 2020. Accessible description of supersegments and multi-loss training [DeepMind-2020].
- **Mobile Century.** Herrera et al., *Evaluation of traffic data obtained via GPS-enabled mobile phones: The Mobile Century field experiment*, Transportation Research Part C, 2009. 100 vehicles; suggests 2–3% phone penetration can be enough for velocity estimation [Herrera-2009].
- **Probe penetration and flow estimation.** Seo et al., *Estimation of flow and density using probe vehicles with spacing measurement equipment*, Transportation Research Part C, 2015. 3.5% penetration for 5-minute flow; 0.2% for hourly flow [Seo-2015].
- **Sampling protocol bias.** Jenelius & Koutsopoulos, *Probe vehicle data sampled by time or space*, Transportation Research Part B, 2015. Sampling protocol affects travel-time bias [Jenelius-2015].
- **Loop vs probe trade-offs.** Mazaré et al., *Trade-offs between inductive loops and GPS probe vehicles for travel time estimation: A Mobile Century case study*, TRB 2012. Penetration 1.5–3% at 275 veh/h; diminishing returns [Mazare-2011].
- **Empirical penetration distribution.** Luo et al., *Empirical distribution of probe vehicle penetration rates and implications for traffic state estimation*, Transportation Research Part C, 2026. Penetration is non-uniform; a reliable penetration estimate matters more than assuming uniformity [Luo-2026].
- **Virtual trip lines.** Hoh et al., *Virtual Trip Lines for Distributed Privacy-Preserving Traffic Monitoring*, MobiSys 2008; and *Enhancing Privacy and Accuracy in Probe Vehicle-Based Traffic Monitoring via Virtual Trip Lines*, IEEE TMC 2012. Spatial triggers reduce raw trajectory exposure and support distributed aggregation [Hoh-2008; Hoh-2012].
- **Differential privacy for traffic flow.** Le Ny et al., *Real-time privacy-preserving model-based estimation of traffic flows*, ICCPS 2014. DP traffic-state estimation with a macroscopic model (useful for the privacy-utility trade-off) [LeNy-2014].
### 8.2 Privacy foundations
- **k-anonymity.** Sweeney, *k-anonymity: A Model for Protecting Privacy*, 2002 [Sweeney-2002].
- **Differential privacy.** Dwork, *Differential Privacy*, ICALP 2006 [Dwork-2006]; Dwork & Roth, *The Algorithmic Foundations of Differential Privacy*, 2014 [Dwork-Roth-2014].
- **Geo-indistinguishability.** Andrés et al., *Geo-Indistinguishability: Differential Privacy for Location-Based Systems*, CCS 2013. Formal location privacy with a radius-dependent guarantee [Andres-2013].
- **Local DP in practice.** Erlingsson et al., *RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response*, CCS 2014. Google's client-side randomized response deployment [RAPPOR-2014].
- **Secure aggregation.** Bonawitz et al., *Practical Secure Aggregation for Privacy-Preserving Machine Learning*, CCS 2017 [Bonawitz-2017].
- **Prio.** Corrigan-Gibbs & Boneh, *Prio: Private, Robust, and Scalable Computation of Aggregate Statistics*, NSDI 2017 [Prio-2017].
- **DAP.** IETF PPM Working Group, *Distributed Aggregation Protocol for Privacy Preserving Measurement*, draft-ietf-ppm-dap-20, 2026 [DAP-2026].
- **Aggregate location privacy.** Pyrgelis et al., *Knock Knock, Who's There? Membership Inference on Aggregate Location Data*, NDSS 2018 [Pyrgelis-2018].
- **Mobility uniqueness.** de Montjoye et al., *Unique in the Crowd: The privacy bounds of human mobility*, Scientific Reports, 2013 [deMontjoye-2013].
### 8.3 Standards and frameworks
- **NIST SP 800-188.** *De-Identifying Government Datasets*, 2023. Practical de-identification and risk assessment [NIST-800-188].
- **ISO/IEC 20889:2018.** *Privacy enhancing data de-identification terminology and classification of techniques* [ISO-20889].
- **DATEX II / EN 16157.** European standard for traffic and travel information exchange; measured and elaborated data publications [DATEX-II].
- **ISO 14819-1:2021.** Traffic and travel information messages via traffic message coding; RDS-TMC/ALERT-C [ISO-14819].
- **H3.** Open-source hexagonal geospatial indexing system, suitable for hierarchical aggregation [H3].
### 8.4 Data-product documentation
- Google Maps traffic/ETA, Timeline, busyness, anonymization, retention [Google-Maps101; Google-NavData; Google-BusyAreas; Google-Timeline2023; Google-Anonymization; Google-Retention].
- HERE, TomTom, INRIX, Waze product and methodology pages [HERE-RTT; HERE-Probe; TomTom-Prediction; TomTom-API; INRIX; Waze-Cities].
- Placer, SafeGraph, Unacast methodology and privacy pages [Placer-Data; Placer-Trust; SafeGraph-Privacy; SafeGraph-Warren; SafeGraph-Bias; Unacast-Method; Unacast-Legal].
- Telco, transit, and fleet examples [Telefonica; Orange-Flux; Orange-Cambridge; Vodafone-Cambridge; TfL-OpenData; TfL-Access; Geotab-FAQ; Geotab-Privacy].
---
## 9. Citations
All URLs accessed 2026-11-14 unless otherwise stated. Company pages may change; the title and access date are part of the citation.
### Google
**[Google-Maps101]** Johann Lau, "Google Maps 101: How AI helps predict traffic and determine routes," Google, 3 September 2020. https://blog.google/products/maps/google-maps-101-how-ai-helps-predict-traffic-and-determine-routes/
**[Google-NavData]** Google Maps Help, "How navigation data makes Maps better for everyone." https://support.google.com/maps/answer/10565726
**[Google-TrafficLegend]** Google Maps Help, "Use layers to find places, traffic, terrain, biking & transit." https://support.google.com/maps/answer/3092439
**[Google-PrivacyPolicy]** Google, "Privacy Policy." https://policies.google.com/privacy
**[Google-LocationData]** Google Privacy & Terms, "How Google uses location information." https://policies.google.com/technologies/location-data
**[Google-LocationAccuracy]** Google Maps Help, "Find & improve your location's accuracy in Google Maps." https://support.google.com/maps/answer/2839911
**[Google-BusyAreas]** Google Maps Help, "Get information about busy areas from Google Maps." https://support.google.com/maps/answer/11323117
**[Google-TimelinePrivacy]** Google Maps Help, "How Google keeps your Timeline data private." https://support.google.com/maps/answer/10077010
**[Google-TimelineManage]** Google Account Help, "Manage your Timeline data." https://support.google.com/accounts/answer/14200149
**[Google-Timeline2023]** Marlo McGriff, "Updates to Location History and new controls coming soon to Maps," Google, 12 December 2023. https://blog.google/products/maps/updates-to-location-history-and-new-controls-coming-soon-to-maps/
**[Google-Incognito]** Google Maps Help, "Use Google Maps in Incognito mode." https://support.google.com/maps/answer/9430563
**[Google-LocationToggle]** Google Maps Help, "Turn location data collection on or off." https://support.google.com/maps/answer/2839958
**[Google-Anonymization]** Google Privacy & Terms, "How Google anonymizes data." https://policies.google.com/technologies/anonymization
**[Google-Retention]** Google Privacy & Terms, "How Google retains data we collect." https://policies.google.com/technologies/retention
**[Google-RMI]** Google for Developers, "Roads Management Insights: Data foundation." https://developers.google.com/maps/documentation/roads-management-insights/data
**[Google-GeolocationAPI]** Google for Developers, "Geolocation API Overview." https://developers.google.com/maps/documentation/geolocation/overview
**[Google-2009]** Google Official Blog, "The bright side of sitting in traffic: Crowdsourcing road congestion data," 25 August 2009. https://googleblog.blogspot.com/2009/08/bright-side-of-sitting-in-traffic.html
**[Google-AU-2009]** Andrew Foster, "Get there faster with traffic information on Google Maps," Google Australia Blog, 15 September 2009. https://australia.googleblog.com/2009/09/get-there-faster-with-traffic.html
### Positioning and terminology
**[GPSgov-Trilateration]** GPS.gov, "Trilateration." https://www.gps.gov/trilateration
**[NOAA-Geodesy]** NOAA National Geodetic Survey, "Geodesy for the Layman" (triangulation and trilateration definitions). https://geodesy.noaa.gov/PUBS_LIB/Geodesy4Layman/TR80003A.HTM
**[Android-RTT]** Android Developers, "Wi-Fi location: ranging with RTT." https://developer.android.com/develop/connectivity/wifi/wifi-rtt
### Alternatives
**[HERE-RTT]** HERE, "Real-Time Traffic," HERE Traffic API v7 documentation. https://docs.here.com/traffic-api/docs/real-time-traffic-here-traffic-api-v7-concepts
**[HERE-OnePager]** HERE, "HERE Real-Time Traffic" product one-pager, 2026. https://assets.here.com/x7rx8ayph7ee/e13d7d3e-c1f8-4562-876c-e4910c7001a8/9e056bbd245dab0a0587e5992b55f4d5/HERE_Traffic_product_one_pager.pdf
**[HERE-Probe]** HERE, "GPS probe data for Traffic Analytics." https://docs.here.com/traffic-analytics/docs/gps-probe-data
**[TomTom-Prediction]** Joao Laranjeira, "What is Traffic Prediction and How Does it Work?" TomTom Newsroom, 24 January 2020. https://www.tomtom.com/newsroom/behind-the-map/road-traffic-prediction
**[TomTom-Stats]** TomTom, "TomTom Traffic Stats explained." https://www.tomtom.com/newsroom/product-focus/tomtom-traffic-stats-explained
**[TomTom-API]** TomTom, "Traffic Stats API: Introduction." https://docs.tomtom.com/traffic-stats/documentation/product-information/introduction
**[INRIX]** INRIX, "INRIX AI Traffic." https://inrix.com/products/ai-traffic/
**[Waze-Cities]** Waze, "Waze for Cities." https://www.waze.com/wazeforcities
**[IowaDOT-2020]** "DOT Traveler Information Website Crowdsourcing Practices," Project 14.4, 2020 (Iowa DOT comparison of Google and Waze traffic layers). https://nwpassage.info/wp-content/uploads/2024/04/14-4-crowdsourcing-practices.pdf
**[Apple-Privacy]** Apple, "Apple Privacy Features: Maps." https://www.apple.com/privacy/features/
**[Placer-Data]** Placer.ai, "Our Data." https://www.placer.ai/resources/our-data
**[Placer-Trust]** Placer.ai, "Trust Center." https://www.placer.ai/company/trust-center
**[SafeGraph-Privacy]** SafeGraph, "Privacy Policy & Data Practices," last updated 2 December 2025. https://www.safegraph.com/privacy-policy
**[SafeGraph-Warren]** SafeGraph, response to Senator Elizabeth Warren, 23 June 2022. https://www.warren.senate.gov/wp-content/uploads/media/doc/SafeGraph%20Response%20%20to%20May%2017%20Letter_Redacted.pdf
**[SafeGraph-Bias]** Wang et al., "Understanding the bias of mobile location data across spatial scales and over time: A comprehensive analysis of SafeGraph data in the United States," PLOS One, 2024. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0294430
**[Unacast-Method]** Unacast, "Our Methodology." https://www.unacast.com/methodology
**[Unacast-Legal]** Unacast, "Legal." https://docs.unacast.com/legal
**[Telefonica]** Telefónica Germany, "Mobility Insights: Anonymization process." https://www.telefonica.de/anonymization_process
**[Orange-Flux]** Orange Business, "Flux Vision." https://www.orange-business.com/en/solutions/data-intelligence-iot/flux-vision
**[Orange-Cambridge]** "The value of network data confirmed by the Covid-19 epidemic and its expanded usages," Data & Policy, 2022. https://www.cambridge.org/core/journals/data-and-policy/article/value-of-network-data-confirmed-by-the-covid19-epidemic-and-its-expanded-usages/4FD137D39AF08DB624981FDA322CABB0
**[Vodafone-Cambridge]** "Using Vodafone mobile phone network data to provide insights into citizens mobility in Italy during the Coronavirus outbreak," Data & Policy, 2021. https://www.cambridge.org/core/journals/data-and-policy/article/using-vodafone-mobile-phone-network-data-to-provide-insights-into-citizens-mobility-in-italy-during-the-coronavirus-outbreak/100EA32E60F4A6E64AE96515B2A80EA4
**[TfL-OpenData]** Transport for London, "Our open data." https://tfl.gov.uk/info-for-open-data-users/our-open-data
**[TfL-Access]** Transport for London, "Access your data." https://lul-trackernet.tfl.gov.uk/corporate/privacy-and-cookies/access-your-data
**[TfL-Oyster]** Transport for London, "Oyster card" privacy page. https://lul-trackernet.tfl.gov.uk/corporate/privacy-and-cookies/oyster-card
**[Geotab-FAQ]** Altitude by Geotab, "Geolocation Data Privacy FAQ," 14 October 2025. https://altitude.geotab.com/news-and-releases/geolocation-data-privacy-faq
**[Geotab-Privacy]** Geotab, "Our Privacy Commitment." https://www.geotab.com/about/privacy
### Probe data and privacy papers
**[LeNy-2014]** J. Le Ny, A. Touati, and G. J. Pappas, "Real-time privacy-preserving model-based estimation of traffic flows," ACM/IEEE ICCPS 2014. DOI: 10.1109/ICCPS.2014.6843714.
**[Hoh-2008]** B. Hoh, M. Gruteser, R. Herring, J. Ban, D. Work, J. C. Herrera, A. M. Bayen, M. Annavaram, and Q. Jacobson, "Virtual Trip Lines for Distributed Privacy-Preserving Traffic Monitoring," MobiSys 2008. DOI: 10.1145/1378600.1378604.
**[Hoh-2012]** B. Hoh, T. Iwuchukwu, Q. Jacobson, D. Work, A. M. Bayen, R. Herring, J.-C. Herrera, M. Gruteser, M. Annavaram, and J. Ban, "Enhancing Privacy and Accuracy in Probe Vehicle-Based Traffic Monitoring via Virtual Trip Lines," IEEE Transactions on Mobile Computing, 11(5), 849–864, 2012. DOI: 10.1109/TMC.2011.116.
**[Herrera-2009]** J. C. Herrera, D. B. Work, R. Herring, X. Ban, Q. Jacobson, and A. M. Bayen, "Evaluation of traffic data obtained via GPS-enabled mobile phones: The Mobile Century field experiment," Transportation Research Part C, 2009.
**[Seo-2015]** T. Seo, T. Kusakabe, and Y. Asakura, "Estimation of flow and density using probe vehicles with spacing measurement equipment," Transportation Research Part C, 2015.
**[Jenelius-2015]** E. Jenelius and H. N. Koutsopoulos, "Probe vehicle data sampled by time or space: Consistent travel time allocation and estimation," Transportation Research Part B, 2015.
**[Mazare-2011]** P.-E. Mazaré, O.-P. Tossavainen, A. M. Bayen, and D. B. Work, "Trade-offs between inductive loops and GPS probe vehicles for travel time estimation: A Mobile Century case study," Transportation Research Board, 2012.
**[Luo-2026]** Y. Luo, J. Balsa-Barreiro, H. Yi, and M. Menéndez, "Empirical distribution of probe vehicle penetration rates and implications for traffic state estimation," Transportation Research Part C, 189, 105729, 2026. DOI: 10.1016/j.trc.2026.105729.
**[Google-ETA-GNN]** A. Derrow-Pinion et al., "ETA Prediction with Graph Neural Networks in Google Maps," CIKM 2021, arXiv:2108.11482. DOI: 10.1145/3459637.3481916.
**[DeepMind-2020]** O. Lange and L. Perez, "Traffic prediction with advanced Graph Neural Networks," Google DeepMind, 3 September 2020. https://deepmind.google/discover/blog/traffic-prediction-with-advanced-graph-neural-networks/
**[Sweeney-2002]** L. Sweeney, "k-anonymity: A Model for Protecting Privacy," International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 10(5), 557–570, 2002. DOI: 10.1142/S0218488502001648.
**[Dwork-2006]** C. Dwork, "Differential Privacy," ICALP 2006, LNCS 4052, pp. 1–12. DOI: 10.1007/11787006_1.
**[Dwork-Roth-2014]** C. Dwork and A. Roth, "The Algorithmic Foundations of Differential Privacy," Foundations and Trends in Theoretical Computer Science, 2014. https://www.cis.upenn.edu/~aaroth/Papers/privacybook.pdf
**[Andres-2013]** M. E. Andrés, N. E. Bordenabe, K. Chatzikokolakis, and C. Palamidessi, "Geo-Indistinguishability: Differential Privacy for Location-Based Systems," CCS 2013. DOI: 10.1145/2508859.2516735.
**[RAPPOR-2014]** Ú. Erlingsson, V. Pihur, and A. Korolova, "RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response," CCS 2014. DOI: 10.1145/2660267.2660348.
**[Bonawitz-2017]** K. Bonawitz et al., "Practical Secure Aggregation for Privacy-Preserving Machine Learning," CCS 2017, pp. 1175–1191. DOI: 10.1145/3133956.3133982.
**[Prio-2017]** H. Corrigan-Gibbs and D. Boneh, "Prio: Private, Robust, and Scalable Computation of Aggregate Statistics," NSDI 2017. https://www.usenix.org/conference/nsdi17/technical-sessions/presentation/corrigan-gibbs
**[DAP-2026]** IETF Privacy Preserving Measurement Working Group, "Distributed Aggregation Protocol for Privacy Preserving Measurement," draft-ietf-ppm-dap-20, October 2026. https://datatracker.ietf.org/doc/draft-ietf-ppm-dap/
**[Pyrgelis-2018]** A. Pyrgelis, C. Troncoso, and E. De Cristofaro, "Knock Knock, Who's There? Membership Inference on Aggregate Location Data," NDSS 2018. https://www.ndss-symposium.org/wp-content/uploads/2018/02/ndss2018_05B-5_Pyrgelis_paper.pdf
**[deMontjoye-2013]** Y.-A. de Montjoye, C. A. Hidalgo, M. Verleysen, and V. D. Blondel, "Unique in the Crowd: The privacy bounds of human mobility," Scientific Reports, 3, 1376, 2013. DOI: 10.1038/srep01376.
**[NIST-800-188]** S. Garfinkel, J. Near, A. Dajani, P. Singer, and B. Guttman, "De-Identifying Government Datasets," NIST Special Publication 800-188, 2023. DOI: 10.6028/NIST.SP.800-188.
**[ISO-20889]** ISO/IEC 20889:2018, "Privacy enhancing data de-identification terminology and classification of techniques." https://www.iso.org/standard/69373.html
**[DATEX-II]** DATEX II, "DATEX II Documentation Portal" (CEN/EN 16157 series). https://docs.datex2.eu/
**[ISO-14819]** ISO 14819-1:2021, "Intelligent transport systems — Traffic and travel information messages via traffic message coding — Part 1: Coding protocol for Radio Data System-Traffic Message Channel (RDS-TMC) using ALERT-C." https://www.iso.org/standard/76781.html
**[H3]** Uber, "H3: A hexagonal hierarchical geospatial indexing system." https://h3geo.org/docs/
---
## Appendix A. One-paragraph summary for the paper's related-work section
Google Maps' traffic and ETA features are built on **aggregated, map-matched probe data**, not, as publicly described, on cell-tower triangulation. Google fuses GPS, Wi-Fi, cellular, and sensor signals on the device; while users navigate, it collects GPS, route, and sensor data; it aggregates those observations into road-segment speeds and combines them with historical traffic patterns and graph-neural-network ETA models [Google-Maps101; Google-NavData; Google-ETA-GNN]. Google's public privacy claims are layered: aggregation and anonymization for traffic, differential privacy for busyness/popular times, and on-device Timeline storage with a three-month default auto-delete [Google-BusyAreas; Google-Timeline2023]. These are meaningful protections, but they are not a substitute for a formal, auditable guarantee: Google does not publish penetration, aggregation thresholds, DP epsilon, traffic-layer coverage, or an independent audit, and aggregate location data remains vulnerable to membership inference [Pyrgelis-2018]. The lesson for a privacy-first sensing layer is to make those parameters explicit: opt-in consent, on-device aggregation, local DP, secure aggregation, a k-threshold, no raw trajectories, and bounded retention.
## Appendix B. Quick fact table
| Fact | Value | Source |
|---|---|---|
| Google Maps daily distance | Over 1 billion km/day | [Google-Maps101] |
| Google Maps country coverage | 220+ countries and territories | [Google-Maps101] |
| Google ETA accuracy claim | "Over 97% of trips" | [Google-Maps101] |
| DeepMind ETA improvement claim | Up to 50% in some cities | [Google-Maps101; DeepMind-2020] |
| Google GPS accuracy (user-facing) | About 20 m | [Google-LocationAccuracy] |
| Google cell accuracy (user-facing) | Up to a few thousand metres | [Google-LocationAccuracy] |
| Google search location threshold | 3 km² or 1,000 people | [Google-LocationData] |
| Timeline default auto-delete | 3 months (2023 update) | [Google-Timeline2023] |
| Google deletion time | ~2 months; backups up to 6 months | [Google-Retention] |
| HERE traffic coverage | 75 countries; flow every minute | [HERE-RTT; HERE-OnePager] |
| TomTom probe base | ~600 million probes | [TomTom-Prediction] |
| Waze driver base | ~200 million drivers | [Waze-Cities] |
| Placer k-anonymity | 50 devices | [Placer-Data; Placer-Trust] |
| TfL Oyster retention | 8–9 weeks | [TfL-Access] |
| Mobile Century penetration | 2–3% enough for velocity | [Herrera-2009] |
| Seo et al. penetration | 3.5% for 5-min flow; 0.2% hourly | [Seo-2015] |
+528
View File
@@ -0,0 +1,528 @@
# Sensing Alternatives for Pedestrian Footfall and Occupancy
**Research note for the PulseCast paper — Heilbronn, November 2026**
**Scope.** This note compares the main sensing approaches that could replace or complement BLE/Wi-Fi MAC sniffing for pedestrian footfall and occupancy measurement. It covers measured accuracy from peer-reviewed studies and official evaluations, planning-level cost ranges, privacy risk, deployment constraints, and the best use of each modality. It closes with a reference sensing stack and a calibration design for Heilbronn.
**What this note does not do.** It does not compare vendor products, does not treat vendor accuracy claims as evidence, and does not assume that a result from a controlled indoor test transfers to a Heilbronn street. Where a number comes from a single site or a small sample, that is stated. Costs marked **(est.)** are planning ranges derived from the cited hardware or labour anchors, not quotations.
---
## Key findings
1. **BLE/Wi-Fi MAC sniffing is an index of discoverable radios, not a people counter.** Only 5–12% of devices are discoverable via Bluetooth, Wi-Fi detection was 69% conditional on Wi-Fi being on in a vehicle-oriented field study, and MAC randomisation is defeatable for ~96% of Android phones (Kurkcu & Ozbay, 2017; Musa & Eriksson, 2012; Martin et al., 2017). It fails both the accuracy test and the privacy test.
2. **No single sensor dominates.** Thermal arrays are the best default for narrow gateways (89–95% accuracy, low privacy risk, ~€150–400 per node). Edge cameras are the accuracy benchmark (MAPE 13.8% day / 29.8% night outdoors) but carry the highest privacy burden among the non-MAC options. mmWave and LiDAR fill wide or weather-exposed gaps; manual counts are the only direct ground truth.
3. **The "right way" is a tiered stack plus a Bayesian hierarchical calibration model.** Fuse a small number of calibrated sensors, model sensor-specific detection probabilities, and report counts with credible intervals and calibration dates. Without calibration, the output is an index, not a measurement.
4. **For Heilbronn:** 8–12 thermal nodes at narrow gateways, 2–4 edge cameras at wide plazas, 1–2 mmWave/LiDAR gap-fillers, twice-yearly manual/video ground truth, and transit validators as covariates. No MAC sniffing, no face recognition. A 12-week, six-site pilot with explicit go/no-go thresholds (MAPE ≤ 20%, |bias| ≤ 10%, interval coverage ≥ 75%).
5. **Cost and legal reality:** a year-1 pilot is roughly €81k–161k (estimate), with €30k–50k recurring; calibration is the largest recurring cost. A DPIA under Art. 35(3)(c) GDPR, BDSG §4 compliance, and a no-raw-video/no-raw-point-cloud design are preconditions, not downstream tasks.
---
## 1. Comparison table
**How to read the accuracy column.** "Accuracy" is not one metric. A study may report exact-count accuracy, mean absolute percentage error (MAPE), mean absolute error (MAE), F1 for presence detection, or correlation with a reference. These are not interchangeable. A method that is excellent at "is anyone here?" can be poor at "how many people are here?". The context in the cell matters more than the headline number.
| Modality | What it measures | Reported performance (source) | Cost (2026 planning estimate) | Privacy risk | Main deployment constraints | Best use |
|---|---|---|---|---|---|---|
| **BLE MAC sniffing** (baseline) | Discoverable Bluetooth radios, not people | 5–12% of devices discoverable via Bluetooth; 97%/92% count accuracy only when penetration is assumed 100% (Kurkcu & Ozbay, 2017). MAC randomisation defeated for ~96% of Android phones (Martin et al., 2017). | €100–500 | **Very high** — MAC addresses are personal data; persistent tracking; re-identification risk | Unknown and changing device penetration; multiple devices per person; staff and passers-by indistinguishable; packet loss; legal basis difficult | None for public footfall. At most a short-lived, calibrated index under a DPIA. |
| **1. Edge AI camera** | People crossing a line or occupying an area | Outdoor 18-site evaluation: MAPE 13.8% day, 29.8% night; best-decile 5.3%/7.3%, worst-decile 30.5%/62.8% (Cyganek et al., 2022). Overhead test: 97% live, 92% over two days (Ahamed et al., 2024). Official review: 2–14% error for video imaging (Ozan et al., 2021). | €500–2,000; commercial systems €1,100–€7,400 | **High** — identifiable images even without face recognition; DPIA required | Lighting, occlusion, rain, mounting height, field of view, compute/power, public acceptance | Reference and validation at 2–4 complex sites; wide plazas; pedestrian/cyclist classification. |
| **2. Thermal / IR array** | Warm bodies crossing a doorway; direction | 93% average occupancy accuracy (Mohammadmoradi et al., 2017); 89–92% doorway accuracy (Maaspuro, 2020); 95% at normal walking speed up to 30 °C, 100% in stable 22–26 °C (Perra et al., 2021). Degrades above 30 °C, beyond ~7.5 m, with fast walking or groups. | €150–400 (sensor ~$50) | **Low** — low-resolution heat maps, not photographic images; aggregate at edge | Narrow doorways only; ambient temperature; mounting height; group counting; limited range | Default permanent counter at store entrances, narrow passages, transit shelters. |
| **3. mmWave radar** | Moving people as point clouds; presence, flow, count | Outdoor Amsterdam: 96% presence, 69% count, −2.5% aggregate bias (Vaidya & Zúñiga, 2024). Three-month outdoor: F1 84%, flow correlation 60–85% (Wang et al., 2023). Single radar: MAE 1.53 up to 21 people (Pallaprolu et al., 2024). Festival: correlation 82.9–94.2%, ±50% small crowds / ±25% large crowds (Sakhnini et al., 2024). | €300–700 ($311 node; $120.87 sensor) | **Low** — point clouds, no images; raw data can still track individuals | Rain, foliage, multipath, groups, range ~8 m, mounting, power | Wide, weather-exposed crossings; transit stops; presence and flow, not exact occupancy. |
| **4. LiDAR** | 3D geometry of people; tracking and direction | 2D LiDAR: >97% disaggregate count; 1.1% false direction; 0.7% overcount; 1.3–2.7% undercount; 1.6–4.3% AAPD at 15 min (Lesani et al., 2020). Official review: 5% or more average error in crowded scenes; $8,000 unit (Ozan et al., 2021). | €1,000–8,000; low-cost 2D from ~$100 | **Low–moderate** — geometric point cloud; individual tracking possible | Cost; eye safety; weather; mounting; calibration; power | High-accuracy fixed-site counting; 1–2 reference sites, not a city-wide network. |
| **5. Wi-Fi CSI** | Channel perturbations caused by bodies; device-free | 80% accuracy up to 7–8 people (Di Domenico et al., cited in Brena et al., 2021); 80% perfect-or-±1 up to 30 people (Xi et al., cited in Brena et al., 2021); 99% with six links in controlled rooms (Brena et al., 2021); 92.7% with RTT in one 80 m² office (Yonekura et al., 2025); 61% cross-domain (VICount, cited in Yonekura et al., 2025). | €200–1,000 per link; €10k–30k engineering | **Moderate–high** — device-free but can infer presence, movement, occupancy; GDPR still applies | Per-room calibration; furniture and crowd changes; CSI extraction hardware (Intel 5300 no longer current); no standard API; poor cross-domain transfer | Indoor rooms where cameras are unacceptable and thermal cannot cover; research pilots only. |
| **6. Wi-Fi probe sniffing** | Probe requests and MAC addresses | 69% detection of passing phones **if Wi-Fi is on**; 67 m mean trajectory error; ~1 device per 5 vehicles (Musa & Eriksson, 2012). No stable people multiplier. Randomisation defeated for ~96% of Android phones (Martin et al., 2017). | €100–500 | **Very high** — MAC addresses are personal data; tracking; DPIA; ePrivacy | Wi-Fi-on rate; randomisation; probe timing; multiple devices; legal basis; public trust | None for public footfall. The privacy cost is not justified by the data quality. |
| **7. Telco network data** | Serving-cell events / CDR; aggregated mobility | Population mapping comparable to census-based methods (Deville et al., 2014). Spatial resolution: 250–1,000 ft urban, 1,500–3,000 ft suburban, 1–3 miles rural; not suitable for intersection-level analysis (BTS, 2025). Urban subscribers typically within ~500 m of a cell tower (Flowminder, n.d.). | No node; bespoke licence, €10k–100k+/year (est.) | **High** — individual mobility traces; aggregation and re-identification risk | Operator cooperation; market share; cell-sector resolution; latency; licensing; regulatory approval | City-wide context, origin–destination, tourism and event flows — not street-level footfall. |
| **8. GPS probe / map providers** | App-based GPS traces; route and volume estimates | Strava users biased toward cyclists (+8%), males (+15.7%), ages 35–54 (+20.4%) (Venter et al., 2023). Model accuracy ±32–160 cyclists on 75% of streets (Strava Metro, n.d.). Pedestrian data are rare (Lee & Sener, 2020). | No node; Strava ~$0.55/user/year (2021); commercial feed €10k–50k/year (est.) | **High** — individual traces; commercial re-use; GDPR | Sample bias; indoor gaps; urban canyon; app penetration; licensing | Cyclist route and origin–destination analysis; recreational trends. Not inner-city footfall. |
| **9. Transit smart cards / APC** | Fare validations, boardings, turnstiles | Transaction-level for validations, but only for passengers who validate. Fare evasion: 5% partial / 37% bus-only in one system (Munizaga et al., 2020); 20–50% in some California agencies (Caltrans, 2025). | €0–5k/year data extract if AFC exists; €5k–15k per new validator/gate (est.) | **High** — individual trip records; pseudonymous but linkable | Only transit users; fare policy; operator cooperation; no pedestrian-zone coverage | Transit demand covariate; commuter and event signal. Not general footfall. |
| **10. Manual counts / video annotation** | Direct human observation; ground truth | Manual clicker average error 11% (8% and 15% on two dates) (Diogenes et al., 2007). Observers accurate for about 2 hours; 2-hour-based annual estimates can be off by up to 60%; one week of counts recommended (Nordback, 2014). | €25–40 per observer-hour loaded (est.); campaign €11k–18k | **Low** for counts; **high** for video annotation (faces) | Labour; training; weather; sampling design; inter-rater reliability | Ground truth and calibration; regular audits; not continuous monitoring. |
| **11. Sensor fusion / Bayesian hierarchical model** | Latent true count, combining sparse modalities | No single accuracy figure. Hierarchical models combine multiple estimates and propagate uncertainty (Parsons et al., 2022); Bayesian spatio-temporal models map pedestrian traffic from sparse counts (Zaouche & Bode, 2023). Performance depends on calibration data and model checks. | €20k–60k development; €5k–15k/year maintenance (est.) | **Depends on inputs** — aggregated outputs can be low-risk; raw inputs inherit their risks | Data engineering; identifiability; drift; validation; explainability | The core of a defensible stack: fuse thermal, camera, radar, manual and transit data. |
**Cost basis.** Cost ranges are 2026 planning estimates in euros, excluding VAT, installation, backend and maintenance unless stated. They are anchored to published figures: a Grid-EYE sensor at ~$50 (Perra et al., 2021), with a complete thermal node estimated at ~€150–400; a 60 GHz radar node at $311, itemised as $120.87 sensor + $49.77 Raspberry Pi 4B + $101.12 PTFE radome + $12.74 back casing + ~$26.50 misc (Wang et al., 2023); commercial radar nodes at ~$600 (Wang et al., 2023); video systems at $1,200–$8,000, depth cameras at $9,900–$12,330, thermal cameras at $4,800 and laser scanning at $8,000 (Ozan et al., 2021); low-cost 2D LiDAR from ~$100 (Umezu et al., 2023). Where a source is in USD, the euro range is a rounded planning conversion, not an invoice.
**Evidence quality.** The strongest evidence in this table is the outdoor camera evaluation (18 sites, 207 minutes, manual reference), the 2D LiDAR evaluation (two sites, disaggregate and 15-minute counts), the three-month outdoor radar deployment, and the official NCDOT technology review. The weakest are the single-room CSI results and the short, controlled radar tests; the three-month radar deployment is stronger but still reports wide outdoor error. No study in this table is a randomised multi-city trial of the kind that would justify a single national accuracy figure.
---
## 2. Modality notes
### 2.0 The baseline: why BLE MAC sniffing is not a people counter
**Principle.** A BLE scanner listens for advertising packets. Each packet carries a device address; the scanner counts unique addresses in a time window. The same logic applies to Wi-Fi probe requests. The output is an index of discoverable radios, not a count of people.
**Measured accuracy.** The fundamental problem is penetration, not classifier quality. Kurkcu and Ozbay (2017) report that 5–12% of devices are discoverable via Bluetooth. Their own pedestrian-counting tests reached 97% and 92% accuracy only when the penetration rate was assumed to be 100%; they explicitly warn that the method depends on the ratio of people with discoverable devices. A separate evaluation cited by Kurkcu and Ozbay (2017) found Bluetooth detection as low as 2.0%; a combined Wi-Fi/Bluetooth system had 15% mean and 9% median pedestrian-flow prediction errors. There is no stable multiplier from devices to people: penetration varies by device brand, OS, battery state, privacy settings, age, and location.
**Randomisation.** Modern phones randomise MAC addresses, but the randomisation is imperfect. Martin et al. (2017) defeated it for approximately 96% of Android phones using probe-request fingerprints, and found a low-level control-frame flaw that applied to 100% of tested devices. In other words, the "privacy-preserving" randomisation is not a reliable anonymisation layer, and the "count" is not a reliable people count. The two failures point in opposite directions: the data are both biased and identifiable.
**Cost.** Very low: a single-board computer or consumer access point can capture packets (Musa & Eriksson, 2012, used $70 access points). The low hardware cost is why the method spread. It is not the relevant cost: legal review, DPIA, signage, retention controls and public consultation dominate.
**Privacy.** MAC addresses are personal data. The Dutch Data Protection Authority states this explicitly and treats Wi-Fi and Bluetooth tracking as processing of indirectly identifying personal data; municipalities may use it only under strict conditions, for specific periods and precisely described areas (Autoriteit Persoonsgegevens, 2025). In Germany, the same conclusion follows from Art. 4(1) GDPR and the official guidance of the Schleswig-Holstein data protection authority (Unabhängiges Landeszentrum für Datenschutz Schleswig-Holstein, 2017). Hashing does not solve the problem if the hash is stable and linkable over time. The GDPR's purpose limitation, data minimisation, storage limitation and DPIA obligations all apply; a large-scale systematic monitoring of a publicly accessible area triggers Art. 35(3)(c) (GDPR, 2016; EDPB, 2020).
**Deployment.** The method requires no cooperation from the public, which is exactly why it is attractive and exactly why it is contested. It also requires a legal basis that most city marketing or retail analytics purposes do not have. The project's own data notes make the same point: the PAX series is an index of estimated devices, the conversion to people is undocumented, and device penetration is unknown (PulseCast data notes, 2026).
**Verdict.** BLE/Wi-Fi MAC sniffing is not a measurement technology for public footfall. It is a low-cost radio-presence index with unknown and unstable coverage and a high privacy cost. The paper's critique is correct; the rest of this note is about what to use instead.
---
### 2.1 Camera-based people counting (edge AI, no face recognition)
**Principle.** A camera frames a doorway, corridor or plaza. A person detector (YOLO-family, SSD-MobileNet, or a newer transformer) locates people; a tracker associates detections across frames; a line-crossing or region-transition rule counts entries and exits. Edge inference means the video never leaves the device: the camera outputs counts, directions and confidence, not frames.
**Measured accuracy.** The most relevant outdoor evaluation is Cyganek et al. (2022): 18 locations, 207 minutes of real footage, almost 1,000 pedestrians, manual annotation as reference. Mean MAPE was 13.8% in daylight and 29.8% at night. The distribution matters more than the mean: the 9th-decile error was 5.3% by day and 7.3% by night, while the 1st-decile error was 30.5% and 62.8%. The reference is manual annotation, and the authors note that the manual counting error is unknown, so the MAPE is a comparison against an imperfect ground truth. In other words, a well-placed camera in good light can be within a few percent; a badly placed camera in the dark can miss a third or more of the crowd. Ahamed et al. (2024) report 97% accuracy in live overhead tests and 92% over a two-day continuous test, with 20–27 FPS on an Intel NUC 12 Pro and no GPU. That is a strong result for a controlled doorway, not a general outdoor guarantee. The NCDOT review reports 2–14% error for video imaging systems, with the caveat that complex intersection movements reduce accuracy (Ozan et al., 2021).
**Cost.** The NCDOT review puts commercial video-imaging systems at $1,200–$8,000 per unit and depth cameras at $9,900–$12,330 (Ozan et al., 2021). A do-it-yourself edge node — industrial camera, Jetson-class compute, weatherproof housing, PoE — is roughly €500–2,000 in hardware (est.). Installation, mounting, power and network add €200–500 per site (est.). The hidden cost is model validation: a generic detector trained on COCO or a European street scene will not be equally accurate on a rainy Heilbronn Saturday; local annotation and threshold tuning are part of the project.
**Privacy.** This is the highest-privacy-cost option among the non-MAC alternatives, even without face recognition. Frames containing identifiable people are personal data under Art. 4(1) GDPR. The EDPB's video-device guidelines explicitly discuss "simple counting algorithms" as less intrusive than biometric analytics, but they do not exempt them from the GDPR: lawfulness, necessity, proportionality, transparency and storage limitation still apply, and a DPIA is required for systematic monitoring of a publicly accessible area on a large scale (EDPB, 2020; GDPR, 2016, Art. 35(3)(c)). In Germany, §4 BDSG adds a necessity/proportionality test for video surveillance of publicly accessible spaces and requires deletion when the purpose is achieved (BDSG, 2017). The AI Act does not ban people counting as such; it prohibits real-time remote biometric identification in publicly accessible spaces for law enforcement, with limited exceptions (AI Act, 2024, Art. 5(1)(h)). Adding face recognition or re-identification would change the legal analysis and should be treated as out of scope.
The practical mitigation is architectural: inference on the device, frames discarded immediately, only counts and confidence transmitted, no remote access to the video stream, signed firmware, encrypted storage for the short buffer needed by the tracker, and a published retention policy (ideally zero frame retention). Even then, a camera in a public square is a visible intrusion; signage and a public explanation are not optional.
**Deployment.** Cameras need a clear line of sight, stable mounting, power and network. They struggle with low light, strong backlight, rain and snow, dense occlusion, and people carrying umbrellas or large bags. They are also the modality most likely to attract public opposition. In a city centre, the acceptable use is sparse: a few reference sites where the accuracy benefit justifies the privacy cost, not a blanket network.
**Best use.** Reference and validation at 2–4 complex sites: wide plazas, intersections, event spaces, and locations where thermal arrays cannot cover the width. Cameras are also the best tool for classifying pedestrians versus cyclists and for validating the thermal and radar counts.
**Verdict for Heilbronn.** Use a small number of edge cameras as the accuracy benchmark and for wide, high-value locations. Do not make them the default. The privacy and governance burden is real, and the accuracy advantage over a well-placed thermal array is modest at a doorway.
---
### 2.2 Thermal / IR array sensors
**Principle.** A thermopile array (typically 8×8, 32×24 or 80×60 pixels) measures infrared radiation and produces a low-resolution heat image. A person crossing the field of view appears as a moving warm blob. On-device processing thresholds the blob, tracks its direction, and increments an in/out counter. No visible image is formed; at 8×8, a person is a few pixels, not a recognisable face.
**Measured accuracy.** The doorway literature is consistent. Mohammadmoradi et al. (2017) report 93% average accuracy for room occupancy, with 96%/94% at a 180 cm door and 100%/96% at a 90 cm door; in uncontrolled classrooms and labs, accuracy was 89–92%. Maaspuro (2020) reports 89–92% for a doorway occupancy counter. Perra et al. (2021) report 95% average detection at normal walking speed for ambient temperatures up to 30 °C, 90% or better for brisk walking up to 26 °C, and 100% in a one-month stable 22–26 °C test. The failure modes are well characterised: accuracy falls with distance (1.5 m, 5 m, 7.5 m), with walking speed (normal, brisk, running), and when ambient temperature approaches body temperature (above 30 °C). At 7.5 m and 30–34 °C, detection can fall to zero. A single 8×8 sensor covers roughly 2.5 m × 2.5 m at the best mounting height; wider doors or plazas need multiple sensors or a higher-resolution array.
**Cost.** The Grid-EYE sensor is about $50 (Perra et al., 2021). A complete node — sensor, microcontroller, radio (LoRaWAN, Zigbee, Wi-Fi), power, housing — is roughly €150–400 (est.). A high-end thermal camera is a different product: the NCDOT review lists a FLIR camera and board at $4,800 (Ozan et al., 2021), which is unnecessary for counting at a doorway.
**Privacy.** This is the lowest-risk optical modality. At 8×8 or 32×24, the data are low-resolution heat maps, not photographic images; they are much harder to link to an individual. The EDPB's video guidelines are about devices that produce images of identifiable people; a low-resolution thermopile array that outputs counts is much easier to justify. That does not mean "no GDPR": if raw frames are stored and linked to a person, they can become personal data. The safe design is to process on the device, output only counts and direction, and never store raw frames. A DPIA is still prudent, but the risk assessment is simpler and the public explanation is easier ("a heat sensor counts entries; it cannot see you").
**Deployment.** Thermal arrays are best at narrow, controlled passages: store entrances, doorway thresholds, corridor mouths, shelter entrances. They need a mounting height of roughly 2.3–3.5 m, a clear view of the floor, and protection from direct sunlight and heat sources (radiators, heaters, ovens). Ambient temperature is the key constraint: in a heatwave, a 30 °C+ street and a 37 °C body are hard to separate. Groups are another constraint: two people walking shoulder-to-shoulder can merge into one blob. Directional counting is reliable for single-file passage; it degrades in dense, bidirectional crowds.
**Best use.** The default permanent counter for Heilbronn's narrow gateways: store entrances, the passages around Stadtgalerie, Marahaus and Götzenturm, and transit shelter entrances. Pair with a second sensor or a camera where the passage is wider than ~2.5 m.
**Verdict for Heilbronn.** The best privacy/cost/accuracy trade-off for the inner-city's narrow entries. It is not a plaza sensor. Use it where the geometry is on your side.
---
### 2.3 mmWave radar and LiDAR
**Principle.** A 60 GHz frequency-modulated continuous-wave (FMCW) radar transmits chirps and measures range, velocity and angle from the reflections. People appear as moving point clouds; clustering and tracking turn the cloud into tracks. LiDAR does the same with laser pulses, producing a geometric point cloud with centimetre-to-decimetre angular resolution. Neither modality forms a photographic image.
**Measured accuracy.** The outdoor evidence is more nuanced than the indoor literature. Vaidya and Zúñiga (2024), working with the municipality of Amsterdam, deployed a radar across a street and compared it with a camera: 96% accuracy for presence detection, 69% for people counting, and an aggregate undercount of only 2.48% (1,005 true vs 980 estimated) over the evaluation. The reference is a camera, so the comparison inherits the camera's own error. Wang et al. (2023) ran a three-month outdoor deployment and report 85% accuracy for detecting empty spaces, an F1 of 84%, and flow correlations of 60–85% depending on weather. They conclude that outdoor mmWave still needs improvement compared with cameras (error below 10% in their framing). Pallaprolu et al. (2024) report an MAE of 1.53 for crowds of up to 21 people using a single radar and a shadowing model, over 22 experiments in four areas. Sakhnini et al. (2024) evaluated a festival crowd of 0–1,500 people and found 82.9–94.2% correlation with bracelet-based access counts, with 95% prediction intervals of ±50% for small crowds and ±25% for large crowds. Ren et al. (2023) report 96.25% average tracking precision for grouped people in outdoor scenarios, but that is a tracking metric, not an exact crowd count. Indoor results are stronger, but they do not transfer unchanged to a street.
LiDAR has the strongest fixed-site counting evidence. Lesani et al. (2020) evaluated a 2D LiDAR system at two high-volume locations: more than 97% of pedestrians counted at the disaggregate level, 1.1% false direction, 0.7% overcount, 1.3% and 2.7% undercount, and 1.6%/4.3% average absolute percentage deviation at 15-minute aggregation. That is better than most camera results, but it comes from a carefully installed system at two sites. The NCDOT review reports an average error of 5% or more for laser scanning in crowded scenes and an $8,000 unit price (Ozan et al., 2021).
**Cost.** The 60 GHz radar node is the cost standout: Wang et al. (2023) itemise $311 for sensor, Raspberry Pi, radome, casing and misc; a commercial indoor node was $600. LiDAR is more expensive: low-cost 2D modules start around $100 (Umezu et al., 2023), but a deployment-grade outdoor unit with weatherproofing, mounting and processing is €1,000–8,000 (est.), and 3D LiDAR is higher. Installation and calibration add €300–800 per site (est.).
**Privacy.** Both modalities are privacy-preserving relative to cameras: they produce geometry, not photographs, and the raw signal does not contain facial features. That does not make them anonymous in all cases. A point cloud can track an individual through a space, and a persistent track can be linked to other data. The GDPR still applies if the data can be related to a person; the safe pattern is the same as thermal — process at the edge, output counts and tracks only for the minimum time, do not store raw point clouds, and document the purpose. mmWave can also pick up small movements, including breathing in some configurations, which is useful for stationary occupancy but raises the possibility of inferring more than presence; keep the inference scope narrow.
**Deployment.** mmWave at 60 GHz is licence-free in the ISM band and has a range of roughly 8 m in the outdoor deployments cited here. It is sensitive to rain, foliage, multipath and moving clutter; a wet street or a swaying tree can produce false points. It undercounts groups because people close together merge into one cluster. LiDAR is more accurate but more expensive, has moving parts in many units, and needs a clear, stable mounting with an unobstructed view. Both need power and backhaul.
**Best use.** mmWave is a good gap-filler at wide, weather-exposed crossings and transit stops where a thermal array cannot cover the width and a camera is politically difficult. Use it for presence and flow, not for exact staffing counts. LiDAR is best at one or two high-value reference sites where its accuracy justifies the cost.
**Verdict for Heilbronn.** Use one or two mmWave nodes at wide, exposed locations (Hagenbuchersee, Neckarufer) and, if the budget allows, one LiDAR reference site. Do not build the network on radar; the outdoor count error is too high for staffing without calibration.
---
### 2.4 Wi-Fi CSI sensing / device-free sensing
**Principle.** Wi-Fi channel state information (CSI) describes how the wireless channel distorts a known signal across subcarriers. People moving in the channel change the amplitude and phase; a classifier maps those changes to a crowd count. The method is "device-free" in the sense that the people do not need to carry a device, but the sensing hardware is a Wi-Fi transmitter and receiver.
**Measured accuracy.** The early results are impressive in controlled rooms. Brena et al. (2021) summarise the state of the art: Di Domenico et al. report about 80% accuracy for up to 7–8 people; Xi et al. report an 80% probability of a perfect count or a one-person error for up to 30 people. Their own multi-link approach reaches 99% with six links in three rooms, versus 44% with one link. Zhou et al. (2020) report a counting error within one person for 100% of cases at a crowd size of 6 and 97.7% at a crowd size of 34. More recent work reports 92.7% with round-trip-time measurements in a single 80 m² office (Yonekura et al., 2025) and 85% for 0–10 people in VICount (cited in Yonekura et al., 2025). But the same literature documents the weakness: environment dependence. Santos et al. (2024) collected data over three months in multiple rooms and found that a model trained without knowledge of the target environment performed substantially worse; partial training on the target environment improved classification accuracy by about 16–20%. Yonekura et al. (2025) report 61% cross-domain accuracy for VICount and note that CSI lacks standardisation, which limits practical deployment.
**Cost.** The radio hardware is cheap: a transmitter, a receiver and two antennas. The expensive part is engineering. CSI extraction is not exposed by all commodity access points; the widely used Intel 5300 NIC is no longer current; different chipsets expose different formats; and each room needs calibration. A realistic pilot budget is €10k–30k for engineering, plus €200–1,000 per link (est.). This is not a plug-and-play sensor.
**Privacy.** CSI does not require MAC addresses, which is a genuine advantage over probe sniffing. But it is not automatically anonymous: the signal changes with presence, movement, and even breathing, and a trained model can infer occupancy patterns over time. If the data are linked to a household or a specific room, they can be personal data. The GDPR's purpose limitation and minimisation principles still apply; a DPIA is appropriate for a building-scale deployment. The privacy story is better than MAC sniffing, but it is not "privacy-free".
**Deployment.** CSI is best indoors, in rooms or corridors with stable geometry and a known transmitter/receiver layout. It needs a calibration dataset per environment, and it drifts when furniture, doors, or the crowd's behaviour change. It is poor at distinguishing people from other moving objects (carts, doors, pets) and at counting dense crowds. Outdoor use is rare and difficult because of multipath, weather and uncontrolled geometry.
**Best use.** Indoor occupancy in offices, libraries, care facilities or transit waiting rooms where cameras are unacceptable and a thermal array cannot cover the room. It is a research-grade modality, not a city-scale footfall sensor.
**Verdict for Heilbronn.** Not ready for the inner-city footfall network. Keep it as an indoor research option if a building-level occupancy use case appears.
---
### 2.5 Wi-Fi probe request sniffing
**Principle.** Wi-Fi devices periodically send probe requests to discover access points. Each request carries a MAC address and information elements. A passive monitor records the requests, deduplicates by MAC, and counts unique devices in a time window.
**Measured accuracy.** Musa and Eriksson (2012) is the canonical field study. In a 12-hour trial on a road with 37,000 average daily traffic, a single roadside monitor detected 7,000 unique devices, or about one device for every five vehicles. Across their deployments, a monitor detected a passing smartphone 69% of the time **if Wi-Fi was turned on**, and their trajectory estimates had a mean error of 67 m against GPS. The conditional is doing a lot of work: Wi-Fi-on rates vary, and the method cannot see devices with Wi-Fi off, devices in power-saving mode, or people without phones. Those deployments were vehicle-oriented, so 69% is not a pedestrian detection rate; it illustrates the conditional nature of the measurement. The count is an index of emitting devices, not people. The same study calls the method "relatively benign" because it only tracks phones in an area of interest, and suggests seeded hashes as a further protection; that is a research-era judgment, not a modern legal position.
**Randomisation.** Martin et al. (2017) show that MAC randomisation is widely deployed but poorly implemented: devices sometimes send their true global address when they should use a random one, and probe-request fingerprints can re-identify devices across address changes. They defeated randomisation for ~96% of Android phones and found a control-frame flaw affecting 100% of tested devices. The practical consequence is that probe sniffing is both biased (you see a non-random subset) and intrusive (you can re-identify that subset).
**Cost.** Low hardware cost: Musa and Eriksson used $70 access points. A modern monitor is €100–500. The compliance cost is high: legal review, DPIA, signage, retention, access control, and likely public consultation.
**Privacy.** MAC addresses are personal data (Autoriteit Persoonsgegevens, 2025). The processing is systematic monitoring of a publicly accessible area, which triggers a DPIA under Art. 35(3)(c) for large-scale deployments. The ePrivacy Directive's rules on location data and terminal equipment add another layer (ePrivacy Directive, 2002). The German supervisory authorities have repeatedly treated Wi-Fi tracking as personal-data processing (Unabhängiges Landeszentrum für Datenschutz Schleswig-Holstein, 2017), and the burden is on the controller to show necessity and proportionality. A city cannot rely on "we only count" if the raw data contain persistent identifiers and the purpose is commercial or operational analytics.
**Deployment.** The method is passive and cheap, which is why it persists. But it fails the two tests that matter for this paper: it does not measure people, and it cannot be made privacy-preserving by hashing alone. The only defensible use is a short, purpose-limited research study with ethics approval, a DPIA, immediate aggregation, and no cross-site linkage. That is not a footfall service.
**Verdict.** Not a solution. The paper should say so plainly and explain why the low hardware cost is a false economy.
---
### 2.6 Telco network data / mobile signalling
**Principle.** Mobile networks record when a device attaches, moves between cells, makes a call, or uses data. Call detail records (CDRs) and passive signalling data can be aggregated to estimate the number of subscribers in a cell or sector, and their flows between cells. The data are collected by the operator, not by the city.
**Measured accuracy.** Deville et al. (2014) used more than 1 billion call records from Portugal and France to produce national-scale population density estimates and compared them with census-based alternatives; they show that mobile-phone-based maps can reproduce population distributions at national scale, while stressing the need for calibration and anonymity. The spatial resolution is the limiting factor. The U.S. Bureau of Transportation Statistics puts cellular probe accuracy at 250–1,000 ft in urban areas, 1,500–3,000 ft in suburban areas and 1–3 miles in rural areas, and states that it is not precise enough for intersection-level or mode-separated analysis (BTS, 2025). Flowminder notes that urban subscribers are generally within about 500 m of a cell tower, with much lower density in rural areas (Flowminder, n.d.). A 2024 comparison of GPS, GSM and CDR for travel trajectories also finds higher measurement errors in GSM and CDR than in GPS, especially for departure times and travel distances (Zannat et al., 2024). That is fine for a district-level estimate and useless for a doorway count. Temporal resolution is also limited by when the network records an event; passive signalling is denser than CDR but still not a continuous people counter.
**Cost.** There is no sensor to buy. The cost is a data licence or a data-sharing agreement with one or more operators. These are bespoke: price depends on spatial and temporal resolution, history, exclusivity, and whether the operator does the aggregation. A planning range of €10k–100k+ per year for an aggregated city-level feed is realistic (est.), but it is an order of magnitude, not a quote. In the U.S., major carriers stopped sharing CDR data with analytics vendors in 2018 after privacy and public-relations concerns (BTS, 2025); the European market is different but still contract-driven.
**Privacy.** This is high-risk data. Even pseudonymised subscriber IDs can be re-identified through mobility patterns, home/work inference, and linkage with other data. The GDPR applies; the ePrivacy Directive applies to location data; a DPIA and a robust data-processing agreement are mandatory; aggregation to a minimum cell size and a minimum count threshold is necessary but not sufficient. The operator is usually the controller or joint controller for the raw data; the city should receive only aggregated, thresholded outputs and should not have access to subscriber-level records. The purpose must be specified and limited; "smart city analytics" is too broad.
**Deployment.** The city needs operator cooperation, a legal basis, a procurement route, and a data governance structure. Coverage and market share vary; prepaid users, children, tourists and multiple-SIM users distort the picture. The data are usually delivered with a lag (hours to days), which is fine for planning and useless for real-time staffing. The BTS guide also warns that the methodology for cellular probe data is dated in the U.S. because of the 2018 withdrawal; in Europe, ask the operator for a current validation.
**Best use.** City-wide context: district-level population, origin–destination flows, tourism and event impacts, and a prior for downscaling. It is not a street-level footfall sensor.
**Verdict for Heilbronn.** Useful as a city-wide covariate or prior, not as the measurement layer. The spatial resolution and governance burden rule it out for doorway or plaza counts.
---
### 2.7 GPS probe data / map providers
**Principle.** Apps and navigation services collect GPS traces from opted-in users. The provider map-matches the traces to a road or path network and sells aggregated counts, speeds or origin–destination matrices. Strava Metro is the best-known pedestrian/cyclist example; INRIX and TomTom are vehicle-oriented.
**Measured accuracy.** GPS itself is accurate to a few metres in open sky, but the sample is not the population. Venter et al. (2023) compared Strava counts with observed recreational use and found that Strava users are biased toward cyclists (+8%), males (+15.7%) and ages 35–54 (+20.4%). Walkers and hikers are underrepresented; pedestrian data are rare (Lee & Sener, 2020). Strava Metro's own case study reports model accuracy of ±32 to ±160 cyclists on 75% of streets — a wide interval that reflects the difficulty of scaling from a biased sample to total volumes (Strava Metro, n.d.). In indoor spaces, in urban canyons, and in dense historic centres, GPS drops out or degrades; the inner city of Heilbronn has exactly the kind of narrow streets and dense building frontage where this matters.
**Cost.** Strava Metro was priced at about $0.55 per app user per year in the NCDOT review (Ozan et al., 2021); a city with 130,000 residents and a 10% Strava user base would pay roughly $7,000 per year at that historical rate, but commercial pricing has changed and should be quoted. A commercial probe-data subscription is typically €10k–50k per year (est.). The hidden cost is validation: without independent counts, you cannot know the sampling fraction.
**Privacy.** GPS traces are among the most sensitive mobility data. Even aggregated and de-identified, they can reveal home, work, health-care visits and associations. The GDPR applies; consent and purpose limitation are central; commercial re-use and onward sharing need careful contracts. Strava's privacy settings and aggregation reduce but do not eliminate the risk.
**Deployment.** Coverage bias is the core problem: fit, male, middle-aged cyclists are overrepresented; children, older people, women, and people without smartphones are underrepresented. The data are good for route choice and recreational demand, weak for total footfall. Indoor and covered areas are invisible. Licensing and data-sharing agreements take time.
**Best use.** Cyclist route and origin–destination analysis, recreational demand, and long-term trend detection where the sample bias is stable and can be calibrated. Not inner-city pedestrian footfall.
**Verdict for Heilbronn.** Not suitable for the inner-city footfall question. It could be a supplementary dataset for cycling or tourism, clearly labelled as a biased sample.
---
### 2.8 Transit smart cards, ticket validators and turnstiles
**Principle.** Automated fare collection (AFC) records every validation: card or ticket ID, time, stop or station, and sometimes the vehicle. Turnstiles physically count entries. Automated passenger counters (APCs) on vehicles use infrared, camera or pressure sensors to count boardings and alightings. The data are administrative by-products, not a research sensor.
**Measured accuracy.** For the population that validates, the record is transaction-level: the validator either read the card or it did not. The problem is coverage. Smart card data capture only trips paid by smart card; cash passengers, non-validating passengers, and fare evaders are missing (Pelletier et al., 2011). Munizaga et al. (2020) estimate a 5% partial evasion rate for bus trip stages before a metro trip and a 37% total fare evasion rate for bus-only trips in their system. Caltrans (2025) reports that fare evasion in some California agencies rose from about 20% to nearly 50% after the pandemic, which directly biases ridership counts that rely on fare equipment. APCs can fill part of the gap, but they need their own validation against manual counts and against fare data.
Heilbronn's transit context is substantial: the Stadtwerke Heilbronn operates 14 city bus lines and the S4/S41/S42 Stadtbahn lines, with about 22.8 million passengers per year (5.5 million Stadtbahn, 17.3 million bus) (Stadtwerke Heilbronn, n.d.). That is a strong, high-frequency signal for commuting, shopping and event flows. It is not a pedestrian count for the Innenstadt, because transit passengers are a subset of pedestrians and the validators do not see people who walk or cycle.
**Cost.** If the AFC system already exists, the marginal cost of an aggregated data extract is low: a data-sharing agreement, an analyst, and a privacy review. New validators or turnstiles are capital projects with vehicle/station installation costs; a planning range of €5k–15k per vehicle or gate is plausible (est.), but it depends heavily on the system. APCs are cheaper than turnstiles but need calibration.
**Privacy.** Individual trip records are personal data. They are pseudonymous, not anonymous, and can be linked across days. The GDPR applies; the legal basis for the transit operator is usually the transport contract or a public task, but sharing with the city for analytics needs a separate basis and a DPIA. Aggregate to stop-level or line-level counts, apply a minimum count threshold, and do not publish trip-level data.
**Deployment.** The city needs the operator's cooperation and a clear data-sharing agreement. The data are usually available at stop/line level, with a delay; real-time feeds exist but are operational, not analytical. Fare policy and validation rules change the denominator. The transit network covers only part of the pedestrian system.
**Best use.** A high-quality covariate for transit demand, commuting patterns and event flows; a cross-check for city-wide models; not a general footfall sensor.
**Verdict for Heilbronn.** Use aggregated validator counts as a covariate in the fusion model. Do not use them as the denominator for inner-city footfall.
---
### 2.9 Manual counts and video annotation as ground truth
**Principle.** A trained observer stands at a cordon or screen line and counts people with a clicker or a tally sheet, usually by direction and sometimes by mode. Video annotation is the same task performed on recorded footage, which allows review and inter-rater checks.
**Measured accuracy.** Manual counting is accurate but not perfect. Diogenes et al. (2007) compared methods at intersections and found an average error of 11% for manual clickers, with 8% and 15% on two different dates. The error is systematic in one direction: the NCDOT review notes that manual entry is prone to undercounting and that error rates tend to be greater at the beginning of a session (Ozan et al., 2021). The TREC guide is blunt: a human can be expected to count accurately for about two hours; estimates of annual average daily pedestrian/bicycle traffic from just two hours of counts can have as much as 60% error on average; one week of counts substantially improves accuracy (Nordback, 2014). Video annotation removes weather and field fatigue but adds digitisation cost and its own inter-rater error; it is the standard way to audit automated counters.
**Cost.** In Germany, the 2026 minimum wage is €13.90 per hour (BMAS, 2026). A loaded observer rate — wage plus employer contributions, training, supervision, travel and equipment — is roughly €25–40 per hour (est.). A campaign at four sites, seven days, four two-hour shifts per day, two observers per shift is 448 observer-hours, or €11k–18k (est.). Video annotation is similar or slightly cheaper per hour but requires a secure viewing environment and blurring/retention controls. These are planning estimates; a procurement quote will differ.
**Privacy.** Count-only manual data are not personal data. Video annotation is different: the footage contains identifiable people, so the same GDPR/DPIA logic as camera counting applies. The mitigation is to record only what is needed, blur faces at ingest, restrict access, delete after annotation, and never publish footage.
**Deployment.** Manual counts need a sampling design, not just a person with a clicker. Choose cordons that capture the relevant flow, stratify by day type and time of day, rotate observers, take breaks every two hours, and measure inter-rater reliability on a subsample. Weather, holidays, events and school terms all affect the count; record them. For Heilbronn, a twice-yearly campaign at 3–4 sites, plus event-specific counts, is a realistic ground-truth program.
**Best use.** Calibration and validation of every automated sensor; estimation of detection probabilities; audit of vendor claims; occasional event evaluation. It is the only modality in this note that can claim to measure people directly.
**Verdict for Heilbronn.** Essential. A sensing stack without a manual/video calibration program is an unvalidated index. Budget for it from the start.
---
### 2.10 Sensor fusion / Bayesian hierarchical models
**Principle.** Each sensor measures a noisy, biased transformation of the same latent quantity: the number of people passing a site in a time interval. A fusion model treats the latent count as the parameter of interest and each sensor as an observation with its own detection probability, bias, and error structure. Hierarchical priors share information across sites and time, which is essential when calibration data are sparse.
**Measured accuracy.** There is no single accuracy number. The value of fusion is that it makes the uncertainty explicit and lets sparse ground truth calibrate many sensors. Parsons et al. (2022) show how a Bayesian hierarchical model can combine multiple estimates from different sources when none is complete; Zaouche and Bode (2023) formulate Bayesian spatio-temporal models to map urban pedestrian traffic from sparse count data. The transport literature has used hierarchical models for traffic counts and small-area estimation for decades; the same machinery applies here (Rao & Molina, 2015; Gelman et al., 2013). The practical test is not "does it beat the best sensor?" but "does it give a calibrated interval and a defensible estimate when the best sensor is missing?".
**Cost.** Software and engineering, not hardware: €20k–60k for model development, data pipeline and validation; €5k–15k per year for maintenance and re-calibration (est.). The biggest cost is the calibration program, which is covered under manual counts.
**Privacy.** The fused output is usually a count with an interval, which can be published as open data. The privacy risk comes from the inputs: if the model ingests camera frames, MAC addresses or individual traces, it inherits their risk. The design rule is to fuse aggregates and counts, not raw personal data.
**Deployment.** Fusion needs a data model, a calibration plan, and someone who can explain the output. It fails when the sensors are correlated in the same way (e.g., all undercount in rain) or when the calibration data are used both to fit and to evaluate. It also fails when the organisation wants a single number and ignores the interval. The model must be versioned, documented and re-estimated when sensor behaviour drifts.
**Best use.** The core of a multi-sensor city network: combine thermal, camera, radar, manual and transit data; downscale city-wide priors to street level; produce counts with credible intervals.
**Verdict for Heilbronn.** This is the "right way" layer. Buy fewer sensors and invest in the model and calibration.
---
## 3. Fusion and calibration
### 3.1 The latent-variable model
Let \(N_t\) be the true number of people crossing a site in interval \(t\) (e.g., 15 minutes). Sensor \(s\) produces \(y_{st}\). A defensible baseline is a negative-binomial observation model:
\[
y_{st} \mid N_t, q_{st} \;\sim\; \mathrm{NegBin}\!\left(N_t q_{st},\ \phi_s\right)
\]
where \(q_{st} \in (0,1)\) is the sensor's detection probability and \(\phi_s\) controls overdispersion. The latent count is modelled on the log scale with time, weather, holiday and event effects:
\[
\log N_t = \alpha + f_{\text{time}}(t) + f_{\text{weather}}(t) + u_t,
\qquad u_t \sim \mathrm{AR}(1)
\]
The detection probability is modelled hierarchically, with partial pooling across sites:
\[
\mathrm{logit}(q_{st}) = \beta_s + \gamma^\top x_t + \varepsilon_{st},
\qquad \beta_s \sim \mathcal{N}(\mu_\beta, \tau_\beta^2)
\]
where \(x_t\) includes light, rain, temperature, crowding and sensor health. Manual counts \(m_t\) enter as a high-quality observation:
\[
m_t \mid N_t \;\sim\; \mathrm{NegBin}(N_t, \phi_m)
\]
or as a lognormal with a known coefficient of variation. The posterior is proportional to the product of the sensor likelihoods, the manual likelihood, the latent-count prior and the detection-probability prior. Report the posterior median and an 80% credible interval for \(N_t\). The interval is the product; a point estimate without it is not a measurement.
This structure does three things that a simple average cannot. It separates the true count from sensor-specific detection, so a sensor that sees 40% of people is not treated as a sensor that sees 100%. It lets a well-calibrated site inform a poorly calibrated one through the hierarchical prior. And it propagates the uncertainty from the manual counts into the final estimate.
### 3.2 Calibration design
Calibration is the difference between a sensor network and a measurement system. A workable program for Heilbronn:
1. **Sites.** Calibrate at 3–4 sites that represent the main geometries: a narrow doorway, a wide plaza, a transit stop, and a street segment.
2. **Ground truth.** Two intensive one-week campaigns per year at 3–4 sites: four two-hour shifts per day (morning, midday, afternoon, evening), two observers per shift. That is 448 observer-hours per campaign at four sites. Between campaigns, run one-day checks at each site to track drift.
3. **Video audit.** Record a subsample and annotate it independently. Use it to estimate inter-rater reliability and to check the field counts. Blur faces at ingest and delete the footage after annotation.
4. **Covariates.** Record weather, light, events, holidays, and sensor health at each calibration interval.
5. **Model.** Fit the hierarchical model with partial pooling. Estimate \(q_{st}\) by site, season and light condition. Check posterior predictive intervals against held-out counts.
6. **Drift.** Track \(q_{st}\) over time with control charts. If a site's detection probability shifts by more than about 5 percentage points, inspect the sensor and re-calibrate.
7. **Validation.** Hold out entire days, not random intervals, to avoid leakage. Report MAPE, bias, 80% interval coverage and rank correlation. A model that is accurate on average but has 50% interval coverage is not calibrated.
### 3.3 Identifiability, confounding and double counting
The model is only identifiable if at least one sensor is calibrated or if the priors are strong. If every sensor is an uncalibrated index, the data identify relative variation, not the level. This is the core problem with the existing PAX network: the device counts are informative about temporal patterns, but the mapping to people is unknown.
Three practical safeguards:
- **Negative controls.** Use closed days, night hours and known-zero periods to estimate false positives. A sensor that counts people in an empty street is not calibrated.
- **Positive controls.** Use events with independent counts (ticketed events, transit validations, manual cordons) to anchor the upper range.
- **Overlapping sensors.** Where two sensors cover the same flow, compare their counts and estimate capture–recapture. Do not assume independence: sensors that share a failure mode (rain, darkness) will be correlated. Use a log-linear model with an interaction term, or treat the overlap as a diagnostic rather than a correction.
Double counting is a design problem, not a modelling problem. Define the cordon, the direction, and the time window before installation. A person who enters and leaves twice is two crossings; a person who stands in the zone is occupancy, not flow. Decide which quantity the decision needs: footfall (crossings), occupancy (people present), or unique visitors (deduplicated over a period). They require different sensors and different models.
### 3.4 Reporting and decision rules
The output should be three numbers, not one: a point estimate, an 80% interval, and a calibration date. For staffing, use the lower bound of the interval for the minimum staff calculation and the central estimate for the expected cost. If the interval width exceeds 50% of the median, label the series "trend only" and do not use it for staffing. If the holdout bias exceeds 10%, re-calibrate before publishing. These thresholds are design choices, not universal truths; state them in the paper and in the dashboard.
### 3.5 A minimal implementation plan
1. Start with a calibration-factor model: one detection probability per sensor, estimated from manual counts, with bootstrap intervals. This is simple, transparent and enough for a pilot.
2. Move to the hierarchical negative-binomial model when there are enough calibration days to estimate partial pooling.
3. Fit in PyMC, Stan or a comparable probabilistic programming language; version the model and the data; publish the model card.
4. Validate on held-out days and on a site that was not used for fitting.
5. Publish the counts, intervals, calibration dates and code; do not publish raw video, MAC addresses or individual traces.
---
## 4. Recommendations for Heilbronn
### 4.1 Context
Heilbronn has about 131,986 residents and 100 km² of city area (Bitkom, 2026). The inner city is a compact retail and leisure cluster; the PulseCast data cover eight PAX sensors in a roughly 580 m × 280 m area, at 15-minute cadence, from June to November 2026. The values are device estimates, not people (PulseCast data notes, 2026). The city already has a digital strategy, a LoRaWAN initiative and an open-data programme (Stadt Heilbronn, n.d.-a, n.d.-b), which lowers the deployment cost for a low-power sensor network. The "Zukunft Innenstadt" programme already frames the inner city as a policy priority (Stadt Heilbronn, n.d.-c). The transit system carries about 22.8 million passengers per year (Stadtwerke Heilbronn, n.d.), a strong covariate for commuter and event flows.
The goal is not a city-wide heatmap. It is a defensible, privacy-preserving measurement layer for the inner city that can support staffing, event operations and urban planning, with uncertainty that is honest enough to make decisions on.
### 4.2 Design principles
1. **Measure people, not devices.** Every sensor must have a documented relationship to a people count, calibrated locally. If it does not, label it an index.
2. **Use the least invasive sensor that meets the accuracy requirement.** Thermal before camera; never face recognition; never MAC addresses.
3. **Process at the edge, store aggregates.** No raw video, no raw point clouds, no MAC addresses. Counts and confidence only.
4. **Calibrate continuously.** A regular manual/video campaign, at least twice a year, is part of the capital plan, not an afterthought.
5. **Publish uncertainty.** Every count comes with an interval and a calibration date. The dashboard should not show a single number without context.
6. **Keep it open and vendor-neutral.** Open data, open code, documented APIs, no lock-in to a single sensor vendor.
7. **Do the legal work first.** DPIA, legal basis, signage, retention, processor agreements, and a public explanation before the first sensor is installed.
### 4.3 Reference stack
| Tier | Sites | Modality | Output | Expected accuracy | Role |
|---|---|---|---|---|---|
| **1. Permanent core** | 8–12 narrow gateways: store entrances, passages at Stadtgalerie, Marahaus, Götzenturm, Soleo | Low-resolution thermal array (8×8 or 32×24), on-device counting, LoRaWAN/Wi-Fi backhaul | 15-min in/out counts, occupancy | 90–95% per passage under good conditions; lower in heat, groups or wide openings | Default measurement layer; privacy-first; low cost |
| **2. Reference / wide areas** | 2–4 wide plazas or complex crossings: Hagenbuchersee, Neckarufer, event spaces | Edge AI camera, on-device inference, no face recognition, frames discarded | Counts, direction, pedestrian/cyclist classification | 10–20% MAPE expected outdoors; validate locally | Accuracy benchmark; wide-area coverage; validation |
| **3. Gap-fill** | 1–2 weather-exposed or wide crossings, transit stops | 60 GHz mmWave (and optionally one LiDAR reference) | Presence, flow, count with wide interval | 69–96% presence/count depending on scene; ±25–50% for small crowds | Cover where thermal/camera cannot; not for exact staffing |
| **4. Ground truth** | 3–4 representative sites, twice yearly | Manual counts + video annotation | Cordon counts by direction and time | ~11% manual error; improve with training and video audit | Calibration, validation, audit |
| **5. Context** | City-wide | Transit validators, weather, events, holidays; optionally telco/GPS aggregates | Covariates and priors | Transit exact for validators; telco 250–1,000 ft urban; GPS biased | Explain demand; downscale city-wide trends |
**Site logic.** Use thermal where the passage is narrow and the flow is single-file or two-file. Use cameras where the area is wide, the count is high-value, and the privacy assessment supports it. Use radar where the width or weather rules out thermal and the budget rules out camera. Do not put a thermal array on an open plaza and expect 90% accuracy; the geometry is wrong.
**A note on the existing PAX sensors.** Keep two or three in place for continuity during the transition. Run them side-by-side with the new sensors for 12 weeks, estimate a calibration ratio, and publish it. Then either phase them out or keep them as a temporal covariate, clearly labelled as a device index. Do not present the historical series as people; the penetration rate is unknown and has probably changed over five months.
### 4.4 Pilot protocol
A 12-week pilot at six sites is enough to estimate accuracy and operational cost without committing to a city-wide rollout.
**Sites.** Two narrow doorways (thermal), two wide plazas (camera), one transit stop (radar), one control site with manual counts only. Use the existing PAX locations where possible to allow side-by-side comparison.
**Ground truth.** Two intensive one-week campaigns (weeks 1 and 12), four two-hour shifts per day, two observers per shift, rotating across 3–4 sites; 448 observer-hours per campaign at four sites. Video a subsample for audit. Record weather, events, holidays and sensor health.
**Metrics.** MAPE and bias against manual counts; 80% interval coverage; correlation with the PAX index; availability (percentage of expected 15-minute intervals with data); cost per site-year; privacy incidents; installation time.
**Go/no-go thresholds.** A sensor is fit for staffing if MAPE ≤ 20%, absolute bias ≤ 10%, 80% interval coverage ≥ 75%, availability ≥ 95%, and there are no unresolved privacy issues. If it misses, use it for trends only. These thresholds are deliberately modest; a 20% error is still useful for a weekly shift plan, but it is not a headcount.
**Analysis.** Fit the hierarchical model on the first eight weeks and validate on the last four. Publish the pre-registered metrics and the model card. Do not tune the model on the validation weeks.
### 4.5 Legal and governance checklist
- **Legal basis.** For a city, Art. 6(1)(e) GDPR (public task) is the likely basis for footfall measurement; document the task and the necessity. For a private operator, Art. 6(1)(f) may apply, but a public-space deployment needs a careful balancing test.
- **DPIA.** Required for systematic monitoring of a publicly accessible area on a large scale (Art. 35(3)(c) GDPR). Do it before procurement, not after. Include a necessity and proportionality assessment, a data-flow diagram, a retention schedule, and a risk register.
- **BDSG §4.** Video surveillance of publicly accessible spaces must be necessary for the task, the purpose must be specified, and data must be deleted when no longer needed (BDSG, 2017). Apply the same logic to any modality that can identify people.
- **AI Act.** No face recognition, no biometric categorisation, no emotion recognition. People counting without identification is outside the Art. 5(1)(h) prohibition, but adding identification would change the legal analysis (AI Act, 2024).
- **Transparency.** Signage at each site, a public web page explaining what is measured, why, how long it is kept, and how to complain. For thermal, say explicitly that no images are formed.
- **Retention.** Zero raw frames; counts retained indefinitely as open data; calibration video deleted after annotation; access logs retained for audit.
- **Procurement.** Require on-device processing, no hidden cloud upload, open APIs, data ownership by the city, security testing, and an exit/portability clause.
- **Data protection by design.** Edge inference, encryption, signed firmware, least privilege, no default passwords, regular updates, incident response.
- **Publication.** Open 15-minute counts with intervals and calibration dates; a data dictionary; a DPIA summary; the model code. Do not publish raw video, MAC addresses or individual traces.
### 4.6 Budget (planning estimate)
| Item | Year-1 range (€) | Basis |
|---|---:|---|
| 8–12 thermal nodes | 2,000–5,000 | €150–400 per node; sensor ~$50 |
| 2–4 edge cameras | 1,500–8,000 | €500–2,000 per node; commercial systems higher |
| 1–2 mmWave nodes | 600–1,400 | €300–700 per node |
| Installation, mounting, power, backhaul | 8,000–20,000 | Site-dependent; assumes use of existing street furniture and LoRaWAN/Wi-Fi |
| Backend, dashboard, integration | 20,000–40,000 | Data pipeline, API, open-data portal, security |
| Calibration (manual + video) | 22,000–36,000 | Two campaigns per year, ~448 observer-hours each; ~€25–40 per observer-hour |
| DPIA, legal, procurement | 8,000–15,000 | External counsel and DPO time |
| Project management, training, comms | 8,000–15,000 | Part-time coordination, training and public communication |
| Contingency (15%) | 10,500–21,000 | Hardware delays, site changes |
| **Total, year 1** | **~81,000–161,000** | All figures are estimates, not quotes |
| Recurring, year 2+ | 30,000–50,000 | Calibration, maintenance, cloud, replacements |
The budget is deliberately hardware-light and calibration-heavy. That is the right ratio. A network of cheap uncalibrated sensors is not cheaper than a smaller network with a real ground-truth program; it is just less honest.
### 4.7 What not to do
- **Do not use BLE or Wi-Fi MAC sniffing for public footfall.** The data are biased, the penetration is unknown, and the privacy cost is high. The paper's critique is correct.
- **Do not buy a "95% accuracy" claim without a local validation.** The same sensor can be 95% in a controlled doorway and 70% on a windy plaza. Ask for the evaluation protocol, the ground truth and the error distribution.
- **Do not use a single sensor for staffing.** Every modality has a failure mode; fuse at least two, and calibrate against manual counts.
- **Do not store raw video or point clouds.** The marginal analytical value is small; the legal and reputational risk is large.
- **Do not use face recognition, gait recognition or re-identification.** It is unnecessary for counting and changes the legal regime.
- **Do not compare counts across sensors without calibration.** A thermal count, a camera count and a radar count are different measurements until they are mapped to the same latent quantity.
- **Do not present device counts as people.** Label the historical PAX series as an index and calibrate it or retire it.
---
## 5. Citations
Ahamed, I., Ranathunga, C. D., Udayantha, D. S., Ng, B. K. K., & Yuen, C. (2024). *Real-Time AI-Driven People Tracking and Counting Using Overhead Cameras.* arXiv:2411.10072. https://arxiv.org/abs/2411.10072
Autoriteit Persoonsgegevens (Dutch Data Protection Authority). (2025). *Wifi and bluetooth.* https://www.autoriteitpersoonsgegevens.nl/en/themes/internet-and-smart-devices/wifi-and-bluetooth
Bitkom. (2026). *Smart City Index 2026: Heilbronn.* https://www.bitkom.org/Smart-City-2026/Heilbronn
Brena, R. F., Escudero, E., Vargas-Rosales, C., Galvan-Tejada, C. E., & Munoz, D. (2021). Device-free crowd counting using multi-link Wi-Fi CSI descriptors in Doppler spectrum. *Electronics, 10*(3), 315. https://doi.org/10.3390/electronics10030315
Bundesdatenschutzgesetz (BDSG). (2017). § 4 Videoüberwachung öffentlich zugänglicher Räume. https://www.gesetze-im-internet.de/bdsg_2018/__4.html
Bundesministerium für Arbeit und Soziales (BMAS). (2026). *Allgemeiner gesetzlicher Mindestlohn.* https://www.bmas.de/DE/Arbeit/Arbeitsrecht/Mindestlohn/mindestlohn.html
California Department of Transportation (Caltrans). (2025). *PI-0393 Fare Evasion and its Impact on Measured Ridership* (T. Monson, author). https://dot.ca.gov/-/media/dot-media/programs/research-innovation-system-information/documents/preliminary-investigations/pi-0393-fare-evasion-and-effects-on-ridership-counts-final-v2-a11y.pdf
Cyganek, B., Janyst, P., Knapik, M., Przebinda, Ł., Ptaszkiewicz, M., & Tajmajer, T. (2022). *Evaluation of an Embedded Computer Vision System for Outdoor Media Audience Analysis.* PP-RAI 2022. https://wydawnictwo.umg.edu.pl/pp-rai2022/pdfs/07_pp-rai-2022-101.pdf
Deville, P., Linard, C., Martin, S., Gilbert, M., Stevens, F. R., Gaughan, A. E., Blondel, V. D., & Tatem, A. J. (2014). Dynamic population mapping using mobile phone data. *Proceedings of the National Academy of Sciences, 111*(45), 15888–15893. https://doi.org/10.1073/pnas.1408439111
Diogenes, M. C., Greene-Roesel, R., Arnold, L. S., & Ragland, D. R. (2007). Pedestrian counting methods at intersections: A comparative study. *Transportation Research Record, 2002*(1), 26–30. https://doi.org/10.3141/2002-04
European Data Protection Board (EDPB). (2020). *Guidelines 3/2019 on processing of personal data through video devices* (Version 2.0). https://www.edpb.europa.eu/sites/default/files/files/file1/edpb_guidelines_201903_video_devices_en_0.pdf
European Union. (2002). *Directive 2002/58/EC of the European Parliament and of the Council of 12 July 2002 concerning the processing of personal data and the protection of privacy in the electronic communications sector (ePrivacy Directive).* https://eur-lex.europa.eu/eli/dir/2002/58/oj
European Union. (2016). *Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 (General Data Protection Regulation).* https://eur-lex.europa.eu/eli/reg/2016/679/oj
European Union. (2024). *Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (AI Act).* https://eur-lex.europa.eu/eli/reg/2024/1689/oj
Flowminder. (n.d.). *Strengths and limitations of Call Detail Records (CDR data).* https://flowgeek.org/what-are-cdrs/what-are-the-strengths-and-limitations-of-cdr-data
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). *Bayesian Data Analysis* (3rd ed.). CRC Press.
Kurkcu, A., & Ozbay, K. (2017). Estimating pedestrian densities, wait times, and flows with Wi-Fi and Bluetooth sensors. *Transportation Research Record, 2644*(1), 72–82. https://doi.org/10.3141/2644-09
Lee, K., & Sener, I. N. (2020). Emerging data for pedestrian and bicycle monitoring: Sources and applications. *Transportation Research Interdisciplinary Perspectives, 4*, 100095. https://doi.org/10.1016/j.trip.2020.100095
Lesani, A., Nateghinia, E., & Miranda-Moreno, L. F. (2020). Development and evaluation of a real-time pedestrian counting system for high-volume conditions based on 2D LiDAR. *Transportation Research Part C: Emerging Technologies, 114*, 20–35. https://doi.org/10.1016/j.trc.2020.01.018
Maaspuro, M. (2020). A low-resolution IR-array as a doorway occupancy counter in a smart building. *International Journal of Online and Biomedical Engineering, 16*(6), 4–18. https://doi.org/10.3991/ijoe.v16i06.13915
Martin, J., Mayberry, T., Donahue, C., Foppe, L., Brown, L., Riggins, C., & Rye, E. C. (2017). A study of MAC address randomization in mobile devices and when it fails. *Proceedings on Privacy Enhancing Technologies, 2017*(4), 365–383. https://doi.org/10.1515/popets-2017-0054
Mohammadmoradi, H., Munir, S., Gnawali, O., & Shelton, C. (2017). Measuring people-flow through doorways using easy-to-install IR array sensors. *Proceedings of the 13th International Conference on Distributed Computing in Sensor Systems (DCOSS),* 35–43. https://doi.org/10.1109/DCOSS.2017.26
Munizaga, M. A., Gschwender, A., & Gallegos, N. (2020). Fare evasion correction for smartcard-based origin-destination matrices. *Transportation Research Part A: Policy and Practice, 141*, 307–322. https://doi.org/10.1016/j.tra.2020.09.008
Musa, A. B. M., & Eriksson, J. (2012). Tracking unmodified smartphones using Wi-Fi monitors. *Proceedings of the 10th ACM Conference on Embedded Network Sensor Systems (SenSys),* 281–294. https://doi.org/10.1145/2426656.2426685
Nordback, K. (2014). *Guide to Bicycle & Pedestrian Count Programs.* Transportation Research and Education Center, Portland State University. https://trec.pdx.edu/sites/default/files/smol-IBPI%20Guide%20to%20Bicycle%20%26%20Pedestrian%20Count%20Programs_final.pdf
Ozan, E., Searcy, S., Geiger, B. C., Vaughan, C., Carnes, C., Baird, C., & Hipp, A. (2021). *State-of-the-Art Approaches to Bicycle and Pedestrian Counters* (NCDOT Report No. FHWA/NC/2020-39). https://connect.ncdot.gov/projects/research/RNAProjDocs/RP2020-39%20Final%20Report.pdf
Pallaprolu, A., Peng, P., Sandhu, S., Hurst, W., & Mostofi, Y. (2024). Crowd analytics with a single mmWave radar. *Proceedings of the 30th Annual International Conference on Mobile Computing and Networking (MobiCom),* 830–844. https://doi.org/10.1145/3636534.3690664
Parsons, J., Niu, X. M., & Bao, L. (2022). A Bayesian hierarchical model for combining multiple data sources in population size estimation. *Annals of Applied Statistics, 16*(3), 1550–1562. https://doi.org/10.1214/21-AOAS1556
Pelletier, M.-P., Trépanier, M., & Morency, C. (2011). Smart card data use in public transit: A literature review. *Transportation Research Part C: Emerging Technologies, 19*(4), 557–568. https://doi.org/10.1016/j.trc.2010.12.003
Perra, C., Kumar, A., Losito, M., Pirino, P., Moradpour, M., & Gatto, G. (2021). Monitoring indoor people presence in buildings using low-cost infrared sensor array in doorways. *Sensors, 21*(12), 4062. https://doi.org/10.3390/s21124062
PulseCast data notes. (2026). *Data findings — PAX counters, Heilbronn.* Internal project document: `OKF_Vault/mission/data.md`.
Rao, J. N. K., & Molina, I. (2015). *Small Area Estimation* (2nd ed.). Wiley.
Ren, L., Yarovoy, A. G., & Fioranelli, F. (2023). Grouped people counting using mm-wave FMCW MIMO radar. *IEEE Internet of Things Journal, 10*(22), 20107–20119. https://doi.org/10.1109/JIOT.2023.3282797
Sakhnini, A., Rykunov, M., Bourdoux, A., Pollin, S., & Berkvens, R. (2024). Crowd counting measurements in a festival area using a mmWave FMCW radar. *Proceedings of the IEEE Radar Conference (RadarConf24),* 1–5. https://doi.org/10.1109/RadarConf2458775.2024.10549396
Santos, J. D. D., Alarcon, R. J. F., Asuncion, K. F., Galang, C. J. S., Oliveros, H. G. V., & Reyes, J. M. R. (2024). Investigation of environment dependence in Wi-Fi CSI-based crowd counting systems. *Proceedings of the IEEE Region 10 Conference (TENCON),* 362–365. https://doi.org/10.1109/TENCON61640.2024.10903043
Stadt Heilbronn. (n.d.-a). *Digitalisierung | KI.* https://www.heilbronn.de/wirtschaft/digitalisierung.html
Stadt Heilbronn. (n.d.-b). *LoRaWAN.* https://www.heilbronn.de/wirtschaft/digitalisierung/digitale-infrastruktur/lorawan.html
Stadt Heilbronn. (n.d.-c). *Zukunft Innenstadt Heilbronn.* https://www.heilbronn.de/wirtschaft/zukunft-innenstadt-heilbronn.html
Stadtwerke Heilbronn. (n.d.). *Bus & Bahn.* https://www.stadtwerke-heilbronn.de/swh/bus-und-bahn/
Strava Metro. (n.d.). *Using Strava data for active transportation planning.* https://metro.strava.com/case-studies/using-strava-data-for-active-transportation-planning
Umezu, N., Koizumi, S., Nakagawa, K., & Nishida, S. (2023). Potential of low-cost light detection and ranging (LiDAR) sensors: Case studies for enhancing visitor experience at a science museum. *Electronics, 12*(15), 3351. https://doi.org/10.3390/electronics12153351
U.S. Department of Transportation, Bureau of Transportation Statistics. (2025). *BTS Transportation Probe Data Guide: Non-LBS Cellular Probe Data.* https://rosap.ntl.bts.gov/view/dot/90548
Unabhängiges Landeszentrum für Datenschutz Schleswig-Holstein (ULD). (2017). *Tätigkeitsbericht 39, Kapitel 7: Neue Medien.* https://www.datenschutzzentrum.de/tb/tb39/kap07.html
Vaidya, G., & Zúñiga, M. A. (2024). Exploiting mmWave and deep-learning models to estimate people count in urban scenarios. *Proceedings of the 9th IEEE/ACM International Conference on Internet-of-Things Design and Implementation (IoTDI),* 73–84. https://doi.org/10.1109/IoTDI61053.2024.00011
Venter, Z. S., Gundersen, V., Scott, S., & Barton, D. N. (2023). Bias and precision of crowdsourced recreational activity data from Strava. *Landscape and Urban Planning, 232*, 104686. https://doi.org/10.1016/j.landurbplan.2023.104686
Wang, W., Vaidya, G., Bhattacharjee, A. K., Fioranelli, F., & Zúñiga, M. A. (2023). A long-term study of mmWave sensing in an outdoor urban scenario. *Proceedings of the International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT),* 240–247. https://doi.org/10.1109/DCOSS-IoT58021.2023.00049
Yonekura, H., Rizk, H., & Yamaguchi, H. (2025). Round trip time meets transformers: High-fidelity human counting in cluttered environments. *Neural Computing and Applications, 37*(28), 23591–23617. https://doi.org/10.1007/s00521-025-11540-8
Zannat, K. E., Choudhury, C. F., Hess, S., & Watling, D. P. (2024). Investigating the relative accuracy of GPS, GSM and CDR data for inferring spatiotemporal travel trajectories. *IET Intelligent Transport Systems, 18*(S1), 3013–3033. https://doi.org/10.1049/itr2.12563
Zaouche, M., & Bode, N. W. F. (2023). Bayesian spatio-temporal models for mapping urban pedestrian traffic. *Journal of Transport Geography, 111*, 103647. https://doi.org/10.1016/j.jtrangeo.2023.103647
Zhou, R., Lu, X. Y., Fu, Y. L., & Tang, M. (2020). Device-free crowd counting with WiFi channel state information and deep neural networks. *Wireless Networks, 26*(5), 3495–3506. https://doi.org/10.1007/s11276-020-02274-7
---
## Appendix A. Evidence gaps to close locally
The literature gives ranges, not a Heilbronn answer. The pilot should answer these questions:
1. What is the actual detection probability of a thermal array at each Heilbronn site, by season, light and crowd density?
2. How does the existing PAX index correlate with manual counts, and is the relationship stable over the five-month window?
3. What is the false-positive rate of mmWave at Hagenbuchersee and Neckarufer in rain and wind?
4. How much does the camera's error rise at night and in the Christmas-market crowd?
5. What is the cost per site-year, including calibration, maintenance and replacement?
6. Does the fused estimate achieve 80% interval coverage on held-out days, or is it overconfident?
7. Can the city publish the counts as open data without creating a re-identification risk, and what aggregation threshold is needed?
Until those questions are answered, the correct claim is "estimated people, with a calibrated interval," not "exact headcount."
File diff suppressed because it is too large. Load diff
+569
View File
@@ -0,0 +1,569 @@
---
title: "Applications matrix: six challenges and one decision-grade urban sensing stack"
status: research-note
date: 2026-11-14
scope: "Future City Hackathon 2026 challenges, datasets, and the PulseCast solution in OKF_Vault"
---
# Applications matrix: six challenges and one decision-grade urban sensing stack
**Purpose.** This note tests the paper's central claim: that a privacy-first sensing + spatial field + forecasting core generalizes across urban problems. It maps all six hackathon challenges and the PulseCast solution onto one stack, identifies where the abstraction holds, and states where it breaks. It is deliberately conservative: a method that is *transferable in principle* is not presented as *empirically validated*.
**Evidence labels.** `[D]` documented in the knowledge bundle; `[M]` measured in the PulseCast baseline; `[P]` proposed or planned, not evaluated; `[G]` gap or assumption; `[X]` external data not supplied in the bundle. Every number below is traceable to a concept file or mission file. No number is estimated here. Where the bundle is silent, the text says so.
**One-paragraph answer.** The stack generalizes as a **contract**, not as a single model. The contract is: declare the observation operator and its privacy class; infer a latent state at the resolution the decision needs; forecast only when the decision is forward-looking; fuse context that is actually available at decision time; separate the decision rule from the predictor; and evaluate against a baseline, and where possible against decision regret. The contract is demonstrated for footfall-to-staffing and is plausible, with caveats, for advertising and waste routing. It becomes a **sibling spatial-design problem** for sensor placement and LoRaWAN planning, where the decision changes the observation process. It does **not** hold for cable-car sentiment or the learning guide, which have no physical field, no forecast target, and no resource-allocation decision.
**Definition.** For this note, a stack is **decision-grade** when (i) the observation operator and its error/bias are stated; (ii) spatial and temporal resolution match the decision unit; (iii) forecasts are validated on a time-based holdout against a naive baseline; (iv) the decision rule is explicit, constrained, and separable from the predictor; (v) evaluation includes a decision-relevant metric, not only accuracy; and (vi) privacy and provenance are auditable. The matrix in Section 3 assesses each challenge against these conditions.
---
## 1. Challenge summaries
The six challenges are not six versions of the same problem. They differ in what is observed, whether a spatial field exists, whether the decision is forward-looking, and whether the decision changes the observation process. The summaries keep those differences explicit.
### C1 — Pulse of the City (Schwarz Digits)
**Decision.** The challenge names three decisions but does not choose one: where and when to place physical advertising, where to locate a store, and how to staff a store. The decision-maker is a retailer or the city. The challenge's stretch goal is explicitly "estimations and predictions, not only historical visualisation." `[D]`
**Data.** PAX counters passively sniff Bluetooth MAC advertisements from nearby smartphones and smartwatches, then transmit aggregated device counts over a LoRaWAN network. The counts are estimates, not a census: not everyone carries a discoverable device. The challenge brief calls the data privacy-preserving because the counters aggregate devices rather than identify people. The raw CSV is in `data/raw/` in the workspace, not in the knowledge bundle; the download and schema are documented in `mission/data.md`. `[D]`
**Spatial resolution.** The challenge describes sensors in Heilbronn's inner city. The delivered data contains 12 device IDs; after merging the two `density-01` IDs and excluding three sensors, **8 usable sensor locations** remain in a **~580 m x 280 m cluster** around Hagenbuchersee, Neckarufer, Stadtgalerie, Marahaus and Götzenturm. The three excluded sensors are `maker-fair` (73 readings, located in Hannover), `niklas-01` (8 readings, outside the inner city), and `hn-bollwerksturm-01` (5 readings). This is an inner-city micro-area, not a city-wide network. `[D]`
**Temporal resolution.** The 8-sensor network covers **2026-06-12 to 2026-11-07**, about five months. Cadence is **~15 minutes** (median 15.3 min; some sensors 30 min). Gaps reach **up to 6 days** per sensor in the 8-sensor network; `density-01`, which has 13 months of history, has a **35-day gap**. It is a single location and cannot establish network-wide annual seasonality. The full file covers **351 days overall**. `[D]`
**Key uncertainty.** Device penetration is unknown; the values are an index of detected devices, not people. The spatial coverage is not city-wide. Gaps, missing events, observed rather than forecast weather, and a short multi-sensor history all limit inference. There is no POS, sales, queue, labour-cost, or event data. `[D]`
**Heatmap / spatial field.** The challenge slide illustrates a heat map over a map of Heilbronn plus a multi-line time-series chart. With eight points in a 580 m cluster, only a **local pulse map with measured points and a clearly labelled interpolation** is honest. A city-wide heatmap is not supported by the data. `[D]`
**Prediction target.** Footfall or device density per location and time. The challenge does not fix a horizon; "estimations and predictions" are the stretch goal. `[D]`
**External covariates.** The brief suggests weather, holidays, events, or other real-world signals that explain patterns. It does not supply them. PulseCast adds Open-Meteo weather and Baden-Württemberg public holidays. `[D]`
**What the challenge leaves open.** The unit of `densityData`, the sensors' catchment areas, the device-penetration correction, the cause of 4–6 day gaps, whether an event log exists, and whether a live API exists. `[D]`
### S1 — PulseCast (our solution to C1)
**Decision.** A weekly shift plan for one inner-city store: how many staff per hour next week, with the manager able to accept, edit, or reject the plan. The forecast and scenario engine is the reusable core; the staffing app is its first consumer. `[D]`
**Data.** The same PAX stream: 192,464 CSV rows, 96,232 density readings, 12 device IDs, **8 usable sensors**, 3 excluded test sensors, ~15-minute cadence, about five months of multi-sensor history. Weather comes from Open-Meteo; holidays from the Baden-Württemberg calendar. `[D]`
**Spatial resolution.** Eight fixed points in the ~580 m x 280 m cluster. The product shows a pulse map of measured points and a labelled interpolation, not a city-wide surface. `[D]`
**Temporal resolution.** Four horizons are exposed: **15 min, 1 h, 48 h, and 7 d**. The 7-day plan is the hero. The documented evaluation uses a **28-day time-based holdout**; 15 min and 1 h are nowcast/replay because the CSV has no live feed. `[D]`
**Key uncertainty.** Device penetration, 4–6 day gaps, observed weather in training, no event log, no POS/labour data, a four-week test period, one store, and no annual seasonality. `[D]`
**Heatmap / spatial field.** A "pulse map" of eight measured points with a clearly marked interpolation. It is evidence for a local staffing decision, not a city-wide measurement. `[D]`
**Prediction target.** Hourly footfall index per sensor, then staff count via a transparent heuristic: `required staff = ceil(footfall / visitors-per-staff-hour x safety factor)`, bounded by minimum staff, shift length, opening hours, and cost. `[D]`
**External covariates.** Open-Meteo temperature and rain are the useful model features; cloud and wind did not improve the short test. Public holidays are a feature and a dashboard badge, but added little in the 28-day test. The what-if scenarios are counterfactual inputs, not observations. `[D]`
**Claim discipline.** The model predicts **estimated devices, not people**. The staffing plan is decision support with explicit assumptions, not an optimizer with proven ROI. The deployment is a hackathon deployment, not a multi-tenant production system. `[D]`
### C2 — WasteRouting.AI (PreZero)
**Decision.** Which glass containers to empty, in what order, with which fleet, under capacity, shift, and depot constraints. The starting point is a fixed area-by-area schedule that ignores actual fill levels. The challenge asks for a configurable route engine, not only a dashboard. `[D]`
**Data.** **11,701 fill-level readings** for **449 anonymized containers**, covering **2025-10-17 to 2026-09-30** (almost one year). Three glass colours: white (4,432 readings), green (4,169), brown (3,100). Numeric fill levels run from 0 to 100; the string `overfull` appears in **1,192 rows**. The file has only four columns: `Location_id`, `material_type`, `date`, `fill_level_perc`. `[D]`
**Spatial resolution.** **No coordinates.** Container IDs are stable and anonymized, so per-container history and grouping work, but route geometry, inter-point distances, and depot distances are absent. The spatial object is a **graph or attribute field over containers**, not a geographic field. `[D]`
**Temporal resolution.** About one year, but readings are **event-like and irregular**, not a complete daily grid. Dates only, no time of day. Values cluster on round percentages, suggesting reported or estimated levels rather than continuous sensor measurements. `[D]`
**Key uncertainty.** Reporting error and discreteness; how to encode `overfull`; missing container capacity and type; no fleet, depot, or shift data; no ground-truth routes; unknown reading protocol; holidays and school vacations are suggested as drivers but not supplied as a structured calendar. `[D]`
**Heatmap / spatial field.** A fill-level map would be the natural visual, but without coordinates it can only be a **container graph or zone-level attribute map**. Any geographic heatmap requires an external location mapping or an explicit synthetic layout. `[D]`
**Prediction target.** Future fill level per container, or time-to-threshold / overflow risk, then a route. The brief emphasizes pattern analysis and route optimization; a forecasting target is implicit rather than specified. `[D]`
**External covariates.** Material type, public holidays, school vacations, seasonal shifts, and the operational variables named in the brief: waste stream and density, container specifications, fleet characteristics, inter-point distances, disposal depot proximity, depot distances, and shift constraints. `[D]`
**Cross-challenge link.** The Pulse of the City brief explicitly names PreZero as a user: historical footfall heatmaps could inform waste collection routes around high-traffic zones, and predictions could support dynamic routing. This is a documented conceptual bridge between C1 and C2, but it is a use-case suggestion, not a shared dataset or a validated pipeline. `[D]`
### C3 — Measure where it matters (AI TRAQC)
**Decision.** Where to place stationary heat sensors, which routes mobile units take and when, and how many measurement locations are needed, given budget, mission time, and priority areas such as day-care centres or care homes. The deliverable is **positions and routes**, not only a corrected map. `[D]`
**Data.** Open satellite data: Sentinel-3 land surface temperature at roughly **1 km per pixel, daily**; Landsat 8/9 at roughly **100 m, every eight days, around midday, cloud-free**; optional ECOSTRESS at different times of day, but only up to about **52 degrees north**. Suggested open inputs: the global Local Climate Zone map at about **100 m** and OpenStreetMap for the path network. No challenge dataset is stored in the bundle; the deck describes data sources and a simulation workflow. `[D]`
**Spatial resolution.** A coarse 1 km baseline, a 100 m target/reference grid, and city-scale placement. Example constraints include `5+5` or `10+2` sensors. `[D]`
**Temporal resolution.** Satellite snapshots around midday; Landsat repeats every eight days; there is no continuous time series. The "when" needs a separate time axis, via ECOSTRESS scenes or a modelled diurnal cycle. `[D]`
**Key uncertainty.** Land surface temperature is not air temperature; clouds and revisit limits; the simulation samples a same-day Landsat reference, so sim-to-real transfer is untested; priority-area weighting versus overall map error; how much time of day changes the optimal placement; transfer to another city. `[D]`
**Heatmap / spatial field.** This is a **literal heat map**: a land surface temperature field. The task is to reconstruct a fine-grained field from sparse ground/mobile samples plus a coarse baseline, and to choose the sampling design that minimizes reconstruction error. `[D]`
**Prediction target.** A reconstructed 100 m land surface temperature field, optionally at different times of day. This is **spatial prediction**, not a temporal forecast of future demand. `[D]`
**External covariates.** Local Climate Zones, OpenStreetMap, built environment and surface characteristics, satellite bands, time of day. The bonus asks whether the approach also works for humidity or noise. `[D]`
**What the challenge tests.** The algorithm must choose placements from coarse data only, without peeking at the fine reference, then be evaluated honestly against Landsat. An error-versus-number-of-measurement-points curve is the suggested justification. `[D]`
### C4 — LoRaWAN network planning (HNVG)
**Decision.** How many gateways a city needs, where they should go, and what coverage and redundancy can be expected. The output is a first planning tool that lets a user select candidate sites and assess coverage and gaps. The brief explicitly says this is not a replacement for professional radio planning or field measurements. `[D]`
**Data.** **8,009 reception rows** with **31 columns**, from **448 distinct devices** and **9 distinct gateway IDs**, over **2026-11-04 to 2026-11-10** (six days). Transmitter metadata has **593 meters**; gateway metadata has **8 rows**; **71 candidate gateway sites**; **23,127 addresses**. RSSI ranges from **-127 to -70 dBm**; SNR from **-21.5 to 12 dB**. Spreading factors: 7 (3,494 rows), 8 (372), 9 (691), 10 (733), 11 (609), 12 (2,110). `[D]`
**Spatial resolution.** Link-level point-to-point receptions between meters and gateways, plus a coverage field over candidate sites and addresses. Transmitter GPS coordinates are of uncertain quality; gateway and candidate-site coordinates use a decimal comma in the delivered files. `[D]`
**Temporal resolution.** Six days of messages. This is enough for a coverage snapshot but not for a temporal forecast of reception, traffic, or interference. `[D]`
**Key uncertainty.** Received messages alone do not establish packet loss; a reliable estimate of expected transmissions is missing. One observed gateway (`00007076FF03010C`) has no metadata row, so the "8 gateways" and "9 gateway IDs" mismatch must be resolved. **145 mapped meters did not transmit** during the six-day window. Radio settings, building/terrain obstructions, vegetation, interference, and gateway height are not fully supplied. `[D]`
**Heatmap / spatial field.** A **coverage or reception-quality field**: expected RSSI, SNR, or link success probability over space, plus redundancy where multiple gateways hear the same message. This is a planning heatmap, not a temporal demand forecast. `[D]`
**Prediction target.** Reception success or quality at a location or link, and coverage/redundancy for a candidate gateway set. This is **spatial prediction and site ranking**, not a time-series forecast. `[D]`
**External covariates.** Distance, buildings and terrain, spreading factor, bandwidth, transmit power, gateway height, vegetation, interference, and network traffic. The brief lists these as examples to test, not a checklist. `[D]`
**What the challenge tests.** Whether a city can get a useful first answer from real urban data rather than a perfect radio-planning model, and whether the approach transfers to other cities and regions. Partial solutions are explicitly welcome. `[D]`
### C5 — Riding the Sentiment (Stadt Heilbronn)
**Decision.** What the city should understand and communicate about public debate around the planned cable car. The brief asks for three to five key insights and an explorable dashboard; it does not ask for a resource-allocation decision, a forecast, or a policy optimizer. `[D]`
**Data.** **39,361 records** from **2024-10-19 to 2026-10-27**, collected from publicly accessible content without login and without circumventing access restrictions. The export is about **210 MB**. It was pre-filtered with the keywords `IPAI`, `Seilbahn`, and `téléphérique`. Fields include media type, source, language, sentiment (`1`, `0`, `-1`, `-99`), reach, engagements, authors, title, and content. `[D]`
**Spatial resolution.** No physical spatial field. "Space" is the comparison between projects and cities: Heilbronn's planned cable car, the Câble C1 in the Paris region, the planned cable car in Bonn, and the IPAI campus in Heilbronn. `[D]`
**Temporal resolution.** About two years, event-driven and irregular. Publication timestamps support trend, peak, and change-point analysis, but not a regular forecasting grid. `[D]`
**Key uncertainty.** The export is a media-monitoring sample, not a representative survey. Keyword filtering captures tourist cable cars, toys, and unrelated projects. Sentiment is not stance: a negative-sounding statement may still support the project. The `-99` value is a sentinel and should be treated as unknown, not as extreme negative. Duplicates, mixed languages, media-type imbalance, missing reach, and platform bias all limit inference. `[D]`
**Heatmap / spatial field.** **None.** The nearest analogues are a topic/stance landscape over time or a comparison matrix across projects and cities. Calling this a heatmap would be a metaphor, not a measurement. `[D]`
**Prediction target.** **None specified.** The task is descriptive and classificatory: topics, arguments, sentiment/stance, temporal peaks, and cross-project comparison. A forecast of attention is possible in principle but is not asked for and has no ground truth in the bundle. `[D]`
**External covariates.** Real-world developments: political decisions and announcements, new cost estimates, route or planning changes, visualisations, construction milestones, media reports, and the opening of a new system. Also media type, language, source, and platform. There is no structured event log in the dataset. `[D]`
**Coverage details.** 28,805 records have a non-null reach; total engagements are 1,308,843; German-language records are 30,812 and French-language records 7,999; news records are 15,594. Sentiment counts: `1` = 20,712, `0` = 9,928, `-99` = 6,797, `-1` = 1,924. `[D]`
### C6 — Learning Guide (aim)
**Decision.** Which learning and leisure offers to present to a family, and how to match them to a child's age, interests, date, and location. The brief is a product-design and data-integration task, not a sensing or forecasting task. It also asks how the application stays current with low maintenance. `[D]`
**Data.** **No dataset is supplied.** The prototype is limited to four institutions: aim, experimenta, Haus der Familie, and IPAI. The thematic focus is AI, coding, and gaming. Teams must check for APIs, RSS feeds, iCalendar feeds, or CSV/JSON exports before scraping, and must check whether extraction is permitted. `[D]`
**Spatial resolution.** Institution locations and, optionally, distance from home or school via OpenStreetMap. There is no sensor field and no continuous spatial measurement. `[D]`
**Temporal resolution.** Event and course dates, opening hours, and content freshness. There is no forecasting target and no regular time series. `[D]`
**Key uncertainty.** Offer coverage and data quality; website structure changes; scraping legality and terms; child privacy and GDPR compliance; age-appropriate design; no attendance or demand ground truth; long-term maintenance burden. `[D]`
**Heatmap / spatial field.** **None.** A map of offer locations or distances is possible as a supporting view, but it is a catalogue map, not a measured spatial field. `[D]`
**Prediction target.** **None.** The relevant target is search or recommendation relevance, and secondarily demand/supply gaps from anonymous queries. Neither has ground truth in the bundle. `[D]`
**External covariates.** Age group, theme (AI, coding, gaming), date, event type, location/distance, language, accessibility, and institution calendars. `[D]`
**Privacy regime.** This is the highest-privacy challenge in the set. The brief asks for data minimisation, ideally no stored data; guest or incognito modes; no self-registration by underage users; parental consent under GDPR, where young people can consent from age 16 in Germany; child-friendly explanations; and caution with AI chatbots and third-party services. `[D]`
---
## 2. Shared abstraction
### 2.1 Formal core
The common structure is a partially observed decision problem, not a universal heatmap:
```text
observation: y_t = H_t x_t + e_t
field: x_hat_t = I(y_<=t, H_t, c_t, G)
forecast: x_hat_t+h = F(x_hat_<=t, c_t+h; theta)
decision: a* = argmax_a E[U(a, x_t+h)] subject to constraints
evaluation: L(y, x_hat) + R(a*, x_oracle) + privacy leakage
```
- `x_t` is a latent state over spatial units `S`. `S` can be a grid (land surface temperature), a set of fixed points (PAX sensors), a graph (glass containers), a set of links or candidate sites (LoRaWAN), or a set of projects (cable-car debate).
- `H_t` is the observation operator: point sampling, spatial aggregation, link reception, text collection, or catalogue extraction. In C1/C2/S1 it is fixed. In C3/C4 it is a **design variable**: the decision changes what is observed.
- `c_t` is exogenous context available at decision time: weather, holidays, events, built environment, radio settings, platform, age/theme.
- `G` is geometry or graph structure. It may be a metric space, a container graph, a radio link graph, or empty.
- `I` is spatial inference: interpolation, kriging, Gaussian-process regression, physics-based propagation, graph smoothing, or simple aggregation.
- `F` is temporal forecasting. It is only meaningful when the decision is forward-looking.
- `U` is the decision utility. It must be stated explicitly; accuracy alone is not a decision metric.
The key distinction the paper must make is **state estimation versus design**. In C1/C2/S1, the pipeline estimates a state and then acts on it. In C3/C4, the pipeline predicts the value of a design and then chooses where to measure or transmit. Calling both "forecasting" hides the inversion.
### 2.2 The seven-layer stack
| Layer | Question | PulseCast implementation | Generalization test |
|---|---|---|---|
| **L0 Sensing and sampling** | What is measured, where, how often, by whom? | PAX counters: 8 fixed points, ~15 min, 5 months | Is `H_t` fixed or chosen? Does the sampling design match the decision? |
| **L1 Observation and privacy** | What does `y` mean, what error/bias, what privacy class? | Aggregated device counts; no MAC/PII in the app; penetration unknown | Is the observation model calibrated? Is the privacy claim formal or only architectural? |
| **L2 Spatial inference** | How do sparse observations become a field? | Local pulse map and labelled interpolation only | Is the field type Euclidean, raster, graph, or link-based? Does resolution match the decision? |
| **L3 Temporal forecast** | What will happen next? | Global HistGB; 7-day hero; 15 min/1 h/48 h/7 d exposed | Is the target forward-looking? Is there enough history for the claimed horizon? |
| **L4 External context** | What explains the signal? | Open-Meteo temperature/rain; BW holidays; scenarios | Is context available at decision time? Is it observed or forecast? |
| **L5 Decision** | What should be done? | Staffing heuristic with visible assumptions; human accepts/edits | Is the objective explicit? Are constraints real? Is uncertainty propagated? |
| **L6 Evaluation** | Did it work, for the decision? | MAE/WAPE/R² vs seasonal naive; plan latency; load test | Is there a decision-relevant metric, not only accuracy? Is there decision regret or outcome data? |
### 2.3 Where the abstraction holds
The stack holds when five conditions are met:
1. **A latent state exists with spatial or temporal structure.** Footfall, fill level, land surface temperature, and radio coverage all have autocorrelation; a sparse sample is informative about unobserved places or times.
2. **The observation operator is known and stable enough to model.** PAX point sampling, container readings, satellite pixels, and radio links each have a defensible `H_t`, even if it is noisy.
3. **Context explains a meaningful share of variance and is available at decision time.** Weather is the clearest example in PulseCast; built environment and radio settings are the analogues in C3/C4.
4. **The decision can be written as an optimization or rule over forecasts with explicit constraints.** Staffing, ad allocation, routing, sensor placement, and gateway placement all qualify; "understand the debate" does not.
5. **Evaluation can use a time-based holdout and a decision-relevant metric.** PulseCast has the first; it lacks the second.
S1 satisfies conditions 1–4 locally. C1 as posed has a latent state and a plausible observation operator, but it does not supply context or a decision rule. C2 has temporal structure and a routing decision, but its observation model is coarse, its context is only suggested, and its spatial part is missing because there are no coordinates. C3 and C4 satisfy 1, 2, and 4 for predictions, but they invert the arrow: the decision is the sampling design.
### 2.4 Where the abstraction breaks
| Break | Mechanism | Challenges affected | Consequence |
|---|---|---|---|
| **B1 — No physical field** | The latent state is opinion, relevance, or a catalogue, not a spatial quantity | C5, C6 | The "heatmap" layer is a metaphor; spatial inference is absent |
| **B2 — Adversarial or non-representative observation** | The sample is selected by keywords, platforms, or self-selection | C5 | `H_t` is not a neutral sampling operator; sentiment is not stance |
| **B3 — Decision changes the observation** | Placement or gateway choice determines what can be measured next | C3, C4 | The pipeline becomes active learning or optimal experimental design, not forecast-then-act |
| **B4 — Spatial units are anonymized or absent** | Container IDs without coordinates | C2 | The field is a graph or attribute vector, not geography; routing needs an external layout |
| **B5 — Resolution mismatch** | Eight points in 580 m cannot rank city-wide store locations or ad sites | C1 ads/store, S1 | The forecast may be valid locally but not at the decision's spatial scale |
| **B6 — Privacy regime prohibits individual-level state** | Children's data, social-media authors, meter/address data | C5, C6, C4 | Aggregation or data minimisation is required; the modelling unit changes |
| **B7 — No decision outcome data** | No POS, labour, ad cost/reach, fleet, packet-loss, or stance labels | All | Accuracy is a proxy; decision regret and ROI cannot be claimed |
### 2.5 What "multimodal" can and cannot mean
The bundle contains at least five modality families:
| Modality family | Examples | Used empirically by PulseCast? |
|---|---|---|
| Point/aggregate time series | PAX device counts, glass fill levels | Yes, PAX only |
| Raster/field | Sentinel-3 and Landsat land surface temperature | No |
| Link/event records | LoRaWAN receptions, RSSI/SNR, spreading factor | No |
| Text | News, social posts, YouTube comments | No |
| Structured catalogue | Institution offers, dates, age groups, themes | No |
PulseCast empirically exercises **one sensor modality** (PAX counts) plus **tabular context** (weather, holidays). That is enough to claim a **context-fused, modality-agnostic pipeline contract**. It is not enough to claim a single multimodal model trained across modalities. A defensible paper sentence is: "The core is modality-agnostic at the interface, and we demonstrate it on one sensing modality; extending it to raster, link, and text modalities is a research agenda, not a result." `[D]`
---
## 3. Matrix table
The matrix is split into two halves for readability. C1 is the challenge as posed; S1 is our implementation. They are separate rows because the challenge leaves the decision open, while PulseCast fixes it. "Ads" and "store location" are decision layers inside C1, not separate challenges.
### 3.1 Matrix A — sensing to field
| ID | Sensing modality | Privacy risk | Spatial inference (field type) | Temporal forecast |
|---|---|---|---|---|
| **C1** Pulse of the City (as posed) | Passive Bluetooth PAX counts, aggregated over LoRaWAN; delivered CSV | **P1**: aggregate counts, but MAC sniffing at the edge; penetration unknown; presence/re-identification risk | **Point field**: 8 usable sensors in ~580 x 280 m; local interpolation only; city-wide heatmap unsupported | ~15 min cadence; 5 months multi-sensor; prediction requested, horizon unspecified |
| **S1** PulseCast | Same PAX stream + Open-Meteo + BW holidays | **P1**: aggregate only in the app; no MAC/PII; no geolocation permission; sensing externality remains | **Point field**: local pulse map; measured points plus labelled interpolation | **Temporal forecast**: global HistGB; 15 min/1 h/48 h/7 d; 7 d hero; 28-day holdout |
| **C2** WasteRouting.AI | Fill-level scalars per glass container; event-like readings | **P1**: anonymized container IDs; low personal risk; operational sensitivity | **Graph/attribute field**: no coordinates; container IDs only; route geometry absent | **Implicit temporal forecast**: irregular, ~1 year, date-only; fill level or time-to-threshold |
| **C3** Measure where it matters | Satellite LST (Sentinel-3, Landsat, ECOSTRESS) + ground/mobile sensors | **P0/P1**: open satellite data; priority-area maps may reveal vulnerable sites | **Raster field**: 1 km baseline to 100 m target; LCZ/OSM; literal heat field | **No temporal forecast**: snapshots/diurnal; spatial reconstruction is the target |
| **C4** LoRaWAN planning | LoRaWAN uplink receptions (RSSI/SNR/SF) from water meters | **P1**: meter/address data; occupancy inference; device IDs | **Link/coverage field**: link-level; 71 candidate sites; 23,127 addresses | **No temporal forecast**: 6-day window; spatial prediction and site ranking |
| **C5** Riding the Sentiment | Text/media mentions: news, social, YouTube, forums | **P2**: author handles and quotes; platform terms; high re-identification risk | **No field**: project/city comparison; no physical space | **No forecast**: 2 years, event-driven; trends and change points |
| **C6** Learning Guide | Institution offers via web/API/RSS/iCal | **P2**: children's data; GDPR age 16; highest risk | **No field**: institution locations and optional distance only | **No forecast**: event dates and freshness only |
**Privacy classes.** P0 = open/no personal data (satellite, OSM, weather). P1 = aggregated or pseudonymous data with residual re-identification risk (PAX counts, container IDs, meter/address data). P2 = personal or special-category data (social-media authors and quotes, children's data). The classes are a paper device, not a legal determination. `[G]`
### 3.2 Matrix B — forecast to decision
| ID | Prediction target | External context | Decision layer | Evaluation metric |
|---|---|---|---|---|
| **C1** Pulse of the City (as posed) | Footfall/device density per location and time; horizon open | Weather, holidays, events suggested but not supplied | Ads, store location, staffing; no rule specified | None specified; business KPI absent |
| **S1** PulseCast | Hourly footfall index per sensor; next-week staff plan | Open-Meteo temp/rain (cloud/wind no gain); BW holidays (little gain); counterfactual scenarios | **Staffing heuristic**: required staff, minimum staff, shift blocks, cost, coverage; human accept/edit | MAE/WAPE/R² vs seasonal naive; plan latency; coverage/cost vs flat schedule; load test target |
| **C2** WasteRouting.AI | Fill level per container / time-to-threshold; route | Holidays, school vacations, season, material; fleet, depot, shift variables | **VRP + fleet selection** under capacity, shift, and depot constraints; configurable | Fill MAE/overflow rate; cost/km; vehicles; no ground-truth routes |
| **C3** Measure where it matters | 100 m LST field; optionally diurnal field | LCZ, OSM, time of day, satellite bands; humidity/noise bonus | **Sampling design**: sensor positions, mobile routes, number of locations; budget/priority | Reconstruction RMSE vs Landsat; error-vs-count curve; priority-area error |
| **C4** LoRaWAN planning | Reception quality/success at link or location; coverage/redundancy | Distance, buildings/terrain, SF/BW/TX power, height, vegetation, interference | **Gateway planning**: number and sites; candidate-site ranking | Link AUC/MAE; coverage % of addresses; redundancy; cost |
| **C5** Riding the Sentiment | Stance/topic/time; no forecast | Political decisions, cost estimates, route changes, construction, opening; platform/language | **Communication/insight**: no resource allocation | Stance F1/accuracy; topic coherence; reproducibility; insight quality |
| **C6** Learning Guide | Offer relevance/search; no forecast | Age, theme, date, location/distance, language, accessibility | **Matching/search/product maintenance**: no resource allocation | Task success; search relevance; accessibility; maintenance effort; privacy compliance |
### 3.3 Matrix C — layer status
Symbols: **●** present and evidenced (data and/or documented result); **◐** partial, prototype, or assumption; **○** absent or unsupported; **–** not applicable.
| ID | L0 sensing | L1 observation/privacy | L2 spatial | L3 temporal | L4 context | L5 decision | L6 evaluation |
|---|---|---|---|---|---|---|---|
| **C1** as posed | ● | ◐ | ◐ | ◐ | ◐ | ◐ | ○ |
| **S1** PulseCast | ● | ● | ◐ | ● | ● | ● | ◐ |
| **C2** waste | ● | ● | ○ | ◐ | ◐ | ◐ | ○ |
| **C3** sensor placement | ◐ | ● | ● | ○ | ● | ● | ● |
| **C4** LoRaWAN | ● | ◐ | ● | ○ | ◐ | ● | ◐ |
| **C5** cable car | ● | ◐ | ○ | ○ | ◐ | ○ | ◐ |
| **C6** learning guide | ○ | ○ | ○ | ○ | ◐ | ◐ | ○ |
**Reading the matrix.** Only PulseCast has an evidenced end-to-end path from sensing to a decision layer, and even there the spatial layer is local and the decision evaluation is a proxy. C3 and C4 are strong on spatial inference and decision design but have no temporal forecast. C5 and C6 have neither a field nor a forecast. C2 has a strong temporal/operational story but no spatial geometry. The matrix therefore supports a **conditional generalization claim**, not a universal one.
---
## 4. Transferability assessment
### 4.1 Verdict scale
| Verdict | Definition | What it permits in the paper |
|---|---|---|
| **Direct** | Same sensing modality, same field, new decision; no new data required | A re-analysis or extension of PulseCast |
| **Conditional** | Same pipeline pattern, but a new dataset, modality, or geometry is required | A method-transfer claim with an explicit data condition |
| **Sibling** | Shares spatial inference and evaluation, but the decision changes the observation process | A separate "spatial design" contribution, not the forecast core |
| **Non-application** | No physical field, no forecast target, no resource-allocation decision | At most a context source or a different problem class |
### 4.2 Transfer verdicts
| Challenge / decision | Verdict | What transfers | What must change | Evidence |
|---|---|---|---|---|
| **C1/S1 staffing** | **Direct** | Observation handling, local field, weather-aware forecast, staffing heuristic, deployment pattern | Store-specific assumptions; city-scale coverage; outcome data | `[M]` MAE 3.36 vs 5.40 naive; WAPE 27.8% vs 44.7% |
| **C1 advertising** | **Direct (local exposure only)** | Same field and forecast; allocation objective replaces staffing objective | Ad inventory, cost, reach, dwell, conversion; city-wide sites for a real campaign | `[P]` no result in bundle |
| **C1 store location** | **Conditional (data-bound)** | Forecast/context pipeline; ranking logic | City-wide field, POI/rent/competition/sales data; eight points cannot rank zones | `[G]` explicitly out of scope in `mission/mvp.md` |
| **C2 waste routing** | **Conditional** | Temporal forecasting, clustering, configurable VRP, holiday/season context | Coordinates or zone map, capacity/type, fleet/depot/shift, route costs, overflow ground truth | `[P]` no result; dataset documented |
| **C3 sensor placement** | **Sibling (spatial design)** | Observation model, spatial inference, error-vs-cost evaluation | Decision changes `H_t`; active learning/optimal design; no temporal forecast; no delivered dataset | `[P]` protocol only |
| **C4 LoRaWAN planning** | **Sibling (spatial design)** | Link/coverage prediction, candidate-site ranking, address-weighted coverage | Packet-loss estimate, buildings/terrain, longer window, GPS calibration; no temporal forecast | `[P]` no result; dataset documented |
| **C5 cable car sentiment** | **Non-application** | At most an event/attention signal for another model | No physical field, no forecast target, no allocation decision; different NLP stack and privacy regime | `[D]` descriptive task |
| **C6 learning guide** | **Non-application** | At most a location/context consumer | No sensing, no field, no forecast; matching/recommendation and child-privacy regime are different | `[D]` product task |
### 4.3 What is truly transferable
**Direct or same-data transfers.**
- **Staffing.** The PulseCast path documented in the bundle is the only end-to-end, evidence-backed instance. It can be reused for another store in the same cluster; another city would require retraining and revalidation on that city's sensors and weather.
- **Advertising.** The same field and forecast can rank the eight locations and time windows by expected exposure. This is a **decision-layer substitution**, not a new sensing study. It is transferable as an analytical extension, but it cannot support city-wide ad planning without more sites and ad-economics data.
- **City operations.** The same pulse map can support crowd management, event planning, or dynamic street infrastructure, but this is a secondary use case and is not evaluated in the bundle.
**Method transfers that require new data or a new geometry.**
- **Waste routing.** The temporal-forecast and decision-rule pattern transfers, but the spatial object changes from a metric field to a **container graph**. The route result is conditional on assumed geometry, capacity, and fleet data. Without coordinates, a geographic heatmap or a credible route-saving claim is not possible.
- **Store location.** The method can rank sites only if the field is observed at city scale. With eight sensors in 580 m, the correct verdict is **not supported**, not "promising." This is a data-acquisition proposal, not a case study.
- **Sensor placement.** The spatial-inference layer transfers, and the evaluation protocol (error versus number of measurement points) is strong. But the decision is a **design** decision, and the temporal forecast layer is absent. It is a sibling, not the same core.
- **LoRaWAN planning.** Same verdict as sensor placement: a spatial-design sibling with real data, a link-level observation model, and no temporal forecast. It is the best available test of the "spatial field" part of the thesis because the data are real and the candidate sites are explicit.
**Not transferable.**
- **Cable-car sentiment.** There is no physical field, no forecast target, and no resource-allocation decision. The dataset is a media-monitoring sample, not a sensor. It can be an **exogenous context signal** (for example, an event/attention feature for footfall or waste), but it is not an application of the sensing-to-field-to-forecast core. Its privacy regime (author handles, quotes) is also different from aggregate counting.
- **Learning guide.** There is no sensing, no field, no forecast, and no allocation. It is a data-integration, search, and child-privacy product problem. It is the clearest example of why "urban" does not imply "sensing stack."
### 4.4 The six challenges are not six instances of one problem
A useful taxonomy for the paper:
| Problem class | Challenges | What the core can offer |
|---|---|---|
| **State estimation + forward decision** | C1 (staffing, ads, city ops), C2 (waste) | The main pipeline: observation, field, forecast, decision rule |
| **Spatial design / active sensing** | C3 (heat sensors), C4 (gateways) | A sibling pipeline: observation model, spatial prediction, optimal design |
| **Text/opinion analysis** | C5 (cable car) | Nothing directly; at most a context signal |
| **Catalogue/matching** | C6 (learning guide) | Nothing directly; a different product class |
This taxonomy is more defensible than "six challenges, one stack." It says the stack covers **one challenge directly** (C1, which contains three decisions), **one conditionally** (C2), **two as siblings** (C3, C4), and **two not at all** (C5, C6). That honesty strengthens the paper: it shows the authors know where the method stops.
### 4.5 What would falsify the generalization claim
The paper should state these falsifiers explicitly:
1. **Waste forecast fails a naive baseline** after accounting for irregular sampling and `overfull` encoding, or the routing result is driven entirely by assumed geometry.
2. **LoRaWAN link model cannot beat a distance-only baseline** on held-out gateways, or candidate-site rankings are unstable under GPS uncertainty.
3. **Advertising allocation is unstable** under plausible weather scenarios or measurement error, so the ranking is noise.
4. **Store-location ranking with eight points is no better than random** at the city scale.
5. **Decision regret is large despite low MAE**, meaning forecast accuracy does not translate into better staffing.
6. **Aggregate counts can be re-identified** or the privacy claim fails a threat model.
A paper that reports these tests, even if some fail, is stronger than one that asserts six-way generalization.
---
## 5. Case studies
Four case studies are enough for a 4–5 page report. They are ordered by evidence strength, not by ambition. A fifth candidate (AI TRAQC) is described as a protocol because the bundle contains no delivered dataset.
### 5.1 Case study 1 — Footfall to staffing (PulseCast, empirical anchor)
**Research question.** Can eight sparse, privacy-preserving PAX counters plus weather support a weekly staffing plan better than a seasonal-naive baseline?
**Why it tests the thesis.** It is the only case with an end-to-end path: real sensor data, a time-based holdout, a baseline, a forecast, a decision rule, and a deployed application. It establishes the upper bound of what the current evidence supports.
**Data we have.** 192,464 CSV rows; 96,232 density readings; 12 device IDs; after merging the two `density-01` IDs and excluding three sensors, **8 usable sensor locations**; 2026-06-12 to 2026-11-07; ~15-minute cadence; ~580 m x 280 m cluster; values 0–292.6; multi-sensor mean ~14.4; median 7.7; p95 46.9; p99 78.4. Peak 18:00–21:00 local, top at 19:00; Saturday busiest (mean 19.1). `[D]`
**Results we have.** Global `HistGradientBoostingRegressor`; train 2026-06-12 to 2026-10-10; test last 28 days. Seasonal naive: MAE 5.40, WAPE 44.7%, R² 0.462. Day-ahead, no weather: MAE 4.20, WAPE 34.8%, R² 0.678. Day-ahead with temperature/rain: MAE **3.36**, WAPE **27.8%**, R² **0.779**. Seven-day-ahead, no weather: MAE 4.37, WAPE 36.2%, R² 0.672. Holidays added little: 4.15 vs 4.20 without weather; 3.44 vs 3.36 with weather. `[M]`
**What else we have.** A documented three-service deployment on SKE (Nuxt SSR, Go API/SSE, FastAPI) behind a Cloudflare Tunnel, with no Ingress, cert-manager, or Jobs; build-time batch; cached fallback; four forecast horizons; a staffing heuristic with configurable assumptions. Plan generation is documented as under 2 seconds; the k6 load-test number is still `[TODO]`. `[D/P]`
**What is missing.** POS/sales/queue data; actual manager decisions; labour cost and schedule data; forecast weather rather than observed weather; an event log; school holidays and bridge days; multi-store and multi-city validation; annual seasonality; per-sensor MAE table (required by `mission/paper.md`, not present in the bundle); a calibrated uncertainty interval; a decision-regret evaluation; the k6 result. `[G]`
**Claim supported.** In one 580 m cluster, over a 28-day holdout, a weather-aware global model reduces day-ahead MAE by about 20% versus a no-weather model and about 38% versus seasonal naive, which is enough to support a relative staffing plan. `[M]`
**Claim not supported.** A city-wide heatmap; a census of people; annual seasonality; causal weather effects; ROI or productivity gains; a general multi-city model; live sensor data. `[G]`
**Effort and risk.** Low: the work is already done. The risk is overclaiming. The paper should present this as a **single-site, short-horizon decision-support result**, not as proof of generalization.
### 5.2 Case study 2 — Footfall to advertising allocation (same field, new decision)
**Research question.** Can the same forecast rank the eight locations and time windows for physical advertising exposure, and how sensitive is the ranking to weather scenarios and measurement error?
**Why it tests the thesis.** It isolates the **decision layer**. The sensing modality, field, and forecast are unchanged; only the objective changes from staffing to exposure allocation. If the core is modular, this should be a cheap and credible extension. If it is not, the "generalizes across decisions" claim is weak.
**Data we have.** The same PAX data and PulseCast forecast. No ad inventory, cost, reach, frequency, dwell time, audience composition, or conversion data. `[D]`
**Analysis we can do now.** Rank the eight locations by forecasted footfall and stability; compute exposure-weighted allocations under an explicit equal-cost/equal-reach assumption; run the sunny-Saturday, rainy-Saturday, and weekday-event scenarios; report how the ranking changes. This is a **re-analysis of the same field**, not an ad-effectiveness study. `[P]`
**Results we have.** **None.** The bundle contains no advertising result. Any lift, CPM, or revenue number here would be invented. `[G]`
**What is missing.** Ad inventory (sites, formats, prices); reach and frequency; audience segmentation; dwell time; store conversion; city-wide sites; a control/treatment design for causal inference. `[G]`
**Claim supported.** The forecast can support a **local, exposure-based ranking** of the eight measured locations and hours, and the decision layer can be swapped without retraining the sensor model. `[P]`
**Claim not supported.** Ad effectiveness, incremental revenue, city-wide optimal placement, or causal impact of advertising. `[G]`
**Effort and risk.** Low for the exposure ranking; high for any credible effectiveness claim. The paper should label this as an **analytical extension**, not a second empirical validation.
### 5.3 Case study 3 — Fill level to waste routing (new modality, graph field)
**Research question.** Can fill-level histories produce container-level forecasts and a fill-level-driven routing policy that reduces overflow or cost relative to a fixed area schedule?
**Why it tests the thesis.** It is the strongest test of **modality transfer**: the sensor stream is scalar and event-like rather than a 15-minute count, and the decision is a vehicle-routing problem rather than a staffing heuristic. It also exposes the spatial gap: the data have no coordinates, so the "field" must become a graph.
**Data we have.** 11,701 readings; 449 anonymized containers; 2025-10-17 to 2026-09-30; white 4,432, green 4,169, brown 3,100; `overfull` 1,192; numeric 0–100; four columns; no coordinates, capacity, fleet, depot, or shift data. `[D]`
**Analysis we can do now.** Per-container fill trajectories; clustering by fill rate and seasonality; time-to-threshold or survival models; holiday/school-vacation effects; a synthetic or assumed geometry for routing; a VRP baseline versus a fixed schedule under explicit assumptions. The forecast part is defensible; the routing part is **conditional on assumed geometry**. `[P]`
**Results we have.** **None.** The bundle documents the dataset but contains no model result, and the raw CSV is not present in this workspace; only the concept file is. `[G]`
**What is missing.** Coordinates or a zone mapping; container type and capacity; fleet capacity/weight limits; depot locations; shift rules; route costs; overflow ground truth; the reading protocol; the raw file for reproduction. `[G]`
**Claim supported.** The temporal-forecast and decision-rule pattern transfers to a second scalar sensor stream, provided the spatial layer is redefined as a **container graph or zone-level attribute field**. `[P]`
**Claim not supported.** Route savings, reduced emissions, or overflow reduction without real geometry and fleet data. A geographic heatmap of fill levels without coordinates. `[G]`
**Effort and risk.** Medium-high. The forecast is feasible; the routing evaluation is the bottleneck. If the paper includes this case, it must show an explicit synthetic-geometry sensitivity analysis and label it as such.
### 5.4 Case study 4 — LoRaWAN reception to gateway planning (inverted arrow)
**Research question.** Can reception data from 448 observed devices and 8 gateways (9 observed gateway IDs) predict coverage and rank 71 candidate gateway sites, compared with a distance-only baseline?
**Why it tests the thesis.** It is the best available **spatial-design** case with real data. It tests whether the observation model, spatial prediction, and decision layer transfer when the decision is "where should the infrastructure go?" rather than "how many staff next week?" It also exposes the limit: there is no temporal forecast.
**Data we have.** 8,009 reception rows; 31 columns; 448 devices; 8 gateway metadata rows and 9 observed gateway IDs; 593 meters; 71 candidate sites; 23,127 addresses; six days (2026-11-04 to 2026-11-10); RSSI -127 to -70 dBm; SNR -21.5 to 12 dB; spreading factors 7–12; 145 mapped meters silent. `[D]`
**Analysis we can do now.** A link-level model of reception quality or success using distance, radio settings, and building/terrain proxies; leave-one-gateway-out validation; candidate-site coverage and redundancy scoring; address-weighted coverage; a distance-only baseline. `[P]`
**Results we have.** **None.** No model metrics are documented in the bundle. `[G]`
**What is missing.** A packet-loss or expected-transmission baseline; building heights and terrain; gateway heights; GPS calibration; resolution of the ninth gateway; a longer window; field validation of predicted coverage. `[G]`
**Claim supported.** Spatial prediction and candidate-site ranking transfer as a **sibling** of the forecast core, with a different observation model and a different decision objective. `[P]`
**Claim not supported.** Temporal forecasting of reception; a replacement for professional radio planning; coverage guarantees without packet-loss data. `[G]`
**Effort and risk.** Medium-high. The data are real and the candidate sites are explicit, but the missing packet-loss baseline is a serious limitation. A leave-one-gateway-out protocol is the minimum credible evaluation.
### 5.5 Candidate protocol, not a case study — Satellite LST to sensor placement (AI TRAQC)
**Research question.** Where should k stationary sensors be placed, and which mobile routes should be run, to reconstruct a 100 m land surface temperature map from a 1 km baseline?
**Why it matters.** This is the purest test of the **literal heatmap** claim and of the spatial-inference layer. It also makes the design inversion explicit: the decision is the sampling operator.
**Data we have.** No delivered dataset in the bundle. Open Sentinel-3, Landsat 8/9, optional ECOSTRESS, Local Climate Zone, and OpenStreetMap are described. `[D]`
**What a minimal protocol would need.** A city boundary and priority areas; cloud-free Landsat reference scenes; a Sentinel-3 baseline; an LCZ/OSM feature layer; a placement algorithm (greedy, GP-based, or Bayesian optimal design); a mobile-route planner; a reconstruction method; and an error-versus-number-of-points curve. `[G]`
**Why it is not a full case study here.** No dataset is supplied, no field deployment exists, and the paper's time budget is already tight. Including it as a protocol is honest; presenting it as a result would not be.
---
## 6. Gaps
### 6.1 Data gaps
| Gap | Challenges affected | Consequence for the paper | What would close it |
|---|---|---|---|
| **Spatial coverage** | C1/S1, ads, store location | Eight points in 580 m cannot support city-wide heatmaps, ad-site ranking, or store-location ranking | More sensors or city-wide mobility data; explicit local-only scope |
| **Temporal coverage** | C1/S1, C2 | Five months of multi-sensor data; no annual seasonality; 28-day test; waste readings irregular | Multi-season data; longer holdout; forecast weather; event calendar |
| **Observation calibration** | C1/S1 | Device penetration unknown; counts are devices, not people | Penetration study or calibration assumption with sensitivity analysis |
| **Spatial geometry** | C2, C4 | No container coordinates; uncertain meter GPS; no building/terrain | Zone mapping; cadastral/OSM/DEM; GPS validation |
| **Decision outcomes** | All | No POS/labour, ad cost/reach, fleet/route, packet-loss, or stance labels | Operational data sharing; pilot or A/B design; labelled validation set |
| **Privacy evaluation** | C1/S1, C4, C5, C6 | No formal threat model, DP/k-anonymity, or leakage metric | Privacy audit; aggregation thresholds; formal guarantee where possible |
| **Multimodal data** | C1/S1 vs C3/C4/C5 | Only one sensor modality is empirically fused with context | Joint dataset or a clearly scoped interface-level claim |
| **Text labels** | C5 | No stance gold set; keyword noise; `-99` sentinel | Manual annotation; relevance classifier; inter-annotator agreement |
| **Offer data** | C6 | No dataset, no attendance, no demand ground truth | Institution APIs/exports; legal review; anonymous query analytics |
| **Reproducibility** | C2, C3, C4 | Raw files are not all in the workspace; PAX raw is git-ignored | Documented data-access procedure; derived aggregates; model card |
**Documentation inconsistency to reconcile.** `mission/data.md` reports the cluster as ~580 m x 280 m; `mission/paper.md` reports 584 m x 284 m for the same cluster. Both are approximations, but the paper should use one number consistently. `[G]`
### 6.2 Method gaps
- **No formal observation model.** PulseCast treats the PAX count as a direct signal; there is no measurement-error model or detection-function correction. C4 has the same issue with packet loss.
- **No spatial-inference baseline.** PulseCast does not compare interpolation methods; with eight points, a GP/kriging baseline and a leave-one-sensor-out test would make the local field claim much stronger.
- **No probabilistic calibration.** The forecast exposes an uncertainty band, but the bundle does not document calibration (coverage of intervals, CRPS, or quantile loss).
- **No decision regret.** The staffing heuristic is evaluated by forecast accuracy and plan latency, not by whether the plan improved coverage or cost in practice.
- **No causal design.** Weather coefficients are predictive, not causal. The what-if scenarios are counterfactual inputs, not identified effects.
- **No active-learning baseline.** C3 and C4 need a comparison between the proposed placement and a random or greedy baseline.
- **No VRP baseline.** C2 needs a fixed-schedule baseline and a distance-only or fill-threshold baseline.
- **No link-budget/physics baseline.** C4 needs a distance-only or free-space-path-loss baseline before a learned model can claim value.
- **No cross-city validation.** Every claim is single-city, and most are single-site.
- **No multimodal fusion experiment.** Cloud/wind did not improve the short test; no second sensor modality was fused.
### 6.3 Claim gaps
| Claim | Risk | Safer formulation |
|---|---|---|
| "City-wide heatmap" | Eight sensors in 580 m cannot support it | "Local pulse map with measured points and labelled interpolation" |
| "Generalizes across six challenges" | Only one end-to-end empirical case | "Transfers as a contract across a subset; two challenges are siblings and two are out of scope" |
| "Privacy-first" | Aggregation is architectural, not a formal guarantee; PAX sensing has externalities | "Aggregate-only handling with no MAC/PII in the app; formal privacy evaluation is future work" |
| "Multimodal forecasting" | Only PAX + tabular context is empirically fused | "Modality-agnostic interface, demonstrated on one sensing modality" |
| "Production-ready" | Hackathon deployment; no auth/HA/live ingestion | "Deployed hackathon system with health checks, limits, and documented gaps" |
| "Real-time" | CSV, no live feed; 15 min/1 h are replay/nowcast | "Replay/nowcast layer; no live sensor ingestion" |
| "Decision-grade" | No decision outcome data | "Decision-support with explicit assumptions; decision-regret evaluation is future work" |
| "Weather improves accuracy" | Observed weather, 28-day test, one site | "Observed weather improves short-horizon accuracy in this test; forecast-weather validation is needed" |
### 6.4 Evidence status
| Claim | Evidence in the bundle | Status | What would settle it |
|---|---|---|---|
| Sparse footfall + weather beats seasonal naive | MAE 3.36 vs 5.40; WAPE 27.8% vs 44.7%; R² 0.779 vs 0.462; 28-day holdout | **Supported, single site, short test** | Multi-season, multi-site, forecast-weather replication |
| Weather is a real signal | 4.20 to 3.36 MAE (~20%); 5.40 to 3.36 (~38%) | **Supported in test** | Ablation across seasons and sites |
| Holidays improve accuracy | 4.15 vs 4.20; 3.44 vs 3.36 | **Not supported; little effect** | Larger sample; event model; school holidays |
| The pipeline transfers to waste | No result; dataset documented | **Untested** | Fill forecast + routing with real geometry |
| The pipeline transfers to ads | No result; same data | **Untested analytical extension** | Ad inventory/cost/reach; decision evaluation |
| The pipeline transfers to store location | No; eight points | **Not supported at city scale** | City-wide field and commercial data |
| The pipeline transfers to sensor placement | No result; protocol only | **Untested sibling** | Satellite pipeline + error-vs-count experiment |
| The pipeline transfers to LoRaWAN | No result; data documented | **Untested sibling** | Link model + leave-one-gateway-out site ranking |
| Cable car / learning guide are applications | No | **Not supported** | Different stack; at most context/consumer |
| Privacy-first | Aggregate counts, no MAC/PII in app | **Partially supported** | Threat model; formal privacy audit |
| Production-ready | Deployed design, health checks, limits | **Partially supported** | HA/auth/live ingestion/ops |
| Real-time | CSV, replay/nowcast | **Not supported** | Live feed |
### 6.5 Minimum additions for a credible paper
**P0 — must do before submission.**
1. **Scope the thesis.** Replace "generalizes across six challenges" with "a decision-grade sensing contract transfers across a subset of urban problems; we show where it stops."
2. **Keep the PulseCast numbers exact and baseline-anchored.** Report MAE/WAPE/R², the 28-day holdout, the weather ablation, and the honest holiday result.
3. **Relabel the heatmap.** Use "local pulse map" or "measured points plus labelled interpolation"; never "city-wide heatmap."
4. **Include the transferability matrix with evidence status.** The matrix in this note is designed to be adapted directly into the paper.
5. **Add one decision-relevant evaluation.** At minimum, compare the staffing plan to a flat schedule on coverage and cost under the stated assumptions; label it as a heuristic comparison, not ROI.
**P1 — should do if time allows.**
6. **Add one additional empirical case.** The cheapest is the **advertising exposure re-analysis** (same data, new objective). The most informative is **waste** (new modality) or **LoRaWAN** (spatial design), but both need substantial data work.
7. **Add a per-sensor error table and a leave-one-sensor-out check.** This tests whether the global model is learning shared structure or memorising sensor IDs.
8. **Add uncertainty calibration.** Report interval coverage or a proper scoring rule, not only a band on a chart.
9. **Add a privacy paragraph with a threat model.** State what is aggregated, what is not stored, and what residual risk remains.
10. **Fill the k6 load-test number.** Do not claim 120 clients without the measured result.
**P2 — future work, not paper claims.**
11. **Waste forecast + synthetic-geometry routing sensitivity.**
12. **LoRaWAN link model with leave-one-gateway-out validation.**
13. **AI TRAQC placement protocol with an error-versus-count curve.**
14. **Cross-city replication of the PulseCast model.**
15. **A joint multimodal dataset** that would justify the word "multimodal" empirically.
### 6.6 Suggested thesis sentence for the paper
> We do not claim that one model solves six challenges. We claim that a decision-grade sensing contract — explicit observation model, resolution-matched spatial inference, context-fused forecast, separated decision rule, and baseline-plus-regret evaluation — transfers across a subset of urban problems, and we show precisely where it stops.
This sentence is consistent with every number and every gap in the bundle. It is stronger than an unsupported six-way generalization because it makes the boundary conditions part of the contribution.
---
## Source map
| Source | Used for |
|---|---|
| `OKF_Vault/knowledge/challenges/*.md` | Challenge decisions, data descriptions, resolutions, uncertainties, deliverables |
| `OKF_Vault/knowledge/data/*.md` | Dataset counts, schemas, coverage, data-quality notes |
| `OKF_Vault/mission/data.md` | PAX counts, sensor profile, model metrics, weather/holiday findings, explicit limitations |
| `OKF_Vault/mission/mvp.md` | PulseCast decision, staffing heuristic, out-of-scope decisions |
| `OKF_Vault/mission/architecture.md` | Horizons, services, deployment, privacy handling, failure modes |
| `OKF_Vault/mission/paper.md` | Paper structure, required figures/tables, facts and numbers, verification checklist |
| `OKF_Vault/mission/README.md` | Product thesis, scope, non-negotiables |
| `OKF_Vault/mission/demo.md` | Claim discipline, Q&A boundaries, "do not say" list |
| `OKF_Vault/mission/questions.md` | Settled facts, deployment constraints, open decisions |
| `OKF_Vault/knowledge/index.md` | Challenge and dataset inventory |
+529
View File
@@ -0,0 +1,529 @@
# Data quality of the Schwarz PAX BLE counter export
**Research note · 14 November 2026**
**Source:** `data/raw/schwarz-pax.csv` (19.4 MB, read-only)
**Scope:** reliability and privacy limits of 15-minute BLE PAX aggregates from eight Heilbronn locations, 18 June – 7 November 2026 (143 days), plus the legacy `density-01` series from October 2025 to March 2026.
---
## Summary
- The export contains **192,464 rows** (96,232 density + 96,232 battery), **12 device IDs**, **11 names**, and **8 usable locations**. The 8 current devices all lie on a **0.7 value grid**; the legacy `density-01` device (`863f75b0`) is on a **1.0 integer grid**. The two `density-01` series are separated by a **35.2-day gap** and must not be pooled without an explicit scale break.
- The nominal “15-minute” cadence is **not a 15:00 wall-clock grid**. Most current devices report every **15.27–15.37 min** (median), the new `density-01` every **16.14 min**, and the legacy device every **15.18 min** with ~18 s jitter. The new device produces ~89 readings/day, not 96.
- Coverage is strongly non-stationary. Median location-month coverage (48 location-months) is **86.1%**, but August 2026 falls to **54.2%** on average and **23.3%** at the worst location. Six of eight locations show a simultaneous multi-week degradation; `density-01` and `hagenbuchersee-01` do not.
- **All eight locations report in the same 15-minute slot only 28.2% of the time** (3,876 / 13,728). The modal count is **seven** locations (4,117 slots), not eight. Aggregating to 30/60/120 min raises complete eight-location coverage to **63.4% / 69.7% / 71.0%**.
- Values are **not raw counts**. Overall, **5.18%** of readings are zero, **10.36%** are `<1`, and **39.57%** are `<5`. There are no negative or NaN values. The effective positive step is **0.7** for all current devices and **1.0** for the legacy device; the unit and any vendor correction are undocumented.
- Raw cross-location Pearson correlation averages **0.58** (range 0.13–0.90), but drops to **0.38** after removing location-specific hour-of-week means and **0.48** at the daily level. Correlation is not evidence of shared unique visitors.
- Battery telemetry is structurally coupled to density telemetry: every density row has a battery value within 60 s, but both disappear together during outages. Levels are coarse and stale (70–89% of consecutive values identical; runs up to 19 days) and do **not** predict daily counts within a device (−0.41 to 0.08). Across devices, however, August coverage is strongly confounded with battery (**r = 0.84**; six degraded devices at 4–10% vs two healthy at 41–70%), so battery, device batch, and gateway are collinear and the outage cause cannot be isolated.
- **What cannot be measured from these aggregates:** MAC-randomization rate, penetration rate, unique-device counts, dwell time, repeat visits, detection radius, or false-positive/false-negative rates. The 0.7 grid and zero fractions are **proxies for vendor processing**, not direct measurements of randomization or penetration.
---
## 1. Data and methods
### 1.1 File and schema
The CSV is a long-format export with one row per `(time, sensor_id, type)` observation. Timestamps are ISO-8601 with sub-second precision and UTC offset (`+00`). The schema is:
| column | dtype | non-null | example |
|---|---|---:|---|
| `time` | datetime64[us, UTC] | 192,464 | 2026-06-12 12:27:15.241799+00:00 |
| `sensor_id` | string | 192,464 | `00091a5f` |
| `name` | string | 192,464 | `hn-goetzentuem-01` |
| `longitude` | float64 | 192,464 | 9.2148042 |
| `latitude` | float64 | 192,464 | 49.1399582 |
| `type` | string | 192,464 | `densityData` |
| `value` | float64 | 192,464 | 10.5 |
There are **no nulls** in any column. `type` has exactly two values: `densityData` (96,232 rows) and `batteryData` (96,232 rows). Density and battery rows are emitted in pairs; battery timestamps are a median **0.033 s** after the density timestamp (p99 5.82 s, max 34.3 s).
### 1.2 Definitions
- **Device ID**: `sensor_id` (12 unique). **Location**: `name`, except that the two `density-01` device IDs are treated as one location for the 8-location panel.
- **Core devices**: nine IDs with >1,000 density readings. They comprise **eight current locations** plus the legacy `density-01` device. The eight current device IDs are the “8 usable sensors”; the legacy ID is retained for the scale-break analysis but excluded from the panel.
- **Episodic devices**: three IDs with 5–73 readings (`hn-bollwerksturm-01`, `maker-fair`, `niklas-01`). They are test/pilot deployments and are excluded from the panel.
- **Panel window**: `2026-06-18 00:00 UTC` to `2026-11-08 00:00 UTC` (exclusive), i.e. **143 days / 13,728 15-minute slots**. This starts at the first full day on which all eight current locations are live, avoiding the partial deployment days of 12–17 June.
- **Slot**: UTC timestamp floored to 15 minutes (`HH:00`, `HH:15`, `HH:30`, `HH:45`). Multiple readings in one slot are averaged (rare).
- **Local time**: `Europe/Berlin` for hourly/weekday profiles. The DST transition on 25 October 2026 is handled by the timezone library; one local day has 25 hours.
- **Coverage**: share of the 13,728 panel slots with at least one reading, **not** share of expected readings. Because the native cadence is 15.27–16.14 min, 96 slots/day is an upper bound that most devices cannot reach even when healthy. A second metric, **coverage vs cadence ceiling**, divides observed slot coverage by the maximum attainable coverage given the median cadence (`100 × 15 / median_interval_min`).
### 1.3 Software and reproducibility
All computations were done with Python 3.12, pandas 3.1.1 and matplotlib 3.11.2. The analysis script is `paper/research/scripts/01_data_quality.py`; it reads the raw CSV read-only and writes tables to `paper/research/outputs/` and figures to `paper/research/figures/`. Run:
```bash
uv run --with pandas --with matplotlib python paper/research/scripts/01_data_quality.py
```
The full per-device tables are in the CSV outputs cited below; the main text shows the values needed for the paper.
---
## 2. Inventory and time range
**Overall range:** 2025-10-16 21:17:59 UTC → 2026-11-07 23:51:57 UTC (**387.1 days**).
**Panel range:** 2026-06-18 → 2026-11-07 (**143 days**).
| device_id | name | role | density readings | first density timestamp | last density timestamp |
|---|---|---|---:|---|---|
| `863f75b0` | density-01 (legacy) | core | 8,075 | 2025-10-16 21:17 | 2026-03-12 18:28 |
| `f8c19730` | density-01 (new) | core | 17,585 | 2026-04-16 23:25 | 2026-11-07 23:48 |
| `00091a5f` | hn-goetzentuem-01 | core | 9,952 | 2026-06-12 12:27 | 2026-11-07 23:39 |
| `e52c55f3` | hn-hagenbuchersee-01 | core | 12,345 | 2026-06-12 12:02 | 2026-11-07 23:46 |
| `0f8519cf` | hn-hagenbuchersee-02 | core | 10,099 | 2026-06-12 12:32 | 2026-11-07 23:51 |
| `8d7eef22` | hn-marahaus-01 | core | 9,254 | 2026-06-12 12:36 | 2026-11-07 23:49 |
| `d4cce74b` | hn-neckarufer-01 | core | 9,527 | 2026-06-12 12:21 | 2026-11-07 23:49 |
| `ba71377f` | hn-soleo-01 | core | 10,327 | 2026-06-12 11:52 | 2026-11-07 23:32 |
| `f4928534` | hn-stadtgalerie-01 | core | 8,982 | 2026-06-12 12:11 | 2026-11-07 23:26 |
| `d3304b39` | hn-bollwerksturm-01 | episodic | 5 | 2026-06-12 11:19 | 2026-08-13 16:01 |
| `2aadbeed` | maker-fair | episodic | 73 | 2026-08-13 16:07 | 2026-08-16 15:06 |
| `062fe216` | niklas-01 | episodic | 8 | 2026-05-01 17:19 | 2026-05-01 19:06 |
**Important deployment artefacts.**
- The first hn-* reading for seven locations is a single test message on **12 June 2026**, followed by a **4–5 day gap** until 16–17 June. The panel therefore starts on 18 June.
- The `maker-fair` device is at **9.77°E, 52.38°N** (Hannover), roughly 350 km from the Heilbronn cluster (9.21–9.22°E, 49.14–49.15°N). Its 73 readings over three days (mean 117.8) are a separate deployment, not a Heilbronn location.
- `hn-bollwerksturm-01` has five readings over 62 days, including a 58-day gap; it is effectively non-operational.
- The two `density-01` IDs share the same name and coordinates within ~4 m. They are a **hardware replacement**, not two simultaneous sensors.
---
## 3. Value quantization and the `density-01` scale discontinuity
This is the most important unit-level finding. The values are not raw counts, and the counting grid changed at the hardware replacement.
### 3.1 Per-device value grid
“Multiple of 0.7” means the value divided by 0.7 is an integer within floating-point tolerance. “Integer” means the value equals its rounded value. The two sets overlap at 0, 7.0, 14.0, 21.0, … because every tenth 0.7 step is an integer. In this export `sensor_id` *is* the device ID, so this per-device table is also the per-sensor table; the only location with two device IDs is `density-01`.
| device_id | period | n | min–max | unique values | min positive | effective grid | % on 0.7 grid | % integer | % both | % integer only | % neither |
|---|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| `f8c19730` | current | 17,585 | 0–63.0 | 84 | 0.7 | 0.7 | 100.00 | 12.72 | 12.72 | 0.00 | 0.00 |
| `00091a5f` | current | 9,952 | 0–210.7 | 152 | 0.7 | 0.7 | 100.00 | 9.90 | 9.90 | 0.00 | 0.00 |
| `e52c55f3` | current | 12,345 | 0–292.6 | 206 | 0.7 | 0.7 | 100.00 | 11.71 | 11.71 | 0.00 | 0.00 |
| `0f8519cf` | current | 10,099 | 0–200.9 | 202 | 0.7 | 0.7 | 100.00 | 8.30 | 8.30 | 0.00 | 0.00 |
| `8d7eef22` | current | 9,254 | 0–194.6 | 140 | 0.7 | 0.7 | 100.00 | 7.64 | 7.64 | 0.00 | 0.00 |
| `d4cce74b` | current | 9,527 | 0–255.5 | 192 | 0.7 | 0.7 | 100.00 | 8.72 | 8.72 | 0.00 | 0.00 |
| `ba71377f` | current | 10,327 | 0–111.3 | 107 | 0.7 | 0.7 | 100.00 | 12.34 | 12.34 | 0.00 | 0.00 |
| `f4928534` | current | 8,982 | 0.7–137.2 | 168 | 0.7 | 0.7 | 100.00 | 6.32 | 6.32 | 0.00 | 0.00 |
| `d3304b39` | episodic | 5 | 6.3–16.1 | 5 | 6.3 | 0.7 | 100.00 | 20.00 | 20.00 | 0.00 | 0.00 |
| `2aadbeed` | episodic | 73 | 9.1–165.2 | 57 | 9.1 | 0.7 | 100.00 | 13.70 | 13.70 | 0.00 | 0.00 |
| `062fe216` | episodic | 8 | 2.1–6.3 | 3 | 2.1 | 0.7 | 100.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| `863f75b0` | legacy | 8,075 | 0–28.0 | 27 | 1.0 | 1.0 | 35.36 | 100.00 | 35.36 | 64.64 | 0.00 |
**Findings.**
- **All 11 non-legacy devices** (8 current + 3 episodic) report **100.00% multiples of 0.7**. For the eight current panel devices, the minimum positive value is 0.7 and the smallest observed spacing is 0.7; the episodic devices lie on the same grid but their observed minima are higher (2.1–9.1) because they have very few readings.
- The **legacy `density-01`** reports **100% integers** and **0% non-integer values**; its minimum positive value is 1.0 and its smallest spacing is 1.0. Its 35.36% “multiple of 0.7” share is coincidental: those are the integers 0, 7, 14, 21, 28, which happen to be divisible by 7.
- **No device uses 0.1 or 0.5 as its finest grid.** All values are representable as multiples of 0.1 because 0.7 = 7 × 0.1, and 15.7–23.3% of current readings are also multiples of 0.5 because every fifth 0.7 step (3.5, 7.0, 10.5, …) is a 0.5 multiple. Those are subsets of the 0.7 grid, not separate grids. The overall “multiple of 0.1” share is 100% to floating-point tolerance, which is an artefact of decimal representation, not evidence of 0.1 resolution.
- **“Occasional exact integers” are not a separate mode.** Current devices hit an integer only when the 0.7-grid index is a multiple of 10 (0, 7.0, 14.0, …). The observed 6.3–12.7% integer share is consistent with that overlap.
- Overall, **94.58%** of all density readings are multiples of 0.7 and **17.63%** are integers. **0.00%** lie on neither grid.
- The 0.7 factor is **undocumented**. It is consistent with a vendor correction/scaling factor (possibly related to MAC-randomization adjustment) or a unit change, but the export contains no metadata to distinguish these. Do not describe values as “devices” or “raw counts.”
**Consistency check:** all eight usable (current) sensors are **100% on the 0.7 grid**; the legacy `density-01` is the **only 1.0-grid device**; no device uses 0.5 or 0.1 as its base grid. The 0.5 share is a subset of the 0.7 grid, and the 0.1 share is a decimal-representation artefact.
### 3.2 `density-01`: two device IDs, two scales, 35.2-day gap
| series | n | first | last | min | max | mean | median | p95 | p99 | zero % | unique | grid |
|---|---:|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| `863f75b0` (legacy) | 8,075 | 2025-10-16 21:17:59 | 2026-03-12 18:28:06 | 0 | 28.0 | 3.008 | 2.0 | 11.0 | 16.26 | 29.77 | 27 | 1.0 |
| `f8c19730` (new) | 17,585 | 2026-04-16 23:25:31 | 2026-11-07 23:48:49 | 0 | 63.0 | 9.741 | 7.7 | 28.0 | 38.50 | 5.87 | 84 | 0.7 |
- The **gap between the last legacy reading and the first new reading is 35.2065 days** (3,041,845.5 s). There is **no overlap**, so a conversion factor cannot be estimated empirically.
- The legacy range is **0–28** with 27 unique values; the new range is **0–63.0** with 84 unique values. The new maximum 63.0 is exactly 90 × 0.7; the legacy maximum 28 is 28 × 1.0. This is a **scale/quantization discontinuity**, not just a change of device ID.
- The level shift is large: mean **3.008 → 9.741** (×3.24) and median **2.0 → 7.7** (×3.85). This is **not** explained by the 0.7 factor alone: if the new values were 0.7 × raw counts, the numeric values would be lower, not higher, for the same raw count. The periods also differ by season (winter vs summer) and possibly by installation/sensitivity (legacy zero share 29.8% vs new 5.9%). The safe conclusion is: **the two series are not on the same scale and cannot be pooled or compared directly**; the cause of the level difference is confounded.
- The legacy device also has far worse coverage (Section 5): 56.5% of its active 15-minute slots, 417 gaps >1 h, and 29.8% zeros. Treat it as a separate, lower-quality series.
![Value grids and the density-01 scale break](figures/fig_quantization.png)
*Figure 1. Left: per-device share of readings on the 0.7 grid, integer grid, both, or neither. Right: reported value ladders for the two `density-01` devices (0–7 zoom), showing the 1.0 vs 0.7 spacing.*
![density-01 before and after the device replacement](figures/fig_density01_swap.png)
*Figure 2. Daily mean density for the legacy (`863f75b0`, 1.0 grid) and new (`f8c19730`, 0.7 grid) `density-01` devices. The shaded band is the 35.2-day gap with no data; the two series do not overlap, so no empirical conversion factor can be estimated.*
### 3.3 Integrity checks
| check | count |
|---|---:|
| exact duplicate rows | 0 |
| duplicate `(time, sensor_id, type)` | 0 |
| duplicate `(time, sensor_id)` | 0 |
| density values negative | 0 |
| density values NaN | 0 |
| battery values negative | 0 |
| battery values >100 | 0 |
| battery values NaN | 0 |
| core inter-reading intervals <60 s | 113 |
- There are **no duplicate timestamps** on the natural key `(time, sensor_id, type)` and no duplicate `(time, sensor_id)` pairs. The 113 sub-minute intervals are **near-duplicates**, not exact duplicates: 106 belong to the legacy `density-01` device, 6 to `stadtgalerie`, 1 to `marahaus`. The median spacing is 7.0 s, most carry identical values, and the maximum is 53.3 s. These look like repeated publications/retries; deduplicating by keeping one reading per 60 s is advisable.
- **No negative or NaN values** in either series. Battery values are bounded 0–100; density has no declared upper bound and reaches 292.6.
- There is **one duplicate sensor name** (`density-01`) but no duplicate `sensor_id`s. The name collision is the hardware replacement, not a duplicated device.
---
## 4. Cadence and time alignment
The 15-minute grid is **nominal, not exact**. Each device has its own timer and phase, and the timestamps carry second-level jitter.
| device | kind | median interval (s) | mean (s) | SD (s) | IQR (s) | p05–p95 (s) | within 900 ± 5 s | within 900 ± 30 s | 14–17 min share |
|---|---|---:|---:|---:|---|---:|---:|---:|---:|
| `863f75b0` | legacy | 910.86 | 916.35 | 18.03 | 909.85–917.25 | 903.10–958.19 | 12.6% | 89.1% | 72.6% |
| `f8c19730` | new | 968.52 | 968.95 | 2.41 | 968.02–969.02 | 967.19–974.44 | 0.0% | 0.01% | 96.6% |
| `e52c55f3` | current reference | 916.08 | 916.77 | 3.44 | 915.61–916.80 | 914.67–922.11 | 0.4% | 99.3% | 97.4% |
Across the core devices, **72.6–97.4%** of intervals fall in the 14–17 min band and **2.0–12.9%** in the 28–34 min band; the latter are missed single ticks, not a slower timer. Longer intervals are dropouts.
- **Legacy `863f75b0`:** the near-cadence median is **910.86 s = 15.181 min**, with SD 18.0 s and IQR 909.85–917.25 s. Only 12.6% of intervals are within 5 s of 900 s; 89.1% are within 30 s. The first intervals are 1021.19 s, 942.45 s, 911.49 s, then a 2089.20 s gap (missed ticks), then 912.77 s. This is a ~15.2-minute timer with tens-of-seconds jitter and frequent missed transmissions, not a clean 15:00 clock.
- **New `density-01` `f8c19730`:** median **968.52 s = 16.142 min**, SD 2.41 s, IQR 968.02–969.02 s. It is extremely regular but **7.6% slower than 15 minutes**; only 0.01% of intervals are within 30 s of 900 s. At 16.14 min it produces ~89 readings/day, so 96 slots/day is unreachable. Its observed panel coverage (88.4%) is 95.2% of its cadence ceiling (92.9%).
- **Current reference `e52c55f3`:** median **916.08 s = 15.268 min**, SD 3.44 s, 99.3% within 900 ± 30 s. The other current devices are similar (15.27–15.37 min).
- Consequence: flooring to `HH:00/15/30/45` is an approximation. A device’s reading can sit anywhere inside the bin, and two devices matched to the same slot can be up to ~15 minutes apart. This attenuates cross-sensor correlation and makes the 15-minute complete-case panel sparse. Section 5 quantifies the remedy: 30–60-minute bins.
![Cadence distribution](figures/fig_cadence.png)
*Figure 3. Left: distribution of inter-reading intervals for core devices (≤60 min), with peaks at ~15 and ~30 min and a small <1 min retry/duplicate bar. Right: median cadence with IQR; annotations give the share of ~30-minute intervals (missed single ticks).*
---
## 5. Coverage and missingness
### 5.1 Per-device coverage and gaps
Panel = 18 Jun–7 Nov 2026 (143 days, 13,728 slots). “Gaps” are inter-reading intervals inside the panel. The legacy `density-01` has no panel rows (it ended in March); its active-span figures are shown for comparison.
| device | first | last | n | active-span coverage | panel coverage | readings/day (panel) | median cadence (min) | cadence ceiling | coverage vs ceiling | time in gaps >1 h | gaps >1 h | gaps >1 d | max panel gap (h) |
|---|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| goetzentuem-01 | 2026-06-12 | 2026-11-07 | 9,952 | 69.8% | 72.3% | 69.4 | 15.36 | 97.7% | 74.0% | 18.6% | 177 | 4 | 42.6 |
| hagenbuchersee-02 | 2026-06-12 | 2026-11-07 | 10,099 | 70.8% | 72.7% | 69.8 | 15.30 | 98.1% | 74.2% | 19.9% | 119 | 5 | 82.3 |
| marahaus-01 | 2026-06-12 | 2026-11-07 | 9,254 | 64.9% | 67.2% | 64.5 | 15.28 | 98.2% | 68.4% | 25.4% | 73 | 9 | 152.3 |
| soleo-01 | 2026-06-12 | 2026-11-07 | 10,327 | 72.4% | 74.4% | 71.4 | 15.28 | 98.2% | 75.8% | 17.8% | 116 | 4 | 58.1 |
| neckarufer-01 | 2026-06-12 | 2026-11-07 | 9,527 | 66.8% | 69.2% | 66.4 | 15.31 | 98.0% | 70.6% | 22.1% | 176 | 4 | 42.0 |
| hagenbuchersee-01 | 2026-06-12 | 2026-11-07 | 12,345 | 86.6% | 89.1% | 85.5 | 15.27 | 98.2% | 90.7% | 7.2% | 36 | 6 | 25.0 |
| stadtgalerie-01 | 2026-06-12 | 2026-11-07 | 8,982 | 63.0% | 65.2% | 62.6 | 15.37 | 97.6% | 66.8% | 23.4% | 149 | 4 | 141.8 |
| density-01 (new) | 2026-04-16 | 2026-11-07 | 17,585 | 89.3% | 88.4% | 84.9 | 16.14 | 92.9% | 95.2% | 0.5% | 13 | 0 | 4.6 |
| density-01 (legacy) | 2025-10-16 | 2026-03-12 | 8,075 | 56.5% | — | — | 15.18 | 98.7% | — | — | 417 | 3 | 58.7 |
**Reading the table.** For the legacy `density-01` row, gap counts and maximum gap are over its full active span (it has no panel rows); all other rows are panel-based. The distribution metrics requested for each sensor (zero fraction, `<1%`, `<5%`, mean, median, p95, p99, max) are in the table in Section 6.
- “Active-span coverage” uses each device’s own first–last span; “panel coverage” uses the common 143-day window.
- “Cadence ceiling” is the maximum slot coverage attainable at the device’s median cadence (`100 × 15 / median_min`). A device at 16.14 min cannot exceed 92.9% of 96 daily slots even with no outages.
- The legacy device’s 417 gaps >1 h and 29.8% zeros make it unsuitable as a continuous series. It is also the only 1.0-grid device.
- For the eight current devices, **roughly 18–25% of panel time falls inside gaps >1 h** for six locations (17.8% at `soleo`); `hagenbuchersee-01` is 7.2% and `density-01` 0.5%. Gaps >1 h are not just single missed 15-minute ticks: they number 36–177 per device and include many 1–6 h intervals (see `outputs/large_gaps.csv`).
- The maximum panel gap is **152.3 h (6.3 days)** at `marahaus-01`, followed by 141.8 h at `stadtgalerie-01` and 82.3 h at `hagenbuchersee-02`. `density-01` (new) never exceeds 4.6 h.
### 5.2 Monthly coverage and the August outage
| month | density-01 | goetzentuem | hagenbuchersee-01 | hagenbuchersee-02 | marahaus | neckarufer | soleo | stadtgalerie | mean |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| 2026-06 (partial) | 86.7 | 79.7 | 90.1 | 94.5 | 78.0 | 77.0 | 92.1 | 73.6 | 83.9 |
| 2026-07 | 87.1 | 86.5 | 95.2 | 88.5 | 85.4 | 83.2 | 90.7 | 78.9 | 87.0 |
| 2026-08 | 84.8 | 52.0 | 94.1 | 49.1 | 28.9 | 46.9 | 54.6 | 23.3 | 54.2 |
| 2026-09 | 90.6 | 57.2 | 66.8 | 51.0 | 56.0 | 52.7 | 53.0 | 66.9 | 61.8 |
| 2026-10 | 91.0 | 86.8 | 97.0 | 89.2 | 88.6 | 86.2 | 88.0 | 84.1 | 88.8 |
| 2026-11 (partial) | 92.7 | 86.8 | 98.2 | 87.5 | 89.3 | 86.0 | 88.7 | 82.6 | 89.0 |
**This is not a stationary missingness process.** July and October are healthy (mean 87–89%), while August collapses to 54.2% and September to 61.8%. Six of eight locations degrade together in August, but `density-01` (84.8%) and `hagenbuchersee-01` (94.1%) remain healthy. This split is **strongly confounded with battery**: the six degraded devices had August median battery levels of **4–10%**, while the two healthy devices had **41% and 70%**; across the eight devices (n = 8), August coverage and August median battery correlate at **r = 0.84**. That is consistent with a power/battery-related failure, but it is not conclusive: four of the six still report in October at 1–3% while two show battery resets to ~32–36% (likely replacement/charging), the battery gauge is coarse/stale, and battery is collinear with device batch, gateway, and installation. The export has no status/error codes, so the cause cannot be isolated.
### 5.3 All-eight complete cases are the exception
The panel has 13,728 15-minute slots. The distribution of the number of locations reporting per slot is:
| locations reporting | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| slots | 69 | 594 | 1,133 | 721 | 615 | 769 | 1,834 | 4,117 | 3,876 |
| share | 0.5% | 4.3% | 8.3% | 5.3% | 4.5% | 5.6% | 13.4% | 30.0% | **28.2%** |
- **All eight locations report in only 3,876 / 13,728 slots (28.2%)**, or 28.4% of the 13,659 slots with any data. The **modal count is seven**, not eight. Even in July (a healthy month), all-eight is only **36.5%**; in August it is **6.7%**; in October it is **40.3%**.
- Under independence, the product of the eight per-location coverage rates would predict only **1,281.5** all-eight slots (9.3%). The observed 3,876 is **3.0×** higher. Presence is therefore positively correlated across devices: they tend to be up together and down together, consistent with shared platform outages.
- **Aggregation fixes much of this.** Re-binning to 30 minutes raises all-eight coverage to **4,350 / 6,864 = 63.4%**; 60 minutes to **2,393 / 3,432 = 69.7%**; 120 minutes to **1,218 / 1,716 = 71.0%**. Pairwise mean correlation changes little (0.58 → 0.60 → 0.62 → 0.63), so 30–60-minute bins are a reasonable default for complete-case cross-sensor analysis.
| bin | expected bins | bins with any data | all-eight bins | all-eight % | mean pairwise r | median CV |
|---|---:|---:|---:|---:|---:|---:|
| 15 min | 13,728 | 13,659 | 3,876 | 28.2% | 0.58 | 0.66 |
| 30 min | 6,864 | 6,854 | 4,350 | 63.4% | 0.60 | 0.64 |
| 60 min | 3,432 | 3,429 | 2,393 | 69.7% | 0.62 | 0.62 |
| 120 min | 1,716 | 1,715 | 1,218 | 71.0% | 0.63 | 0.60 |
### 5.4 Missingness is not MCAR
For each location, we compared the median daily mean density on high-coverage days (≥80% of 96 slots) with low-coverage days (<50%). The ratio (low/high) ranges from **0.38 to 1.49** (median **0.69**): for six of eight locations, low-coverage days also have lower density (38–77% of high-coverage days). This is confounded with the August holiday period, which has both lower footfall and the outages. Either way, missingness is **not missing completely at random** with respect to the measured quantity. Complete-case seasonal means will be biased, and the direction differs by location. Missingness by local hour is relatively flat (roughly 25–40% for most devices), so there is no simple night-time shutdown pattern.
![Daily coverage heatmap](figures/fig_missingness_heatmap.png)
*Figure 4. Daily share of the 96 possible 15-minute slots with a reading. The dark August–September band is a systemic multi-location outage; note that `density-01` and `hagenbuchersee-01` remain mostly green.*
![Per-location daily series](figures/fig_time_series.png)
*Figure 5. Daily mean density per location. Gaps are days with <80% slot coverage (partial days excluded, not interpolated). The series show a genuine multi-day high-traffic period on 18–20 June, the August outage, and a broad seasonal decline into November.*
---
## 6. Value distribution, zero inflation, and low counts
| device | zero % | `<1` % | `<5` % | mean | median | p95 | p99 | max |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| goetzentuem-01 | 2.97 | 11.28 | 37.82 | 11.12 | 8.40 | 29.40 | 53.90 | 210.70 |
| hagenbuchersee-02 | 0.69 | 3.03 | 30.42 | 19.28 | 11.20 | 64.40 | 96.61 | 200.90 |
| marahaus-01 | 0.01 | 1.76 | 33.93 | 13.73 | 9.10 | 38.50 | 58.10 | 194.60 |
| soleo-01 | 4.77 | 9.66 | 32.13 | 10.58 | 8.40 | 28.00 | 39.20 | 111.30 |
| neckarufer-01 | 0.07 | 0.50 | 23.24 | 19.20 | 11.90 | 61.60 | 86.80 | 255.50 |
| hagenbuchersee-01 | 5.56 | 14.34 | 45.69 | 12.97 | 6.30 | 49.70 | 85.40 | 292.60 |
| stadtgalerie-01 | 0.00 | 9.26 | 40.79 | 21.14 | 8.40 | 67.20 | 88.90 | 137.20 |
| density-01 (new) | 5.87 | 13.20 | 39.68 | 9.74 | 7.70 | 28.00 | 38.50 | 63.00 |
| density-01 (legacy) | 29.77 | 29.77 | 77.78 | 3.01 | 2.00 | 11.00 | 16.26 | 28.00 |
- Overall: **5.18% zeros, 10.36% `<1`, 39.57% `<5`**. The distribution is heavy-tailed and zero-inflated; a log1p or negative-binomial treatment is more appropriate than Gaussian.
- A `<1` reading is not necessarily zero: on the 0.7 grid it means 0 or 0.7. `stadtgalerie-01` has **no zeros** (minimum 0.7) yet **9.26% `<1`**, because all of those are 0.7 readings.
- The legacy `density-01` is an outlier: **29.8% zeros** and **77.8% `<5`**, consistent with a lower-sensitivity or differently scaled deployment. It should not be pooled with the new series.
- There are **no negative or NaN values**. Density has no declared upper bound; the observed maximum is **292.6** at `hagenbuchersee-01` on 2026-08-14 19:19. The distribution is not truncated at 100 or any other ceiling.
![Density distributions](figures/fig_distribution.png)
*Figure 6. Left: histogram of density values (0.7 bin width; log y). Middle: CCDF on log-log axes. Right: per-device boxplots on a symlog scale. The legacy device is visibly lower and more zero-inflated; current devices share a similar heavy-tailed shape.*
---
## 7. Cross-location correlation and ratio stability
### 7.1 Pairwise correlation at matching 15-minute slots
The matrix uses the 8-location panel, pairwise complete observations. Each cell is Pearson *r*; *N* ranges from **7,261** (`neckarufer`–`stadtgalerie`) to **10,827** (`density-01`–`hagenbuchersee-01`). The diagonal is 1.00.
| location | density-01 | goetzentuem | hagenbuchersee-01 | hagenbuchersee-02 | marahaus | neckarufer | soleo | stadtgalerie |
|---|---:|---:|---:|---:|---:|---:|---:|---:|
| density-01 | 1.00 | 0.60 | 0.64 | 0.69 | 0.74 | 0.73 | 0.49 | 0.44 |
| goetzentuem | 0.60 | 1.00 | 0.75 | 0.63 | 0.78 | 0.72 | 0.55 | 0.37 |
| hagenbuchersee-01 | 0.64 | 0.75 | 1.00 | 0.83 | 0.79 | 0.87 | 0.48 | 0.15 |
| hagenbuchersee-02 | 0.69 | 0.63 | 0.83 | 1.00 | 0.73 | 0.90 | 0.45 | 0.13 |
| marahaus | 0.74 | 0.78 | 0.79 | 0.73 | 1.00 | 0.80 | 0.53 | 0.41 |
| neckarufer | 0.73 | 0.72 | 0.87 | 0.90 | 0.80 | 1.00 | 0.52 | 0.19 |
| soleo | 0.49 | 0.55 | 0.48 | 0.45 | 0.53 | 0.52 | 1.00 | 0.40 |
| stadtgalerie | 0.44 | 0.37 | 0.15 | 0.13 | 0.41 | 0.19 | 0.40 | 1.00 |
**Robustness to transformation and time scale:**
| estimator | mean off-diagonal *r* | min | max | N |
|---|---:|---:|---:|---:|
| Pearson, 15-min raw | 0.582 | 0.130 | 0.904 | 7,261–10,827 pairs |
| Spearman, 15-min | 0.678 | 0.438 | 0.871 | same pairs |
| Pearson, log1p | 0.670 | 0.435 | 0.862 | same pairs |
| Pearson, hour-of-week mean removed | 0.385 | 0.031 | 0.785 | same pairs |
| Pearson, daily means (69 high-coverage days) | 0.477 | −0.063 | 0.937 | 69 days |
- Raw Pearson is **0.58** on average but ranges from **0.13** (`hagenbuchersee-02` vs `stadtgalerie`) to **0.90** (`hagenbuchersee-02` vs `neckarufer`). Spearman and log1p are higher (0.67–0.68), indicating that raw Pearson is attenuated by spikes and zero inflation; rank/log transforms capture the shared ordering better.
- Removing location-specific **hour-of-week means** drops the average to **0.385**. At the **daily** level, the average is **0.477** (69 days with all eight locations at ≥80% coverage). So roughly **15–23% of variance is shared after removing the common weekly profile**, versus ~34% raw (0.58²). The rest is location-specific level, semantics, and noise.
- The low pair is interpretable: `stadtgalerie` (the name and profile suggest a shopping center) has a midday/afternoon peak and Sunday near-closure; `hagenbuchersee-02` (a lake location) has an evening peak. Their raw *r* = 0.13 is not a sensor failure; it is a different population/flow.
- **Correlation is not deduplication.** Two sensors can correlate because they see the same city rhythm, weather, or events, not because they see the same devices. With aggregates only, unique-visitor overlap is not identifiable.
![Correlation heatmaps](figures/fig_correlation_heatmap.png)
*Figure 7. Left: raw Pearson. Middle: Spearman. Right: Pearson after removing each location’s hour-of-week mean. The third panel is the conservative view: shared sub-weekly structure is much weaker than the raw matrix suggests.*
### 7.2 Cross-sensor dispersion and ratio stability
- **Raw coefficient of variation across the eight locations at the same 15-minute slot:** median **0.66**, p90 **1.05**, p95 **1.18**. This is dominated by the different mean levels of the locations.
- **Scale-free share CV** (each location’s share of the eight-location total at that slot): median **0.216**, p95 **0.463**. Even after removing level differences, the cross-sectional composition varies substantially from slot to slot.
- **Daily share stability:** over the 69 days on which all eight locations have ≥80% coverage, the daily share CV per location ranges from **0.159** (`neckarufer`) to **0.508** (`soleo`); the median is **0.216**.
- **Pairwise ratio stability:** the median SD of the daily log ratio between two locations is **0.399** (a 1-SD multiplicative change of **e^0.399 = 1.49×**); the p90 is **0.711** (**2.04×**). A fixed conversion factor between two sensors is not supported. Ratios drift with season, events, and location-specific behavior.
### 7.3 Hourly and weekday profiles
| location | peak hour (local) | weekend/weekday ratio |
|---|---:|---:|
| density-01 | 19 | 1.18 |
| goetzentuem-01 | 19 | 1.12 |
| hagenbuchersee-01 | 20 | 1.24 |
| hagenbuchersee-02 | 21 | 1.19 |
| marahaus-01 | 19 | 1.17 |
| neckarufer-01 | 20 | 1.14 |
| soleo-01 | 19 | 0.98 |
| stadtgalerie-01 | 16 | 0.81 |
- Seven locations peak in the **late afternoon/evening (16:00–21:00)**; `stadtgalerie` peaks at **16:00** and drops sharply after 20:00, consistent with a shopping-center closing time.
- Weekend/weekday ratios are **1.12–1.24** for the outdoor/leisure locations, **0.98** for `soleo`, and **0.81** for `stadtgalerie`. The mall’s Sunday value is only ~0.3 of its mean, i.e. near-closure, while the outdoor locations peak on Saturday (~1.4 of mean). This heterogeneity is a semantic difference, not a sensor error.
- Mean pairwise correlation of the **normalized hourly shapes** is **0.79**, so the broad day/evening rhythm is shared even where the levels and weekend behavior differ.
![Hourly and weekday profiles](figures/fig_hourly_profile.png)
*Figure 8. Normalized hourly (left) and weekday (right) profiles. The mall (`stadtgalerie`, grey) is the clear outlier: midday peak and Sunday near-closure.*
---
## 8. Battery telemetry
Battery rows are emitted with density rows, not independently. Every one of the 96,232 density rows has a battery value within 60 s (**100%**; median offset 0.033 s, p99 5.82 s). But during a device outage **both** series disappear, so battery missingness is structurally identical to density missingness. In the panel, the eight active devices contribute **82,170** paired rows out of **109,824** expected device-slots (**74.8%**); the other 25.2% have neither density nor battery. “0% battery missing conditional on a density row” is therefore not evidence of reliable battery telemetry.
| device | first → last (%) | median (%) | share `<20%` | consecutive-equal share | longest stale run (days) | corr(battery, daily readings) |
|---|---:|---:|---:|---:|---:|---:|
| goetzentuem-01 | 99 → 0 | 4 | 100% | 89% | 11.5 | 0.02 |
| hagenbuchersee-02 | 92 → 27 | 8 | 53% | 85% | 19.1 | −0.16 |
| marahaus-01 | 52 → 87 | 7 | 85% | 81% | 10.8 | −0.06 |
| soleo-01 | 94 → 26 | 8 | 51% | 83% | 10.0 | −0.04 |
| neckarufer-01 | 100 → 11 | 8 | 89% | 84% | 11.2 | 0.08 |
| hagenbuchersee-01 | 92 → 43 | 57 | 16% | 74% | 3.6 | −0.02 |
| stadtgalerie-01 | 99 → 75 | 4 | 93% | 88% | 11.6 | 0.07 |
| density-01 (new) | 89 → 11 | 46 | 9% | 70% | 3.7 | −0.41 |
| density-01 (legacy) | 44.9 → 69.4 | 31.9 | 7% | 52% | 0.5 | — |
- Overall battery median is **17%**, with **51.3%** of readings below 20% and **1.6%** at 0%. Values are bounded 0–100.
- The current devices are **coarse and stale**: 70–89% of consecutive battery readings are identical, and the longest constant run is **3.6–19.1 days**. Values can also **increase** (e.g. 4 → 36, 6 → 74), indicating battery replacement, charging, or gauge reset rather than monotonic discharge.
- **Within a device, battery does not predict availability.** Daily correlation between battery level and daily reading count ranges from **−0.41 to 0.08**; several devices keep reporting for months at 1–4%.
- At the reading level, corr(battery, density) ranges from **−0.14 to 0.42** across the eight current devices (0.53 for the Hannover `maker-fair` pilot), again with no consistent sign.
- **Across devices, August coverage is strongly confounded with battery.** The six degraded locations had August median battery **4–10%**, the two healthy locations **41%** and **70%**; the cross-device correlation is **r = 0.84** (n = 8; Figure 9, right). But four of the six still report in October at 1–3% while two show battery resets to ~32–36%, so a simple threshold does not explain the outage. Battery, device batch, gateway, and installation are collinear; do not read the association as causal.
- The legacy `density-01` battery values are fractional and increase over the deployment (44.9 → 69.4), another firmware difference. Its battery series is not comparable to the integer-percent current devices.
- **Interpretation:** battery telemetry here is a coarse gauge, not a reliable power-state or duty-cycle measurement. It cannot be used to explain missingness cleanly, and it cannot be used to infer scanning duty cycle or power-saving behavior.
![Battery levels](figures/fig_battery.png)
*Figure 9. Left: daily median battery per device. Middle: distribution of paired battery readings. Right: August slot coverage vs August median battery; the six low-battery devices are also the six degraded ones (r = 0.84). The comb pattern in the middle panel is integer-percent quantization; the long low plateaus are stale values, not necessarily a linear discharge.*
---
## 9. Duplicates, spikes, flatlines, and dropouts
### 9.1 Duplicates
- **Exact duplicate rows: 0.** Duplicate `(time, sensor_id, type)`: 0. Duplicate `(time, sensor_id)`: 0. The export is unique on its natural key.
- **Near-duplicates:** 113 inter-reading intervals **<60 s**, of which **106** are the legacy `density-01`. The median spacing is 7.0 s (range 5.1–53.3 s), and most pairs carry identical values. This looks like repeated publication/retry, especially in the legacy firmware. Deduplicate by retaining one reading per 60 s (or averaging) before any cadence or correlation analysis.
- **Duplicate sensor name:** `density-01` appears twice (two IDs). This is the hardware replacement, not a duplicated device; see Section 3.2.
### 9.2 Spikes
We flag a reading when its value is at least **5×** the centered 9-point rolling median (plus 0.7) and at least 10. This is a screen for single-slot outliers, not a confirmed error label.
| device | candidate spikes | max ratio | examples |
|---|---:|---:|---|
| goetzentuem-01 | 3 | 10.0 | 2026-08-01 08:09; 2026-10-07 20:16 |
| hagenbuchersee-02 | 7 | 22.0 | 2026-08-22 17:23 (**46.2** between 1.4 and 0.7); 2026-09-19 21:29 |
| density-01 (legacy) | 4 | 10.0 | 2025-11-22 02:43 (**24.0** between 7.0 and 1.0) |
| marahaus-01 | 1 | 5.1 | 2026-10-17 10:01 |
| soleo-01 | 7 | 9.3 | 2026-06-29 23:40; 2026-07-19 10:26 |
| neckarufer-01 | 6 | 11.0 | 2026-08-14 16:57; 2026-09-08 17:07 (**77.0** between 11.2 and 69.3) |
| hagenbuchersee-01 | 4 | 8.0 | 2026-08-14 19:19 (**292.6** between 52.5 and 65.8); 2026-10-17 10:04 |
| stadtgalerie-01 | 6 | 10.5 | 2026-08-11 10:43; 2026-08-13 11:16 |
| density-01 (new) | 1 | 16.0 | 2026-10-09 02:19 (**11.2** between 0.0 and 0.0) |
There are **39 candidate spikes** across the nine core devices, i.e. **0.041%** of the 96,146 core readings. Some are clearly isolated single-slot outliers (46.2 between 1.4 and 0.7; 11.2 at 02:19 between zeros; 292.6 between 52.5 and 65.8). Others sit inside genuine high-traffic periods: 18–20 June shows a sustained 3–6× evening peak at several locations, and 255.5/262.5 values are surrounded by 200–250, so they are more likely real events than glitches. **Without ground truth we cannot separate real surges from double counts.** For the paper, report spike counts and run a winsorized/robust sensitivity analysis rather than silently deleting them.
### 9.3 Flatlines
- The longest run of identical consecutive readings is **20 readings (5.0 h)**, mostly zeros or 0.7 at night. The next longest are 17 (4.25 h, 1.4) and 16 (4.0 h, 0.0). There is no evidence of a multi-day stuck nonzero value.
- Adjacent-equal share is **7.1–29.3%**, driven by zero inflation and the 0.7 grid (at low counts, 0.7 and 1.4 repeat naturally). The legacy device has the highest share (29.3%) because of its 29.8% zeros.
- **Conclusion:** flatlines are not a major failure mode in this export. The apparent “stuck” runs are short and occur at night/low traffic; they should not be mistaken for sensor freezes.
### 9.4 Dropouts
- Within the panel, **seven of the eight current devices** have **4–9 gaps >1 day** each, with maximum gaps of **25–152 h**; `density-01` (new) has **none** and a maximum gap of 4.6 h. The long gaps are dominated by the August–September systemic outage and a few device-specific multi-day losses (e.g. `marahaus` 152.3 h, `stadtgalerie` 141.8 h).
- The legacy `density-01` has **417 gaps >1 h** and 3 gaps >1 day over its 147-day life, plus 29.8% zeros; it is the least reliable series in the export.
- `density-01` (new) is the most reliable: 13 gaps >1 h, **0 gaps >1 day**, max 4.6 h, and 95.2% of its cadence ceiling.
---
## 10. What these aggregates cannot measure
This section is deliberately explicit because the paper is about **reliability and privacy** of BLE PAX counters. The export is an aggregate product; several quantities of interest are not identifiable from it.
1. **MAC randomization.** There are no MAC addresses, address-type flags (public vs random), RSSI, or per-detection records. The 0.7 value grid may encode a vendor correction for randomization, but the export provides no metadata to confirm this. Any statement about randomization rate from these aggregates would be speculation.
2. **Penetration rate.** There is no independent ground-truth count (manual counts, turnstiles, sales, vehicle counts) and no denominator. The values cannot be converted to “share of visitors carrying a BLE device” or “share of devices detected.”
3. **Unique visitors, dwell time, repeat visits.** The counters report a 15-minute aggregate per location. They do not expose device trajectories, session lengths, or cross-location matches. Cross-sensor correlation is not deduplication; it can be driven by shared city rhythm, weather, or events.
4. **False positives/negatives and detection radius.** There is no labeled validation set. The 0.7 grid, zero inflation, and spikes are consistent with vendor processing, but the false-positive rate, missed-detection rate, and effective radius are unknown.
5. **Missingness mechanism.** The export has no status/error codes. The August outage is visible as missing rows, but its cause (backend, gateway, network, power, firmware) cannot be determined from the CSV. Battery telemetry is not independent evidence because it disappears with density and is confounded with device identity in August.
6. **Unit and scale.** Values are not raw counts; the 0.7 factor is undocumented. The legacy/new `density-01` break proves that the unit can change across hardware/firmware. Summing or averaging across generations, or comparing absolute levels across sites, is not defensible without vendor documentation or an overlap calibration.
7. **Causal effects.** The hourly/weekday profiles and correlations are observational and confounded by season, events, holidays, weather, and the August outage. They describe co-movement, not causal impact.
8. **Privacy risk.** The data are location-level aggregates, but low counts (0–5) at specific times and places can be sensitive. We did not attempt a k-anonymity or differencing attack; the note only assesses data quality, not disclosure risk.
**Bottom line for the paper:** the 0.7 grid and zero/low-count shares are **proxies for vendor processing**, not measurements of MAC randomization or penetration. Use them to motivate uncertainty and to rule out naïve “counts = devices” interpretations; do not present them as direct estimates of privacy-protection effectiveness.
---
## 11. Implications and recommendations
1. **Do not treat values as raw counts.** Describe them as vendor-scaled 15-minute indices. Report the 0.7 grid and the legacy 1.0 grid in the methods; do not sum across device generations.
2. **Handle `density-01` as a structural break.** Exclude the legacy series from the panel, or model it with a separate scale/level. There is no overlap, so a conversion factor cannot be estimated; any pooling is an assumption, not a calibration.
3. **Use 30–60-minute bins for complete-case cross-sensor analysis.** At 15 minutes, 72% of slots lack at least one location; at 30/60 minutes, all-eight coverage rises to 63%/70% with little change in correlation. Report the binning choice and the all-eight coverage.
4. **Model missingness explicitly.** Do not listwise delete 15-minute slots without reporting how many are lost. Use hierarchical/state-space models, multiple imputation, or inverse-coverage weighting; run sensitivity analyses with and without the August–September outage. Report monthly coverage alongside every estimate.
5. **Use robust correlation summaries.** Report Spearman/log1p and daily/detrended correlations alongside raw Pearson. The raw 15-minute matrix overstates shared structure because of the common daily/weekly profile. Use block bootstrap or daily-level aggregation for uncertainty; nominal 15-minute *N* is not an effective sample size.
6. **Screen spikes and near-duplicates.** Deduplicate intervals <60 s (106 of 113 are legacy), flag isolated spikes (39 candidates, 0.041% of core readings), and run winsorized sensitivity analyses. Do not silently remove the 292.6 maximum or the 46.2 outlier; report both as candidate errors/real events.
7. **Treat battery as a confounded covariate, not a cause.** It is coarse, stale, and structurally missing with density. Within a device it does not predict daily availability; across devices it is collinear with the August outage (r = 0.84) and with device batch/gateway. If it is used at all, model it with measurement error and do not interpret the association causally.
8. **For privacy claims, collect the right data.** To measure MAC randomization or penetration, the study needs raw scan records with hashed MACs (salted), address-type flags, RSSI, timestamps, and independent ground truth. The aggregate export cannot answer those questions.
9. **Pre-specify exclusions.** The three episodic devices (especially the Hannover `maker-fair` deployment) and the legacy `density-01` should be excluded or analyzed separately. The June 18–20 high-traffic period and the August outage should be pre-specified as events/robustness checks, not discovered post hoc.
10. **Report the limits with the estimates.** A reliability claim based on this export should carry the coverage, quantization, missingness, and unit caveats in the same table or paragraph as the estimate.
---
## 12. Outputs and reproducibility
**Script:** `paper/research/scripts/01_data_quality.py`
**Command:** `uv run --with pandas --with matplotlib python paper/research/scripts/01_data_quality.py`
**Raw data:** `data/raw/schwarz-pax.csv` (read-only; never modified)
**Tables** (`paper/research/outputs/`):
- `schema.csv`, `sensor_inventory.csv`
- `coverage_metrics.csv`, `monthly_coverage.csv`, `monthly_diagnostics.csv`, `coverage_bias.csv`
- `cadence_by_sensor.csv`, `cadence_bins.csv`, `cadence_jitter.csv`
- `zero_lowcount.csv`, `quantization_by_device.csv`, `density01_scale_comparison.csv`
- `correlation_pearson.csv`, `correlation_spearman.csv`, `correlation_log1p.csv`, `correlation_detrended.csv`, `correlation_daily.csv`, `correlation_pair_n.csv`
- `aggregation_sensitivity.csv`, `locations_per_slot.csv`, `panel_15min_matrix.csv`, `hourly_profile.csv`, `weekday_profile.csv`, `cv_by_hour.csv`, `missingness_by_hour.csv`
- `battery_summary.csv`, `battery_vs_coverage.csv`, `august_battery_coverage.csv`
- `spikes.csv`, `top_spikes.csv`, `flatlines.csv`, `large_gaps.csv`, `near_duplicate_intervals.csv`
- `integrity_checks.csv`, `summary.json`
**Figures** (`paper/research/figures/`):
- `fig_time_series.png` — per-location daily series
- `fig_correlation_heatmap.png` — Pearson, Spearman, hour-of-week-removed
- `fig_missingness_heatmap.png` — daily slot coverage
- `fig_distribution.png` — histogram, CCDF, per-device boxplots
- `fig_cadence.png` — interval distribution and per-device cadence
- `fig_hourly_profile.png` — hourly and weekday profiles
- `fig_battery.png` — battery time series and distribution
- `fig_quantization.png` — value-grid membership and density-01 ladders
- `fig_density01_swap.png` — legacy/new `density-01` daily means and 35.2-day gap
All figures are generated from the raw CSV by the script; no manual edits or external data were used.
---
## Appendix A. Largest gaps (>24 h)
| device | gap start | gap end | hours |
|---|---|---|---:|
| hn-bollwerksturm-01 | 2026-06-16 18:20 | 2026-08-13 16:00 | 1,389.7 |
| hn-marahaus-01 | 2026-08-19 12:25 | 2026-08-25 20:45 | 152.3 |
| hn-stadtgalerie-01 | 2026-08-20 04:46 | 2026-08-26 02:33 | 141.8 |
| hn-stadtgalerie-01 | 2026-06-12 12:11 | 2026-06-17 16:31 | 124.3 |
| hn-neckarufer-01 | 2026-06-12 12:21 | 2026-06-17 16:33 | 124.2 |
| hn-goetzentuem-01 | 2026-06-12 12:27 | 2026-06-17 16:36 | 124.1 |
| hn-marahaus-01 | 2026-06-12 12:36 | 2026-06-17 16:37 | 124.0 |
| hn-marahaus-01 | 2026-09-04 10:55 | 2026-09-09 07:50 | 116.9 |
| hn-soleo-01 | 2026-09-18 00:13 | 2026-09-20 10:18 | 58.1 |
| density-01 (legacy) | 2025-11-28 17:20 | 2025-12-01 03:59 | 58.7 |
The 12–17 June gaps are the initial deployment/test-message gaps, not operational outages; the panel starts after them. The August–September gaps are the systemic outage. The full list is in `outputs/large_gaps.csv`.
## Appendix B. Exact cadence sequences
**Legacy `863f75b0`** (first readings):
```
2025-10-16 21:17:59.316
2025-10-16 21:35:00.506 +1021.190 s (17.020 min)
2025-10-16 21:50:42.960 + 942.454 s (15.708 min)
2025-10-16 22:05:54.445 + 911.485 s (15.191 min)
2025-10-16 22:40:43.640 +2089.195 s (34.820 min; missed ticks)
2025-10-16 22:55:56.406 + 912.766 s (15.213 min)
```
**New `f8c19730`** (first readings):
```
2026-04-16 23:25:31.574
2026-04-16 23:41:38.142 +966.568 s (16.109 min)
2026-04-16 23:57:47.513 +969.371 s (16.156 min)
2026-04-17 00:13:56.263 +968.751 s (16.146 min)
2026-04-17 00:30:05.097 +968.833 s (16.147 min)
2026-04-17 00:46:12.591 +967.495 s (16.125 min)
```
The legacy series starts with startup transients and missed ticks, then settles near 911 s. The new series is stable near 968.5 s (16.14 min) with only ±2.4 s jitter. Neither is a 900 s clock.
@@ -0,0 +1,684 @@
# Privacy-Enhancing Technologies for Mobility Data
### A technical assessment for a privacy-first alternative to raw Bluetooth MAC sniffing
**Status:** research memo for the PulseCast technical report. **Date:** 2026-11-14.
**Scope.** This memo assesses privacy-enhancing technologies (PETs) for a system that estimates urban footfall from Bluetooth Low Energy (BLE) and Wi-Fi observations, publishes heatmaps, and forecasts short-term demand. It assumes:
- the pedestrian's phone is **not** under our control and does not run our code;
- the sensing layer observes ephemeral radio addresses at fixed locations;
- the product needs per-location counts at 15 min to 1 h resolution, historical aggregates for forecasting, and a public or municipal dashboard;
- raw trajectories are not a product requirement.
The current PulseCast pilot uses PAX counters and does not sniff MAC addresses. The PET stack below is for a future BLE/Wi-Fi sensing layer that would otherwise collect raw MAC addresses. The memo separates **formal guarantees** (differential privacy, secure computation), **syntactic privacy models** (k-anonymity and its extensions), and **operational controls** (minimization, retention, edge aggregation). It does not treat them as interchangeable.
---
## 1. PET overview
| PET | What it protects | Guarantee | Trust assumption | Main cost | Fit for footfall |
|---|---|---|---|---|---|
| **Central differential privacy (DP)** | The contribution of one privacy unit to a published aggregate | `(ε, δ)`-DP: a formal bound on one unit's influence, independent of the adversary's auxiliary information | A trusted curator holds the raw data | Noise; budget accounting; governance | **Strong.** Best utility for counts; needs a trusted aggregator |
| **Local DP** | Each user's report against the curator and the world | `ε`-DP per report, no trusted curator | None for the data, but devices must run the mechanism | Very high noise for sparse, high-dimensional data | **Weak** for 15-min counts; usable for large-scale telemetry |
| **Shuffle DP** | Individual reports against the server, via anonymity | Amplified DP, between local and central | A trusted shuffler or a mixnet | Latency; shuffler operation; validation | **Medium.** Good at scale; overkill for 8–50 sensors |
| **k-anonymity** | Identity disclosure under a quasi-identifier model | Syntactic: each QI combination appears at least `k` times | Adversary's auxiliary knowledge is limited to the QIs | Generalization/suppression destroys local signal | **Weak alone**; useful as a secondary check |
| **l-diversity / t-closeness** | Attribute disclosure within an equivalence class | Syntactic; stronger than k-anonymity for specific attacks | Known sensitive attribute and distance metric | Hard to apply to high-dimensional mobility | **Weak alone**; research/interim use |
| **Secure aggregation** | Individual contributions to a server | Cryptographic: server learns only the sum, below a collusion threshold | Threshold of honest participants; correct protocol | Communication rounds; dropout handling | **Good** for multi-operator aggregation |
| **Secure multi-party computation (MPC)** | Inputs to a joint computation | Cryptographic: correctness and input privacy under a corruption model | Honest majority or preprocessing assumptions | Network rounds; malicious security overhead | **Good** for cross-city queries; heavy for a pilot |
| **Homomorphic encryption (HE)** | Data in computation | Cryptographic: computation on ciphertexts | Key holder; threshold decryption | Ciphertext expansion; limited depth; key management | **Good** for encrypted sums; not for full heatmaps |
| **Trusted execution environment (TEE)** | Data in use against a privileged operator | Hardware-based: confidentiality/integrity, conditional on attestation | Hardware vendor and attestation chain; no side channels | Attestation infrastructure; side-channel risk | **Good** for production aggregation; not a DP substitute |
| **Federated learning / analytics** | Raw data locality | No privacy guarantee by itself; DP and SecAgg must be added | Device-side code; participation | Communication; device availability; poisoning | **Not applicable** to passive BLE sniffing |
| **DP synthetic data** | Individual records in a released dataset | `(ε, δ)`-DP if the generator is DP | Correct DP training and evaluation | Utility loss; high-dimensional data is hard | **Good** for public release, not live counts |
| **Non-DP synthetic data** | Nothing formally | None; empirical privacy only | Generator and evaluation | False sense of anonymity | **Do not claim anonymity** |
| **Edge aggregation, ephemeral IDs, minimization** | Raw data collection and retention | Operational, not formal | Sensor firmware/hardware; operator | Firmware work; audit; calibration | **Essential first layer** |
**Guarantee hierarchy.** DP is the only technique in this table that gives a formal, composable, adversary-agnostic bound on the influence of one privacy unit's data on a published output. Secure computation protects data *during* a computation but says nothing about the output. k-anonymity and its extensions are syntactic properties that hold only under a specified quasi-identifier and background-knowledge model. Hashing, MAC randomization, and non-DP synthetic data are **not** privacy guarantees. A production system should compose layers: minimization and edge aggregation to avoid collecting raw data, DP to bound the output, and secure computation or a TEE to reduce trust in the aggregator. No single layer is sufficient.
---
## 2. Threat models
### 2.1 Privacy unit and neighboring relation
A DP guarantee is meaningless without a **privacy unit** and a **neighboring relation**. For footfall, the practical choices are:
- **Event-level:** one sensor observation of one device in one time bin. This is the weakest unit; it protects a single ping, not a person.
- **Device-day:** all observations of one device on one calendar day. This is the natural unit for a daily heatmap and the one this memo recommends.
- **Person-day:** all observations of one person on one day. This is the strongest unit, but it is not measurable from MAC addresses: one person may carry two devices, and one device may be shared. A person-day guarantee cannot be honestly claimed from device data alone.
The choice matters. A device that is observed by six sensors in a day contributes six sensor-bin records. If the release is per sensor, the same device appears in six releases. Treating those releases as independent would undercount the privacy loss. The correct approach is to **bound each device's total contribution** across sensors and bins (for example, at most four sensor-bin pairs per day, analogous to the contribution bound used by Google's COVID-19 Community Mobility Reports) and to set the sensitivity of the release to that bound. Contribution bounding is not an engineering afterthought; it is the step that makes the sensitivity calculation true.
A second distinction is **add/remove** versus **replace** neighboring datasets. Add/remove (one device added or removed) gives sensitivity 1 for a count; replace (one device replaced by another) gives sensitivity 2. Most deployed systems use add/remove and should say so.
### 2.2 Risk categories and attacks
The Article 29 Working Party's anonymisation opinion (2014) identifies three risk categories that a release must resist: **singling out**, **linkability**, and **inference**. Reconstruction is a fourth, emphasized in the DP literature (Dinur and Nissim, 2003). Membership inference (Shokri et al., 2017) and differencing attacks are related risks that matter for an interactive dashboard.
| Risk | What the adversary does | Footfall example | What actually helps | What does not |
|---|---|---|---|---|
| **Singling out** | Isolates one individual's record in the release | A cell or time bin that contains one device | DP; minimum aggregation; suppression (partially) | Removing names; pseudonyms |
| **Linkability** | Links two records about the same person across releases or datasets | Same device in Monday's and Tuesday's heatmaps; same device across two sensors | Rotating identifiers; aggregation; contribution bounds; DP composition across releases | Stable pseudonyms; hashed MACs |
| **Inference** | Deduces a sensitive attribute from the release | Home/work location, religion, health, protest participation | DP bounds the influence of one person; coarsening; purpose limitation | k-anonymity alone; "anonymized" labels |
| **Reconstruction** | Recovers the input data from many aggregate answers | Arbitrary filters on an API, differenced to isolate a person | Query restrictions; budget ledger; DP composition | Noise without accounting; unlimited queries |
| **Membership inference** | Decides whether a specific person is in the data | "Was this device in the dataset on Saturday?" | DP bounds the advantage; aggregation | Pseudonymization; synthetic data without DP |
| **Differencing** | Subtracts two releases that differ by one person | "All visitors" minus "all visitors except sensor S" | Consistent noise; budget accounting; no overlapping releases | Independent noise that can be averaged out |
| **Sensor compromise** | Reads raw MACs at the edge | Malicious or buggy sensor firmware | Secure boot; no raw storage; attestation; audit | DP at the aggregator; hashing after collection |
### 2.3 Re-identification case studies
**Netflix Prize (2008).** Narayanan and Shmatikov linked the "anonymized" Netflix Prize ratings to public IMDb ratings and de-anonymized a large fraction of subscribers, with up to 99% of records recovered under their attack for users with enough ratings. The dataset was high-dimensional and sparse, which made each user's rating pattern nearly unique. Removing names did not make the data anonymous. The lesson for mobility is direct: a trajectory is a high-dimensional, sparse behavioral signature, and auxiliary data (social media, check-ins, transit records) is abundant.
**AOL search logs (2006).** AOL released search logs with pseudonymous user IDs. The New York Times re-identified user 4417749 from the content of the queries. The lesson: pseudonyms plus rich behavior are not anonymization; the sensitivity is in the behavior, not the identifier.
**NYC taxi data (2014).** The New York City Taxi and Limousine Commission released 173 million trip records with medallion and driver-license numbers pseudonymized. Anthony Tockar showed that a medallion number, a timestamp, and a public photo of a celebrity entering a cab were enough to identify trips, including pickup and drop-off locations and tips. Pseudonymization was not enough; the linkage came from public information, not from reversing the hash. The lesson: pseudonymous IDs plus spatio-temporal data plus public information equals re-identification; a single auxiliary point can be enough.
**Mobility uniqueness (2013, 2015).** de Montjoye et al. (2013, 2015) showed that four spatio-temporal points uniquely identify 95% of 1.5 million individuals in a mobile-phone dataset at hourly and cell-tower resolution, and that four points uniquely re-identify 90% of individuals in credit-card metadata. This is the strongest empirical argument against k-anonymity as a primary guarantee for mobility: the data is inherently high-dimensional, and generalization to achieve k-anonymity destroys the local, peaked signal that footfall analysis needs.
**Google COVID-19 Community Mobility Reports (2020).** Google published a production mobility release using DP. The published description reports Laplace noise at ε = 0.11 per count at the coarser geographic levels (0.22 at the finest), a contribution bound of four category-location pairs per user per day per geographic level, a maximum daily contribution of ε = 1.76, δ = 0, no regions smaller than 3 km², and a post-noise threshold of 100 contributing users. This is a useful existence proof: a city-scale mobility release can use DP, but at coarse spatial and temporal resolution and with an explicit utility cost. It is not a template for a 15-minute, 100-metre public heatmap.
**Apple/Google Exposure Notification (2020).** The Exposure Notification framework uses rolling proximity identifiers that change about every 15 minutes, on-device matching, and no central storage of raw identifiers. It is a strong example of an architecture that avoids collecting the sensitive data in the first place. It does not transfer to passive BLE sniffing: in Exposure Notification, the phone runs the protocol; in our system, the pedestrian's phone does not.
**Wi-Fi and BLE tracking.** MAC address randomization is not sufficient. Vanhoef et al. showed that probe-request content, information elements, timing, and sequence numbers can fingerprint devices across MAC rotations, and that some devices leak their real MAC address. Franklin et al. showed that 802.11 driver fingerprints can identify devices passively. Martin et al. showed that Apple's BLE Continuity protocol leaks identifying fields and that predictable sequence numbers allow tracking across address rotations; in some macOS versions, a bug broadcast the public MAC address entirely. The design implication is not "randomize better"; it is "do not collect raw MACs."
### 2.4 What DP does and does not protect
DP protects the **influence of one privacy unit on the output**. The bound holds regardless of what auxiliary information the adversary has about the other records; that is what "adversary-agnostic" means. It does not protect:
- population-level facts that are true regardless of any one person;
- inferences that an adversary can make from external data without using the release;
- a sensor or edge node that retains raw MACs before aggregation;
- the utility of the release;
- the legal question of whether the output is "anonymous" under GDPR.
A DP release can still be harmful if the privacy unit is too coarse, the contribution bound is wrong, the budget is not enforced, or the released statistic is itself sensitive at the population level (for example, a cell that reveals a protest or a clinic visit even with no individual identified). DP is a necessary component of a privacy argument, not the whole argument.
---
## 3. Differential privacy
### 3.1 Definition and semantics
A randomized mechanism `M` is `(ε, δ)`-differentially private if for all neighboring datasets `D` and `D'` that differ in one privacy unit, and for all output sets `S`:
```text
Pr[M(D) ∈ S] ≤ e^ε · Pr[M(D') ∈ S] + δ.
```
`ε = 0, δ = 0` is exact privacy; `δ = 0` is **pure DP**; `δ > 0` is **approximate DP**. The guarantee is with respect to the neighboring relation and the privacy unit, not with respect to a particular adversary or auxiliary dataset.
Interpretation:
- `ε` bounds the multiplicative change in the probability of any output when one privacy unit's data is added or removed. `ε = 1` permits a change of factor `e ≈ 2.72`; `ε = 0.1` permits about 11%; `ε = 10` permits a factor of 22,026.
- Under pure DP, the optimal membership-inference advantage is at most `(e^ε − 1)/(e^ε + 1)`. This is about 46% at `ε = 1`, 5% at `ε = 0.1`, and 0.5% at `ε = 0.01`. Approximate DP relaxes this bound by a `δ`-dependent term. These are worst-case bounds, not predictions of a specific attack, but they show why "ε = 1" is not a mild guarantee.
- `δ` is the probability of a catastrophic failure of the bound. It should be much smaller than `1/n`. A common conservative convention is `δ ≤ 1/n²`; for `n = 10,000` devices, that is `1e-8`. If `δ = 1e-6`, the expected number of affected individuals under the failure event is small, but the choice should be justified, not defaulted.
There is no universal "safe" ε. The common guidance (Dwork and Roth, 2014; Dwork, Kohli, and Mulligan, 2019) that ε around 1 is a reasonable starting point and that ε above 10 is hard to justify is a heuristic, not a theorem. NIST SP 800-226 explicitly says that ε alone is not a sufficient descriptor of a DP guarantee; the evaluation must include the privacy unit, the neighboring relation, the mechanism, the composition, the data, and the threat model. A public release should publish all of these, not just a number.
### 3.2 Mechanisms for counts
For a function `f` with L1 sensitivity `Δf`, the **Laplace mechanism** releases `f(D) + Lap(Δf/ε)`. For a count with add/remove neighboring data, `Δf = 1`, so the noise scale is `1/ε`. The noise standard deviation is `√2/ε`, and the 95% confidence interval is approximately `±3/ε`.
For a function with L2 sensitivity `Δ₂f`, the **Gaussian mechanism** adds `N(0, σ²)` with
```text
σ = Δ₂f · sqrt(2 ln(1.25/δ)) / ε (for 0 < ε < 1).
```
Using the classical calibration at the `ε = 1` boundary, `δ = 1e-6` gives `σ ≈ 5.3 · Δ₂f`, and the 95% interval is about `±10.4 · Δ₂f`; the exact guarantee should be computed with a modern accountant rather than read off this approximation. The Gaussian mechanism is worse than Laplace for a single count but composes better over many queries. The formula above is conservative; analytical calibration (Balle and Wang, 2018) and Rényi/zCDP accounting can reduce the noise. For integer counts, a discrete Gaussian avoids floating-point rounding leaks; a vetted library should be used rather than a hand-rolled Laplace sampler.
**The histogram mechanism is the key utility fact for heatmaps.** If each device contributes to at most one cell of a fixed spatial grid in a release, the L1 sensitivity of the entire histogram vector is 1, not the number of cells. Adding `Lap(1/ε)` to every cell satisfies `ε`-DP for the whole histogram, and the number of cells does not multiply the budget. This is why a 1,000-cell daily heatmap is feasible while 1,000 separate per-cell queries at `ε/1000` each are not. The caveats are:
- the grid must be fixed in advance; a data-dependent grid requires a different analysis;
- each device must be counted at most once (deduplicate within the release window);
- if a device can contribute to `k` cells, sensitivity is `k`, and the noise scale is `k/ε`;
- if the release is a time series of `T` bins and a device can appear in every bin, sensitivity is `T` unless contributions are bounded. This is the central difficulty for 15-minute heatmaps.
For range queries (rectangles, arbitrary map filters), the **matrix mechanism** and hierarchical decompositions (quadtrees, PrivTree) can answer a workload with less noise than independent per-cell perturbation (Li et al., 2015; Cormode et al., 2012; Zhang et al., 2016). Constrained inference (for example, non-negativity and consistency across a hierarchy) is post-processing and does not consume budget, but it can amplify bias as well as reduce variance (Hay et al., 2010).
### 3.3 Local, central, and shuffle models
| Model | Who must be trusted | Utility for counts | Typical examples | Fit |
|---|---|---|---|---|
| **Central DP** | A curator that sees raw data | Best; error scales like `1/(εn)` for a bounded count | Census TopDown (Abowd et al., 2022), Google Mobility Reports, OpenDP | Strong for a city-owned aggregator |
| **Local DP** | No curator; each device randomizes | Worst; error scales like `1/(ε√n)` for a bounded mean, and worse in high dimension | RAPPOR, Apple's count-mean sketch | Weak for sparse 15-min footfall |
| **Shuffle DP** | A shuffler or mixnet; not the analyst | Between local and central; amplification by anonymity | Prochlo, Google's shuffle deployments, Apple's distributed aggregation | Good at large scale; complex at city scale |
The local-versus-central gap is not a constant factor. For bounded means, and for the relative error of counts, local DP's error decreases as `1/√n` while central DP's decreases as `1/n`; local DP needs roughly quadratically more users for the same accuracy (Duchi, Jordan, and Wainwright, 2013). In high-dimensional settings the gap is worse. A 15-minute footfall bin with 14 devices per sensor is exactly the regime where local DP fails: randomized response on a count with that few contributors produces noise larger than the signal. Local DP is appropriate when the number of contributors per release is large (thousands), or when the query is low-dimensional and the privacy requirement is modest.
The shuffle model recovers some of the central utility by hiding individual messages among many. If each user sends a locally randomized message and a trusted shuffler permutes the messages and strips identities, the effective privacy loss can be amplified from the local `ε` toward the central regime; in the small-`ε`, large-`n` regime the shuffled guarantee is on the order of `ε/√n` for suitable mechanisms, and frequency estimation approaches the central rate (Erlingsson et al., 2019; Feldman, McMillan, and Talwar, 2021). The guarantee depends on the mechanism, the number of messages, and the shuffler's honesty. Single-message shuffle has strong lower bounds and cannot match central DP for every task; multi-message shuffle is more powerful but requires more communication (Ghazi et al., 2021). A cryptographic mixnet can replace the trusted shuffler, at the cost of latency and operational complexity. For a small city with 8–50 sensors, the shuffle model adds machinery without solving the binding problem, which is the sparse count, not the trust in the aggregator.
### 3.4 Composition and the privacy budget
DP composes. If a mechanism is applied `k` times adaptively to the same data, the basic bound is `(kε, kδ)`. Advanced composition gives approximately
```text
( ε · sqrt(2k ln(1/δ')) + kε(e^ε − 1), kδ + δ' ).
```
The optimal composition theorem (Kairouz, Oh, and Viswanath, 2015) characterizes the exact degradation. Rényi DP (Mironov, 2017), zero-concentrated DP (Bun and Steinke, 2016), and Gaussian DP (Dong, Roth, and Su, 2022) give tighter accounting for many queries and are the right tools for a production system. The choice of accounting method must be consistent across the system; mixing bounds from different frameworks without care can understate the total.
Two further properties matter for a deployed system. **Post-processing immunity:** any function of a DP output is DP with the same parameters, so forecasting, smoothing, and visualization that use only DP aggregates do not consume additional budget. **Group privacy:** protecting a group of `k` devices (or `k` events from one device) costs roughly `kε` under pure DP; the linear degradation is one reason contribution bounding and a device-day unit matter, and one reason repeated releases are dangerous. Neither property removes the need for a budget ledger.
Two structural rules matter more than the exact bound:
- **Sequential composition** applies when the same data is queried repeatedly. The budget adds.
- **Parallel composition** applies when the queries touch disjoint subsets of the data. The budget is the maximum, not the sum.
For a heatmap, spatial cells in the same time bin are disjoint if each device is counted once, so a single histogram release is one query with sensitivity 1. Time bins are not disjoint for a device that appears in several bins; a device that visits a location at 09:00 and 17:00 contributes to two bins. Unless the release deduplicates per device per window, the time series composes sequentially.
**A privacy budget is a ledger, not a suggestion.** It must include every release, every query, model selection, hyperparameter tuning, retries, failed queries, and internal dashboards. If the same data is queried twice with independent noise, the privacy loss accumulates; under basic composition it doubles. Averaging repeated queries reduces the noise but not the privacy loss. The budget must be attached to a privacy unit and a time window. "ε = 1" without a unit and a window is not a guarantee.
A worked composition example shows why release cadence dominates the design. Assume 96 fifteen-minute bins per day, 365 days per year, and advanced composition with `δ' = 1e-5`:
| Release cadence | Releases/year | ε per release | Annual ε (advanced) | Comment |
|---|---:|---:|---:|---|
| Every 15 min | 35,040 | 0.01 | ≈ 12.5 | Already a large annual budget |
| Every 15 min | 35,040 | 0.1 | ≈ 458 | Unusable |
| Hourly | 8,760 | 0.1 | ≈ 137 | Unusable |
| Daily | 365 | 0.1 | ≈ 13 | High but arguable |
| Weekly | 52 | 0.1 | ≈ 4.0 | Defensible |
The arithmetic is not a measured result, but the conclusion is robust for a small city with the counts assumed here: a live 15-minute public dashboard cannot have a meaningful annual DP budget for a device-day privacy unit. The release cadence must be part of the privacy design, not an afterthought. A system that publishes every 15 minutes and claims "ε = 0.1" per release is either not accounting for composition or is choosing an annual budget in the hundreds.
### 3.5 Utility loss for counts and heatmaps
For a single count with Laplace noise at `ε = 1`, the 95% interval is about `±3`, independent of the true count. The relative error therefore depends entirely on the count:
| True count `c` | 95% interval (ε = 1) | Relative half-width |
|---:|---:|---:|
| 1 | ±3.0 | ±300% |
| 5 | ±3.0 | ±60% |
| 10 | ±3.0 | ±30% |
| 50 | ±3.0 | ±6% |
| 100 | ±3.0 | ±3% |
| 1,000 | ±3.0 | ±0.3% |
At `ε = 0.1`, multiply the interval by 10; at `ε = 10`, divide by 10. The Gaussian mechanism at `ε = 1, δ = 1e-6` has a 95% interval of about `±10.4` for sensitivity 1, which is worse for a single count but better when many queries compose.
This table is the core utility argument for footfall. A cell with 1–5 devices is not usable at ε ≤ 1. A cell with 100 devices is usable at ε = 1 and marginal at ε = 0.1. A daily total per sensor with ~1,000 device-visits is usable at ε = 0.1. The design question is therefore not "what is the best DP mechanism?" but "what spatial and temporal aggregation gives each released cell enough counts to survive the noise?"
**Illustrative sizing for a small city.** Assume 8 sensors, about 14 devices per sensor per 15-minute bin (consistent with the observed density index scale in the PulseCast pilot, though the true unique-device count is unknown), a 10-cell heatmap, Laplace noise, and advanced composition with `δ' = 1e-5`:
| Release | Releases/year | Count per cell | ε/release | 95% interval | Annual ε | Verdict |
|---|---:|---:|---:|---:|---:|---|
| 15-min per sensor | 35,040 | 14 | 0.1 | ±30 (±214%) | ≈ 458 | Unusable |
| Hourly per sensor | 8,760 | 56 | 0.1 | ±30 (±54%) | ≈ 137 | Unusable |
| Daily per sensor | 365 | 1,344 | 0.1 | ±30 (±2.2%) | ≈ 13 | Borderline |
| Daily 10-cell heatmap | 365 | 1,075 | 0.1 | ±30 (±2.8%) | ≈ 13 | Borderline |
| Weekly 10-cell heatmap | 52 | 7,525 | 0.1 | ±30 (±0.4%) | ≈ 4 | Viable |
These are design calculations, not measurements. They assume independent Laplace noise, a fixed grid, one count per device per cell, and the advanced-composition bound. They show the trade-off that matters: **fine-grained public heatmaps and strong DP for a device-day unit are incompatible at small-city scale.** The honest options are coarser releases, a larger budget with a public justification, or a trusted curator that publishes a one-shot aggregate rather than an interactive dashboard.
Utility loss is not only variance. DP noise can produce negative counts, false hotspots, and spurious peaks; constrained inference and smoothing can repair some of this, but they can also bias the result. Forecasting models trained on noisy aggregates inherit the noise; the model's error may increase, and the increase must be measured, not assumed. A paper that reports a WAPE on raw counts and then claims the same WAPE under DP is making an unverified claim.
### 3.6 Implementations: Google, Apple, OpenDP
**Google.** Google's open-source DP library (C++, Java, Go) implements Laplace and Gaussian mechanisms, bounded count/sum/mean, quantiles, selection, and partition selection, with contribution bounding and accounting. PipelineDP and Privacy on Beam provide pipeline-level DP. Google's COVID-19 Community Mobility Reports used the C++ library with Laplace noise, `ε = 0.11` per count at the coarser levels (0.22 at the finest), a contribution bound of four category-location pairs per user per day per geographic level, a maximum daily `ε = 1.76`, and `δ = 0`; the release also applied a minimum region size of 3 km² and a post-noise threshold of 100 users. Google's Privacy Sandbox Private Aggregation API uses a different architecture: browser worklets contribute encrypted reports, a TEE-backed aggregation service decrypts and sums them, and noise is added before release. The trust model is a TEE plus DP, not a trusted curator alone.
**Apple.** Apple's local DP deployment uses on-device privatization with count-mean sketches and Hadamard count-mean sketches, designed for large domains and limited communication. The 2017 "Learning with Privacy at Scale" paper describes the client-side algorithms and the server-side estimation; Apple reports task-specific ε values rather than a single global value. Apple's Exposure Notification framework uses rolling proximity identifiers and on-device matching, a different architecture that avoids central raw identifiers. Apple's Private Cloud Compute is a TEE-based cloud architecture with attested, reproducible node software and a transparency log; it is relevant to the "trusted aggregator" layer, not a substitute for DP.
**OpenDP.** OpenDP is an open-source library with a Rust core and Python bindings. It provides transformations and measurements, sequential and parallel composition, privacy filters that stop a computation when the budget would be exceeded, and contribution-bounding primitives. It is used by DP Creator and SmartNoise. Its guarantee is only as good as the caller's use: the library can enforce composition and sensitivity for the queries it understands, but it cannot choose the privacy unit, the contribution bound, or the release cadence for the analyst. A common failure mode is to use a DP library for the noise but compute the sensitivity by hand and get it wrong.
| Implementation | Model | Strengths | Limitations |
|---|---|---|---|
| Google DP library | Central | Mature; count/sum/mean/quantiles; contribution bounding; used in production mobility | C++/Java/Go focus; pipeline integration requires care |
| OpenDP | Central | Open source; Rust/Python; composition and privacy filters; auditable | Smaller mechanism set; caller must define the threat model |
| Apple CMS/HCMS | Local | On-device; large domains; production deployment at scale | Local-DP utility; task-specific ε; not for sparse counts |
| Google Private Aggregation | Shuffle/TEE | Browser-scale; encrypted reports; TEE aggregation | TEE trust; complex; not a general DP query system |
### 3.7 What ε should a municipal footfall release use?
There is no theorem that maps ε to a legal or ethical standard. A defensible process is:
1. Define the privacy unit (device-day) and the contribution bound (for example, one count per sensor per 15-minute bin and at most four sensor-bin pairs per day).
2. Choose a release cadence that gives each cell enough counts (daily or weekly, not 15-minute).
3. Choose a total annual budget in the range of roughly 1–10 for public releases, with a written justification. Below 1 is strong; above 10 is hard to defend for public data. This is a policy judgment, not a mathematical result.
4. Publish the mechanism, `ε`, `δ`, the privacy unit, the contribution bound, the release cadence, and the utility metrics. Do not publish only "ε = 1".
5. Have the data protection authority or an independent reviewer check the accounting and the utility claims.
The "expose your epsilons" argument (Dwork, Kohli, and Mulligan, 2019) is the right norm: a DP claim without its parameters and threat model is not verifiable. NIST SP 800-226 provides a structured evaluation checklist; it does not certify a particular ε.
---
## 4. k-anonymity, l-diversity, and t-closeness
### 4.1 Definitions
Let `QI` be a set of quasi-identifiers (age, postal code, timestamp, location) that can be linked to external data. A release is **k-anonymous** if every combination of QI values appears in at least `k` records (Sweeney, 2002; Samarati and Sweeney, 1998). The classic implementation generalizes QI values (for example, age 34 → 30–39, a 100 m cell → a 500 m cell) and suppresses records that cannot be generalized.
**l-diversity** strengthens k-anonymity by requiring each equivalence class to contain at least `l` distinct values of a sensitive attribute (Machanavajjhala et al., 2006). **t-closeness** requires the earth mover's distance between the sensitive-attribute distribution in each class and the global distribution to be at most `t` (Li, Li, and Venkatasubramanian, 2007). Both are attempts to prevent attribute disclosure, not just identity disclosure.
### 4.2 Why k-anonymity alone is weak
- **Homogeneity attack.** If all `k` records in a class share the same sensitive value, the class reveals that value even though no individual is singled out. k-anonymity does not prevent this; l-diversity does, in principle.
- **Background-knowledge attack.** The adversary may know that the target is not in a class, or may know additional attributes that narrow the class. k-anonymity's guarantee is only with respect to the quasi-identifiers it models.
- **Composition attack.** Two independently k-anonymous releases can be joined to re-identify individuals (Ganta, Thompson, and Kumar, 2008). k-anonymity does not compose; the privacy loss from multiple releases is not bounded by `k`.
- **High-dimensional uniqueness.** Mobility traces are high-dimensional and sparse. de Montjoye et al. (2013) show that four spatio-temporal points uniquely identify 95% of individuals in a large mobile-phone dataset. Achieving `k = 20` on such data would likely require generalization so coarse that the local footfall signal disappears.
- **Suppression leakage.** Suppressing cells with counts below `k` reveals that those cells have between 1 and `k−1` devices. Complementary suppression can reduce this, but it is a patch, not a guarantee.
- **Computational hardness.** Optimal k-anonymization is NP-hard (Meyerson and Williams, 2004); practical algorithms use heuristics whose privacy properties are hard to verify.
l-diversity addresses homogeneity but not skewness or similarity attacks: a class can have `l` distinct values that are all close to the sensitive value, or the overall distribution can be skewed. t-closeness addresses the distribution but requires a sensitive attribute, a distance metric, and a global distribution; it is difficult to apply to continuous, high-dimensional mobility data, and it still does not compose.
### 4.3 Spatial and temporal generalization for mobility
The standard tools are grid coarsening, time binning, and cloaking. Gruteser and Grunwald (2003) introduced spatial and temporal cloaking for location-based services; Casper (Mokbel, Chow, and Aref, 2006) generalized the idea with a privacy-aware query processor. For footfall, these techniques have a direct utility cost:
- Coarsening a 100 m grid to 500 m merges distinct storefronts and sensors; the heatmap no longer identifies the location that matters.
- Binning 15 minutes to 1 hour hides the lunch and evening peaks that drive staffing.
- Cloaking a trajectory to a region and a time window destroys the sequence information that makes mobility data useful for forecasting.
The practical result is that k-anonymity for mobility is either weak (small k, high uniqueness) or useless (large k, no local signal). It is not a good primary guarantee for a public footfall API.
### 4.4 Practical thresholds
Common practice is not a guarantee:
- `k = 5` is common in health data and is generally considered weak for public release.
- `k = 10–20` is a common public-release range; the choice is policy, not derived from a threat model.
- Google's COVID-19 Community Mobility Reports used a post-noise threshold of 100 contributing users, but that is a DP release with a threshold, not a k-anonymity guarantee. The threshold is a reliability and privacy filter on top of DP, not a substitute for it.
- `l = 2–3` and `t = 0.2` appear in the literature as illustrative choices; they are not universal standards.
NIST SP 800-188 treats k-anonymity as one technique among several and emphasizes that de-identification must be evaluated against a threat model, with re-identification testing and governance. It does not endorse a universal k. The UK ICO's anonymisation code makes a similar point: anonymisation is a risk assessment, not a checkbox.
**Recommendation.** Do not use k-anonymity as the primary privacy guarantee for a public footfall release. Use DP for the published aggregates. If k-anonymity is used at all, use it as a secondary check on internal data or as a sanity check on cell counts, document the quasi-identifiers and the adversary, and do not claim that it composes or that it prevents attribute inference.
---
## 5. Secure aggregation, MPC, federated learning, HE, and TEEs
### 5.1 Secure aggregation
Secure aggregation lets a server compute the sum of client updates without seeing any individual update. In the Bonawitz et al. (2017) protocol, each client masks its update with pairwise random masks that cancel in the sum, secret-shares the mask with the other clients, and the server reconstructs the sum even if some clients drop out. The server learns the sum, not the inputs, provided it does not collude with enough clients to reconstruct the masks.
- **Guarantee:** cryptographic confidentiality of individual contributions against the server and colluding clients below the threshold. It does not protect the sum; if the sum is sensitive (a cell with one device), DP is still required. It does not prevent a malicious client from poisoning the sum; robust aggregation is a separate problem.
- **Cost:** communication grows with the number of participants and the dropout threshold; the protocol needs multiple rounds and secret sharing. It is practical for hundreds to thousands of clients per round, not for per-request queries.
- **Fit:** good when multiple operators (city, transit authority, private sensor vendors) want to aggregate counts without revealing their raw data to each other. For a single city-owned aggregator, it adds complexity without removing the main trust assumption.
### 5.2 Secure multi-party computation
MPC computes an arbitrary function over private inputs (Yao, 1986; Goldreich, Micali, and Wigderson, 1987). Secret-sharing-based protocols are efficient for addition and counts; garbled circuits are efficient for comparisons and branching; SPDZ-style protocols provide malicious security with preprocessing (Damgård et al., 2012). The guarantee depends on the corruption model: semi-honest (parties follow the protocol) versus malicious (parties may deviate), and honest-majority versus dishonest-majority.
- **Guarantee:** correctness and input privacy, conditional on the corruption model and the protocol assumptions. MPC does not provide DP; the output can still leak.
- **Cost:** network rounds and bandwidth; malicious security adds substantial overhead; cross-city WAN latency dominates.
- **Fit:** joint cross-city forecasting or benchmarking without sharing raw counts. It is overkill for a single-city pilot with a trusted aggregator, and it does not solve the sparse-cell utility problem.
### 5.3 Homomorphic encryption
Additive HE (for example, Paillier, 1999) supports encrypted sums: each party encrypts its count, the aggregator adds ciphertexts and multiplies them by plaintext weights, and the key holder decrypts the total. Fully homomorphic encryption (Gentry, 2009; BGV/BFV; CKKS for approximate arithmetic, Cheon et al., 2017) supports arbitrary computation but is orders of magnitude slower, has limited multiplicative depth, and requires bootstrapping or large parameters.
- **Guarantee:** confidentiality of inputs during computation; the output is decryptable by the key holder. HE does not provide DP, and the output can leak. Threshold decryption distributes the key, but collusion and key management remain.
- **Cost:** ciphertext expansion, key management, limited operations, and a much higher computational cost than plaintext aggregation. Additive HE is practical for counts; FHE is not yet practical for full heatmap smoothing or model training at city scale.
- **Fit:** encrypted aggregation of counts across mutually distrustful parties. It is a complement to DP, not a replacement.
### 5.4 Trusted execution environments
A TEE provides isolated execution with attestation: Intel SGX, AMD SEV-SNP, ARM Confidential Compute Architecture. The code and data are protected from the host operating system and the cloud operator, conditional on the hardware and the attestation chain.
- **Guarantee:** confidentiality and integrity against a privileged operator, subject to the TEE's threat model. It is not a mathematical guarantee against a malicious hardware vendor, a physical attacker, or side-channel leakage. SGX in particular has a history of side-channel and fault attacks (Van Bulck et al., 2018; Kocher et al., 2020) and a limited enclave memory.
- **Cost:** attestation infrastructure, reproducible builds, key management, and operational complexity. Performance is near-native for many workloads, but I/O and memory limits matter.
- **Fit:** a TEE-backed aggregation service for a multi-tenant production system, especially when the aggregator is a cloud provider rather than the city. Google's Private Aggregation API and Apple's Private Cloud Compute are production examples. A TEE does not replace DP: it protects the data in use, while DP protects the output.
### 5.5 Federated learning and federated analytics
Federated learning trains a model across devices without centralizing raw data. FedAvg aggregates local updates (McMahan et al., 2017); secure aggregation hides individual updates; DP-FedAvg adds clipping and noise at the user level (McMahan et al., 2018). Federated analytics computes aggregates on devices and returns only the aggregates.
- **FL is not a privacy guarantee.** Gradients can leak training data (Zhu, Liu, and Han, 2019), membership inference can succeed (Shokri et al., 2017), and the server sees the updates unless SecAgg is used. FL must be combined with DP and SecAgg to make a privacy claim.
- **Federated analytics requires device-side code.** It works when the phone runs the app and the app can compute the aggregate. It does not work for passive BLE sniffing, where the pedestrian's phone is not participating. In our architecture, "on-device" means the **sensor or edge node**, not the pedestrian's phone.
- **Fit:** relevant only if PulseCast ships a companion app or partners with a transit app. For the sensing layer described here, FL is not applicable, and claiming it would be misleading.
### 5.6 Trade-off summary
| PET | What it removes | What it does not remove | Cost | Best use |
|---|---|---|---|---|
| Central DP | Trust in the analyst; bounds output leakage | Trust in the curator | Low compute; governance | City-owned aggregator; published aggregates |
| Local DP | Trust in the curator | Utility loss; device compromise | Low compute; high noise | Large-scale telemetry |
| Shuffle DP | Trust in the server (with a shuffler) | Trust in the shuffler/mixnet; validation | Latency; infrastructure | Large-scale analytics |
| Secure aggregation | Server's view of individual updates | Sum leakage; poisoning; collusion | Communication | Multi-operator aggregation |
| MPC | Parties' view of each other's inputs | Output leakage; corruption assumptions | Network rounds | Cross-city queries |
| HE | Computation on plaintext | Output leakage; key management | Compute; ciphertext size | Encrypted sums |
| TEE | Cloud operator's view of data in use | Side channels; vendor trust; output leakage | Attestation; operations | Production aggregation |
| FL | Raw data centralization | Gradient leakage; membership inference | Device coordination | App-based sensing |
The composition rule is simple: **use the weakest trust assumption that the deployment can support, and add DP at the output regardless.** For a city-owned pilot, central DP is sufficient and honest. For a multi-operator production system, add SecAgg or a TEE, but keep DP as the output guarantee.
---
## 6. Synthetic data for mobility
### 6.1 Why synthetic data
Synthetic data can support public release, model development, scenario simulation, and cross-city sharing without publishing raw records. The main families are:
- **Statistical synthesis:** independent sampling, iterative proportional fitting, Bayesian networks (PrivBayes; Zhang et al., 2017), and marginal-based methods (AIM; McKenna et al., 2022).
- **Deep generative models:** GANs, VAEs, and diffusion models, including DP variants (DP-GAN; Xie et al., 2018; PATE-GAN; Jordon, Yoon, and van der Schaar, 2019).
- **Mobility-specific generators:** sequence models and GANs for trajectories (MoveSim; Feng et al., 2020), and flow models such as Deep Gravity (Simini et al., 2021).
- **Agent-based simulation:** activity-based travel demand models that generate plausible daily schedules from population and land-use data.
These are not equivalent. A synthetic **count** table (sensor × time) is low-dimensional and relatively easy to synthesize with DP. A synthetic **trajectory** dataset is high-dimensional, sparse, and much harder to protect.
### 6.2 Fidelity metrics
There is no single fidelity score. A credible evaluation reports several:
| Dimension | Metric | What it checks |
|---|---|---|
| Marginal distributions | Kolmogorov–Smirnov, total variation, chi-square | Whether each column's distribution is preserved |
| Correlations | Pairwise correlation difference, mutual information | Whether dependencies survive |
| Predictive utility | Train-on-synthetic, test-on-real (TSTR); propensity mean squared error (pMSE) | Whether the synthetic data is useful for the downstream task |
| Coverage | Precision/recall, density, discriminator AUC | Whether rare modes are represented or collapsed |
| Mobility-specific | Trip-length distribution, origin-destination matrix similarity, radius of gyration, temporal profile, unique locations | Whether the synthetic data looks like mobility, not just like a table |
For a footfall count table, the relevant metrics are the hourly/weekly profile, the spatial correlation between sensors, the weather/holiday response, and the forecast error of a model trained on synthetic data. For a trajectory dataset, the relevant metrics are trip lengths, OD flows, radius of gyration, and the distribution of dwell times. Reporting only a discriminator AUC is not enough; a synthetic dataset can fool a discriminator and still leak.
### 6.3 Privacy metrics
**Formal.** If the generator is `(ε, δ)`-DP, then the synthetic dataset is `(ε, δ)`-DP by post-processing, and any model trained on it is also DP. This is the only formal guarantee. A non-DP generator has no formal guarantee, regardless of how realistic the output looks.
**Empirical.** Privacy is evaluated with attacks:
- **Membership inference:** can an adversary tell whether a target record was in the training data? Report AUC or advantage over a baseline (Shokri et al., 2017; van Breugel et al., 2023).
- **Attribute inference:** can an adversary infer a sensitive attribute from the synthetic data plus auxiliary knowledge?
- **Singling out:** can an adversary isolate a unique record?
- **Linkability:** can an adversary link a synthetic record to a real record or to another dataset?
- **Distance to closest record (DCR) and nearest-neighbor distance ratio (NNDR):** heuristics for memorization. A small DCR suggests the generator copied a training record; a large DCR does not prove privacy.
- **Anonymeter:** a framework that quantifies singling out, linkability, and inference risk for synthetic tabular data, aligned with the GDPR risk categories (Giomi et al., 2023).
The empirical results must be compared with a baseline (random guessing, or a non-private synthetic dataset) and reported with confidence intervals. A single attack with a low success rate does not prove privacy; a single attack with a high success rate disproves it.
### 6.4 Risks
- **Synthetic data is not automatically anonymous.** Stadler, Oprisanu, and Troncoso (2022) evaluated a range of generative models and found that synthetic data either fails to prevent inference attacks or fails to retain utility; the privacy-utility trade-off is unpredictable. The paper's title, "Anonymisation Groundhog Day," is the right warning: the same claims recur without the guarantees.
- **Memorization.** GANs and diffusion models can reproduce rare training records, especially with small datasets or overfitting. DCR is a useful screen but not a proof.
- **Attribute inference.** Even if no record is copied, the synthetic data can preserve correlations that reveal sensitive attributes (for example, a rare diagnosis correlated with a location).
- **Mobility-specific rare events.** A VIP's commute, a protest, a clinic visit, or a night-shift pattern may be rare enough to be memorized or inferred. Aggregation and DP are essential.
- **Privacy washing.** A vendor may call a dataset "synthetic" and imply anonymity without a DP guarantee or an attack evaluation. Do not repeat that claim.
### 6.5 Fit for footfall
Synthetic data is a good fit for **public release** and **model development**, not for the live dashboard:
- A DP-synthesized daily count table can be released for research and reproducibility, with a formal guarantee and an Anonymeter evaluation.
- A synthetic trajectory dataset should be avoided unless there is a clear need and a DP generator; the risk is high and the utility is hard to validate.
- For forecasting, train on DP aggregates rather than synthetic raw trajectories. Synthetic data can augment rare scenarios, but the model must be validated on real aggregates, and the evaluation must report the noise level.
---
## 7. Recommended PET stack
### 7.1 Architecture
```text
Pedestrian devices
│ BLE / Wi-Fi advertisements with ephemeral addresses
▼
Sensor / edge node
│ 1. Count unique ephemeral addresses per time bin
│ 2. Rotate a secret salt per bin; destroy it after the bin
│ 3. Never write raw MACs, RSSI, probe requests, or sequences
│ 4. Emit only (sensor_id, bin_start, count)
▼
Edge aggregator (city-owned in the pilot)
│ 5. Enforce contribution bounds across sensors and bins
│ 6. Add DP noise (Laplace for single releases; Gaussian/RDP for many)
│ 7. Maintain a privacy budget ledger
▼
Aggregate store (counts only)
│ 8. Train forecasting models on DP aggregates
│ 9. No raw trajectories; no per-device records
▼
Dashboard / API
│ 10. Fixed grid and time bins; no arbitrary filters
│ 11. Post-processing: smoothing, constrained inference, visualization
│ 12. Publish the budget, mechanism, and utility metrics
```
The design principle is **collect less, aggregate early, publish with a bound**. The sensor is the privacy boundary: if raw MACs never leave the sensor, and the firmware is not compromised, the aggregator cannot leak them. The aggregator is the trust boundary: if it is city-owned and audited, central DP is realistic. The dashboard is the attack surface: if it allows arbitrary queries, even a DP release can be differenced; a fixed release schedule is safer than an interactive query API.
### 7.2 Layer-by-layer guarantees
| Layer | Mechanism | Guarantee | Trust assumption | Failure mode |
|---|---|---|---|---|
| Capture | Count unique ephemeral addresses; rotating salt; no raw storage | Operational: no raw MAC retention | Sensor firmware and hardware | Compromised sensor; firmware bug; physical access |
| Edge aggregation | Sum counts; drop identifiers; contribution bounds | Architectural: only counts leave | Edge operator | Small counts; cross-bin linkage |
| Central DP | Laplace/Gaussian; OpenDP; budget ledger | Formal: `(ε, δ)`-DP for each release | Aggregator is trusted and correct | Wrong sensitivity; budget overrun; implementation bug |
| Query restrictions | Fixed grid/time bins; no raw exports; rate limits | Operational: reduces attack surface | API operator | Differencing through allowed queries; metadata leaks |
| Secure aggregation / TEE (production) | Encrypted aggregation; attestation | Cryptographic/hardware: confidentiality against the server/cloud | Threshold or hardware vendor | Collusion; side channels; attestation bypass |
| Forecasting | Train on DP aggregates; post-processing | DP inherited from the aggregates | Model pipeline | Noise propagation; overfitting to noise; utility loss |
| Synthetic data (optional) | DP synthesis; Anonymeter evaluation | DP if the generator is DP | Synthesis pipeline | Utility loss; memorization if not DP |
| Governance | DPIA; transparency; audit; retention | Accountability, not a technical guarantee | Institution | Policy drift; insider access; scope creep |
### 7.3 On-device processing, minimization, and identifier hygiene
The cheapest privacy control is not collecting the data. For a passive sensing layer, this means the sensor, not the cloud, is where the privacy boundary sits.
**On-device means the sensor or edge node.** The pedestrian's phone does not run our code, so "on-device processing" cannot mean federated learning on the phone. It means the sensor counts locally and emits only a count. This is a weaker architecture than a participant app, but it is the only one available for passive BLE/Wi-Fi sensing.
**Data minimization.** Collect only what the product needs: a count per sensor per time bin. Do not collect or retain raw MAC addresses, RSSI time series, probe-request payloads, sequence numbers, or device fingerprints. Apply purpose limitation (footfall analysis for urban operations, not law enforcement or advertising) and storage limitation (aggregates retained for a defined period, raw data never stored). These are GDPR Article 5 principles, but they are also good engineering: data that does not exist cannot leak.
**Ephemeral identifiers.** BLE devices that support privacy use resolvable private addresses that typically rotate about every 15 minutes; many Wi-Fi devices randomize MAC addresses per network. These are device-side mitigations, not guarantees. Vanhoef et al. (2016), Franklin et al. (2006), and Martin et al. (2019) show that fingerprinting, probe-request content, sequence numbers, and protocol bugs can link devices across rotations. The sensor should treat the address as an ephemeral token, count unique tokens within a bin, and never store the token.
**Rotating salts.** If the sensor hashes the ephemeral address before counting, the salt determines linkability. A secret random salt per 15-minute bin, destroyed after the bin, prevents cross-bin linkage but also prevents unique daily counts. A daily salt enables unique daily visitors but allows linkage within the day. The rotation period is a privacy/utility knob and should be documented. Hashing without a salt is not anonymization: the MAC space is 2^48, and a known OUI prefix reduces the search space to about 2^24, which is trivial to brute-force. A secret salt makes the hash a keyed pseudonym, and the key becomes a single point of failure; a public salt is brute-forceable. Do not describe either as anonymization.
**Edge aggregation.** The sensor or a city-owned gateway sums counts and drops identifiers before transmission. The edge node is a trust boundary: secure boot, signed firmware, disabled debug interfaces, no persistent storage, and remote attestation where available. A compromised sensor can still record raw MACs before aggregation; no output-side PET can prevent that. This residual risk must be stated, not hidden.
**No raw trajectory retention.** If trajectories are not a product requirement, do not store them. If aggregate flows are needed, compute origin-destination matrices at the edge or aggregator and protect them with DP; do not store per-device sequences. Trajectory data is among the most re-identifiable forms of mobility data (de Montjoye et al., 2013), and a "temporary" raw table is a permanent liability.
### 7.4 Explicit guarantees
**What we can guarantee, if the implementation is correct:**
- `(ε, δ)`-DP for the sequence of published aggregates, with privacy unit = device-day, contribution bound = at most one count per sensor per 15-minute bin and at most `k` sensor-bin pairs per day, mechanism = Laplace or Gaussian, annual budget = `ε_total`, and `δ = δ_total`. The guarantee is with respect to the stated neighboring relation; the bound holds regardless of the auxiliary information the adversary has about the other devices.
- Post-processing immunity: forecasting, smoothing, and visualization that use only the DP aggregates do not consume additional privacy budget and do not weaken the guarantee, provided they do not access raw data.
- No raw MAC retention at the sensor, if the firmware is correct and audited. This is an operational guarantee, not a mathematical one.
- Edge aggregation: only counts leave the sensor, so the aggregator does not receive device-level records. This is an architectural guarantee, conditional on the firmware.
**What we cannot guarantee:**
- Anonymity of every pedestrian. DP protects the contribution of a device-day, not the identity of a person. Shared devices, multiple devices per person, and non-smartphone users break the device-person mapping.
- Protection against a compromised sensor that records raw MACs before aggregation. No output-side PET can prevent that.
- Protection against inference from external data. DP bounds the influence of the release, not the adversary's other knowledge.
- Utility for sparse cells. At 15-minute/100 m resolution, noise dominates; the system should not promise a live fine-grained public heatmap.
- That a particular `ε` is "safe" in a legal or ethical sense. `ε` is a parameter, not a certificate.
- That non-DP synthetic data is anonymous. It is not.
- That k-anonymity, l-diversity, or t-closeness composes or resists arbitrary auxiliary data. They do not.
**What we do not claim:**
- We do not claim federated learning. The pedestrian's phone does not run our code.
- We do not claim MPC, HE, or TEE in the pilot. The pilot's trust boundary is a city-owned aggregator; those technologies add complexity without a clear threat model at that scale.
- We do not claim that hashing MAC addresses is anonymization. The MAC space is small; a hash with a public salt is brute-forceable, and a hash with a secret salt is pseudonymization with a key-management problem.
- We do not claim that MAC randomization protects the sensing layer. It is a device-side mitigation that fingerprinting can defeat.
### 7.5 What is realistic: pilot versus production
| Capability | Small-city pilot (8–50 sensors) | Production (100–1,000 sensors) | Reason |
|---|---|---|---|
| Per-sensor 15-min counts, internal | Yes | Yes | Low-dimensional; access control or DP |
| Public 15-min heatmap at 100 m | No | No (with strong DP) | Sparse cells; noise dominates |
| Public hourly heatmap at 250 m | Maybe | Yes | More counts per cell; coarser grid |
| Public daily heatmap at 100–250 m | Yes | Yes | Enough counts; DP feasible |
| Forecasting on DP aggregates | Yes | Yes | Post-processing; measure noise impact |
| Cross-city federated query | No | Maybe | MPC/SecAgg complexity |
| Synthetic trajectory release | No | Maybe | High risk; needs DP and attack evaluation |
| Secure aggregation / TEE | No | Yes | Multi-operator trust boundary |
| Formal DP accounting and audit | Basic | Full (RDP/zCDP, independent audit) | Governance maturity |
**Pilot.** The realistic pilot is 8–50 sensors, a city-owned aggregator, OpenDP or the Google DP library, a fixed daily release plus per-sensor 15-minute counts for internal use, and a DPIA. The main engineering work is sensor firmware (no raw storage, signed firmware, rotating salt), the aggregation pipeline, and the governance materials. The compute cost is negligible; the privacy budget and the release cadence are the binding constraints. A pilot should not attempt MPC, HE, or a TEE unless a specific multi-party trust problem requires it.
**Production.** A production system with 100–1,000 sensors and multiple operators needs secure aggregation or a TEE-backed aggregation service, formal DP accounting (RDP or zCDP), a budget ledger, independent audit, a transparency report, query restrictions, and incident response. The compute cost of DP remains small relative to the data pipeline; secure computation and governance dominate. Fine-grained public heatmaps remain statistically hard: the production system should publish hourly or daily aggregates at a coarse grid, not a 15-minute 100 m public map.
**Performance.** DP noise addition is negligible compared with data ingestion, aggregation, and model inference; the latency budget of a dashboard is not threatened by DP. Secure aggregation adds communication rounds and dropout handling; MPC adds network round trips; HE adds orders of magnitude in computation and ciphertext size; a TEE is near-native but requires attestation and careful I/O. For a city-scale count workload (10^4–10^5 counts per day), the DP release is computationally trivial. The hard problems are the privacy budget, the query interface, and the governance, not throughput.
### 7.6 Recommended stack for PulseCast
**Pilot (deployable now, honest about limits):**
1. **Sensor firmware:** count unique ephemeral addresses per 15-minute bin; use a secret salt that rotates per bin and is destroyed; never store raw MACs, RSSI, probe requests, or sequences; sign firmware and disable debug interfaces.
2. **Edge aggregation:** emit only `(sensor_id, bin_start, count)`; aggregate at a city-owned gateway; enforce contribution bounds across sensors and bins.
3. **Central DP:** use OpenDP or the Google DP library; Laplace noise for per-sensor counts; Gaussian/RDP for multi-query releases; publish the privacy unit, contribution bound, `ε`, `δ`, and annual budget.
4. **Release policy:** per-sensor 15-minute counts for internal operations; daily or weekly heatmaps at 250 m for public release; no arbitrary public filters; no raw exports.
5. **Forecasting:** train on DP aggregates; report WAPE under the same noise level; do not compare a DP-trained model to a raw-data model without saying so.
6. **Governance:** DPIA, legal basis, retention policy, data protection officer sign-off, public privacy budget, and an audit of the firmware and the aggregation code. A DPIA is required for systematic monitoring of a publicly accessible area on a large scale under GDPR Article 35(3)(c); the DPIA should cover the sensor layer, the aggregation, the release policy, and the re-identification risk.
**Production (additional layers):**
1. **Secure aggregation or TEE** across sensor operators; attestation and reproducible builds; threshold decryption or a hardware root of trust.
2. **Formal accounting** with RDP or zCDP; a budget ledger that includes every release, query, model selection, and retry; a privacy filter that blocks over-budget queries.
3. **Independent audit** of the DP implementation, the sensitivity calculations, and the firmware; a public transparency report.
4. **Query restrictions:** a fixed release schedule, a documented API, rate limits, and no differencing queries. If an interactive API is required, use a DP query system with per-analyst budgets and query restriction.
5. **DP synthetic data** for open release, with Anonymeter and membership-inference evaluation; never release non-DP synthetic trajectories.
6. **Incident response:** key rotation, firmware update, side-channel review, and a process for revoking a compromised sensor.
**What not to do:**
- Do not store raw MACs "for calibration" or "for debugging" in production.
- Do not hash MACs and call the result anonymous.
- Do not rely on k-anonymity alone for a public API.
- Do not use local DP for sparse 15-minute counts and expect useful heatmaps.
- Do not release non-DP synthetic data with an anonymity claim.
- Do not claim federated learning without device-side code.
- Do not claim a TEE makes the output private; TEEs protect data in use, not the released statistic.
- Do not publish "ε = 1" without the privacy unit, contribution bound, release cadence, and annual budget.
### 7.7 Open problems
- **Utility of DP at fine spatio-temporal resolution.** Adaptive grids, matrix mechanisms for heatmap range queries, and hierarchical models may improve the trade-off, but the sparse-cell limit is fundamental.
- **Forecasting under DP noise.** How much WAPE degradation does a given `(ε, δ)` cause? This is an empirical question for the PulseCast evaluation, not an assumption.
- **Calibration of device counts to people.** Device penetration, shared devices, and MAC randomization bias the counts; DP protects devices, not people. A calibration study against manual counts is needed.
- **Cross-sensor contribution bounding.** A device observed by many sensors creates a sensitivity problem; the right bound and the right privacy unit for a multi-sensor heatmap are open design questions.
- **Standard privacy budgets for municipal data.** There is no accepted `ε` for city footfall; a public, participatory process is needed, not a vendor default.
- **Regulatory acceptance.** DP is not automatically anonymization under GDPR. The legal test remains whether re-identification is reasonably likely in context; a DP release with a strong budget and no auxiliary data may support an anonymization argument, but that is a legal determination, not a mathematical one.
- **Auditing.** Independent audits of DP implementations, sensitivity calculations, and firmware are rare; they should be standard for public deployments.
---
## 8. Citations
### DP foundations and interpretation
- Abowd, J. M., and Schmutte, I. M. (2019). An Economic Analysis of Privacy Protection and Statistical Accuracy as Social Choices. *American Economic Review*, 109(1), 171–202. https://doi.org/10.1257/aer.20170627
- Abowd, J. M., Ashmead, R., Cumings-Menon, R., Garfinkel, S., Heineck, M., Heiss, C., Johns, R., Kifer, D., Leclerc, P., Machanavajjhala, A., Moran, B., Sexton, W., Spence, M., and Zhuravlev, P. (2022). The 2020 Census Disclosure Avoidance System TopDown Algorithm. *Harvard Data Science Review*, Special Issue 2. https://doi.org/10.1162/99608f92.529e3cb9
- Dwork, C. (2006). Differential Privacy. *ICALP 2006*, LNCS 4052, 1–12. https://doi.org/10.1007/11787006_1
- Dwork, C. (2011). A Firm Foundation for Private Data Analysis. *Communications of the ACM*, 54(1), 86–95. https://doi.org/10.1145/1866739.1866758
- Dwork, C., Kohli, N., and Mulligan, D. (2019). Differential Privacy in Practice: Expose your Epsilons! *Journal of Privacy and Confidentiality*, 9(2). https://doi.org/10.29012/jpc.689
- Dwork, C., McSherry, F., Nissim, K., and Smith, A. (2006). Calibrating Noise to Sensitivity in Private Data Analysis. *TCC 2006*, LNCS 3876, 265–284. https://doi.org/10.1007/11681878_14
- Dwork, C., and Roth, A. (2014). The Algorithmic Foundations of Differential Privacy. *Foundations and Trends in Theoretical Computer Science*, 9(3–4), 211–407. https://doi.org/10.1561/0400000042
- Hsu, J., Gaboardi, M., Haeberlen, A., Khanna, S., Narayan, A., Pierce, B. C., and Roth, A. (2014). Differential Privacy: An Economic Method for Choosing Epsilon. *IEEE CSF 2014*. https://doi.org/10.1109/CSF.2014.35
- Lee, J., and Clifton, C. (2011). How Much Is Enough? Choosing ε for Differential Privacy. *ISC 2011*, LNCS 7001, 325–340. https://doi.org/10.1007/978-3-642-24861-0_22
- NIST (2025). *Guidelines for Evaluating Differential Privacy Guarantees*. NIST Special Publication 800-226. https://doi.org/10.6028/NIST.SP.800-226
- Wood, A., Altman, M., Bembenek, A., Bun, M., Gaboardi, M., Honaker, J., Nissim, K., O'Brien, D. R., Steinke, T., and Vadhan, S. (2018). Differential Privacy: A Primer for a Non-Technical Audience. *Vanderbilt Journal of Entertainment & Technology Law*, 21(1), 209–276.
### Mechanisms, composition, and accounting
- Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L. (2016). Deep Learning with Differential Privacy. *ACM CCS 2016*, 308–318. https://doi.org/10.1145/2976749.2978318
- Balle, B., and Wang, Y.-X. (2018). Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising. *ICML 2018*, PMLR 80, 394–403.
- Bun, M., and Steinke, T. (2016). Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds. *TCC 2016*, LNCS 9563, 635–658. https://doi.org/10.1007/978-3-662-53641-4_24
- Canonne, C. L., Kamath, G., and Steinke, T. (2020). The Discrete Gaussian for Differential Privacy. *NeurIPS 2020*.
- Dong, J., Roth, A., and Su, W. J. (2022). Gaussian Differential Privacy. *Journal of the Royal Statistical Society: Series B*, 84(1), 3–37. https://doi.org/10.1111/rssb.12454
- Dwork, C., Naor, M., Pitassi, T., and Rothblum, G. N. (2010a). Differential Privacy under Continual Observation. *STOC 2010*, 715–724. https://doi.org/10.1145/1806689.1806787
- Dwork, C., Rothblum, G. N., and Vadhan, S. (2010b). Boosting and Differential Privacy. *FOCS 2010*, 51–60. https://doi.org/10.1109/FOCS.2010.12
- Kairouz, P., Oh, S., and Viswanath, P. (2015). The Composition Theorem for Differential Privacy. *ICML 2015*, PMLR 37, 1376–1385.
- Mironov, I. (2017). Rényi Differential Privacy. *IEEE CSF 2017*, 263–275. https://doi.org/10.1109/CSF.2017.11
### Counting, histograms, and spatial queries
- Cormode, G., Procopiuc, C., Srivastava, D., Shen, E., and Yu, T. (2012). Differentially Private Spatial Decompositions. *IEEE ICDE 2012*, 20–31. https://doi.org/10.1109/ICDE.2012.28
- Hay, M., Rastogi, V., Miklau, G., and Suciu, D. (2010). Boosting the Accuracy of Differentially Private Histograms through Consistency. *PVLDB*, 3(1), 1021–1032. https://doi.org/10.14778/1920841.1920970
- Li, C., Miklau, G., Hay, M., McGregor, A., and Rastogi, V. (2015). The Matrix Mechanism: Optimizing Linear Counting Queries under Differential Privacy. *The VLDB Journal*, 24(6), 757–781. https://doi.org/10.1007/s00778-015-0398-x
- Zhang, J., Xiao, X., and Xie, X. (2016). PrivTree: A Differentially Private Algorithm for Hierarchical Decompositions. *ACM SIGMOD 2016*, 155–170. https://doi.org/10.1145/2882903.2882928
### Local and shuffle models
- Balle, B., Bell, J., Gascón, A., and Nissim, K. (2019). The Privacy Blanket of the Shuffler. *CRYPTO 2019*, LNCS 11694, 157–186. https://doi.org/10.1007/978-3-030-26953-1_9
- Bittau, A., Erlingsson, Ú., Maniatis, P., Mironov, I., Raghunathan, A., Lie, D., Rudominer, M., Kifer, D., Balle, B., Kairouz, P., and Kenthapadi, K. (2017). Prochlo: Strong Privacy for Analytics in the Crowd. *ACM SOSP 2017*, 441–459. https://doi.org/10.1145/3132747.3132769
- Duchi, J. C., Jordan, M. I., and Wainwright, M. J. (2013). Local Privacy and Statistical Minimax Rates. *FOCS 2013*, 429–438. https://doi.org/10.1109/FOCS.2013.53
- Erlingsson, Ú., Feldman, V., Mironov, I., Raghunathan, A., Talwar, K., and Thakurta, A. (2019). Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity. *SODA 2019*, 2468–2479. https://doi.org/10.1137/1.9781611975482.151
- Erlingsson, Ú., Pihur, V., and Korolova, A. (2014). RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response. *ACM CCS 2014*, 1054–1067. https://doi.org/10.1145/2660267.2660348
- Feldman, V., McMillan, A., and Talwar, K. (2021). Hiding among the Clones: A Simple and Nearly Optimal Analysis of Privacy Amplification by Shuffling. *FOCS 2021*, 954–964. https://doi.org/10.1109/FOCS52979.2021.00095
- Ghazi, B., Golowich, N., Kumar, R., Pagh, R., and Velingker, A. (2021). On the Power of Multiple Anonymous Messages: Frequency Estimation and Selection in the Shuffle Model of Differential Privacy. *EUROCRYPT 2021*, LNCS 12696, 463–488. https://doi.org/10.1007/978-3-030-77883-5_16
- Kairouz, P., Oh, S., and Viswanath, P. (2016). Extremal Mechanisms for Local Differential Privacy. *Journal of Machine Learning Research*, 17(1), 1–51.
- Warner, S. L. (1965). Randomized Response: A Survey Technique for Eliminating Evasive Answer Bias. *Journal of the American Statistical Association*, 60(309), 63–69. https://doi.org/10.1080/01621459.1965.10480775
### k-anonymity, l-diversity, t-closeness, and re-identification
- de Montjoye, Y.-A., Hidalgo, C. A., Verleysen, M., and Blondel, V. D. (2013). Unique in the Crowd: The Privacy Bounds of Human Mobility. *Scientific Reports*, 3, 1376. https://doi.org/10.1038/srep01376
- de Montjoye, Y.-A., Radaelli, L., Singh, V. K., and Pentland, A. (2015). Unique in the Shopping Mall: On the Reidentifiability of Credit Card Metadata. *Science*, 347(6221), 536–539. https://doi.org/10.1126/science.1256297
- Ganta, S. R., Thompson, S., and Kumar, T. N. V. (2008). Composition Attacks on Anonymized Databases. *PVLDB*, 1(1). Proceedings of VLDB 2008.
- Gruteser, M., and Grunwald, D. (2003). Anonymous Usage of Location-Based Services through Spatial and Temporal Cloaking. *MobiSys 2003*.
- Li, N., Li, T., and Venkatasubramanian, S. (2007). t-Closeness: Privacy Beyond k-Anonymity and l-Diversity. *IEEE ICDE 2007*, 106–115. https://doi.org/10.1109/ICDE.2007.367856
- Machanavajjhala, A., Kifer, D., Gehrke, J., and Venkitasubramaniam, M. (2006). l-Diversity: Privacy Beyond k-Anonymity. *IEEE ICDE 2006*. https://doi.org/10.1109/ICDE.2006.1
- Meyerson, A., and Williams, R. (2004). On the Complexity of Optimal K-Anonymity. *ACM PODS 2004*, 223–228. https://doi.org/10.1145/1055558.1055591
- Mokbel, M. F., Chow, C.-Y., and Aref, W. G. (2006). The New Casper: Query Processing for Location Services without Compromising Privacy. *VLDB 2006*.
- Narayanan, A., and Shmatikov, V. (2008). Robust De-anonymization of Large Sparse Datasets. *IEEE Symposium on Security and Privacy 2008*, 111–125. https://doi.org/10.1109/SP.2008.33
- Ohm, P. (2010). Broken Promises of Privacy: Responding to the Surprising Failure of Anonymization. *UCLA Law Review*, 57, 1701–1777.
- Samarati, P., and Sweeney, L. (1998). Protecting Privacy when Disclosing Information: k-Anonymity and Its Enforcement through Generalization and Suppression. Technical Report SRI-CSL-98-04, SRI Computer Science Laboratory.
- Sweeney, L. (2002). k-anonymity: A Model for Protecting Privacy. *International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems*, 10(5), 557–570. https://doi.org/10.1142/S0218488502001648
- Tockar, A. (2014). Riding with the Stars: Passenger Privacy in the NYC Taxicab Dataset. Neustar Research, September 15, 2014.
- Barbaro, M., and Zeller, T. (2006). A Face Is Exposed for AOL Searcher No. 4417749. *The New York Times*, August 9, 2006.
- Dinur, I., and Nissim, K. (2003). Revealing Information while Preserving Privacy. *ACM PODS 2003*, 202–210. https://doi.org/10.1145/773153.773173
- Shokri, R., Stronati, M., Song, C., and Shmatikov, V. (2017). Membership Inference Attacks Against Machine Learning Models. *IEEE Symposium on Security and Privacy 2017*, 3–18. https://doi.org/10.1109/SP.2017.41
### Secure computation, federated learning, HE, and TEEs
- Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., and Seth, K. (2017). Practical Secure Aggregation for Privacy-Preserving Machine Learning. *ACM CCS 2017*, 1175–1191. https://doi.org/10.1145/3133956.3133982
- Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., Van Overveldt, T., Petrou, D., Ramage, D., and Roselander, J. (2019). Towards Federated Learning at Scale: System Design. *MLSys 2019*.
- Cheon, J. H., Kim, A., Kim, M., and Song, Y. (2017). Homomorphic Encryption for Arithmetic of Approximate Numbers. *ASIACRYPT 2017*, LNCS 10624, 409–437. https://doi.org/10.1007/978-3-319-70694-8_15
- Damgård, I., Pastro, V., Smart, N., and Zakarias, S. (2012). Multiparty Computation from Somewhat Homomorphic Encryption. *CRYPTO 2012*, LNCS 7417, 643–662. https://doi.org/10.1007/978-3-642-32009-5_38
- Gentry, C. (2009). Fully Homomorphic Encryption Using Ideal Lattices. *STOC 2009*, 169–178. https://doi.org/10.1145/1536414.1536440
- Goldreich, O., Micali, S., and Wigderson, A. (1987). How to Play any Mental Game. *STOC 1987*, 218–229. https://doi.org/10.1145/28395.28420
- Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., et al. (2021). Advances and Open Problems in Federated Learning. *Foundations and Trends in Machine Learning*, 14(1–2), 1–210. https://doi.org/10.1561/2200000083
- Lindell, Y. (2020). Secure Multiparty Computation. *Communications of the ACM*, 63(9), 76–85. https://doi.org/10.1145/3387108
- McMahan, H. B., Moore, E., Ramage, D., Hampson, S., and Agüera y Arcas, B. (2017). Communication-Efficient Learning of Deep Networks from Decentralized Data. *AISTATS 2017*, PMLR 54, 1273–1282.
- McMahan, H. B., Ramage, D., Talwar, K., and Zhang, L. (2018). Learning Differentially Private Recurrent Language Models. *ICLR 2018*.
- NIST (2022). *Hardware-Enabled Security: Enabling a Layered Approach to Platform Security for Cloud and Edge Computing Use Cases*. NIST IR 8320. https://doi.org/10.6028/NIST.IR.8320
- Paillier, P. (1999). Public-Key Cryptosystems Based on Composite Degree Residuosity Classes. *EUROCRYPT 1999*, LNCS 1592, 223–238. https://doi.org/10.1007/3-540-48910-X_16
- Van Bulck, J., Minkin, M., Weisse, O., Genkin, D., Kasikci, B., Piessens, F., Silberstein, M., Wenisch, T. F., Yarom, Y., and Strackx, R. (2018). Foreshadow: Extracting the Keys to the Intel SGX Kingdom with Transient Out-of-Order Execution. *USENIX Security 2018*.
- Kocher, P., Horn, J., Fogh, A., Genkin, D., Gruss, D., Haas, W., Hamburg, M., Lipp, M., Mangard, S., Prescher, T., Schwarz, M., and Yarom, Y. (2020). Plundervolt: Software-based Fault Injection Attacks against Intel SGX. *IEEE Symposium on Security and Privacy 2020*. https://doi.org/10.1109/SP40000.2020.00057
- Yao, A. C.-C. (1986). How to Generate and Exchange Secrets. *FOCS 1986*, 162–167. https://doi.org/10.1109/SFCS.1986.25
- Zhu, L., Liu, Z., and Han, S. (2019). Deep Leakage from Gradients. *NeurIPS 2019*.
- Nasr, M., Shokri, R., and Houmansadr, A. (2019). Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning. *IEEE Symposium on Security and Privacy 2019*, 739–753. https://doi.org/10.1109/SP.2019.00065
### Synthetic data
- Feng, J., Yang, Z., Xu, F., Yu, H., Wang, M., and Li, Y. (2020). Learning to Simulate Human Mobility. *ACM SIGKDD 2020*.
- Giomi, M., Boenisch, F., Wehmeyer, C., and Tasnádi, B. (2023). A Unified Framework for Quantifying Privacy Risk in Synthetic Data. *Proceedings on Privacy Enhancing Technologies*, 2023(3). https://doi.org/10.56553/popets-2023-0055
- Jordon, J., Yoon, J., and van der Schaar, M. (2019). PATE-GAN: Generating Synthetic Data with Differential Privacy Guarantees. *ICLR 2019*.
- McKenna, R., Mullins, B., Sheldon, D., and Miklau, G. (2022). AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Data. *PVLDB*, 15(11), 2599–2612. https://doi.org/10.14778/3551793.3551817
- Simini, F., Barlacchi, G., Luca, M., and Pappalardo, L. (2021). A Deep Gravity Model for Mobility Flows Generation. *Nature Communications*, 12, 6576. https://doi.org/10.1038/s41467-021-26752-4
- Stadler, T., Oprisanu, B., and Troncoso, C. (2022). Synthetic Data – Anonymisation Groundhog Day. *USENIX Security 2022*, 1451–1468.
- van Breugel, B., Sun, H., Qian, Z., and van der Schaar, M. (2023). Membership Inference Attacks Against Synthetic Data through Overfitting Detection. *AISTATS 2023*.
- Xie, L., Lin, K., Wang, S., Wang, F., and Zhou, J. (2018). Differentially Private Generative Adversarial Network. arXiv:1802.06739.
- Zhang, J., Cormode, G., Procopiuc, C. M., Srivastava, D., and Xiao, X. (2017). PrivBayes: Private Data Release via Bayesian Networks. *ACM Transactions on Database Systems*, 42(4), 25. https://doi.org/10.1145/3132704
### Deployments and implementations
- Aktay, A., Bavadekar, S., Cossoul, G., Davis, J., Desfontaines, D., Fabrikant, A., et al. (2020). Google COVID-19 Community Mobility Reports: Anonymization Process Description (version 1.1). arXiv:2004.04145. https://doi.org/10.48550/arXiv.2004.04145
- Apple Differential Privacy Team (2017). Learning with Privacy at Scale. *Apple Machine Learning Journal*, 1(8). https://machinelearning.apple.com/research/learning-with-privacy-at-scale
- Apple (2024). Private Cloud Compute: A New Frontier for AI Privacy in the Cloud. Apple Security Blog. https://security.apple.com/blog/private-cloud-compute/
- Apple and Google (2020). Exposure Notification: Bluetooth Specification. https://www.apple.com/covid19/contacttracing
- Google (2024a). Private Aggregation API Fundamentals. Privacy Sandbox documentation. https://privacysandbox.google.com/private-advertising/private-aggregation/fundamentals
- Google (2024b). Differential Privacy Library. https://github.com/google/differential-privacy
- OpenDP Team (2020). The OpenDP White Paper. https://projects.iq.harvard.edu/files/opendp/files/opendp_white_paper_11may2020.pdf
- OpenDP (2025). OpenDP Library Documentation. https://docs.opendp.org/
- Wilson, R. J., Zhang, C. Y., Lam, W., Desfontaines, D., Simmons-Marengo, D., and Gipson, B. (2020). Differentially Private SQL with Bounded User Contribution. *Proceedings on Privacy Enhancing Technologies*, 2020(2), 230–250. https://doi.org/10.2478/popets-2020-0025
### Regulatory and governance
- Article 29 Working Party (2011). Opinion 13/2011 on Geolocation Services on Smart Mobile Devices. WP185.
- Article 29 Working Party (2014). Opinion 05/2014 on Anonymisation Techniques. WP216.
- European Parliament and Council (2016). Regulation (EU) 2016/679 (General Data Protection Regulation).
- UK Information Commissioner's Office (2012). Anonymisation: Managing Data Protection Risk Code of Practice.
- U.S. Department of Health and Human Services. HIPAA Privacy Rule, 45 CFR §164.514.
- NIST (2015). De-Identification of Personal Information. NIST IR 8053. https://doi.org/10.6028/NIST.IR.8053
- NIST (2023). De-Identifying Government Datasets: Techniques and Governance. NIST SP 800-188. https://doi.org/10.6028/NIST.SP.800-188
### Wireless tracking and ephemeral identifiers
- Franklin, J., McCoy, D., Tabriz, P., Neagoe, V., Van Randwyk, J., and Snoeren, A. C. (2006). Passive Data Link Layer 802.11 Wireless Device Driver Fingerprinting. *USENIX Security 2006*.
- Martin, J., Alpuche, D., Bodeman, K., Brown, L., Fenske, E., Foppe, L., Mayberry, T., Rye, E. C., Sipes, B., and Teplov, S. (2019). Handoff All Your Privacy: A Review of Apple's Bluetooth Low Energy Continuity Protocol. *Proceedings on Privacy Enhancing Technologies*, 2019(4). https://doi.org/10.2478/popets-2019-0057
- Vanhoef, M., Matte, C., Cunche, M., Cardoso, L. S., and Piessens, F. (2016). Why MAC Address Randomization is not Enough: An Analysis of Wi-Fi Network Discovery Mechanisms. *ACM AsiaCCS 2016*, 413–424. https://doi.org/10.1145/2897845.2897883
---
*Empirical and deployment numbers in this memo are sourced from the cited primary literature. Derived quantities (confidence intervals, composition bounds, release-cadence arithmetic) are shown with their assumptions and are labelled as illustrative design arithmetic where they are not direct measurements. The PulseCast pilot's own measurements (8 sensors, ~5 months, ~580 m cluster) are used only as sizing assumptions, not as privacy results.*
+345
View File
@@ -0,0 +1,345 @@
---
title: "Source-code audit: how the reference PAX counter actually counts"
status: research-note
date: 2026-11-14
scope: "ESP32-Paxcounter / libpax reference implementation; not confirmed to be the exact firmware deployed in Heilbronn"
---
# Source-code audit: how the reference PAX counter actually counts
**Purpose.** The challenge brief calls the PAX counters "privacy-preserving sensors" that "estimate crowd density by passively sniffing Bluetooth signals (MAC address advertisements)" and transmit "aggregated device counts" (Schwarz Digits, *The Pulse of the City*, challenge brief). This note audits the widely used open-source reference implementation to separate the marketing description from the actual measurement semantics. It is a code audit, not a claim about the exact firmware or configuration used by Code for Heilbronn.
**Scope and caveat.** The implementation audited is:
- `ESP32-Paxcounter` v3.8.0 (`platformio_orig.ini`, `release_version = 3.8.0`), <https://github.com/cyberman54/ESP32-Paxcounter>
- `libpax`, the counting library extracted from Paxcounter, <https://github.com/dbinfrago/libpax>
The challenge says the data come from "LoRa-based PAX Counters" deployed by Code for Heilbronn. The upstream project is the canonical open-source design, but the deployed firmware version, build flags, sensor enclosure, antenna, and backend aggregation are not documented in the challenge package. Treat the findings below as **properties of the reference design**, and verify them against the deployed devices before making definitive claims.
**Important update from the operator.** Code for Heilbronn's own project post (18 Sep 2025) describes the deployed design:
- Hardware: Heltec Wireless Paper board with an ESP32 (Bluetooth/WLAN), battery input, and an SX1262 LoRa transceiver.
- Counting: the sensor receives Bluetooth advertisement packets and remembers device addresses.
- Privacy: "Diese MAC-Adressen werden gehashed und für maximal eine Stunde gespeichert" — the MAC addresses are hashed and stored for at most one hour. "Außerdem verlassen diese MAC Adressen auch gehashed nicht die Sensoren, da nur die Anzahl der gezählten Adressen gesendet wird" — even hashed, the addresses do not leave the sensors; only the number of counted addresses is transmitted.
Source: <https://codeforheilbronn.de/blog/2025/09/passantenzahlen-messen-mit-lora/>
This is **stronger** than the upstream libpax design in one respect (a one-hour hashed dedup cache rather than a 60-second 16-bit bitmap) and **different** in another (the deployed firmware may be a fork). The exact hash length, cache eviction rule, and aggregation from one-hour hashed counts to 15-minute CSV rows are not documented. The paper must not assume the upstream code is the deployed firmware; it should cite both and state the uncertainty.
---
## 1. Deployed firmware audit (Code for Heilbronn `lorax`)
The canonical deployment repository is **`cfhn/lorax`** (mirror name: `cfhn/lora-frequenzmessung`), <https://codeberg.org/cfhn/lorax>. It contains the `sensor-pax` ESP32 firmware, the LoRaWAN/TTN aggregator, the Go backend, TimescaleDB, and Grafana. This is a much closer match to the challenge than the generic upstream Paxcounter.
### 1.1 Exact cycle
`sensor-pax/src/main.cpp`:
```c
paxSetup();
unsigned long scanStart = millis();
while (!new_data_available && millis() - scanStart < 35000) {
delay(100);
}
uint16_t paxCount = current_count;
paxStop();
...
int payloadLen = snprintf(payload, sizeof(payload), "%s,0,%.4f,1,%.4f",
cfg.sensor_id, paxCount * cfg.factor, battPct);
...
goToSleep(); // deep sleep
```
`sensor-pax/src/pax.h`:
```c
configuration.blecounter = 1;
configuration.blescantime = 30; // seconds
configuration.wificounter = 0;
configuration.wifi_rssi_threshold = -80;
configuration.ble_rssi_threshold = -80;
libpax_update_config(&configuration);
libpax_counter_init(log, &count_from_libpax, 10, 1); // 10 s report, cumulative
libpax_counter_start();
```
`sensor-pax/src/provision.h`:
```c
cfg.factor = p.getFloat(KEY_FACTOR, 0.7f);
cfg.sleep_time_sec = p.getInt(KEY_SLEEP_TIME, 900);
```
**Therefore each CSV density row is:**
```text
value = (unique locally administered BLE MAC hashes seen in a ~10-second scan)
× (per-sensor factor, default 0.7)
```
and the next row is produced after a **15-minute deep sleep** (default `SLEEP_SEC=900`).
**The actual scan is shorter than 30 seconds.** `blescantime = 30` is passed to `start_BLE_scan()`, but this fork never uses it to stop the scan. Instead, the main loop exits on the first `new_data_available` callback, and the report timer is configured for **10 seconds** (`libpax_counter_init(..., 10, 1)`). So the effective scan is approximately **10 seconds**, followed by a 15-minute sleep. That is a **~1.1% duty cycle** (10 s / ~910 s), not 3.3%. The 35-second loop bound is only a safety timeout. The target is a **10-second snapshot**, so a crowd that peaks between snapshots is invisible, and the forecast is predicting a sparse sample rather than an integrated footfall flow.
### 1.2 The factor is an undocumented calibration knob
`cfg.factor` is provisioned per sensor (`.env` requires `FACTOR`), defaults to **0.7**, and is multiplied into the payload. The repository contains no calibration procedure, no ground-truth counts, and no explanation of what 0.7 means. It could be a device-to-person correction, a normalization to a manual count, or an arbitrary scale. The paper must not treat the CSV values as raw device counts or as people.
### 1.3 The hash is still 16-bit
`sensor-pax/lib/paxlib/libpax.cpp`:
```c
#define LIBPAX_MAX_SIZE 0xFFFF
#define LIBPAX_MAP_SIZE (LIBPAX_MAX_SIZE / BITS_PER_WORD)
...
IRAM_ATTR int mac_add(uint8_t *paddr, snifftype_t sniff_type) {
uint16_t *id;
// mac addresses are 6 bytes long, we only use the last two bytes
id = (uint16_t *)(paddr + 4);
// if it is NOT a locally administered ("random") mac, we don't count it
if (!(paddr[0] & 0b10)) return false;
int added = add_to_bucket(*id);
...
}
```
This confirms the 16-bit ID and the locally-administered-only filter in the deployed firmware. Two important precision points:
- The code does **not** apply a cryptographic hash. It truncates the last two bytes of the MAC address into a `uint16_t` (`id = (uint16_t *)(paddr + 4)`). The operator's blog calls this "gehashed"; the paper should call it a **16-bit truncation**, which is weaker than a hash and has a small ID space.
- The operator's blog statement that MACs are "hashed and stored for at most one hour" is **not** what this firmware does literally: the bitmap is reset at each wake (`libpax_counter_init` calls `libpax_counter_reset`), and the device deep-sleeps after each ~10 s scan. The blog may describe an earlier version or the intended privacy policy. The paper should report both and flag the discrepancy.
### 1.4 The CSV is a raw export, not an aggregate
The aggregator (`backend/Aggregator/main.go`) parses the payload as `sensor_id,type,value,...`, maps type 0 to `densityData`, and posts the value to the backend. The backend stores it in `sensor_data(time, sensor_id, name, longitude, latitude, type, value)` with no time-bucket aggregation in the schema. The challenge CSV has the same columns and ~15-minute spacing. The simplest reading is that the CSV is a direct export of these per-wake values, not a 15-minute mean. This should be verified with the data provider, but the code strongly supports it.
### 1.5 What this changes
The measured `value` is a **~10-second sample**, scaled by a **fixed, undocumented factor**, from a sensor that sees only **randomized BLE addresses**, hashed into **16 bits**, with an **RSSI cut-off at -80 dBm**, repeated every **15 minutes**. That is a chain of six lossy transformations before the number reaches the CSV. The paper's reliability critique is not speculative; it is a property of the deployed design.
---
## 2. What the upstream reference code counts
### 2.1 BLE path
`libpax/lib/libpax/blescan.cpp` starts a passive BLE scan:
```c
uint8_t scan_type = 0x00; // passive scan, since we don't need / want answers
...
uint16_t scan_interval = BLESCANINTERVAL * 1000 / 625;
uint16_t scan_window = BLESCANWINDOW * 1000 / 625;
...
uint8_t filter_duplicates = 0x00; // duplicate filtering disabled
```
Defaults in `libpax_api.cpp` and `shared/paxcounter_orig.conf`:
```c
#define BLESCANWINDOW 80 // milliseconds
#define BLESCANINTERVAL 80 // milliseconds (100% duty cycle)
#define SENDCYCLE 30 // payload send cycle [seconds/2] => 60 s
#define BLECOUNTER 0 // upstream default: BLE sniffer off
#define WIFICOUNTER 1 // upstream default: Wi-Fi sniffer on
```
The BLE scanner is **passive**: it listens for advertising reports and never sends scan requests. With an 80 ms window every 80 ms, it is effectively continuous. Every advertising report above the RSSI threshold is passed to `mac_add()`.
### 2.2 The `mac_add()` filter is the key
`libpax/lib/libpax/libpax.cpp`:
```c
IRAM_ATTR int mac_add(uint8_t *paddr, snifftype_t sniff_type) {
// Check locally administered bit first (cheapest check)
if (!(paddr[0] & 0b10)) return 0;
// Use last 2 bytes of MAC address as ID (little-endian)
uint16_t id = (paddr[5] << 8) | paddr[4];
int added = add_to_bucket(id, sniff_type);
...
}
```
Interpretation:
- Bit 1 of the first MAC octet is the **locally administered** bit.
- `if (!(paddr[0] & 0b10)) return 0;` returns early when the bit is **not** set, i.e. it **ignores universally administered (burned-in/static) MAC addresses** and only counts **locally administered (randomized) addresses**.
- The ID is not the MAC address. It is a 16-bit value made from the **last two bytes** of the address.
- The ID is stored in a 65,536-bit bitmap (`seen_ids_map_ble`), so a device is counted at most once per bitmap epoch.
- The bitmap is cleared every report cycle (`libpax_counter_reset()`), which defaults to 60 seconds.
This is a privacy choice: the code does not persist MAC addresses and does not use the full address. It is also a measurement choice with large consequences: the count is **not** a count of all nearby devices, and it is **not** a count of people.
The upstream documentation says:
> "Identifiers are generated by using the last 2 bytes of universal MAC adresses. Personal MAC adresses remain untouched and are not evaluated." — <https://paxcounter.org/legalnote/>
The code's bit test appears to do the opposite of the documentation's wording: it counts locally administered addresses and skips universal ones. This may be a documentation lag, a byte-order subtlety, or a change in the implementation. The paper should report the code behavior and flag the discrepancy, not silently repeat the documentation.
### 2.3 Wi-Fi path
`libpax/lib/libpax/wifiscan.cpp` uses ESP32 promiscuous mode and captures management and data frames:
```c
const wifi_ieee80211_packet_t* const ipkt =
(wifi_ieee80211_packet_t*)ppkt->payload;
mac_add((uint8_t*)ipkt->hdr.addr2, MAC_SNIFF_WIFI);
```
It hops channels on a timer. Upstream default channel map is 1/3/6/9/11 (or all channels in some configs), with a 0.5 s switch interval. The default build config enables Wi-Fi counting and disables BLE counting; a deployment that follows the challenge brief ("Bluetooth signals") must have changed this. The CSV does not expose the Wi-Fi/BLE split.
**Paper implication.** The public description says "Bluetooth"; the reference implementation defaults to Wi-Fi and supports both. The delivered CSV has only `densityData` and `batteryData`, not separate `wifi`/`ble` counts. The unit of `value` is therefore ambiguous: it may be `pax = wifi + ble`, BLE-only, or Wi-Fi-only depending on the deployment. This is an open provenance question, not a detail to guess.
---
## 3. Failure modes visible in the code
### 2.1 Only randomized addresses are counted
If the deployed firmware uses the current `mac_add()` logic, devices that transmit a **universal/static** MAC are invisible. Modern phones and watches usually use resolvable private addresses (randomized) for BLE, so this may be intentional: the counter counts the privacy-preserving addresses, not the stable identity. But the consequence is that the population is not "all devices"; it is "devices currently using a locally administered address that the scanner hears." Devices with static addresses, some IoT devices, and some older hardware are excluded. The count is a **sample of a particular address regime**, not a device census.
### 2.2 A 16-bit ID space collides
The bitmap has 2^16 = 65,536 slots. With `n` distinct IDs in one 60 s epoch, the expected number of colliding pairs is approximately:
```text
E[collisions] ≈ n(n - 1) / (2 × 65,536)
```
Examples (assuming uniform hashing, which is an approximation):
| Distinct IDs in 60 s | Expected colliding pairs | Approx. probability of at least one collision |
|---:|---:|---:|
| 50 | 0.019 | ~1.9% |
| 100 | 0.076 | ~7.3% |
| 200 | 0.304 | ~26% |
| 300 | 0.685 | ~50% |
| 500 | 1.90 | ~85% |
At a busy inner-city corner, hundreds of devices in a minute are plausible. Collisions cause **undercounting**, and the error grows quadratically with density. The 16-bit hash is a deliberate privacy/space trade-off, but it makes the counter non-linear at exactly the moments a staffing decision cares about.
### 2.3 The counting window is not a 15 min count
The upstream report timer resets the bitmap every `SENDCYCLE` (default 60 s). The operator's blog says the deployed firmware hashes MACs and stores them for **at most one hour**, which suggests a one-hour dedup cache rather than a 60-second bitmap. Either way, the CSV has a ~15 min cadence (median 15.3 min), and the aggregation rule is undocumented. Plausible rules:
- mean of 15 one-minute unique counts,
- maximum of 15 one-minute counts,
- sum of 15 one-minute counts,
- a count over a different window entirely.
These are not equivalent. A mean of per-minute unique counts is not the number of unique devices in 15 minutes; a sum double-counts devices that persist across minutes. The challenge's own open question asks what the unit of `densityData` is (see `OKF_Vault/mission/data.md`). The paper must state that the unit is undocumented and that this limits absolute-count claims.
### 2.4 Queue drops under density
`blescan.cpp`:
```c
#define BLE_ADV_QUEUE_SIZE 60
...
if (xQueueSend(adv_queue, (void *)&send_data, (TickType_t)0) != pdTRUE) {
ble_adv_dropped = ble_adv_dropped + 1;
ESP_LOGD(TAG, "Failed to enqueue advertising report. Queue full.");
}
```
The code counts dropped advertising reports in `ble_adv_dropped`, but the variable is not part of the normal payload and is not in the CSV. Under high device density, reports can be dropped before `mac_add()` ever sees them. The code comments explicitly raise the queue from 30 to 60 "to give more headroom under high device density, where accurate counting matters most." The deployed data cannot tell us how many reports were dropped.
### 2.5 RSSI threshold and physical layer
Both BLE and Wi-Fi paths apply an RSSI filter (`ble_rssi_threshold`, `wifi_rssi_threshold`). Upstream default is 0 (disabled), but a deployment can set a threshold. Range, antenna orientation, body shielding, enclosure, mounting height, and crowding all change what is heard. A person behind a wall or in a crowd may be invisible; a person near the sensor may be counted once per 60 s. The count is a **detection process**, not a sample of a well-defined catchment area.
### 2.6 Clock and time sync
The device can sync time from LoRaWAN, GPS, or an RTC. If time sync fails, the 60 s report cycle still runs, but the timestamp attached by the backend may drift. The CSV has UTC timestamps and a 15 min cadence; the paper should not assume perfect clock alignment across sensors. The `mission/data.md` notes DST handling as a preprocessing requirement.
### 2.7 LoRaWAN delivery
The payload is sent over LoRaWAN. The CSV is the backend's view, not the device's view. Lost uplinks, network congestion, and retransmissions can remove readings. The 4–6 day gaps in the 8-sensor network are consistent with power, maintenance, or LoRaWAN coverage problems; the challenge package does not document the cause. The paper should treat gaps as **missing not at random** until proven otherwise.
---
## 3. Privacy reading
The reference implementation's privacy disclosure is real but narrow:
- It does not persistently store MAC addresses.
- It does not transfer MAC addresses over LoRaWAN.
- It stores a 16-bit ID derived from the last two bytes of the address, in RAM, for one report cycle.
- It does no fingerprinting beyond the address hash.
This is better than storing raw MACs. It is not a formal anonymization guarantee:
- The raw MAC is observable at the sensor before hashing.
- A 16-bit ID space is small; collisions are both a counting error and a privacy ambiguity.
- If an adversary can observe the same device at the sensor and correlate the 16-bit ID with another source, the "anonymous" count can become linkable.
- The code's legal note warns that sniffing MAC addresses may be illegal depending on jurisdiction and cites the Dutch DPA's Wi-Fi tracking enforcement (see the legal research note).
- The challenge's "privacy-preserving" label describes the **aggregation architecture**, not a legal or mathematical guarantee. The paper should say exactly that.
---
## 4. What this means for the paper
### 4.1 Claims the code supports
- The reference design counts **unique locally administered MAC hashes per scan cycle**, not people.
- The upstream implementation uses a **16-bit hash**, so collisions are expected at high density; the deployed hash length is unknown.
- It **does not persist raw MACs**, which is a meaningful privacy-by-design choice; the operator says the deployed cache is capped at one hour.
- It has **no exposed drop counter** in the delivered data, so undercount from queue overflow cannot be quantified.
- The upstream default config is **Wi-Fi on, BLE off**; the challenge's "Bluetooth" description implies a deployment-specific configuration.
### 4.2 Claims the code does not support
- "The sensor counts all nearby smartphones."
- "The count is a census of people."
- "The count is anonymous in a formal sense."
- "The count is linear in crowd density."
- "The count is comparable across sensors without calibration."
### 4.3 A useful paper table
| Property | Reference implementation | Consequence |
|---|---|---|
| Radio | BLE passive scan; Wi-Fi promiscuous mode | Detection depends on address type and channel |
| Address filter | Locally administered bit only | Static/universal devices excluded |
| ID | Last 2 bytes, 16-bit | Collisions grow with density |
| Dedup window | Upstream: 60 s bitmap reset; operator: hashed cache up to 1 h | Count is not a 15 min count; exact window undocumented |
| Hash length | Upstream: 16-bit ID; deployed: unknown hash | Collision rate depends on the deployed hash |
| Aggregation to CSV | Undocumented | Unit ambiguity; mean/max/sum differ |
| Drop visibility | Internal counter, not in payload | Undercount cannot be corrected |
| Raw MAC storage | None persistent; RAM only | Privacy-by-design, not formal anonymity |
| Legal status | Upstream warns may be illegal | Requires jurisdiction-specific analysis |
### 4.4 Wording for the paper
> The reference PAX counter is a privacy-by-design device: it hashes the last two bytes of locally administered MAC addresses into a 16-bit bitmap and clears the bitmap every 60 seconds. That design avoids persistent MAC storage, but it also means the published count is not a device census. It excludes universally administered addresses, collides at high density, and depends on an undocumented aggregation from 60-second scan cycles to 15-minute CSV rows. The challenge's "privacy-preserving" label is therefore an architectural claim, not a measurement guarantee.
---
## 5. Open questions to resolve before submission
1. Which firmware version and build flags did Code for Heilbronn deploy? BLE only, Wi-Fi only, or both?
2. What is the exact `SENDCYCLE`, `BLESCANWINDOW`, `BLESCANINTERVAL`, RSSI threshold, and channel map?
3. How does the backend aggregate 60 s counts into the 15 min `value`? Mean, max, sum, or something else?
4. Is `value` `pax`, `ble`, or `wifi`? The CSV does not say.
5. What is the sensor mounting height, antenna, enclosure, and catchment area?
6. Were the 4–6 day gaps caused by power, LoRaWAN, maintenance, or something else?
7. Can the deployment expose `ble_adv_dropped` or a health metric to estimate undercount?
8. Is there any ground-truth count (manual, camera, POS) for even a few hours?
Until these are answered, the paper should present the code audit as evidence of **structural limitations**, and the empirical data analysis as evidence of **observed instability**, not as a calibrated error rate.
---
## 6. Sources
- Schwarz Digits / Code for Heilbronn, *The Pulse of the City*, challenge brief, 2026. Local: `OKF_Vault/Schwarz Digits/FCH Challenge The _Pulse of the City_.pdf`.
- Code for Heilbronn, "Passantenzahlen messen mit LORA", 18 Sep 2025, <https://codeforheilbronn.de/blog/2025/09/passantenzahlen-messen-mit-lora/>.
- ESP32-Paxcounter v3.8.0, <https://github.com/cyberman54/ESP32-Paxcounter>; `platformio_orig.ini`, `shared/paxcounter_orig.conf`, `src/senddata.cpp`.
- libpax, <https://github.com/dbinfrago/libpax>; `lib/libpax/blescan.cpp`, `lib/libpax/wifiscan.cpp`, `lib/libpax/libpax.cpp`, `lib/libpax/libpax_api.cpp`.
- Paxcounter legal note and privacy disclosure, <https://paxcounter.org/legalnote/>.
- Paxcounter payload format, <https://paxcounter.org/payloadformat/>.
- `OKF_Vault/mission/data.md` for the delivered CSV coverage and the open unit question.
Binary file not shown.

After

Width:  |  Height:  |  Size: 134 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 232 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 112 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 95 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 249 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 63 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 326 KiB

@@ -0,0 +1,5 @@
bin_minutes,expected_bins,bins_with_any,all8_bins,all8_pct,mean_pairwise_r,median_cv
15,13728,13659,3876,28.234265734265733,0.5820317643141067,0.6614114460889224
30,6864,6854,4350,63.37412587412587,0.599387864124681,0.6446924498066315
60,3432,3429,2393,69.72610722610723,0.6166113899636757,0.6238904213712391
120,1716,1715,1218,70.97902097902097,0.6298106340884699,0.6040265827688449
1 bin_minutes expected_bins bins_with_any all8_bins all8_pct mean_pairwise_r median_cv
2 15 13728 13659 3876 28.234265734265733 0.5820317643141067 0.6614114460889224
3 30 6864 6854 4350 63.37412587412587 0.599387864124681 0.6446924498066315
4 60 3432 3429 2393 69.72610722610723 0.6166113899636757 0.6238904213712391
5 120 1716 1715 1218 70.97902097902097 0.6298106340884699 0.6040265827688449
@@ -0,0 +1,9 @@
location,aug_coverage_pct,aug_battery_median,aug_battery_p10,aug_battery_p90
density-01,84.84543010752688,41.0,36.0,44.0
hn-goetzentuem-01,51.982526881720425,8.0,4.0,8.0
hn-hagenbuchersee-01,94.08602150537635,70.0,68.0,74.0
hn-hagenbuchersee-02,49.05913978494624,4.0,2.0,4.0
hn-marahaus-01,28.864247311827956,10.0,8.0,12.0
hn-neckarufer-01,46.94220430107527,9.0,6.0,10.0
hn-soleo-01,54.56989247311827,4.0,2.0,46.0
hn-stadtgalerie-01,23.31989247311828,8.0,4.0,8.0
1 location aug_coverage_pct aug_battery_median aug_battery_p10 aug_battery_p90
2 density-01 84.84543010752688 41.0 36.0 44.0
3 hn-goetzentuem-01 51.982526881720425 8.0 4.0 8.0
4 hn-hagenbuchersee-01 94.08602150537635 70.0 68.0 74.0
5 hn-hagenbuchersee-02 49.05913978494624 4.0 2.0 4.0
6 hn-marahaus-01 28.864247311827956 10.0 8.0 12.0
7 hn-neckarufer-01 46.94220430107527 9.0 6.0 10.0
8 hn-soleo-01 54.56989247311827 4.0 2.0 46.0
9 hn-stadtgalerie-01 23.31989247311828 8.0 4.0 8.0
+13
View File
@@ -0,0 +1,13 @@
sensor_id,name,n,battery_n,battery_missing,first,last,min,p05,median,mean,max,share_lt10,share_lt20,share_zero,stale_share,max_stale_run,max_stale_run_days,n_resets_gt10,max_drop,corr_density
00091a5f,hn-goetzentuem-01,9952,9952,0,99.0,0.0,0.0,0.0,4.0,6.046523311897106,99.0,0.7470860128617364,0.9985932475884244,0.0947548231511254,0.8906752411575563,1103,11.489583333333334,0,-82.0,0.19474412100982358
062fe216,niklas-01,8,8,0,95.0,67.0,47.0,54.0,95.5,86.75,100.0,0.0,0.0,0.0,0.0,8,0.08333333333333333,1,-51.0,0.1806649295212645
0f8519cf,hn-hagenbuchersee-02,10099,10099,0,92.0,27.0,2.0,3.0,8.0,18.749579166254083,92.0,0.5265867907713635,0.5265867907713635,0.0,0.8477076938310724,1831,19.072916666666668,2,-84.0,-0.13934563219980173
2aadbeed,maker-fair,73,73,0,100.0,100.0,0.0,89.4,98.0,94.3013698630137,100.0,0.0273972602739726,0.0273972602739726,0.0136986301369863,0.1917808219178082,17,0.17708333333333334,2,-98.0,0.5299264598388426
863f75b0,density-01,8075,8075,0,44.9334,69.4334,0.0,5.1889,31.8667,30.79530138699691,69.4334,0.05609907120743034,0.07331269349845201,0.009411764705882352,0.5226006191950464,44,0.4583333333333333,4,-5.988900000000001,-0.02000437007310333
8d7eef22,hn-marahaus-01,9254,9254,0,52.0,87.0,1.0,2.0,7.0,9.986276204884375,91.0,0.549492111519343,0.8451480440890425,0.0,0.8107845256105468,1038,10.8125,2,-48.0,0.20301922389918353
ba71377f,hn-soleo-01,10327,10327,0,94.0,26.0,1.0,3.0,8.0,19.34511474774862,94.0,0.5078919337658565,0.5078919337658565,0.0,0.8310254672218457,960,10.0,1,-86.0,0.42256702455879164
d3304b39,hn-bollwerksturm-01,5,5,0,100.0,0.0,0.0,0.0,8.0,41.4,100.0,0.6,0.6,0.4,0.2,4,0.041666666666666664,0,-91.0,-0.8438257093687824
d4cce74b,hn-neckarufer-01,9527,9527,0,100.0,11.0,1.0,2.0,8.0,8.652356460585704,100.0,0.5469717644589063,0.8938805500157447,0.0,0.8404534480948882,1075,11.197916666666666,1,-78.0,0.20098294808864203
e52c55f3,hn-hagenbuchersee-01,12345,12345,0,92.0,43.0,4.0,5.0,57.0,52.250384771162416,92.0,0.1610368570271365,0.1610368570271365,0.0,0.7402187120291616,349,3.6354166666666665,1,-84.0,-0.1279408792404342
f4928534,hn-stadtgalerie-01,8982,8982,0,99.0,75.0,0.0,0.0,4.0,8.651636606546425,99.0,0.720663549320864,0.9305277221108884,0.05900690269427744,0.8847695390781564,1117,11.635416666666666,1,-79.0,0.04186791960806407
f8c19730,density-01,17585,17585,0,89.0,11.0,11.0,14.0,46.0,49.6245663918112,91.0,0.0,0.09240830253056583,0.0,0.7030423656525447,351,3.65625,0,-6.0,0.03483563721394326
1 sensor_id name n battery_n battery_missing first last min p05 median mean max share_lt10 share_lt20 share_zero stale_share max_stale_run max_stale_run_days n_resets_gt10 max_drop corr_density
2 00091a5f hn-goetzentuem-01 9952 9952 0 99.0 0.0 0.0 0.0 4.0 6.046523311897106 99.0 0.7470860128617364 0.9985932475884244 0.0947548231511254 0.8906752411575563 1103 11.489583333333334 0 -82.0 0.19474412100982358
3 062fe216 niklas-01 8 8 0 95.0 67.0 47.0 54.0 95.5 86.75 100.0 0.0 0.0 0.0 0.0 8 0.08333333333333333 1 -51.0 0.1806649295212645
4 0f8519cf hn-hagenbuchersee-02 10099 10099 0 92.0 27.0 2.0 3.0 8.0 18.749579166254083 92.0 0.5265867907713635 0.5265867907713635 0.0 0.8477076938310724 1831 19.072916666666668 2 -84.0 -0.13934563219980173
5 2aadbeed maker-fair 73 73 0 100.0 100.0 0.0 89.4 98.0 94.3013698630137 100.0 0.0273972602739726 0.0273972602739726 0.0136986301369863 0.1917808219178082 17 0.17708333333333334 2 -98.0 0.5299264598388426
6 863f75b0 density-01 8075 8075 0 44.9334 69.4334 0.0 5.1889 31.8667 30.79530138699691 69.4334 0.05609907120743034 0.07331269349845201 0.009411764705882352 0.5226006191950464 44 0.4583333333333333 4 -5.988900000000001 -0.02000437007310333
7 8d7eef22 hn-marahaus-01 9254 9254 0 52.0 87.0 1.0 2.0 7.0 9.986276204884375 91.0 0.549492111519343 0.8451480440890425 0.0 0.8107845256105468 1038 10.8125 2 -48.0 0.20301922389918353
8 ba71377f hn-soleo-01 10327 10327 0 94.0 26.0 1.0 3.0 8.0 19.34511474774862 94.0 0.5078919337658565 0.5078919337658565 0.0 0.8310254672218457 960 10.0 1 -86.0 0.42256702455879164
9 d3304b39 hn-bollwerksturm-01 5 5 0 100.0 0.0 0.0 0.0 8.0 41.4 100.0 0.6 0.6 0.4 0.2 4 0.041666666666666664 0 -91.0 -0.8438257093687824
10 d4cce74b hn-neckarufer-01 9527 9527 0 100.0 11.0 1.0 2.0 8.0 8.652356460585704 100.0 0.5469717644589063 0.8938805500157447 0.0 0.8404534480948882 1075 11.197916666666666 1 -78.0 0.20098294808864203
11 e52c55f3 hn-hagenbuchersee-01 12345 12345 0 92.0 43.0 4.0 5.0 57.0 52.250384771162416 92.0 0.1610368570271365 0.1610368570271365 0.0 0.7402187120291616 349 3.6354166666666665 1 -84.0 -0.1279408792404342
12 f4928534 hn-stadtgalerie-01 8982 8982 0 99.0 75.0 0.0 0.0 4.0 8.651636606546425 99.0 0.720663549320864 0.9305277221108884 0.05900690269427744 0.8847695390781564 1117 11.635416666666666 1 -79.0 0.04186791960806407
13 f8c19730 density-01 17585 17585 0 89.0 11.0 11.0 14.0 46.0 49.6245663918112 91.0 0.0 0.09240830253056583 0.0 0.7030423656525447 351 3.65625 0 -6.0 0.03483563721394326
+9
View File
@@ -0,0 +1,9 @@
sensor_id,days,corr_battery_readings,corr_battery_density,median_readings,median_battery
00091a5f,142,0.019973340974513105,0.3591425899083509,83.0,4.0
0f8519cf,138,-0.1619976874693108,-0.27156362070695567,85.0,17.5
8d7eef22,127,-0.05521993942391102,0.3929069212649879,84.0,8.0
ba71377f,142,-0.04087317991800462,0.6967070266222465,85.0,17.0
d4cce74b,142,0.084123819465874,0.3848942442923485,82.0,8.0
e52c55f3,143,-0.020004451679533346,-0.34620169674910267,93.0,59.0
f4928534,133,0.07331223332341452,0.09426247125533256,80.0,4.0
f8c19730,143,-0.4125027055019088,0.20959385602454614,86.0,36.0
1 sensor_id days corr_battery_readings corr_battery_density median_readings median_battery
2 00091a5f 142 0.019973340974513105 0.3591425899083509 83.0 4.0
3 0f8519cf 138 -0.1619976874693108 -0.27156362070695567 85.0 17.5
4 8d7eef22 127 -0.05521993942391102 0.3929069212649879 84.0 8.0
5 ba71377f 142 -0.04087317991800462 0.6967070266222465 85.0 17.0
6 d4cce74b 142 0.084123819465874 0.3848942442923485 82.0 8.0
7 e52c55f3 143 -0.020004451679533346 -0.34620169674910267 93.0 59.0
8 f4928534 133 0.07331223332341452 0.09426247125533256 80.0 4.0
9 f8c19730 143 -0.4125027055019088 0.20959385602454614 86.0 36.0
+10
View File
@@ -0,0 +1,10 @@
sensor_id,<5m,5-10m,10-14m,14-17m,17-20m,20-28m,28-34m,34-40m,40-60m,1-6h,6-24h,>24h
00091a5f,0,2,1,8722,1,3,900,1,143,154,19,5
0f8519cf,1,0,0,9128,0,1,775,0,73,89,25,6
863f75b0,106,0,5,5862,271,127,937,46,303,378,36,3
8d7eef22,1,0,1,8352,0,0,761,0,64,53,11,10
ba71377f,0,0,1,9327,0,0,809,0,72,90,22,5
d4cce74b,1,0,2,8297,1,0,940,0,108,144,28,5
e52c55f3,0,0,0,12028,0,0,248,0,31,27,3,7
f4928534,6,0,1,7515,2,0,1138,1,168,128,17,5
f8c19730,0,0,0,16985,0,0,534,0,50,15,0,0
1 sensor_id <5m 5-10m 10-14m 14-17m 17-20m 20-28m 28-34m 34-40m 40-60m 1-6h 6-24h >24h
2 00091a5f 0 2 1 8722 1 3 900 1 143 154 19 5
3 0f8519cf 1 0 0 9128 0 1 775 0 73 89 25 6
4 863f75b0 106 0 5 5862 271 127 937 46 303 378 36 3
5 8d7eef22 1 0 1 8352 0 0 761 0 64 53 11 10
6 ba71377f 0 0 1 9327 0 0 809 0 72 90 22 5
7 d4cce74b 1 0 2 8297 1 0 940 0 108 144 28 5
8 e52c55f3 0 0 0 12028 0 0 248 0 31 27 3 7
9 f4928534 6 0 1 7515 2 0 1138 1 168 128 17 5
10 f8c19730 0 0 0 16985 0 0 534 0 50 15 0 0
+10
View File
@@ -0,0 +1,10 @@
sensor_id,name,n,median_s,p05_s,p95_s,median_min,p05_min,p25_min,p75_min,p95_min,share_14_17min,share_28_34min,max_gap_h
00091a5f,hn-goetzentuem-01,9952,921.604873,915.4190644,1833.0257384000001,15.360081216666666,15.256984406666668,15.270784066666666,15.384300416666667,30.550428973333336,0.8924588151028343,0.09209045328967563,124.14629634222223
0f8519cf,hn-hagenbuchersee-02,10099,917.89295,915.407152,1833.0765724999999,15.298215833333334,15.256785866666666,15.271158575,15.315042404166666,30.55127620833333,0.9148125876929244,0.07767087592703949,101.52013999083333
863f75b0,density-01,8075,911.494487,903.0216793999999,1951.8050363999998,15.191574783333333,15.050361323333332,15.168427366666666,16.093452816666666,32.53008394,0.7655739845892647,0.12237168603891864,58.654426167500006
8d7eef22,hn-marahaus-01,9254,916.670523,915.304101,1832.6957782999998,15.27784205,15.255068349999998,15.268433850000001,15.375232366666667,30.54492963833333,0.9099030395467916,0.08290663470966336,152.32702305222222
ba71377f,hn-soleo-01,10327,916.778465,915.2240456,1832.4121752,15.279641083333333,15.253734093333332,15.267592400000002,15.375628616666667,30.540202920000002,0.9136056420805172,0.07924380448623763,102.40406208583333
d4cce74b,hn-neckarufer-01,9527,918.311709,915.5379656,1835.9932654,15.30519515,15.258966093333333,15.277160083333333,15.377997883333334,30.599887756666664,0.8874745962134988,0.10054551288907905,124.21339694944444
e52c55f3,hn-hagenbuchersee-01,12345,916.104139,914.6834951,924.4434573999999,15.268402316666668,15.244724918333333,15.260342125,15.282031316666666,15.407390956666665,0.9773299748110831,0.020151133501259445,102.12467683444444
f4928534,hn-stadtgalerie-01,8982,922.188528,915.630675,1844.618643,15.3698088,15.26051125,15.280674108333333,15.478208791666669,30.74364405,0.8509795040199298,0.12886422828671726,141.77853083055555
f8c19730,density-01,17585,968.545989,967.2141428,975.5509224,16.14243315,16.120235713333333,16.1339909,16.15156115,16.25918204,0.9667596334452729,0.030394444760657977,4.5767124775
1 sensor_id name n median_s p05_s p95_s median_min p05_min p25_min p75_min p95_min share_14_17min share_28_34min max_gap_h
2 00091a5f hn-goetzentuem-01 9952 921.604873 915.4190644 1833.0257384000001 15.360081216666666 15.256984406666668 15.270784066666666 15.384300416666667 30.550428973333336 0.8924588151028343 0.09209045328967563 124.14629634222223
3 0f8519cf hn-hagenbuchersee-02 10099 917.89295 915.407152 1833.0765724999999 15.298215833333334 15.256785866666666 15.271158575 15.315042404166666 30.55127620833333 0.9148125876929244 0.07767087592703949 101.52013999083333
4 863f75b0 density-01 8075 911.494487 903.0216793999999 1951.8050363999998 15.191574783333333 15.050361323333332 15.168427366666666 16.093452816666666 32.53008394 0.7655739845892647 0.12237168603891864 58.654426167500006
5 8d7eef22 hn-marahaus-01 9254 916.670523 915.304101 1832.6957782999998 15.27784205 15.255068349999998 15.268433850000001 15.375232366666667 30.54492963833333 0.9099030395467916 0.08290663470966336 152.32702305222222
6 ba71377f hn-soleo-01 10327 916.778465 915.2240456 1832.4121752 15.279641083333333 15.253734093333332 15.267592400000002 15.375628616666667 30.540202920000002 0.9136056420805172 0.07924380448623763 102.40406208583333
7 d4cce74b hn-neckarufer-01 9527 918.311709 915.5379656 1835.9932654 15.30519515 15.258966093333333 15.277160083333333 15.377997883333334 30.599887756666664 0.8874745962134988 0.10054551288907905 124.21339694944444
8 e52c55f3 hn-hagenbuchersee-01 12345 916.104139 914.6834951 924.4434573999999 15.268402316666668 15.244724918333333 15.260342125 15.282031316666666 15.407390956666665 0.9773299748110831 0.020151133501259445 102.12467683444444
9 f4928534 hn-stadtgalerie-01 8982 922.188528 915.630675 1844.618643 15.3698088 15.26051125 15.280674108333333 15.478208791666669 30.74364405 0.8509795040199298 0.12886422828671726 141.77853083055555
10 f8c19730 density-01 17585 968.545989 967.2141428 975.5509224 16.14243315 16.120235713333333 16.1339909 16.15156115 16.25918204 0.9667596334452729 0.030394444760657977 4.5767124775
+10
View File
@@ -0,0 +1,10 @@
sensor_id,name,kind,n_intervals,median_all_s,mean_all_s,sd_all_s,median_near_s,mean_near_s,sd_near_s,iqr_low_s,iqr_high_s,p05_near_s,p95_near_s,share_900_5s_pct,share_900_30s_pct,share_900_60s_pct,share_14_17min_pct
00091a5f,hn-goetzentuem-01,current,9951,921.782886,1289.0723037585167,5446.192039143962,917.2627275,919.9161463750288,4.964963449069862,916.1766015000001,922.6046269999999,915.3619444,929.02304175,0.11465260261407935,97.48910800275166,99.98853473973858,87.64948246407397
0f8519cf,hn-hagenbuchersee-02,current,10098,917.9281625000001,1270.3465660968507,5705.008905844744,917.571286,918.158909025745,3.7197107257700033,916.21645025,918.51619225,915.34253135,923.12839875,0.14241893076248904,98.97020157756354,99.98904469763366,90.39413745296098
863f75b0,density-01,legacy,8074,911.9082165,1571.7868114988853,4227.889156365839,910.859239,916.3502143522688,18.0341892874005,909.84656225,917.24860225,903.1048513000001,958.1886803499999,12.589559877175024,89.11634254520642,95.27465029000342,72.60341837998514
8d7eef22,hn-marahaus-01,current,9253,916.68365,1386.3180153460498,10250.415193914416,916.5590425,918.3382902341954,4.093862303372867,916.04752,921.90853625,915.23724575,924.26676165,0.14367816091954022,98.77873563218391,99.98802681992338,90.26261752944991
ba71377f,hn-soleo-01,current,10326,916.829096,1242.4168086068178,5052.519543484417,916.536683,919.2410932057466,5.162059772829141,916.008293,922.212844,915.173724,929.0847739,0.053607805296451164,95.9472499195883,99.98927843894072,90.32539221382918
d4cce74b,hn-neckarufer-01,current,9526,918.337582,1346.6827457801808,5675.4683216621515,918.124619,919.0656111636736,3.971172282574465,916.522059,921.838361,915.458807,928.3190741999999,0.13257804025551403,98.73448234301556,100.0,87.09846735250892
e52c55f3,hn-hagenbuchersee-01,current,12344,916.1065775,1039.3286537004212,3949.1071736846056,916.0835645,916.7677576030096,3.4375048157133317,915.6054875,916.7954025,914.67033005,922.10836255,0.35749916860658465,99.27668772863318,99.99168606584637,97.4400518470512
f4928534,hn-stadtgalerie-01,current,8981,922.253235,1428.3143157327693,8812.84129244029,918.627691,920.913361045376,5.482441628648548,916.591935,922.9360875,915.5222092,929.7222238,0.05322687957418496,95.4224883566201,100.0,83.67665070704821
f8c19730,density-01,current,17584,968.5470675,1007.3588255524908,245.21410608924393,968.515136,968.9461660125993,2.407473545387518,968.017988,969.022319,967.1916012,974.4429233999999,0.0,0.0058875478363261706,0.1589637915808066,96.5934940855323
1 sensor_id name kind n_intervals median_all_s mean_all_s sd_all_s median_near_s mean_near_s sd_near_s iqr_low_s iqr_high_s p05_near_s p95_near_s share_900_5s_pct share_900_30s_pct share_900_60s_pct share_14_17min_pct
2 00091a5f hn-goetzentuem-01 current 9951 921.782886 1289.0723037585167 5446.192039143962 917.2627275 919.9161463750288 4.964963449069862 916.1766015000001 922.6046269999999 915.3619444 929.02304175 0.11465260261407935 97.48910800275166 99.98853473973858 87.64948246407397
3 0f8519cf hn-hagenbuchersee-02 current 10098 917.9281625000001 1270.3465660968507 5705.008905844744 917.571286 918.158909025745 3.7197107257700033 916.21645025 918.51619225 915.34253135 923.12839875 0.14241893076248904 98.97020157756354 99.98904469763366 90.39413745296098
4 863f75b0 density-01 legacy 8074 911.9082165 1571.7868114988853 4227.889156365839 910.859239 916.3502143522688 18.0341892874005 909.84656225 917.24860225 903.1048513000001 958.1886803499999 12.589559877175024 89.11634254520642 95.27465029000342 72.60341837998514
5 8d7eef22 hn-marahaus-01 current 9253 916.68365 1386.3180153460498 10250.415193914416 916.5590425 918.3382902341954 4.093862303372867 916.04752 921.90853625 915.23724575 924.26676165 0.14367816091954022 98.77873563218391 99.98802681992338 90.26261752944991
6 ba71377f hn-soleo-01 current 10326 916.829096 1242.4168086068178 5052.519543484417 916.536683 919.2410932057466 5.162059772829141 916.008293 922.212844 915.173724 929.0847739 0.053607805296451164 95.9472499195883 99.98927843894072 90.32539221382918
7 d4cce74b hn-neckarufer-01 current 9526 918.337582 1346.6827457801808 5675.4683216621515 918.124619 919.0656111636736 3.971172282574465 916.522059 921.838361 915.458807 928.3190741999999 0.13257804025551403 98.73448234301556 100.0 87.09846735250892
8 e52c55f3 hn-hagenbuchersee-01 current 12344 916.1065775 1039.3286537004212 3949.1071736846056 916.0835645 916.7677576030096 3.4375048157133317 915.6054875 916.7954025 914.67033005 922.10836255 0.35749916860658465 99.27668772863318 99.99168606584637 97.4400518470512
9 f4928534 hn-stadtgalerie-01 current 8981 922.253235 1428.3143157327693 8812.84129244029 918.627691 920.913361045376 5.482441628648548 916.591935 922.9360875 915.5222092 929.7222238 0.05322687957418496 95.4224883566201 100.0 83.67665070704821
10 f8c19730 density-01 current 17584 968.5470675 1007.3588255524908 245.21410608924393 968.515136 968.9461660125993 2.407473545387518 968.017988 969.022319 967.1916012 974.4429233999999 0.0 0.0058875478363261706 0.1589637915808066 96.5934940855323
+9
View File
@@ -0,0 +1,9 @@
,density-01,hn-goetzentuem-01,hn-hagenbuchersee-01,hn-hagenbuchersee-02,hn-marahaus-01,hn-neckarufer-01,hn-soleo-01,hn-stadtgalerie-01
density-01,1.0,0.4873724449187153,0.5796747808543874,0.6812436263053844,0.7726268916690955,0.7383627222318554,0.049581091516022684,0.37539023342505085
hn-goetzentuem-01,0.4873724449187153,1.0,0.8611299407317917,0.7661526488681863,0.7925289649493704,0.8247220366128696,0.21482136724110018,0.16481429468703024
hn-hagenbuchersee-01,0.5796747808543873,0.8611299407317917,0.9999999999999999,0.9371083479781734,0.8837337772362686,0.9072172200826037,-0.020904512837028842,0.09274374114930728
hn-hagenbuchersee-02,0.6812436263053844,0.7661526488681865,0.9371083479781733,1.0,0.8756117683253667,0.9374415786978159,-0.0327709027985336,0.13348077591878818
hn-marahaus-01,0.7726268916690955,0.7925289649493704,0.8837337772362686,0.8756117683253667,1.0,0.9185101437403713,-0.06344788459084313,0.14142083656956148
hn-neckarufer-01,0.7383627222318554,0.8247220366128695,0.9072172200826037,0.9374415786978159,0.9185101437403713,1.0,-0.0004364394920098867,0.18585993021125707
hn-soleo-01,0.049581091516022684,0.2148213672411002,-0.02090451283702884,-0.0327709027985336,-0.06344788459084313,-0.0004364394920098867,0.9999999999999998,0.1653131327503826
hn-stadtgalerie-01,0.37539023342505085,0.16481429468703024,0.0927437411493073,0.13348077591878818,0.14142083656956148,0.1858599302112571,0.16531313275038256,1.0
1 density-01 hn-goetzentuem-01 hn-hagenbuchersee-01 hn-hagenbuchersee-02 hn-marahaus-01 hn-neckarufer-01 hn-soleo-01 hn-stadtgalerie-01
2 density-01 1.0 0.4873724449187153 0.5796747808543874 0.6812436263053844 0.7726268916690955 0.7383627222318554 0.049581091516022684 0.37539023342505085
3 hn-goetzentuem-01 0.4873724449187153 1.0 0.8611299407317917 0.7661526488681863 0.7925289649493704 0.8247220366128696 0.21482136724110018 0.16481429468703024
4 hn-hagenbuchersee-01 0.5796747808543873 0.8611299407317917 0.9999999999999999 0.9371083479781734 0.8837337772362686 0.9072172200826037 -0.020904512837028842 0.09274374114930728
5 hn-hagenbuchersee-02 0.6812436263053844 0.7661526488681865 0.9371083479781733 1.0 0.8756117683253667 0.9374415786978159 -0.0327709027985336 0.13348077591878818
6 hn-marahaus-01 0.7726268916690955 0.7925289649493704 0.8837337772362686 0.8756117683253667 1.0 0.9185101437403713 -0.06344788459084313 0.14142083656956148
7 hn-neckarufer-01 0.7383627222318554 0.8247220366128695 0.9072172200826037 0.9374415786978159 0.9185101437403713 1.0 -0.0004364394920098867 0.18585993021125707
8 hn-soleo-01 0.049581091516022684 0.2148213672411002 -0.02090451283702884 -0.0327709027985336 -0.06344788459084313 -0.0004364394920098867 0.9999999999999998 0.1653131327503826
9 hn-stadtgalerie-01 0.37539023342505085 0.16481429468703024 0.0927437411493073 0.13348077591878818 0.14142083656956148 0.1858599302112571 0.16531313275038256 1.0
@@ -0,0 +1,9 @@
,density-01,hn-goetzentuem-01,hn-hagenbuchersee-01,hn-hagenbuchersee-02,hn-marahaus-01,hn-neckarufer-01,hn-soleo-01,hn-stadtgalerie-01
density-01,1.0,0.26159987436567617,0.3530815656646536,0.3856303535322388,0.44056032164512365,0.44577014800122533,0.1237113115651267,0.14889467404601534
hn-goetzentuem-01,0.26159987436567617,0.9999999999999999,0.6805872542951652,0.5204886746917748,0.652374284261973,0.6231859582324983,0.340813169159997,0.05096097463096638
hn-hagenbuchersee-01,0.3530815656646536,0.6805872542951653,1.0,0.7308115126237007,0.7112640963971736,0.7725046744065927,0.2911800415356195,0.048993681123086434
hn-hagenbuchersee-02,0.38563035353223873,0.5204886746917748,0.7308115126237007,0.9999999999999999,0.5903074160531134,0.7851102480184055,0.22543651535234185,0.030668399194183717
hn-marahaus-01,0.4405603216451236,0.6523742842619729,0.7112640963971735,0.5903074160531134,1.0,0.7021424672336278,0.2549655979625577,0.0985182597453256
hn-neckarufer-01,0.4457701480012254,0.6231859582324983,0.7725046744065927,0.7851102480184055,0.7021424672336278,1.0,0.3020919932566273,0.08490864561351638
hn-soleo-01,0.1237113115651267,0.340813169159997,0.2911800415356195,0.22543651535234185,0.2549655979625577,0.3020919932566273,1.0,0.11886065015600784
hn-stadtgalerie-01,0.14889467404601534,0.05096097463096638,0.04899368112308644,0.03066839919418372,0.0985182597453256,0.0849086456135164,0.11886065015600783,1.0
1 density-01 hn-goetzentuem-01 hn-hagenbuchersee-01 hn-hagenbuchersee-02 hn-marahaus-01 hn-neckarufer-01 hn-soleo-01 hn-stadtgalerie-01
2 density-01 1.0 0.26159987436567617 0.3530815656646536 0.3856303535322388 0.44056032164512365 0.44577014800122533 0.1237113115651267 0.14889467404601534
3 hn-goetzentuem-01 0.26159987436567617 0.9999999999999999 0.6805872542951652 0.5204886746917748 0.652374284261973 0.6231859582324983 0.340813169159997 0.05096097463096638
4 hn-hagenbuchersee-01 0.3530815656646536 0.6805872542951653 1.0 0.7308115126237007 0.7112640963971736 0.7725046744065927 0.2911800415356195 0.048993681123086434
5 hn-hagenbuchersee-02 0.38563035353223873 0.5204886746917748 0.7308115126237007 0.9999999999999999 0.5903074160531134 0.7851102480184055 0.22543651535234185 0.030668399194183717
6 hn-marahaus-01 0.4405603216451236 0.6523742842619729 0.7112640963971735 0.5903074160531134 1.0 0.7021424672336278 0.2549655979625577 0.0985182597453256
7 hn-neckarufer-01 0.4457701480012254 0.6231859582324983 0.7725046744065927 0.7851102480184055 0.7021424672336278 1.0 0.3020919932566273 0.08490864561351638
8 hn-soleo-01 0.1237113115651267 0.340813169159997 0.2911800415356195 0.22543651535234185 0.2549655979625577 0.3020919932566273 1.0 0.11886065015600784
9 hn-stadtgalerie-01 0.14889467404601534 0.05096097463096638 0.04899368112308644 0.03066839919418372 0.0985182597453256 0.0849086456135164 0.11886065015600783 1.0
+9
View File
@@ -0,0 +1,9 @@
,density-01,hn-goetzentuem-01,hn-hagenbuchersee-01,hn-hagenbuchersee-02,hn-marahaus-01,hn-neckarufer-01,hn-soleo-01,hn-stadtgalerie-01
density-01,1.0,0.778289790845024,0.7891290486342575,0.7796663075783844,0.7960438557301058,0.7865858080675852,0.500899329195123,0.6702324550370826
hn-goetzentuem-01,0.778289790845024,1.0,0.759360907580583,0.6999126285760182,0.7967512924476091,0.7170313966918529,0.5716120555398148,0.7278111796565614
hn-hagenbuchersee-01,0.7891290486342575,0.759360907580583,0.9999999999999998,0.846801348692525,0.8118105747549154,0.8398248873778403,0.45311668692281293,0.5468057356235985
hn-hagenbuchersee-02,0.7796663075783844,0.6999126285760183,0.8468013486925251,1.0,0.7685308867032483,0.862408503604677,0.43540483488616927,0.4698601855370546
hn-marahaus-01,0.7960438557301057,0.7967512924476091,0.8118105747549154,0.7685308867032483,0.9999999999999998,0.7885602819946863,0.4648185052861985,0.657328965364906
hn-neckarufer-01,0.7865858080675853,0.7170313966918529,0.8398248873778402,0.8624085036046769,0.7885602819946864,1.0,0.48539959205638067,0.4811166095232261
hn-soleo-01,0.500899329195123,0.5716120555398148,0.45311668692281293,0.4354048348861692,0.46481850528619856,0.48539959205638067,1.0,0.48015749801450425
hn-stadtgalerie-01,0.6702324550370827,0.7278111796565614,0.5468057356235985,0.46986018553705455,0.6573289653649059,0.4811166095232261,0.4801574980145043,1.0
1 density-01 hn-goetzentuem-01 hn-hagenbuchersee-01 hn-hagenbuchersee-02 hn-marahaus-01 hn-neckarufer-01 hn-soleo-01 hn-stadtgalerie-01
2 density-01 1.0 0.778289790845024 0.7891290486342575 0.7796663075783844 0.7960438557301058 0.7865858080675852 0.500899329195123 0.6702324550370826
3 hn-goetzentuem-01 0.778289790845024 1.0 0.759360907580583 0.6999126285760182 0.7967512924476091 0.7170313966918529 0.5716120555398148 0.7278111796565614
4 hn-hagenbuchersee-01 0.7891290486342575 0.759360907580583 0.9999999999999998 0.846801348692525 0.8118105747549154 0.8398248873778403 0.45311668692281293 0.5468057356235985
5 hn-hagenbuchersee-02 0.7796663075783844 0.6999126285760183 0.8468013486925251 1.0 0.7685308867032483 0.862408503604677 0.43540483488616927 0.4698601855370546
6 hn-marahaus-01 0.7960438557301057 0.7967512924476091 0.8118105747549154 0.7685308867032483 0.9999999999999998 0.7885602819946863 0.4648185052861985 0.657328965364906
7 hn-neckarufer-01 0.7865858080675853 0.7170313966918529 0.8398248873778402 0.8624085036046769 0.7885602819946864 1.0 0.48539959205638067 0.4811166095232261
8 hn-soleo-01 0.500899329195123 0.5716120555398148 0.45311668692281293 0.4354048348861692 0.46481850528619856 0.48539959205638067 1.0 0.48015749801450425
9 hn-stadtgalerie-01 0.6702324550370827 0.7278111796565614 0.5468057356235985 0.46986018553705455 0.6573289653649059 0.4811166095232261 0.4801574980145043 1.0
+9
View File
@@ -0,0 +1,9 @@
,density-01,hn-goetzentuem-01,hn-hagenbuchersee-01,hn-hagenbuchersee-02,hn-marahaus-01,hn-neckarufer-01,hn-soleo-01,hn-stadtgalerie-01
density-01,12139,8898,10827,8864,8286,8507,9109,8011
hn-goetzentuem-01,8898,9926,9459,8271,8055,8259,8628,7487
hn-hagenbuchersee-01,10827,9459,12229,9547,8781,9112,9742,8244
hn-hagenbuchersee-02,8864,8271,9547,9983,8074,8152,8793,7385
hn-marahaus-01,8286,8055,8781,8074,9224,7888,8193,7350
hn-neckarufer-01,8507,8259,9112,8152,7888,9496,8441,7261
hn-soleo-01,9109,8628,9742,8793,8193,8441,10215,7547
hn-stadtgalerie-01,8011,7487,8244,7385,7350,7261,7547,8947
1 density-01 hn-goetzentuem-01 hn-hagenbuchersee-01 hn-hagenbuchersee-02 hn-marahaus-01 hn-neckarufer-01 hn-soleo-01 hn-stadtgalerie-01
2 density-01 12139 8898 10827 8864 8286 8507 9109 8011
3 hn-goetzentuem-01 8898 9926 9459 8271 8055 8259 8628 7487
4 hn-hagenbuchersee-01 10827 9459 12229 9547 8781 9112 9742 8244
5 hn-hagenbuchersee-02 8864 8271 9547 9983 8074 8152 8793 7385
6 hn-marahaus-01 8286 8055 8781 8074 9224 7888 8193 7350
7 hn-neckarufer-01 8507 8259 9112 8152 7888 9496 8441 7261
8 hn-soleo-01 9109 8628 9742 8793 8193 8441 10215 7547
9 hn-stadtgalerie-01 8011 7487 8244 7385 7350 7261 7547 8947
+9
View File
@@ -0,0 +1,9 @@
,density-01,hn-goetzentuem-01,hn-hagenbuchersee-01,hn-hagenbuchersee-02,hn-marahaus-01,hn-neckarufer-01,hn-soleo-01,hn-stadtgalerie-01
density-01,0.9999999999999999,0.5961802455384488,0.6394648008448646,0.6883768724231473,0.7429997368890553,0.7345703704515069,0.4926965186879815,0.4376066153983442
hn-goetzentuem-01,0.5961802455384488,1.0,0.7494925802615117,0.6251991142877771,0.7811625037344821,0.7173984775989862,0.5504883065310853,0.37435496746996616
hn-hagenbuchersee-01,0.6394648008448645,0.7494925802615118,1.0,0.8318335989388819,0.7902702361523158,0.8664628436386933,0.48135903690323684,0.14743612987302893
hn-hagenbuchersee-02,0.6883768724231472,0.6251991142877771,0.8318335989388819,0.9999999999999999,0.7273924631095717,0.9035816465731277,0.4505285642135351,0.13049968301306178
hn-marahaus-01,0.7429997368890553,0.7811625037344823,0.7902702361523158,0.7273924631095717,1.0,0.7996617776661145,0.5292353672250621,0.40546283521231663
hn-neckarufer-01,0.734570370451507,0.7173984775989862,0.8664628436386934,0.9035816465731278,0.7996617776661143,1.0,0.5157827892323491,0.1915068696548477
hn-soleo-01,0.4926965186879815,0.5504883065310853,0.4813590369032369,0.4505285642135351,0.5292353672250621,0.5157827892323491,0.9999999999999999,0.3958844492716839
hn-stadtgalerie-01,0.4376066153983442,0.37435496746996616,0.14743612987302893,0.13049968301306178,0.40546283521231663,0.1915068696548477,0.3958844492716839,0.9999999999999999
1 density-01 hn-goetzentuem-01 hn-hagenbuchersee-01 hn-hagenbuchersee-02 hn-marahaus-01 hn-neckarufer-01 hn-soleo-01 hn-stadtgalerie-01
2 density-01 0.9999999999999999 0.5961802455384488 0.6394648008448646 0.6883768724231473 0.7429997368890553 0.7345703704515069 0.4926965186879815 0.4376066153983442
3 hn-goetzentuem-01 0.5961802455384488 1.0 0.7494925802615117 0.6251991142877771 0.7811625037344821 0.7173984775989862 0.5504883065310853 0.37435496746996616
4 hn-hagenbuchersee-01 0.6394648008448645 0.7494925802615118 1.0 0.8318335989388819 0.7902702361523158 0.8664628436386933 0.48135903690323684 0.14743612987302893
5 hn-hagenbuchersee-02 0.6883768724231472 0.6251991142877771 0.8318335989388819 0.9999999999999999 0.7273924631095717 0.9035816465731277 0.4505285642135351 0.13049968301306178
6 hn-marahaus-01 0.7429997368890553 0.7811625037344823 0.7902702361523158 0.7273924631095717 1.0 0.7996617776661145 0.5292353672250621 0.40546283521231663
7 hn-neckarufer-01 0.734570370451507 0.7173984775989862 0.8664628436386934 0.9035816465731278 0.7996617776661143 1.0 0.5157827892323491 0.1915068696548477
8 hn-soleo-01 0.4926965186879815 0.5504883065310853 0.4813590369032369 0.4505285642135351 0.5292353672250621 0.5157827892323491 0.9999999999999999 0.3958844492716839
9 hn-stadtgalerie-01 0.4376066153983442 0.37435496746996616 0.14743612987302893 0.13049968301306178 0.40546283521231663 0.1915068696548477 0.3958844492716839 0.9999999999999999
@@ -0,0 +1,9 @@
,density-01,hn-goetzentuem-01,hn-hagenbuchersee-01,hn-hagenbuchersee-02,hn-marahaus-01,hn-neckarufer-01,hn-soleo-01,hn-stadtgalerie-01
density-01,1.0,0.7853849582586121,0.8082308555143052,0.7895024116479034,0.8072895202832425,0.806706596005652,0.5067244939915168,0.6657859361662315
hn-goetzentuem-01,0.7853849582586122,1.0,0.7655157904889967,0.7028470065566969,0.8061828548942369,0.7229318951648043,0.5498626047127119,0.7456881355216162
hn-hagenbuchersee-01,0.8082308555143052,0.7655157904889967,1.0,0.8514772525953683,0.8141149748915264,0.8369378319771817,0.4520242207872377,0.5920086980833652
hn-hagenbuchersee-02,0.7895024116479034,0.7028470065566967,0.8514772525953684,1.0,0.7729797284328626,0.8705938725928254,0.4383718655564834,0.490460022077018
hn-marahaus-01,0.8072895202832425,0.8061828548942369,0.8141149748915265,0.7729797284328624,1.0,0.7913812580572691,0.4509950130506562,0.6771596104932338
hn-neckarufer-01,0.806706596005652,0.7229318951648044,0.8369378319771817,0.8705938725928254,0.7913812580572691,0.9999999999999998,0.4841239452686533,0.5192375806931034
hn-soleo-01,0.5067244939915166,0.5498626047127119,0.4520242207872377,0.4383718655564834,0.45099501305065626,0.48412394526865327,1.0,0.4869344704233597
hn-stadtgalerie-01,0.6657859361662315,0.7456881355216162,0.5920086980833652,0.4904600220770179,0.6771596104932338,0.5192375806931033,0.48693447042335963,1.0
1 density-01 hn-goetzentuem-01 hn-hagenbuchersee-01 hn-hagenbuchersee-02 hn-marahaus-01 hn-neckarufer-01 hn-soleo-01 hn-stadtgalerie-01
2 density-01 1.0 0.7853849582586121 0.8082308555143052 0.7895024116479034 0.8072895202832425 0.806706596005652 0.5067244939915168 0.6657859361662315
3 hn-goetzentuem-01 0.7853849582586122 1.0 0.7655157904889967 0.7028470065566969 0.8061828548942369 0.7229318951648043 0.5498626047127119 0.7456881355216162
4 hn-hagenbuchersee-01 0.8082308555143052 0.7655157904889967 1.0 0.8514772525953683 0.8141149748915264 0.8369378319771817 0.4520242207872377 0.5920086980833652
5 hn-hagenbuchersee-02 0.7895024116479034 0.7028470065566967 0.8514772525953684 1.0 0.7729797284328626 0.8705938725928254 0.4383718655564834 0.490460022077018
6 hn-marahaus-01 0.8072895202832425 0.8061828548942369 0.8141149748915265 0.7729797284328624 1.0 0.7913812580572691 0.4509950130506562 0.6771596104932338
7 hn-neckarufer-01 0.806706596005652 0.7229318951648044 0.8369378319771817 0.8705938725928254 0.7913812580572691 0.9999999999999998 0.4841239452686533 0.5192375806931034
8 hn-soleo-01 0.5067244939915166 0.5498626047127119 0.4520242207872377 0.4383718655564834 0.45099501305065626 0.48412394526865327 1.0 0.4869344704233597
9 hn-stadtgalerie-01 0.6657859361662315 0.7456881355216162 0.5920086980833652 0.4904600220770179 0.6771596104932338 0.5192375806931033 0.48693447042335963 1.0
+9
View File
@@ -0,0 +1,9 @@
location,median_density_high_cov_days,median_density_low_cov_days,ratio_low_to_high,days_below_50pct
density-01,9.488888888888889,14.175,1.4938524590163935,1
hn-goetzentuem-01,10.200000000000001,7.833333333333333,0.7679738562091503,32
hn-hagenbuchersee-01,11.977653165854909,4.55,0.37987408192540045,10
hn-hagenbuchersee-02,17.871939321037154,11.1,0.6210853674359856,29
hn-marahaus-01,13.505882352941176,7.811702127659574,0.5783925791385574,40
hn-neckarufer-01,18.05125,17.966666666666665,0.9953142672483438,32
hn-soleo-01,10.2102189244784,7.7,0.7541464151703645,27
hn-stadtgalerie-01,24.12117647058824,12.051931818181817,0.49964112790589393,38
1 location median_density_high_cov_days median_density_low_cov_days ratio_low_to_high days_below_50pct
2 density-01 9.488888888888889 14.175 1.4938524590163935 1
3 hn-goetzentuem-01 10.200000000000001 7.833333333333333 0.7679738562091503 32
4 hn-hagenbuchersee-01 11.977653165854909 4.55 0.37987408192540045 10
5 hn-hagenbuchersee-02 17.871939321037154 11.1 0.6210853674359856 29
6 hn-marahaus-01 13.505882352941176 7.811702127659574 0.5783925791385574 40
7 hn-neckarufer-01 18.05125 17.966666666666665 0.9953142672483438 32
8 hn-soleo-01 10.2102189244784 7.7 0.7541464151703645 27
9 hn-stadtgalerie-01 24.12117647058824 12.051931818181817 0.49964112790589393 38
+13
View File
@@ -0,0 +1,13 @@
sensor_id,name,role,n,first,last,span_days,readings_per_day,expected_slots_active,observed_slots_active,coverage_active_pct,panel_slots,panel_readings_per_day,coverage_panel_pct,median_cadence_min,cadence_ceiling_pct,coverage_vs_ceiling_pct,time_in_gaps_gt1h_pct,time_in_gaps_gt6h_pct,panel_max_gap_h,panel_gaps_gt_1h,panel_gaps_gt_1d,zero_frac,lt1_frac,lt5_frac,mean,median,p95,p99,max,min,max_gap_h,gaps_gt_1h,gaps_gt_6h,gaps_gt_1d,gaps_gt_7d,max_zero_run,zero_runs_ge5
00091a5f,hn-goetzentuem-01,core,9952,2026-06-12 12:27,2026-11-07 23:39,148.4671122071875,67.03168029638694,14254.0,9952.0,69.81899817595061,9926.0,69.41258741258741,72.30477855477855,15.360081216666666,97.65573364106984,74.04048473029981,18.607461249943345,9.71331604822261,42.57558039722222,177.0,4.0,0.0297427652733119,0.11284163987138264,0.3782154340836013,11.12403536977492,8.4,29.4,53.9,210.7,0.0,124.14629634222223,178,24,5,0,9.0,8.0
0f8519cf,hn-hagenbuchersee-02,core,10099,2026-06-12 12:32,2026-11-07 23:51,148.47175491256945,68.01967152571879,14254.0,10098.0,70.84327206398204,9984.0,69.81818181818181,72.72727272727273,15.298215833333334,98.05064958827715,74.17316767676769,19.875976461983615,14.361207058056527,82.32757577416666,119.0,5.0,0.006931379344489554,0.030300029705911478,0.3041885335181701,19.2822655708486,11.2,64.4,96.6140000000003,200.9,0.0,101.52013999083333,120,31,6,0,3.0,0.0
863f75b0,density-01,core,8075,2025-10-16 21:17,2026-03-12 18:28,146.88202217641205,54.97609496621414,14101.0,7967.0,56.49953903978441,0.0,0.0,0.0,15.191574783333333,98.73894058999393,0.0,0.0,0.0,,0.0,0.0,0.2977089783281734,0.2977089783281734,0.7778328173374613,3.0080495356037154,2.0,11.0,16.26000000000022,28.0,0.0,58.654426167500006,417,39,3,0,20.0,126.0
8d7eef22,hn-marahaus-01,core,9254,2026-06-12 12:36,2026-11-07 23:49,148.46759949070602,62.3300978243357,14254.0,9253.0,64.91511154763575,9224.0,64.5034965034965,67.19114219114219,15.27784205,98.18140514157233,68.43571050369076,25.401452488871083,22.417963597643094,152.32702305222222,73.0,9.0,0.0001080613788631943,0.01761400475470067,0.3393127296304301,13.72571860816944,9.1,38.5,58.1,194.6,0.0,152.32702305222222,74,21,10,0,1.0,0.0
ba71377f,hn-soleo-01,core,10327,2026-06-12 11:52,2026-11-07 23:32,148.48606441752315,69.54861414443475,14256.0,10326.0,72.43265993265993,10216.0,71.44055944055944,74.41724941724942,15.279641083333333,98.16984520900587,75.80459076696452,17.830078570011008,11.898643418868168,58.09414319527777,116.0,4.0,0.04773893676769633,0.09663987605306479,0.3212936961363416,10.580323424034086,8.4,28.0,39.2,111.3,0.0,102.40406208583333,117,27,5,0,9.0,17.0
d4cce74b,hn-neckarufer-01,core,9527,2026-06-12 12:21,2026-11-07 23:49,148.47800736460647,64.16438480754427,14255.0,9525.0,66.8186601192564,9498.0,66.41958041958041,69.18706293706293,15.30519515,98.00593754598418,70.59476667380535,22.092291606384357,12.581170542548888,42.011142987499994,176.0,4.0,0.0007347538574577516,0.0050383121654245825,0.23239214863020888,19.200440852314475,11.9,61.6,86.8,255.5,0.0,124.21339694944444,177,33,5,0,1.0,0.0
e52c55f3,hn-hagenbuchersee-01,core,12345,2026-06-12 12:02,2026-11-07 23:46,148.4892696907176,83.13732046573405,14256.0,12345.0,86.59511784511784,12229.0,85.51748251748252,89.08071095571096,15.268402316666668,98.24210607567181,90.67467556843273,7.242715835567211,5.644689397694898,25.026753832222223,36.0,6.0,0.05556905629809639,0.1433778857837181,0.456865127582017,12.973673552045362,6.3,49.7,85.4,292.6,0.0,102.12467683444444,37,10,7,0,14.0,20.0
f4928534,hn-stadtgalerie-01,core,8982,2026-06-12 12:11,2026-11-07 23:26,148.4686443240278,60.497622517499984,14254.0,8975.0,62.9647818156307,8954.0,62.61538461538461,65.22435897435898,15.3698088,97.59392712809804,66.8323951025641,23.449999491129244,16.626900938851655,141.77853083055555,149.0,4.0,0.0,0.09262970385214875,0.4079269650411935,21.13513694054776,8.4,67.2,88.9,137.2,0.7,141.77853083055555,150,22,5,0,0.0,0.0
f8c19730,density-01,core,17585,2026-04-16 23:25,2026-11-07 23:48,205.01617579299767,85.77371971739127,19683.0,17585.0,89.34105573337398,12139.0,84.8881118881119,88.42511655011656,16.14243315,92.92279460361279,95.15976884608098,0.541116472764504,0.0,4.5767124775,13.0,0.0,0.058686380437873185,0.13204435598521466,0.3967586010804663,9.740608473130509,7.7,28.0,38.5,63.0,0.0,4.5767124775,15,0,0,0,16.0,46.0
d3304b39,hn-bollwerksturm-01,episodic,5,2026-06-12 11:19,2026-08-13 16:01,62.195593056273154,,,,,,,,,,,,,,,,0.0,0.0,0.0,10.78,9.1,15.96,16.072000000000003,16.1,6.3,1389.6733469847222,2,2,2,1,,
2aadbeed,maker-fair,episodic,73,2026-08-13 16:07,2026-08-16 15:06,2.9573475597800924,,,,,,,,,,,,,,,,0.0,0.0,0.0,117.83013698630137,127.4,157.78,161.672,165.2,9.1,38.09907913694445,2,2,1,0,,
062fe216,niklas-01,episodic,8,2026-05-01 17:19,2026-05-01 19:06,0.07439584600694445,,,,,,,,,,,,,,,,0.0,0.0,0.75,4.55,4.9,6.3,6.3,6.3,2.1,0.723217388888889,0,0,0,0,,
1 sensor_id name role n first last span_days readings_per_day expected_slots_active observed_slots_active coverage_active_pct panel_slots panel_readings_per_day coverage_panel_pct median_cadence_min cadence_ceiling_pct coverage_vs_ceiling_pct time_in_gaps_gt1h_pct time_in_gaps_gt6h_pct panel_max_gap_h panel_gaps_gt_1h panel_gaps_gt_1d zero_frac lt1_frac lt5_frac mean median p95 p99 max min max_gap_h gaps_gt_1h gaps_gt_6h gaps_gt_1d gaps_gt_7d max_zero_run zero_runs_ge5
2 00091a5f hn-goetzentuem-01 core 9952 2026-06-12 12:27 2026-11-07 23:39 148.4671122071875 67.03168029638694 14254.0 9952.0 69.81899817595061 9926.0 69.41258741258741 72.30477855477855 15.360081216666666 97.65573364106984 74.04048473029981 18.607461249943345 9.71331604822261 42.57558039722222 177.0 4.0 0.0297427652733119 0.11284163987138264 0.3782154340836013 11.12403536977492 8.4 29.4 53.9 210.7 0.0 124.14629634222223 178 24 5 0 9.0 8.0
3 0f8519cf hn-hagenbuchersee-02 core 10099 2026-06-12 12:32 2026-11-07 23:51 148.47175491256945 68.01967152571879 14254.0 10098.0 70.84327206398204 9984.0 69.81818181818181 72.72727272727273 15.298215833333334 98.05064958827715 74.17316767676769 19.875976461983615 14.361207058056527 82.32757577416666 119.0 5.0 0.006931379344489554 0.030300029705911478 0.3041885335181701 19.2822655708486 11.2 64.4 96.6140000000003 200.9 0.0 101.52013999083333 120 31 6 0 3.0 0.0
4 863f75b0 density-01 core 8075 2025-10-16 21:17 2026-03-12 18:28 146.88202217641205 54.97609496621414 14101.0 7967.0 56.49953903978441 0.0 0.0 0.0 15.191574783333333 98.73894058999393 0.0 0.0 0.0 0.0 0.0 0.2977089783281734 0.2977089783281734 0.7778328173374613 3.0080495356037154 2.0 11.0 16.26000000000022 28.0 0.0 58.654426167500006 417 39 3 0 20.0 126.0
5 8d7eef22 hn-marahaus-01 core 9254 2026-06-12 12:36 2026-11-07 23:49 148.46759949070602 62.3300978243357 14254.0 9253.0 64.91511154763575 9224.0 64.5034965034965 67.19114219114219 15.27784205 98.18140514157233 68.43571050369076 25.401452488871083 22.417963597643094 152.32702305222222 73.0 9.0 0.0001080613788631943 0.01761400475470067 0.3393127296304301 13.72571860816944 9.1 38.5 58.1 194.6 0.0 152.32702305222222 74 21 10 0 1.0 0.0
6 ba71377f hn-soleo-01 core 10327 2026-06-12 11:52 2026-11-07 23:32 148.48606441752315 69.54861414443475 14256.0 10326.0 72.43265993265993 10216.0 71.44055944055944 74.41724941724942 15.279641083333333 98.16984520900587 75.80459076696452 17.830078570011008 11.898643418868168 58.09414319527777 116.0 4.0 0.04773893676769633 0.09663987605306479 0.3212936961363416 10.580323424034086 8.4 28.0 39.2 111.3 0.0 102.40406208583333 117 27 5 0 9.0 17.0
7 d4cce74b hn-neckarufer-01 core 9527 2026-06-12 12:21 2026-11-07 23:49 148.47800736460647 64.16438480754427 14255.0 9525.0 66.8186601192564 9498.0 66.41958041958041 69.18706293706293 15.30519515 98.00593754598418 70.59476667380535 22.092291606384357 12.581170542548888 42.011142987499994 176.0 4.0 0.0007347538574577516 0.0050383121654245825 0.23239214863020888 19.200440852314475 11.9 61.6 86.8 255.5 0.0 124.21339694944444 177 33 5 0 1.0 0.0
8 e52c55f3 hn-hagenbuchersee-01 core 12345 2026-06-12 12:02 2026-11-07 23:46 148.4892696907176 83.13732046573405 14256.0 12345.0 86.59511784511784 12229.0 85.51748251748252 89.08071095571096 15.268402316666668 98.24210607567181 90.67467556843273 7.242715835567211 5.644689397694898 25.026753832222223 36.0 6.0 0.05556905629809639 0.1433778857837181 0.456865127582017 12.973673552045362 6.3 49.7 85.4 292.6 0.0 102.12467683444444 37 10 7 0 14.0 20.0
9 f4928534 hn-stadtgalerie-01 core 8982 2026-06-12 12:11 2026-11-07 23:26 148.4686443240278 60.497622517499984 14254.0 8975.0 62.9647818156307 8954.0 62.61538461538461 65.22435897435898 15.3698088 97.59392712809804 66.8323951025641 23.449999491129244 16.626900938851655 141.77853083055555 149.0 4.0 0.0 0.09262970385214875 0.4079269650411935 21.13513694054776 8.4 67.2 88.9 137.2 0.7 141.77853083055555 150 22 5 0 0.0 0.0
10 f8c19730 density-01 core 17585 2026-04-16 23:25 2026-11-07 23:48 205.01617579299767 85.77371971739127 19683.0 17585.0 89.34105573337398 12139.0 84.8881118881119 88.42511655011656 16.14243315 92.92279460361279 95.15976884608098 0.541116472764504 0.0 4.5767124775 13.0 0.0 0.058686380437873185 0.13204435598521466 0.3967586010804663 9.740608473130509 7.7 28.0 38.5 63.0 0.0 4.5767124775 15 0 0 0 16.0 46.0
11 d3304b39 hn-bollwerksturm-01 episodic 5 2026-06-12 11:19 2026-08-13 16:01 62.195593056273154 0.0 0.0 0.0 10.78 9.1 15.96 16.072000000000003 16.1 6.3 1389.6733469847222 2 2 2 1
12 2aadbeed maker-fair episodic 73 2026-08-13 16:07 2026-08-16 15:06 2.9573475597800924 0.0 0.0 0.0 117.83013698630137 127.4 157.78 161.672 165.2 9.1 38.09907913694445 2 2 1 0
13 062fe216 niklas-01 episodic 8 2026-05-01 17:19 2026-05-01 19:06 0.07439584600694445 0.0 0.0 0.75 4.55 4.9 6.3 6.3 6.3 2.1 0.723217388888889 0 0 0 0
+25
View File
@@ -0,0 +1,25 @@
,median_cv
0,0.7949545035546963
1,0.8252083725529498
2,0.8513506881157319
3,0.8440971508067069
4,0.8660254037844388
5,0.7886068896771579
6,0.7221786137191952
7,0.6021308086219471
8,0.5771597649592951
9,0.6330874783831757
10,0.722547728019157
11,0.7272190599496808
12,0.6734848363010881
13,0.6353527324110486
14,0.6490264486168573
15,0.6544356459537732
16,0.6013460797080393
17,0.523390197173663
18,0.458528241586464
19,0.44380170979843625
20,0.5080222353089345
21,0.5997308123562638
22,0.6843112667531579
23,0.7869445110781751
1 median_cv
2 0 0.7949545035546963
3 1 0.8252083725529498
4 2 0.8513506881157319
5 3 0.8440971508067069
6 4 0.8660254037844388
7 5 0.7886068896771579
8 6 0.7221786137191952
9 7 0.6021308086219471
10 8 0.5771597649592951
11 9 0.6330874783831757
12 10 0.722547728019157
13 11 0.7272190599496808
14 12 0.6734848363010881
15 13 0.6353527324110486
16 14 0.6490264486168573
17 15 0.6544356459537732
18 16 0.6013460797080393
19 17 0.523390197173663
20 18 0.458528241586464
21 19 0.44380170979843625
22 20 0.5080222353089345
23 21 0.5997308123562638
24 22 0.6843112667531579
25 23 0.7869445110781751
@@ -0,0 +1,3 @@
series,n,first,last,min,max,mean,median,p95,p99,zero_pct,n_unique,effective_quantum,gap_after_days
legacy 863f75b0,8075,2025-10-16T21:17:59.316153+00:00,2026-03-12T18:28:06.032195+00:00,0.0,28.0,3.0080495356037154,2.0,11.0,16.26000000000022,29.77089783281734,27,1.0,35.20654562179398
new f8c19730,17585,2026-04-16T23:25:31.573918+00:00,2026-11-07T23:48:49.162433+00:00,0.0,63.0,9.740608473130509,7.7,28.0,38.5,5.8686380437873185,84,0.7,
1 series n first last min max mean median p95 p99 zero_pct n_unique effective_quantum gap_after_days
2 legacy 863f75b0 8075 2025-10-16T21:17:59.316153+00:00 2026-03-12T18:28:06.032195+00:00 0.0 28.0 3.0080495356037154 2.0 11.0 16.26000000000022 29.77089783281734 27 1.0 35.20654562179398
3 new f8c19730 17585 2026-04-16T23:25:31.573918+00:00 2026-11-07T23:48:49.162433+00:00 0.0 63.0 9.740608473130509 7.7 28.0 38.5 5.8686380437873185 84 0.7
+10
View File
@@ -0,0 +1,10 @@
sensor_id,name,adjacent_equal_share,runs_ge3,max_run,max_run_hours,max_run_value,max_run_start,share_in_runs_ge3
00091a5f,hn-goetzentuem-01,0.1275120578778135,198,17,4.25,1.4,2026-11-04 00:31,0.08440514469453377
0f8519cf,hn-hagenbuchersee-02,0.08010694128131499,96,5,1.25,3.5,2026-07-30 22:49,0.031686305574809386
863f75b0,density-01,0.2926315789473684,438,20,5.0,0.0,2025-10-27 22:09,0.2486687306501548
8d7eef22,hn-marahaus-01,0.09811973200778042,128,12,3.0,0.7,2026-10-18 23:43,0.04960017289820618
ba71377f,hn-soleo-01,0.10574222910816307,156,9,2.25,0.0,2026-08-10 00:17,0.0556792873051225
d4cce74b,hn-neckarufer-01,0.07085126482628319,64,6,1.5,2.1,2026-10-17 01:39,0.021622756376613834
e52c55f3,hn-hagenbuchersee-01,0.12839206156338598,264,14,3.5,0.0,2026-08-24 00:11,0.07995139732685298
f4928534,hn-stadtgalerie-01,0.125250501002004,165,20,5.0,0.7,2026-09-29 22:48,0.07893564907592963
f8c19730,density-01,0.13818595393801536,386,16,4.0,0.0,2026-10-01 22:25,0.08928063690645437
1 sensor_id name adjacent_equal_share runs_ge3 max_run max_run_hours max_run_value max_run_start share_in_runs_ge3
2 00091a5f hn-goetzentuem-01 0.1275120578778135 198 17 4.25 1.4 2026-11-04 00:31 0.08440514469453377
3 0f8519cf hn-hagenbuchersee-02 0.08010694128131499 96 5 1.25 3.5 2026-07-30 22:49 0.031686305574809386
4 863f75b0 density-01 0.2926315789473684 438 20 5.0 0.0 2025-10-27 22:09 0.2486687306501548
5 8d7eef22 hn-marahaus-01 0.09811973200778042 128 12 3.0 0.7 2026-10-18 23:43 0.04960017289820618
6 ba71377f hn-soleo-01 0.10574222910816307 156 9 2.25 0.0 2026-08-10 00:17 0.0556792873051225
7 d4cce74b hn-neckarufer-01 0.07085126482628319 64 6 1.5 2.1 2026-10-17 01:39 0.021622756376613834
8 e52c55f3 hn-hagenbuchersee-01 0.12839206156338598 264 14 3.5 0.0 2026-08-24 00:11 0.07995139732685298
9 f4928534 hn-stadtgalerie-01 0.125250501002004 165 20 5.0 0.7 2026-09-29 22:48 0.07893564907592963
10 f8c19730 density-01 0.13818595393801536 386 16 4.0 0.0 2026-10-01 22:25 0.08928063690645437
+25
View File
@@ -0,0 +1,25 @@
,density-01,hn-goetzentuem-01,hn-hagenbuchersee-01,hn-hagenbuchersee-02,hn-marahaus-01,hn-neckarufer-01,hn-soleo-01,hn-stadtgalerie-01
0,6.448648648648649,5.044495412844037,8.394605009633912,22.1655421686747,7.99501246882793,18.190047393364928,7.241726618705036,3.1171875
1,2.7521484375,2.38,3.9459566074950687,10.09464720194647,4.792307692307692,8.05593220338983,5.037757437070939,2.2907455012853473
2,1.4948919449901767,1.7771764705882351,1.7580459770114942,4.76595744680851,3.1752577319587627,5.225742574257425,4.41588785046729,1.7815
3,1.2915686274509803,1.272877358490566,1.366019417475728,3.3920481927710844,2.7102094240837693,4.33419689119171,3.9267123287671235,1.6507731958762886
4,1.0583333333333333,1.2579710144927536,1.128598848368522,2.7590243902439027,2.6127499999999997,4.081201044386423,3.940449438202247,1.6266839378238342
5,1.8308737864077669,1.7541966426858513,1.2447470817120623,2.902398081534772,2.792764857881137,4.32976501305483,4.358876404494382,1.6789340101522843
6,1.689041095890411,2.6485576923076923,1.699421965317919,3.202582159624413,3.379373368146214,4.891315136476427,4.750925925925926,2.8614987080103362
7,2.83996062992126,5.351184834123222,3.1768172888015718,4.852347417840376,5.129156010230179,6.236519607843137,6.638359201773836,5.516708860759493
8,4.306640625,7.623150357995226,5.129239766081872,7.568235294117647,7.119689119170984,8.433084577114428,9.575870069605568,9.906561679790027
9,6.046868686868686,7.84,4.37810650887574,6.534441805225653,7.42257217847769,7.645866666666667,8.668604651162791,17.21926121372032
10,8.103155818540435,9.241981132075471,5.042745098039216,7.353225806451613,8.041902313624679,8.85438596491228,8.634411085450346,27.046747967479675
11,10.301183431952662,10.995620437956203,7.530511811023622,10.39,11.72185089974293,10.992727272727272,11.160563380281689,37.804768392370576
12,12.260894941634241,13.921227621483375,10.559486166007906,14.305727923627686,16.742894736842107,14.819072164948453,13.449180327868852,48.398870056497174
13,13.142430278884461,14.05072463768116,11.458333333333334,15.793961352657004,20.102374670184698,15.987786259541984,13.850710900473933,48.399715099715095
14,12.5875,14.607125307125308,10.317693836978131,15.818333333333333,18.036482939632545,16.199007444168736,12.732701421800947,47.38328767123288
15,12.140361445783132,15.941990291262137,10.83759842519685,17.9095,20.326344086021503,18.440105540897097,13.377403846153847,49.892897727272725
16,13.28719512195122,18.533416458852866,13.73891129032258,21.97758620689655,21.428083989501314,22.947481108312342,15.842061611374408,52.44930362116992
17,17.049601593625496,20.626903553299492,18.65401606425703,26.896356783919597,23.638904109589042,29.22171581769437,16.863793103448277,50.816326530612244
18,21.001408450704226,22.8643391521197,26.957623762376237,34.10217391304348,26.523636363636363,37.82126582278481,17.94037558685446,45.26
19,22.536653386454184,24.952119700748128,35.206126482213435,42.26909975669099,31.27034120734908,44.21221374045802,18.06,34.45566298342542
20,22.266074950690335,24.29051094890511,40.57459677419355,49.171601941747575,29.8997375328084,47.758933002481385,17.76142506142506,16.1
21,18.91408450704225,19.23579952267303,39.76712328767123,50.12235576923077,25.453543307086612,47.154060913705585,15.788321167883211,9.363215258855586
22,14.849805447470818,14.436633663366337,29.800773694390717,48.49292682926829,18.5590206185567,41.41295843520783,13.306779661016948,6.5698113207547175
23,10.288047808764942,9.023877068557919,18.447378640776698,38.72,12.709660574412533,30.884482758620692,9.869176470588235,4.682697201017811
1 density-01 hn-goetzentuem-01 hn-hagenbuchersee-01 hn-hagenbuchersee-02 hn-marahaus-01 hn-neckarufer-01 hn-soleo-01 hn-stadtgalerie-01
2 0 6.448648648648649 5.044495412844037 8.394605009633912 22.1655421686747 7.99501246882793 18.190047393364928 7.241726618705036 3.1171875
3 1 2.7521484375 2.38 3.9459566074950687 10.09464720194647 4.792307692307692 8.05593220338983 5.037757437070939 2.2907455012853473
4 2 1.4948919449901767 1.7771764705882351 1.7580459770114942 4.76595744680851 3.1752577319587627 5.225742574257425 4.41588785046729 1.7815
5 3 1.2915686274509803 1.272877358490566 1.366019417475728 3.3920481927710844 2.7102094240837693 4.33419689119171 3.9267123287671235 1.6507731958762886
6 4 1.0583333333333333 1.2579710144927536 1.128598848368522 2.7590243902439027 2.6127499999999997 4.081201044386423 3.940449438202247 1.6266839378238342
7 5 1.8308737864077669 1.7541966426858513 1.2447470817120623 2.902398081534772 2.792764857881137 4.32976501305483 4.358876404494382 1.6789340101522843
8 6 1.689041095890411 2.6485576923076923 1.699421965317919 3.202582159624413 3.379373368146214 4.891315136476427 4.750925925925926 2.8614987080103362
9 7 2.83996062992126 5.351184834123222 3.1768172888015718 4.852347417840376 5.129156010230179 6.236519607843137 6.638359201773836 5.516708860759493
10 8 4.306640625 7.623150357995226 5.129239766081872 7.568235294117647 7.119689119170984 8.433084577114428 9.575870069605568 9.906561679790027
11 9 6.046868686868686 7.84 4.37810650887574 6.534441805225653 7.42257217847769 7.645866666666667 8.668604651162791 17.21926121372032
12 10 8.103155818540435 9.241981132075471 5.042745098039216 7.353225806451613 8.041902313624679 8.85438596491228 8.634411085450346 27.046747967479675
13 11 10.301183431952662 10.995620437956203 7.530511811023622 10.39 11.72185089974293 10.992727272727272 11.160563380281689 37.804768392370576
14 12 12.260894941634241 13.921227621483375 10.559486166007906 14.305727923627686 16.742894736842107 14.819072164948453 13.449180327868852 48.398870056497174
15 13 13.142430278884461 14.05072463768116 11.458333333333334 15.793961352657004 20.102374670184698 15.987786259541984 13.850710900473933 48.399715099715095
16 14 12.5875 14.607125307125308 10.317693836978131 15.818333333333333 18.036482939632545 16.199007444168736 12.732701421800947 47.38328767123288
17 15 12.140361445783132 15.941990291262137 10.83759842519685 17.9095 20.326344086021503 18.440105540897097 13.377403846153847 49.892897727272725
18 16 13.28719512195122 18.533416458852866 13.73891129032258 21.97758620689655 21.428083989501314 22.947481108312342 15.842061611374408 52.44930362116992
19 17 17.049601593625496 20.626903553299492 18.65401606425703 26.896356783919597 23.638904109589042 29.22171581769437 16.863793103448277 50.816326530612244
20 18 21.001408450704226 22.8643391521197 26.957623762376237 34.10217391304348 26.523636363636363 37.82126582278481 17.94037558685446 45.26
21 19 22.536653386454184 24.952119700748128 35.206126482213435 42.26909975669099 31.27034120734908 44.21221374045802 18.06 34.45566298342542
22 20 22.266074950690335 24.29051094890511 40.57459677419355 49.171601941747575 29.8997375328084 47.758933002481385 17.76142506142506 16.1
23 21 18.91408450704225 19.23579952267303 39.76712328767123 50.12235576923077 25.453543307086612 47.154060913705585 15.788321167883211 9.363215258855586
24 22 14.849805447470818 14.436633663366337 29.800773694390717 48.49292682926829 18.5590206185567 41.41295843520783 13.306779661016948 6.5698113207547175
25 23 10.288047808764942 9.023877068557919 18.447378640776698 38.72 12.709660574412533 30.884482758620692 9.869176470588235 4.682697201017811
+10
View File
@@ -0,0 +1,10 @@
check,count
exact_duplicate_rows,0
duplicate_time_sensor_type,0
duplicate_time_sensor,0
density_negative,0
density_nan,0
battery_negative,0
battery_over_100,0
battery_nan,0
core_intervals_lt_60s,113
1 check count
2 exact_duplicate_rows 0
3 duplicate_time_sensor_type 0
4 duplicate_time_sensor 0
5 density_negative 0
6 density_nan 0
7 battery_negative 0
8 battery_over_100 0
9 battery_nan 0
10 core_intervals_lt_60s 113
+50
View File
@@ -0,0 +1,50 @@
sensor_id,name,gap_start,gap_end,gap_hours
d3304b39,hn-bollwerksturm-01,2026-06-16 18:20,2026-08-13 16:00,1389.6733469847222
8d7eef22,hn-marahaus-01,2026-08-19 12:25,2026-08-25 20:45,152.32702305222222
f4928534,hn-stadtgalerie-01,2026-08-20 04:46,2026-08-26 02:33,141.77853083055555
f4928534,hn-stadtgalerie-01,2026-06-12 12:11,2026-06-17 16:31,124.33075184361111
d4cce74b,hn-neckarufer-01,2026-06-12 12:21,2026-06-17 16:33,124.21339694944444
00091a5f,hn-goetzentuem-01,2026-06-12 12:27,2026-06-17 16:36,124.14629634222223
8d7eef22,hn-marahaus-01,2026-06-12 12:36,2026-06-17 16:37,124.01588750027778
8d7eef22,hn-marahaus-01,2026-09-04 10:55,2026-09-09 07:50,116.91929981472222
d3304b39,hn-bollwerksturm-01,2026-06-12 11:21,2026-06-16 18:20,102.97919121972222
ba71377f,hn-soleo-01,2026-06-12 11:52,2026-06-16 18:16,102.40406208583333
e52c55f3,hn-hagenbuchersee-01,2026-06-12 12:02,2026-06-16 18:09,102.12467683444444
0f8519cf,hn-hagenbuchersee-02,2026-06-12 12:32,2026-06-16 18:03,101.52013999083333
0f8519cf,hn-hagenbuchersee-02,2026-08-27 18:48,2026-08-31 05:07,82.32757577416666
f4928534,hn-stadtgalerie-01,2026-08-26 10:27,2026-08-29 20:28,82.01580273750001
f4928534,hn-stadtgalerie-01,2026-08-04 10:17,2026-08-07 18:16,79.98217900388889
8d7eef22,hn-marahaus-01,2026-08-15 04:31,2026-08-18 11:59,79.47032030694444
8d7eef22,hn-marahaus-01,2026-08-27 09:55,2026-08-30 17:08,79.22193586222222
863f75b0,density-01,2025-11-28 17:20,2025-12-01 03:59,58.654426167500006
ba71377f,hn-soleo-01,2026-09-18 00:13,2026-09-20 10:18,58.09414319527777
8d7eef22,hn-marahaus-01,2026-09-17 08:15,2026-09-19 10:03,49.80453973
0f8519cf,hn-hagenbuchersee-02,2026-08-16 00:34,2026-08-18 00:49,48.24859280333334
8d7eef22,hn-marahaus-01,2026-09-19 10:03,2026-09-21 09:00,46.958265823333335
8d7eef22,hn-marahaus-01,2026-09-13 10:39,2026-09-15 08:59,46.32689665638888
00091a5f,hn-goetzentuem-01,2026-09-17 08:21,2026-09-19 02:56,42.57558039722222
863f75b0,density-01,2025-11-22 02:43,2025-11-23 21:11,42.454578017777784
d4cce74b,hn-neckarufer-01,2026-08-10 14:35,2026-08-12 08:35,42.011142987499994
f4928534,hn-stadtgalerie-01,2026-09-13 10:17,2026-09-15 00:59,38.69393264972222
2aadbeed,maker-fair,2026-08-13 17:21,2026-08-15 07:27,38.09907913694445
ba71377f,hn-soleo-01,2026-08-19 12:29,2026-08-20 23:34,35.07558270027778
863f75b0,density-01,2025-11-14 17:55,2025-11-16 03:12,33.28355225333333
d4cce74b,hn-neckarufer-01,2026-09-15 00:07,2026-09-16 08:16,32.1404086875
00091a5f,hn-goetzentuem-01,2026-08-20 01:11,2026-08-21 08:24,31.219955259444443
00091a5f,hn-goetzentuem-01,2026-09-13 10:45,2026-09-14 16:48,30.03560134861111
8d7eef22,hn-marahaus-01,2026-08-11 05:45,2026-08-12 11:17,29.53876039
00091a5f,hn-goetzentuem-01,2026-09-19 04:13,2026-09-20 07:50,27.604858111944445
0f8519cf,hn-hagenbuchersee-02,2026-08-19 22:17,2026-08-21 01:53,27.600140854444444
0f8519cf,hn-hagenbuchersee-02,2026-09-14 08:35,2026-09-15 11:52,27.295206438055555
ba71377f,hn-soleo-01,2026-08-24 11:40,2026-08-25 14:31,26.849423225833334
ba71377f,hn-soleo-01,2026-08-16 00:38,2026-08-17 03:28,26.829599543055554
8d7eef22,hn-marahaus-01,2026-07-01 09:37,2026-07-02 12:22,26.748956767777777
d4cce74b,hn-neckarufer-01,2026-08-21 12:58,2026-08-22 14:00,25.04180304222222
e52c55f3,hn-hagenbuchersee-01,2026-09-17 22:54,2026-09-18 23:56,25.026753832222223
e52c55f3,hn-hagenbuchersee-01,2026-09-15 20:52,2026-09-16 21:53,25.024294579444444
e52c55f3,hn-hagenbuchersee-01,2026-09-19 00:12,2026-09-20 01:13,25.021727040277778
e52c55f3,hn-hagenbuchersee-01,2026-09-13 16:30,2026-09-14 17:31,25.021333215000002
e52c55f3,hn-hagenbuchersee-01,2026-09-14 18:02,2026-09-15 19:03,25.018525909166666
e52c55f3,hn-hagenbuchersee-01,2026-09-20 01:13,2026-09-21 02:14,25.017016116388888
d4cce74b,hn-neckarufer-01,2026-08-24 13:43,2026-08-25 14:31,24.80185189361111
0f8519cf,hn-hagenbuchersee-02,2026-09-15 11:52,2026-09-16 12:22,24.487326584722222
1 sensor_id name gap_start gap_end gap_hours
2 d3304b39 hn-bollwerksturm-01 2026-06-16 18:20 2026-08-13 16:00 1389.6733469847222
3 8d7eef22 hn-marahaus-01 2026-08-19 12:25 2026-08-25 20:45 152.32702305222222
4 f4928534 hn-stadtgalerie-01 2026-08-20 04:46 2026-08-26 02:33 141.77853083055555
5 f4928534 hn-stadtgalerie-01 2026-06-12 12:11 2026-06-17 16:31 124.33075184361111
6 d4cce74b hn-neckarufer-01 2026-06-12 12:21 2026-06-17 16:33 124.21339694944444
7 00091a5f hn-goetzentuem-01 2026-06-12 12:27 2026-06-17 16:36 124.14629634222223
8 8d7eef22 hn-marahaus-01 2026-06-12 12:36 2026-06-17 16:37 124.01588750027778
9 8d7eef22 hn-marahaus-01 2026-09-04 10:55 2026-09-09 07:50 116.91929981472222
10 d3304b39 hn-bollwerksturm-01 2026-06-12 11:21 2026-06-16 18:20 102.97919121972222
11 ba71377f hn-soleo-01 2026-06-12 11:52 2026-06-16 18:16 102.40406208583333
12 e52c55f3 hn-hagenbuchersee-01 2026-06-12 12:02 2026-06-16 18:09 102.12467683444444
13 0f8519cf hn-hagenbuchersee-02 2026-06-12 12:32 2026-06-16 18:03 101.52013999083333
14 0f8519cf hn-hagenbuchersee-02 2026-08-27 18:48 2026-08-31 05:07 82.32757577416666
15 f4928534 hn-stadtgalerie-01 2026-08-26 10:27 2026-08-29 20:28 82.01580273750001
16 f4928534 hn-stadtgalerie-01 2026-08-04 10:17 2026-08-07 18:16 79.98217900388889
17 8d7eef22 hn-marahaus-01 2026-08-15 04:31 2026-08-18 11:59 79.47032030694444
18 8d7eef22 hn-marahaus-01 2026-08-27 09:55 2026-08-30 17:08 79.22193586222222
19 863f75b0 density-01 2025-11-28 17:20 2025-12-01 03:59 58.654426167500006
20 ba71377f hn-soleo-01 2026-09-18 00:13 2026-09-20 10:18 58.09414319527777
21 8d7eef22 hn-marahaus-01 2026-09-17 08:15 2026-09-19 10:03 49.80453973
22 0f8519cf hn-hagenbuchersee-02 2026-08-16 00:34 2026-08-18 00:49 48.24859280333334
23 8d7eef22 hn-marahaus-01 2026-09-19 10:03 2026-09-21 09:00 46.958265823333335
24 8d7eef22 hn-marahaus-01 2026-09-13 10:39 2026-09-15 08:59 46.32689665638888
25 00091a5f hn-goetzentuem-01 2026-09-17 08:21 2026-09-19 02:56 42.57558039722222
26 863f75b0 density-01 2025-11-22 02:43 2025-11-23 21:11 42.454578017777784
27 d4cce74b hn-neckarufer-01 2026-08-10 14:35 2026-08-12 08:35 42.011142987499994
28 f4928534 hn-stadtgalerie-01 2026-09-13 10:17 2026-09-15 00:59 38.69393264972222
29 2aadbeed maker-fair 2026-08-13 17:21 2026-08-15 07:27 38.09907913694445
30 ba71377f hn-soleo-01 2026-08-19 12:29 2026-08-20 23:34 35.07558270027778
31 863f75b0 density-01 2025-11-14 17:55 2025-11-16 03:12 33.28355225333333
32 d4cce74b hn-neckarufer-01 2026-09-15 00:07 2026-09-16 08:16 32.1404086875
33 00091a5f hn-goetzentuem-01 2026-08-20 01:11 2026-08-21 08:24 31.219955259444443
34 00091a5f hn-goetzentuem-01 2026-09-13 10:45 2026-09-14 16:48 30.03560134861111
35 8d7eef22 hn-marahaus-01 2026-08-11 05:45 2026-08-12 11:17 29.53876039
36 00091a5f hn-goetzentuem-01 2026-09-19 04:13 2026-09-20 07:50 27.604858111944445
37 0f8519cf hn-hagenbuchersee-02 2026-08-19 22:17 2026-08-21 01:53 27.600140854444444
38 0f8519cf hn-hagenbuchersee-02 2026-09-14 08:35 2026-09-15 11:52 27.295206438055555
39 ba71377f hn-soleo-01 2026-08-24 11:40 2026-08-25 14:31 26.849423225833334
40 ba71377f hn-soleo-01 2026-08-16 00:38 2026-08-17 03:28 26.829599543055554
41 8d7eef22 hn-marahaus-01 2026-07-01 09:37 2026-07-02 12:22 26.748956767777777
42 d4cce74b hn-neckarufer-01 2026-08-21 12:58 2026-08-22 14:00 25.04180304222222
43 e52c55f3 hn-hagenbuchersee-01 2026-09-17 22:54 2026-09-18 23:56 25.026753832222223
44 e52c55f3 hn-hagenbuchersee-01 2026-09-15 20:52 2026-09-16 21:53 25.024294579444444
45 e52c55f3 hn-hagenbuchersee-01 2026-09-19 00:12 2026-09-20 01:13 25.021727040277778
46 e52c55f3 hn-hagenbuchersee-01 2026-09-13 16:30 2026-09-14 17:31 25.021333215000002
47 e52c55f3 hn-hagenbuchersee-01 2026-09-14 18:02 2026-09-15 19:03 25.018525909166666
48 e52c55f3 hn-hagenbuchersee-01 2026-09-20 01:13 2026-09-21 02:14 25.017016116388888
49 d4cce74b hn-neckarufer-01 2026-08-24 13:43 2026-08-25 14:31 24.80185189361111
50 0f8519cf hn-hagenbuchersee-02 2026-09-15 11:52 2026-09-16 12:22 24.487326584722222
+10
View File
@@ -0,0 +1,10 @@
,slots
0,69
1,594
2,1133
3,721
4,615
5,769
6,1834
7,4117
8,3876
1 slots
2 0 69
3 1 594
4 2 1133
5 3 721
6 4 615
7 5 769
8 6 1834
9 7 4117
10 8 3876
+25
View File
@@ -0,0 +1,25 @@
,density-01,hn-goetzentuem-01,hn-hagenbuchersee-01,hn-hagenbuchersee-02,hn-marahaus-01,hn-neckarufer-01,hn-soleo-01,hn-stadtgalerie-01
0,9.44,23.78,9.27,27.45,29.9,26.22,27.1,32.87
1,9.86,27.82,10.74,27.64,31.34,27.29,23.06,31.51
2,11.63,26.22,9.38,26.56,32.64,29.86,25.69,30.56
3,10.84,25.87,9.97,27.45,33.22,32.52,23.43,32.17
4,11.89,27.62,8.92,28.32,30.07,33.04,22.2,32.52
5,9.97,27.1,10.14,27.1,32.34,33.04,22.2,31.12
6,10.66,27.27,9.27,25.52,33.04,29.55,24.48,32.34
7,11.19,26.22,11.01,25.52,31.64,28.67,21.15,30.94
8,10.49,26.75,10.31,25.7,32.52,29.72,24.65,33.39
9,13.46,24.83,11.36,26.4,33.39,34.44,24.83,33.74
10,11.36,25.87,10.84,24.13,31.99,30.24,24.3,35.49
11,11.36,28.15,11.19,26.57,31.99,32.69,25.52,35.84
12,10.14,31.64,11.54,26.75,33.57,32.17,25.35,38.11
13,12.24,27.62,11.89,27.62,33.74,31.29,26.22,38.64
14,11.89,28.85,12.06,26.57,33.39,29.55,26.22,36.19
15,12.94,27.97,11.19,30.07,34.97,33.74,27.27,38.46
16,13.99,29.9,13.29,29.02,33.39,30.59,26.22,37.24
17,12.24,31.12,12.94,30.42,36.19,34.79,29.02,40.03
18,13.11,29.9,11.71,27.62,32.69,30.94,25.52,38.81
19,12.24,29.9,11.54,28.15,33.39,31.29,29.2,36.71
20,11.36,28.15,13.29,27.97,33.39,29.55,28.85,36.89
21,13.11,26.75,10.66,27.27,33.39,31.12,28.15,35.84
22,10.14,29.37,9.62,28.32,32.17,28.5,27.8,35.14
23,12.24,26.05,9.97,26.57,33.04,29.02,25.7,31.29
1 density-01 hn-goetzentuem-01 hn-hagenbuchersee-01 hn-hagenbuchersee-02 hn-marahaus-01 hn-neckarufer-01 hn-soleo-01 hn-stadtgalerie-01
2 0 9.44 23.78 9.27 27.45 29.9 26.22 27.1 32.87
3 1 9.86 27.82 10.74 27.64 31.34 27.29 23.06 31.51
4 2 11.63 26.22 9.38 26.56 32.64 29.86 25.69 30.56
5 3 10.84 25.87 9.97 27.45 33.22 32.52 23.43 32.17
6 4 11.89 27.62 8.92 28.32 30.07 33.04 22.2 32.52
7 5 9.97 27.1 10.14 27.1 32.34 33.04 22.2 31.12
8 6 10.66 27.27 9.27 25.52 33.04 29.55 24.48 32.34
9 7 11.19 26.22 11.01 25.52 31.64 28.67 21.15 30.94
10 8 10.49 26.75 10.31 25.7 32.52 29.72 24.65 33.39
11 9 13.46 24.83 11.36 26.4 33.39 34.44 24.83 33.74
12 10 11.36 25.87 10.84 24.13 31.99 30.24 24.3 35.49
13 11 11.36 28.15 11.19 26.57 31.99 32.69 25.52 35.84
14 12 10.14 31.64 11.54 26.75 33.57 32.17 25.35 38.11
15 13 12.24 27.62 11.89 27.62 33.74 31.29 26.22 38.64
16 14 11.89 28.85 12.06 26.57 33.39 29.55 26.22 36.19
17 15 12.94 27.97 11.19 30.07 34.97 33.74 27.27 38.46
18 16 13.99 29.9 13.29 29.02 33.39 30.59 26.22 37.24
19 17 12.24 31.12 12.94 30.42 36.19 34.79 29.02 40.03
20 18 13.11 29.9 11.71 27.62 32.69 30.94 25.52 38.81
21 19 12.24 29.9 11.54 28.15 33.39 31.29 29.2 36.71
22 20 11.36 28.15 13.29 27.97 33.39 29.55 28.85 36.89
23 21 13.11 26.75 10.66 27.27 33.39 31.12 28.15 35.84
24 22 10.14 29.37 9.62 28.32 32.17 28.5 27.8 35.14
25 23 12.24 26.05 9.97 26.57 33.04 29.02 25.7 31.29
+7
View File
@@ -0,0 +1,7 @@
,density-01,hn-goetzentuem-01,hn-hagenbuchersee-01,hn-hagenbuchersee-02,hn-marahaus-01,hn-neckarufer-01,hn-soleo-01,hn-stadtgalerie-01
2026-06-01 00:00:00+00:00,86.7,79.65,90.06,94.47,77.96,77.0,92.07,73.64
2026-07-01 00:00:00+00:00,87.1,86.46,95.23,88.54,85.42,83.23,90.73,78.9
2026-08-01 00:00:00+00:00,84.85,51.98,94.09,49.06,28.86,46.94,54.57,23.32
2026-09-01 00:00:00+00:00,90.59,57.19,66.84,50.97,56.01,52.74,52.99,66.94
2026-10-01 00:00:00+00:00,90.99,86.76,96.98,89.15,88.61,86.16,88.04,84.11
2026-11-01 00:00:00+00:00,92.71,86.76,98.21,87.5,89.29,86.01,88.69,82.59
1 density-01 hn-goetzentuem-01 hn-hagenbuchersee-01 hn-hagenbuchersee-02 hn-marahaus-01 hn-neckarufer-01 hn-soleo-01 hn-stadtgalerie-01
2 2026-06-01 00:00:00+00:00 86.7 79.65 90.06 94.47 77.96 77.0 92.07 73.64
3 2026-07-01 00:00:00+00:00 87.1 86.46 95.23 88.54 85.42 83.23 90.73 78.9
4 2026-08-01 00:00:00+00:00 84.85 51.98 94.09 49.06 28.86 46.94 54.57 23.32
5 2026-09-01 00:00:00+00:00 90.59 57.19 66.84 50.97 56.01 52.74 52.99 66.94
6 2026-10-01 00:00:00+00:00 90.99 86.76 96.98 89.15 88.61 86.16 88.04 84.11
7 2026-11-01 00:00:00+00:00 92.71 86.76 98.21 87.5 89.29 86.01 88.69 82.59
+7
View File
@@ -0,0 +1,7 @@
month,slots,all8_slots,all8_pct,any_pct,mean_pairwise_r,median_cv
2026-06,1248,449,35.9775641025641,100.0,0.6680303302249962,0.6860567953468348
2026-07,2976,1086,36.49193548387097,100.0,0.6047122860200611,0.6689481262481496
2026-08,2976,200,6.720430107526882,99.59677419354838,0.6269011158420487,0.5604577203874694
2026-09,2880,692,24.02777777777778,98.57638888888889,0.6928758374356724,0.6168586794743283
2026-10,2976,1198,40.255376344086024,99.46236559139786,0.6232924446528996,0.7482833577857352
2026-11,672,251,37.351190476190474,100.0,0.5740676625577704,0.7768660102948879
1 month slots all8_slots all8_pct any_pct mean_pairwise_r median_cv
2 2026-06 1248 449 35.9775641025641 100.0 0.6680303302249962 0.6860567953468348
3 2026-07 2976 1086 36.49193548387097 100.0 0.6047122860200611 0.6689481262481496
4 2026-08 2976 200 6.720430107526882 99.59677419354838 0.6269011158420487 0.5604577203874694
5 2026-09 2880 692 24.02777777777778 98.57638888888889 0.6928758374356724 0.6168586794743283
6 2026-10 2976 1198 40.255376344086024 99.46236559139786 0.6232924446528996 0.7482833577857352
7 2026-11 672 251 37.351190476190474 100.0 0.5740676625577704 0.7768660102948879
@@ -0,0 +1,114 @@
sensor_id,name,prev_time,time,delta_s,prev_value,value
863f75b0,density-01,2025-10-17 17:42:41,2025-10-17 17:42:46,5.188711,4.0,4.0
863f75b0,density-01,2025-10-18 17:45:43,2025-10-18 17:45:49,5.17891,4.0,4.0
863f75b0,density-01,2025-10-19 22:10:05,2025-10-19 22:10:10,5.170571,1.0,1.0
863f75b0,density-01,2025-10-21 01:57:30,2025-10-21 01:57:35,5.227865,0.0,0.0
863f75b0,density-01,2025-10-23 14:57:52,2025-10-23 14:57:57,5.308622,13.0,13.0
863f75b0,density-01,2025-10-25 01:03:16,2025-10-25 01:03:21,5.35228,0.0,0.0
863f75b0,density-01,2025-10-26 04:30:22,2025-10-26 04:30:27,5.176728,0.0,0.0
863f75b0,density-01,2025-10-27 05:57:41,2025-10-27 05:57:53,12.169911,0.0,0.0
863f75b0,density-01,2025-10-28 07:00:49,2025-10-28 07:01:01,12.182164,2.0,2.0
863f75b0,density-01,2025-10-30 14:50:35,2025-10-30 14:51:28,53.119628,7.0,10.0
863f75b0,density-01,2025-11-03 19:52:05,2025-11-03 19:52:11,5.681216,1.0,1.0
863f75b0,density-01,2025-11-04 21:34:00,2025-11-04 21:34:53,53.287809,1.0,1.0
863f75b0,density-01,2025-11-06 01:43:34,2025-11-06 01:43:40,5.354458,1.0,1.0
863f75b0,density-01,2025-11-07 03:54:27,2025-11-07 03:54:32,5.286414,0.0,0.0
863f75b0,density-01,2025-11-08 07:09:57,2025-11-08 07:10:02,5.284831,1.0,1.0
863f75b0,density-01,2025-11-10 10:11:16,2025-11-10 10:11:22,5.225233,2.0,2.0
863f75b0,density-01,2025-11-11 20:41:22,2025-11-11 20:41:43,21.830494,1.0,4.0
863f75b0,density-01,2025-11-13 22:47:11,2025-11-13 22:47:18,6.49613,2.0,2.0
863f75b0,density-01,2025-11-18 04:24:37,2025-11-18 04:24:43,5.55562,1.0,1.0
863f75b0,density-01,2025-12-01 16:21:30,2025-12-01 16:21:36,6.520454,7.0,7.0
863f75b0,density-01,2025-12-02 23:55:39,2025-12-02 23:55:44,5.572952,1.0,1.0
863f75b0,density-01,2025-12-04 08:07:42,2025-12-04 08:07:47,5.515536,2.0,2.0
863f75b0,density-01,2025-12-05 09:03:47,2025-12-05 09:03:52,5.166214,6.0,6.0
863f75b0,density-01,2025-12-07 11:26:49,2025-12-07 11:26:55,5.253181,0.0,0.0
863f75b0,density-01,2025-12-09 03:47:22,2025-12-09 03:47:38,15.507519,0.0,0.0
863f75b0,density-01,2025-12-10 03:54:47,2025-12-10 03:54:54,6.287352,2.0,2.0
863f75b0,density-01,2025-12-11 04:48:30,2025-12-11 04:48:42,12.182643,1.0,1.0
863f75b0,density-01,2025-12-12 10:59:27,2025-12-12 10:59:40,12.157672,2.0,4.0
863f75b0,density-01,2025-12-13 14:17:17,2025-12-13 14:17:29,12.183899,2.0,2.0
863f75b0,density-01,2025-12-14 16:28:23,2025-12-14 16:28:28,5.17072,0.0,0.0
863f75b0,density-01,2025-12-16 00:38:21,2025-12-16 00:38:26,5.167143,3.0,3.0
863f75b0,density-01,2025-12-17 07:14:10,2025-12-17 07:14:15,5.183621,2.0,2.0
863f75b0,density-01,2025-12-18 07:21:20,2025-12-18 07:21:32,12.193084,0.0,0.0
863f75b0,density-01,2025-12-19 08:50:03,2025-12-19 08:50:09,6.315329,0.0,0.0
863f75b0,density-01,2025-12-20 09:07:15,2025-12-20 09:07:20,5.215,1.0,1.0
863f75b0,density-01,2025-12-21 10:33:09,2025-12-21 10:33:14,5.188037,2.0,2.0
863f75b0,density-01,2025-12-22 10:51:25,2025-12-22 10:51:31,6.401524,3.0,3.0
863f75b0,density-01,2025-12-23 11:26:31,2025-12-23 11:26:37,5.174973,4.0,4.0
863f75b0,density-01,2025-12-24 12:21:21,2025-12-24 12:21:26,5.231738,2.0,2.0
863f75b0,density-01,2025-12-25 14:02:13,2025-12-25 14:02:19,5.226044,1.0,1.0
863f75b0,density-01,2025-12-26 14:37:45,2025-12-26 14:37:50,5.223747,0.0,0.0
863f75b0,density-01,2025-12-27 19:54:33,2025-12-27 19:54:38,5.18015,3.0,3.0
863f75b0,density-01,2025-12-28 21:10:13,2025-12-28 21:10:18,5.137496,0.0,0.0
863f75b0,density-01,2025-12-31 23:13:36,2025-12-31 23:13:52,15.525371,0.0,0.0
863f75b0,density-01,2026-01-02 13:44:05,2026-01-02 13:44:30,24.987623,6.0,7.0
863f75b0,density-01,2026-01-03 18:35:48,2026-01-03 18:36:11,22.563346,2.0,2.0
863f75b0,density-01,2026-01-04 21:46:45,2026-01-04 21:47:07,21.8652,0.0,0.0
863f75b0,density-01,2026-01-06 15:12:27,2026-01-06 15:13:03,35.807956,1.0,1.0
863f75b0,density-01,2026-01-07 16:42:02,2026-01-07 16:42:53,50.618583,2.0,2.0
863f75b0,density-01,2026-01-08 22:37:45,2026-01-08 22:38:21,35.80642,0.0,0.0
863f75b0,density-01,2026-01-10 15:04:05,2026-01-10 15:04:33,28.740012,4.0,5.0
863f75b0,density-01,2026-01-11 19:19:24,2026-01-11 19:19:31,7.051725,1.0,3.0
863f75b0,density-01,2026-01-14 09:17:33,2026-01-14 09:18:02,28.819672,0.0,0.0
863f75b0,density-01,2026-01-15 19:27:18,2026-01-15 19:28:09,50.619776,4.0,4.0
863f75b0,density-01,2026-01-17 04:44:59,2026-01-17 04:45:28,28.802725,0.0,0.0
863f75b0,density-01,2026-01-18 20:58:40,2026-01-18 20:59:31,50.60705,0.0,2.0
863f75b0,density-01,2026-01-19 23:14:35,2026-01-19 23:14:42,7.000839,0.0,1.0
863f75b0,density-01,2026-01-21 08:37:14,2026-01-21 08:37:36,21.819446,0.0,0.0
863f75b0,density-01,2026-01-22 19:13:35,2026-01-22 19:14:04,28.565642,0.0,2.0
863f75b0,density-01,2026-01-23 19:28:52,2026-01-23 19:29:04,12.176994,2.0,2.0
863f75b0,density-01,2026-01-24 20:03:25,2026-01-24 20:03:31,5.178072,10.0,14.0
863f75b0,density-01,2026-01-25 20:17:16,2026-01-25 20:17:21,5.166376,1.0,1.0
863f75b0,density-01,2026-01-26 21:46:37,2026-01-26 21:46:42,5.209291,1.0,1.0
863f75b0,density-01,2026-01-27 22:28:48,2026-01-27 22:29:00,12.137812,2.0,2.0
863f75b0,density-01,2026-01-28 23:20:23,2026-01-28 23:20:29,5.263236,0.0,0.0
863f75b0,density-01,2026-01-29 23:23:30,2026-01-29 23:23:42,12.166944,1.0,2.0
863f75b0,density-01,2026-01-31 01:21:12,2026-01-31 01:21:25,12.217324,0.0,0.0
863f75b0,density-01,2026-02-01 01:38:20,2026-02-01 01:38:25,5.193612,0.0,0.0
863f75b0,density-01,2026-02-02 02:29:24,2026-02-02 02:29:29,5.194286,0.0,0.0
863f75b0,density-01,2026-02-03 02:39:20,2026-02-03 02:39:25,5.202508,0.0,0.0
863f75b0,density-01,2026-02-04 02:47:17,2026-02-04 02:47:22,5.203643,0.0,0.0
863f75b0,density-01,2026-02-05 02:49:04,2026-02-05 02:49:17,12.182387,4.0,4.0
863f75b0,density-01,2026-02-06 02:50:49,2026-02-06 02:50:54,5.185432,2.0,4.0
863f75b0,density-01,2026-02-07 04:08:33,2026-02-07 04:08:39,5.214555,0.0,0.0
863f75b0,density-01,2026-02-08 04:55:54,2026-02-08 04:56:06,12.258212,2.0,2.0
863f75b0,density-01,2026-02-09 05:27:56,2026-02-09 05:28:08,12.210633,3.0,3.0
863f75b0,density-01,2026-02-10 05:28:49,2026-02-10 05:28:54,5.178586,6.0,6.0
863f75b0,density-01,2026-02-11 22:52:32,2026-02-11 22:52:40,8.563582,0.0,2.0
863f75b0,density-01,2026-02-12 23:00:01,2026-02-12 23:00:16,15.525545,3.0,3.0
863f75b0,density-01,2026-02-14 00:37:30,2026-02-14 00:38:21,50.708598,2.0,2.0
863f75b0,density-01,2026-02-15 02:17:10,2026-02-15 02:17:32,21.825311,1.0,1.0
863f75b0,density-01,2026-02-16 02:31:03,2026-02-16 02:31:08,5.482081,2.0,2.0
863f75b0,density-01,2026-02-17 02:48:13,2026-02-17 02:48:19,5.478879,0.0,0.0
863f75b0,density-01,2026-02-18 06:36:41,2026-02-18 06:37:09,28.189355,4.0,4.0
863f75b0,density-01,2026-02-19 06:51:05,2026-02-19 06:51:10,5.468128,1.0,1.0
863f75b0,density-01,2026-02-20 07:15:22,2026-02-20 07:15:44,21.872177,2.0,4.0
863f75b0,density-01,2026-02-21 08:12:15,2026-02-21 08:12:37,21.832343,4.0,4.0
863f75b0,density-01,2026-02-22 08:17:45,2026-02-22 08:18:14,28.849793,0.0,0.0
863f75b0,density-01,2026-02-23 09:55:45,2026-02-23 09:55:50,5.546099,6.0,6.0
863f75b0,density-01,2026-02-24 10:33:57,2026-02-24 10:34:02,5.555163,2.0,2.0
863f75b0,density-01,2026-02-25 11:02:34,2026-02-25 11:02:56,21.835883,6.0,7.0
863f75b0,density-01,2026-02-26 11:50:05,2026-02-26 11:50:10,5.644904,17.0,17.0
863f75b0,density-01,2026-02-27 12:03:35,2026-02-27 12:04:04,28.823213,4.0,8.0
863f75b0,density-01,2026-02-28 12:09:53,2026-02-28 12:10:15,21.97868,14.0,22.0
863f75b0,density-01,2026-03-01 12:12:38,2026-03-01 12:12:59,21.515399,7.0,11.0
863f75b0,density-01,2026-03-02 12:17:18,2026-03-02 12:17:24,5.5465219999999995,7.0,7.0
863f75b0,density-01,2026-03-03 13:30:14,2026-03-03 13:30:36,21.837461,7.0,10.0
863f75b0,density-01,2026-03-04 14:16:41,2026-03-04 14:16:47,5.615973,9.0,9.0
863f75b0,density-01,2026-03-05 14:37:06,2026-03-05 14:37:35,28.855245,15.0,18.0
863f75b0,density-01,2026-03-06 15:34:53,2026-03-06 15:34:58,5.591055,20.0,20.0
863f75b0,density-01,2026-03-07 15:53:43,2026-03-07 15:53:48,5.547173,23.0,23.0
863f75b0,density-01,2026-03-08 17:08:50,2026-03-08 17:08:56,6.603965,10.0,10.0
863f75b0,density-01,2026-03-09 17:10:15,2026-03-09 17:10:44,28.875973,9.0,12.0
863f75b0,density-01,2026-03-10 17:22:46,2026-03-10 17:22:54,8.567725,10.0,11.0
863f75b0,density-01,2026-03-11 17:37:24,2026-03-11 17:37:30,5.31581,4.0,4.0
863f75b0,density-01,2026-03-12 18:27:57,2026-03-12 18:28:06,8.861128,17.0,18.0
8d7eef22,hn-marahaus-01,2026-06-12 12:36:14,2026-06-12 12:36:42,27.980657,6.3,7.7
f4928534,hn-stadtgalerie-01,2026-10-16 08:08:25,2026-10-16 08:08:41,16.382029,24.5,25.2
f4928534,hn-stadtgalerie-01,2026-10-21 10:45:45,2026-10-21 10:46:01,16.370418,47.6,44.8
f4928534,hn-stadtgalerie-01,2026-10-21 17:54:16,2026-10-21 17:54:32,16.259417,11.2,12.6
f4928534,hn-stadtgalerie-01,2026-10-23 00:18:14,2026-10-23 00:18:30,16.346725,2.1,0.7
f4928534,hn-stadtgalerie-01,2026-10-24 09:42:12,2026-10-24 09:42:29,16.372927,65.1,65.8
f4928534,hn-stadtgalerie-01,2026-10-24 17:36:26,2026-10-24 17:36:42,16.350135,51.8,44.1
1 sensor_id name prev_time time delta_s prev_value value
2 863f75b0 density-01 2025-10-17 17:42:41 2025-10-17 17:42:46 5.188711 4.0 4.0
3 863f75b0 density-01 2025-10-18 17:45:43 2025-10-18 17:45:49 5.17891 4.0 4.0
4 863f75b0 density-01 2025-10-19 22:10:05 2025-10-19 22:10:10 5.170571 1.0 1.0
5 863f75b0 density-01 2025-10-21 01:57:30 2025-10-21 01:57:35 5.227865 0.0 0.0
6 863f75b0 density-01 2025-10-23 14:57:52 2025-10-23 14:57:57 5.308622 13.0 13.0
7 863f75b0 density-01 2025-10-25 01:03:16 2025-10-25 01:03:21 5.35228 0.0 0.0
8 863f75b0 density-01 2025-10-26 04:30:22 2025-10-26 04:30:27 5.176728 0.0 0.0
9 863f75b0 density-01 2025-10-27 05:57:41 2025-10-27 05:57:53 12.169911 0.0 0.0
10 863f75b0 density-01 2025-10-28 07:00:49 2025-10-28 07:01:01 12.182164 2.0 2.0
11 863f75b0 density-01 2025-10-30 14:50:35 2025-10-30 14:51:28 53.119628 7.0 10.0
12 863f75b0 density-01 2025-11-03 19:52:05 2025-11-03 19:52:11 5.681216 1.0 1.0
13 863f75b0 density-01 2025-11-04 21:34:00 2025-11-04 21:34:53 53.287809 1.0 1.0
14 863f75b0 density-01 2025-11-06 01:43:34 2025-11-06 01:43:40 5.354458 1.0 1.0
15 863f75b0 density-01 2025-11-07 03:54:27 2025-11-07 03:54:32 5.286414 0.0 0.0
16 863f75b0 density-01 2025-11-08 07:09:57 2025-11-08 07:10:02 5.284831 1.0 1.0
17 863f75b0 density-01 2025-11-10 10:11:16 2025-11-10 10:11:22 5.225233 2.0 2.0
18 863f75b0 density-01 2025-11-11 20:41:22 2025-11-11 20:41:43 21.830494 1.0 4.0
19 863f75b0 density-01 2025-11-13 22:47:11 2025-11-13 22:47:18 6.49613 2.0 2.0
20 863f75b0 density-01 2025-11-18 04:24:37 2025-11-18 04:24:43 5.55562 1.0 1.0
21 863f75b0 density-01 2025-12-01 16:21:30 2025-12-01 16:21:36 6.520454 7.0 7.0
22 863f75b0 density-01 2025-12-02 23:55:39 2025-12-02 23:55:44 5.572952 1.0 1.0
23 863f75b0 density-01 2025-12-04 08:07:42 2025-12-04 08:07:47 5.515536 2.0 2.0
24 863f75b0 density-01 2025-12-05 09:03:47 2025-12-05 09:03:52 5.166214 6.0 6.0
25 863f75b0 density-01 2025-12-07 11:26:49 2025-12-07 11:26:55 5.253181 0.0 0.0
26 863f75b0 density-01 2025-12-09 03:47:22 2025-12-09 03:47:38 15.507519 0.0 0.0
27 863f75b0 density-01 2025-12-10 03:54:47 2025-12-10 03:54:54 6.287352 2.0 2.0
28 863f75b0 density-01 2025-12-11 04:48:30 2025-12-11 04:48:42 12.182643 1.0 1.0
29 863f75b0 density-01 2025-12-12 10:59:27 2025-12-12 10:59:40 12.157672 2.0 4.0
30 863f75b0 density-01 2025-12-13 14:17:17 2025-12-13 14:17:29 12.183899 2.0 2.0
31 863f75b0 density-01 2025-12-14 16:28:23 2025-12-14 16:28:28 5.17072 0.0 0.0
32 863f75b0 density-01 2025-12-16 00:38:21 2025-12-16 00:38:26 5.167143 3.0 3.0
33 863f75b0 density-01 2025-12-17 07:14:10 2025-12-17 07:14:15 5.183621 2.0 2.0
34 863f75b0 density-01 2025-12-18 07:21:20 2025-12-18 07:21:32 12.193084 0.0 0.0
35 863f75b0 density-01 2025-12-19 08:50:03 2025-12-19 08:50:09 6.315329 0.0 0.0
36 863f75b0 density-01 2025-12-20 09:07:15 2025-12-20 09:07:20 5.215 1.0 1.0
37 863f75b0 density-01 2025-12-21 10:33:09 2025-12-21 10:33:14 5.188037 2.0 2.0
38 863f75b0 density-01 2025-12-22 10:51:25 2025-12-22 10:51:31 6.401524 3.0 3.0
39 863f75b0 density-01 2025-12-23 11:26:31 2025-12-23 11:26:37 5.174973 4.0 4.0
40 863f75b0 density-01 2025-12-24 12:21:21 2025-12-24 12:21:26 5.231738 2.0 2.0
41 863f75b0 density-01 2025-12-25 14:02:13 2025-12-25 14:02:19 5.226044 1.0 1.0
42 863f75b0 density-01 2025-12-26 14:37:45 2025-12-26 14:37:50 5.223747 0.0 0.0
43 863f75b0 density-01 2025-12-27 19:54:33 2025-12-27 19:54:38 5.18015 3.0 3.0
44 863f75b0 density-01 2025-12-28 21:10:13 2025-12-28 21:10:18 5.137496 0.0 0.0
45 863f75b0 density-01 2025-12-31 23:13:36 2025-12-31 23:13:52 15.525371 0.0 0.0
46 863f75b0 density-01 2026-01-02 13:44:05 2026-01-02 13:44:30 24.987623 6.0 7.0
47 863f75b0 density-01 2026-01-03 18:35:48 2026-01-03 18:36:11 22.563346 2.0 2.0
48 863f75b0 density-01 2026-01-04 21:46:45 2026-01-04 21:47:07 21.8652 0.0 0.0
49 863f75b0 density-01 2026-01-06 15:12:27 2026-01-06 15:13:03 35.807956 1.0 1.0
50 863f75b0 density-01 2026-01-07 16:42:02 2026-01-07 16:42:53 50.618583 2.0 2.0
51 863f75b0 density-01 2026-01-08 22:37:45 2026-01-08 22:38:21 35.80642 0.0 0.0
52 863f75b0 density-01 2026-01-10 15:04:05 2026-01-10 15:04:33 28.740012 4.0 5.0
53 863f75b0 density-01 2026-01-11 19:19:24 2026-01-11 19:19:31 7.051725 1.0 3.0
54 863f75b0 density-01 2026-01-14 09:17:33 2026-01-14 09:18:02 28.819672 0.0 0.0
55 863f75b0 density-01 2026-01-15 19:27:18 2026-01-15 19:28:09 50.619776 4.0 4.0
56 863f75b0 density-01 2026-01-17 04:44:59 2026-01-17 04:45:28 28.802725 0.0 0.0
57 863f75b0 density-01 2026-01-18 20:58:40 2026-01-18 20:59:31 50.60705 0.0 2.0
58 863f75b0 density-01 2026-01-19 23:14:35 2026-01-19 23:14:42 7.000839 0.0 1.0
59 863f75b0 density-01 2026-01-21 08:37:14 2026-01-21 08:37:36 21.819446 0.0 0.0
60 863f75b0 density-01 2026-01-22 19:13:35 2026-01-22 19:14:04 28.565642 0.0 2.0
61 863f75b0 density-01 2026-01-23 19:28:52 2026-01-23 19:29:04 12.176994 2.0 2.0
62 863f75b0 density-01 2026-01-24 20:03:25 2026-01-24 20:03:31 5.178072 10.0 14.0
63 863f75b0 density-01 2026-01-25 20:17:16 2026-01-25 20:17:21 5.166376 1.0 1.0
64 863f75b0 density-01 2026-01-26 21:46:37 2026-01-26 21:46:42 5.209291 1.0 1.0
65 863f75b0 density-01 2026-01-27 22:28:48 2026-01-27 22:29:00 12.137812 2.0 2.0
66 863f75b0 density-01 2026-01-28 23:20:23 2026-01-28 23:20:29 5.263236 0.0 0.0
67 863f75b0 density-01 2026-01-29 23:23:30 2026-01-29 23:23:42 12.166944 1.0 2.0
68 863f75b0 density-01 2026-01-31 01:21:12 2026-01-31 01:21:25 12.217324 0.0 0.0
69 863f75b0 density-01 2026-02-01 01:38:20 2026-02-01 01:38:25 5.193612 0.0 0.0
70 863f75b0 density-01 2026-02-02 02:29:24 2026-02-02 02:29:29 5.194286 0.0 0.0
71 863f75b0 density-01 2026-02-03 02:39:20 2026-02-03 02:39:25 5.202508 0.0 0.0
72 863f75b0 density-01 2026-02-04 02:47:17 2026-02-04 02:47:22 5.203643 0.0 0.0
73 863f75b0 density-01 2026-02-05 02:49:04 2026-02-05 02:49:17 12.182387 4.0 4.0
74 863f75b0 density-01 2026-02-06 02:50:49 2026-02-06 02:50:54 5.185432 2.0 4.0
75 863f75b0 density-01 2026-02-07 04:08:33 2026-02-07 04:08:39 5.214555 0.0 0.0
76 863f75b0 density-01 2026-02-08 04:55:54 2026-02-08 04:56:06 12.258212 2.0 2.0
77 863f75b0 density-01 2026-02-09 05:27:56 2026-02-09 05:28:08 12.210633 3.0 3.0
78 863f75b0 density-01 2026-02-10 05:28:49 2026-02-10 05:28:54 5.178586 6.0 6.0
79 863f75b0 density-01 2026-02-11 22:52:32 2026-02-11 22:52:40 8.563582 0.0 2.0
80 863f75b0 density-01 2026-02-12 23:00:01 2026-02-12 23:00:16 15.525545 3.0 3.0
81 863f75b0 density-01 2026-02-14 00:37:30 2026-02-14 00:38:21 50.708598 2.0 2.0
82 863f75b0 density-01 2026-02-15 02:17:10 2026-02-15 02:17:32 21.825311 1.0 1.0
83 863f75b0 density-01 2026-02-16 02:31:03 2026-02-16 02:31:08 5.482081 2.0 2.0
84 863f75b0 density-01 2026-02-17 02:48:13 2026-02-17 02:48:19 5.478879 0.0 0.0
85 863f75b0 density-01 2026-02-18 06:36:41 2026-02-18 06:37:09 28.189355 4.0 4.0
86 863f75b0 density-01 2026-02-19 06:51:05 2026-02-19 06:51:10 5.468128 1.0 1.0
87 863f75b0 density-01 2026-02-20 07:15:22 2026-02-20 07:15:44 21.872177 2.0 4.0
88 863f75b0 density-01 2026-02-21 08:12:15 2026-02-21 08:12:37 21.832343 4.0 4.0
89 863f75b0 density-01 2026-02-22 08:17:45 2026-02-22 08:18:14 28.849793 0.0 0.0
90 863f75b0 density-01 2026-02-23 09:55:45 2026-02-23 09:55:50 5.546099 6.0 6.0
91 863f75b0 density-01 2026-02-24 10:33:57 2026-02-24 10:34:02 5.555163 2.0 2.0
92 863f75b0 density-01 2026-02-25 11:02:34 2026-02-25 11:02:56 21.835883 6.0 7.0
93 863f75b0 density-01 2026-02-26 11:50:05 2026-02-26 11:50:10 5.644904 17.0 17.0
94 863f75b0 density-01 2026-02-27 12:03:35 2026-02-27 12:04:04 28.823213 4.0 8.0
95 863f75b0 density-01 2026-02-28 12:09:53 2026-02-28 12:10:15 21.97868 14.0 22.0
96 863f75b0 density-01 2026-03-01 12:12:38 2026-03-01 12:12:59 21.515399 7.0 11.0
97 863f75b0 density-01 2026-03-02 12:17:18 2026-03-02 12:17:24 5.5465219999999995 7.0 7.0
98 863f75b0 density-01 2026-03-03 13:30:14 2026-03-03 13:30:36 21.837461 7.0 10.0
99 863f75b0 density-01 2026-03-04 14:16:41 2026-03-04 14:16:47 5.615973 9.0 9.0
100 863f75b0 density-01 2026-03-05 14:37:06 2026-03-05 14:37:35 28.855245 15.0 18.0
101 863f75b0 density-01 2026-03-06 15:34:53 2026-03-06 15:34:58 5.591055 20.0 20.0
102 863f75b0 density-01 2026-03-07 15:53:43 2026-03-07 15:53:48 5.547173 23.0 23.0
103 863f75b0 density-01 2026-03-08 17:08:50 2026-03-08 17:08:56 6.603965 10.0 10.0
104 863f75b0 density-01 2026-03-09 17:10:15 2026-03-09 17:10:44 28.875973 9.0 12.0
105 863f75b0 density-01 2026-03-10 17:22:46 2026-03-10 17:22:54 8.567725 10.0 11.0
106 863f75b0 density-01 2026-03-11 17:37:24 2026-03-11 17:37:30 5.31581 4.0 4.0
107 863f75b0 density-01 2026-03-12 18:27:57 2026-03-12 18:28:06 8.861128 17.0 18.0
108 8d7eef22 hn-marahaus-01 2026-06-12 12:36:14 2026-06-12 12:36:42 27.980657 6.3 7.7
109 f4928534 hn-stadtgalerie-01 2026-10-16 08:08:25 2026-10-16 08:08:41 16.382029 24.5 25.2
110 f4928534 hn-stadtgalerie-01 2026-10-21 10:45:45 2026-10-21 10:46:01 16.370418 47.6 44.8
111 f4928534 hn-stadtgalerie-01 2026-10-21 17:54:16 2026-10-21 17:54:32 16.259417 11.2 12.6
112 f4928534 hn-stadtgalerie-01 2026-10-23 00:18:14 2026-10-23 00:18:30 16.346725 2.1 0.7
113 f4928534 hn-stadtgalerie-01 2026-10-24 09:42:12 2026-10-24 09:42:29 16.372927 65.1 65.8
114 f4928534 hn-stadtgalerie-01 2026-10-24 17:36:26 2026-10-24 17:36:42 16.350135 51.8 44.1
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,13 @@
sensor_id,name,period,n,first,last,min,max,n_unique,min_positive,effective_quantum,pct_mult_0_7,pct_integer,pct_mult_0_7_and_integer,pct_mult_0_7_only,pct_integer_only,pct_neither,pct_mult_0_5,pct_mult_0_1,mean,median,p95,zero_pct
f8c19730,density-01,current,17585,2026-04-16,2026-11-07,0.0,63.0,84,0.7,0.7,100.0,12.715382428205857,12.715382428205857,87.28461757179414,0.0,0.0,21.60363946545351,100.0,9.740608473130509,7.7,28.0,5.8686380437873185
00091a5f,hn-goetzentuem-01,current,9952,2026-06-12,2026-11-07,0.0,210.7,152,0.7,0.7,100.0,9.89750803858521,9.89750803858521,90.10249196141478,0.0,0.0,18.860530546623792,100.0,11.12403536977492,8.4,29.4,2.97427652733119
e52c55f3,hn-hagenbuchersee-01,current,12345,2026-06-12,2026-11-07,0.0,292.6,206,0.7,0.7,100.0,11.70514378290806,11.70514378290806,88.29485621709195,0.0,0.0,20.639935196435804,100.0,12.973673552045362,6.3,49.7,5.5569056298096395
0f8519cf,hn-hagenbuchersee-02,current,10099,2026-06-12,2026-11-07,0.0,200.9,202,0.7,0.7,100.0,8.297851272403209,8.297851272403209,91.70214872759679,0.0,0.0,19.60590157441331,100.0,19.2822655708486,11.2,64.4,0.6931379344489553
8d7eef22,hn-marahaus-01,current,9254,2026-06-12,2026-11-07,0.0,194.6,140,0.7,0.7,100.0,7.639939485627837,7.639939485627837,92.36006051437217,0.0,0.0,19.11605792089907,100.0,13.72571860816944,9.1,38.5,0.01080613788631943
d4cce74b,hn-neckarufer-01,current,9527,2026-06-12,2026-11-07,0.0,255.5,192,0.7,0.7,100.0,8.722577936391309,8.722577936391309,91.2774220636087,0.0,0.0,19.92232602078304,100.0,19.200440852314475,11.9,61.6,0.07347538574577515
ba71377f,hn-soleo-01,current,10327,2026-06-12,2026-11-07,0.0,111.3,107,0.7,0.7,100.0,12.336593395952358,12.336593395952358,87.66340660404765,0.0,0.0,21.710080371840807,100.0,10.580323424034086,8.4,28.0,4.773893676769633
f4928534,hn-stadtgalerie-01,current,8982,2026-06-12,2026-11-07,0.7,137.2,168,0.7,0.7,100.0,6.323758628367846,6.323758628367846,93.67624137163216,0.0,0.0,15.72032954798486,100.0,21.13513694054776,8.4,67.2,0.0
d3304b39,hn-bollwerksturm-01,episodic,5,2026-06-12,2026-08-13,6.3,16.1,5,6.3,0.7,100.0,20.0,20.0,80.0,0.0,0.0,20.0,100.0,10.78,9.1,15.96,0.0
2aadbeed,maker-fair,episodic,73,2026-08-13,2026-08-16,9.1,165.2,57,9.1,0.7,100.0,13.698630136986301,13.698630136986301,86.3013698630137,0.0,0.0,23.28767123287671,100.0,117.83013698630137,127.4,157.78,0.0
062fe216,niklas-01,episodic,8,2026-05-01,2026-05-01,2.1,6.3,3,2.1,0.7,100.0,0.0,0.0,100.0,0.0,0.0,0.0,100.0,4.55,4.9,6.3,0.0
863f75b0,density-01,legacy,8075,2025-10-16,2026-03-12,0.0,28.0,27,1.0,1.0,35.356037151702786,100.0,35.356037151702786,0.0,64.6439628482972,0.0,100.0,100.0,3.0080495356037154,2.0,11.0,29.77089783281734
1 sensor_id name period n first last min max n_unique min_positive effective_quantum pct_mult_0_7 pct_integer pct_mult_0_7_and_integer pct_mult_0_7_only pct_integer_only pct_neither pct_mult_0_5 pct_mult_0_1 mean median p95 zero_pct
2 f8c19730 density-01 current 17585 2026-04-16 2026-11-07 0.0 63.0 84 0.7 0.7 100.0 12.715382428205857 12.715382428205857 87.28461757179414 0.0 0.0 21.60363946545351 100.0 9.740608473130509 7.7 28.0 5.8686380437873185
3 00091a5f hn-goetzentuem-01 current 9952 2026-06-12 2026-11-07 0.0 210.7 152 0.7 0.7 100.0 9.89750803858521 9.89750803858521 90.10249196141478 0.0 0.0 18.860530546623792 100.0 11.12403536977492 8.4 29.4 2.97427652733119
4 e52c55f3 hn-hagenbuchersee-01 current 12345 2026-06-12 2026-11-07 0.0 292.6 206 0.7 0.7 100.0 11.70514378290806 11.70514378290806 88.29485621709195 0.0 0.0 20.639935196435804 100.0 12.973673552045362 6.3 49.7 5.5569056298096395
5 0f8519cf hn-hagenbuchersee-02 current 10099 2026-06-12 2026-11-07 0.0 200.9 202 0.7 0.7 100.0 8.297851272403209 8.297851272403209 91.70214872759679 0.0 0.0 19.60590157441331 100.0 19.2822655708486 11.2 64.4 0.6931379344489553
6 8d7eef22 hn-marahaus-01 current 9254 2026-06-12 2026-11-07 0.0 194.6 140 0.7 0.7 100.0 7.639939485627837 7.639939485627837 92.36006051437217 0.0 0.0 19.11605792089907 100.0 13.72571860816944 9.1 38.5 0.01080613788631943
7 d4cce74b hn-neckarufer-01 current 9527 2026-06-12 2026-11-07 0.0 255.5 192 0.7 0.7 100.0 8.722577936391309 8.722577936391309 91.2774220636087 0.0 0.0 19.92232602078304 100.0 19.200440852314475 11.9 61.6 0.07347538574577515
8 ba71377f hn-soleo-01 current 10327 2026-06-12 2026-11-07 0.0 111.3 107 0.7 0.7 100.0 12.336593395952358 12.336593395952358 87.66340660404765 0.0 0.0 21.710080371840807 100.0 10.580323424034086 8.4 28.0 4.773893676769633
9 f4928534 hn-stadtgalerie-01 current 8982 2026-06-12 2026-11-07 0.7 137.2 168 0.7 0.7 100.0 6.323758628367846 6.323758628367846 93.67624137163216 0.0 0.0 15.72032954798486 100.0 21.13513694054776 8.4 67.2 0.0
10 d3304b39 hn-bollwerksturm-01 episodic 5 2026-06-12 2026-08-13 6.3 16.1 5 6.3 0.7 100.0 20.0 20.0 80.0 0.0 0.0 20.0 100.0 10.78 9.1 15.96 0.0
11 2aadbeed maker-fair episodic 73 2026-08-13 2026-08-16 9.1 165.2 57 9.1 0.7 100.0 13.698630136986301 13.698630136986301 86.3013698630137 0.0 0.0 23.28767123287671 100.0 117.83013698630137 127.4 157.78 0.0
12 062fe216 niklas-01 episodic 8 2026-05-01 2026-05-01 2.1 6.3 3 2.1 0.7 100.0 0.0 0.0 100.0 0.0 0.0 0.0 100.0 4.55 4.9 6.3 0.0
13 863f75b0 density-01 legacy 8075 2025-10-16 2026-03-12 0.0 28.0 27 1.0 1.0 35.356037151702786 100.0 35.356037151702786 0.0 64.6439628482972 0.0 100.0 100.0 3.0080495356037154 2.0 11.0 29.77089783281734
+8
View File
@@ -0,0 +1,8 @@
column,dtype,non_null,null,example
time,"datetime64[us, UTC]",192464,0,2026-06-12 12:27:15.241799+00:00
sensor_id,str,192464,0,00091a5f
name,str,192464,0,hn-goetzentuem-01
longitude,float64,192464,0,9.2148042
latitude,float64,192464,0,49.1399582
type,str,192464,0,densityData
value,float64,192464,0,10.5
1 column dtype non_null null example
2 time datetime64[us, UTC] 192464 0 2026-06-12 12:27:15.241799+00:00
3 sensor_id str 192464 0 00091a5f
4 name str 192464 0 hn-goetzentuem-01
5 longitude float64 192464 0 9.2148042
6 latitude float64 192464 0 49.1399582
7 type str 192464 0 densityData
8 value float64 192464 0 10.5
+13
View File
@@ -0,0 +1,13 @@
sensor_id,name,longitude,latitude,rows,first,last,readings_density,readings_battery,location,role
863f75b0,density-01,9.217473,49.14344,16150,2025-10-16 21:17:59.316153+00:00,2026-03-12 18:28:06.042384+00:00,8075,8075,density-01,core
f8c19730,density-01,9.2174649,49.1434044,35170,2026-04-16 23:25:31.573918+00:00,2026-11-07 23:48:49.164923+00:00,17585,17585,density-01,core
00091a5f,hn-goetzentuem-01,9.2148042,49.1399582,19904,2026-06-12 12:27:15.241799+00:00,2026-11-07 23:39:53.740331+00:00,9952,9952,hn-goetzentuem-01,core
e52c55f3,hn-hagenbuchersee-01,9.217198,49.144402,24690,2026-06-12 12:02:16.143036+00:00,2026-11-07 23:46:49.051578+00:00,12345,12345,hn-hagenbuchersee-01,core
0f8519cf,hn-hagenbuchersee-02,9.2169499,49.1439097,20198,2026-06-12 12:32:37.714826+00:00,2026-11-07 23:51:57.342194+00:00,10099,10099,hn-hagenbuchersee-02,core
8d7eef22,hn-marahaus-01,9.2159119,49.141541,18508,2026-06-12 12:36:14.714757+00:00,2026-11-07 23:49:35.315114+00:00,9254,9254,hn-marahaus-01,core
d4cce74b,hn-neckarufer-01,9.216612,49.1433132,19054,2026-06-12 12:21:00.992399+00:00,2026-11-07 23:49:20.832221+00:00,9527,9527,hn-neckarufer-01,core
ba71377f,hn-soleo-01,9.2173925,49.1449327,20654,2026-06-12 11:52:36.889419+00:00,2026-11-07 23:32:32.859699+00:00,10327,10327,hn-soleo-01,core
f4928534,hn-stadtgalerie-01,9.2186987,49.1402425,17964,2026-06-12 12:11:38.733449+00:00,2026-11-07 23:26:29.605846+00:00,8982,8982,hn-stadtgalerie-01,core
d3304b39,hn-bollwerksturm-01,9.2180336,49.1456643,10,2026-06-12 11:19:46.253032+00:00,2026-08-13 16:01:25.495505+00:00,5,5,hn-bollwerksturm-01,episodic
2aadbeed,maker-fair,9.769265797482834,52.37609756665297,146,2026-08-13 16:07:26.589683+00:00,2026-08-16 15:06:01.464353+00:00,73,73,maker-fair,episodic
062fe216,niklas-01,9.1497052,49.2239178,16,2026-05-01 17:19:43.603332+00:00,2026-05-01 19:06:51.407202+00:00,8,8,niklas-01,episodic
1 sensor_id name longitude latitude rows first last readings_density readings_battery location role
2 863f75b0 density-01 9.217473 49.14344 16150 2025-10-16 21:17:59.316153+00:00 2026-03-12 18:28:06.042384+00:00 8075 8075 density-01 core
3 f8c19730 density-01 9.2174649 49.1434044 35170 2026-04-16 23:25:31.573918+00:00 2026-11-07 23:48:49.164923+00:00 17585 17585 density-01 core
4 00091a5f hn-goetzentuem-01 9.2148042 49.1399582 19904 2026-06-12 12:27:15.241799+00:00 2026-11-07 23:39:53.740331+00:00 9952 9952 hn-goetzentuem-01 core
5 e52c55f3 hn-hagenbuchersee-01 9.217198 49.144402 24690 2026-06-12 12:02:16.143036+00:00 2026-11-07 23:46:49.051578+00:00 12345 12345 hn-hagenbuchersee-01 core
6 0f8519cf hn-hagenbuchersee-02 9.2169499 49.1439097 20198 2026-06-12 12:32:37.714826+00:00 2026-11-07 23:51:57.342194+00:00 10099 10099 hn-hagenbuchersee-02 core
7 8d7eef22 hn-marahaus-01 9.2159119 49.141541 18508 2026-06-12 12:36:14.714757+00:00 2026-11-07 23:49:35.315114+00:00 9254 9254 hn-marahaus-01 core
8 d4cce74b hn-neckarufer-01 9.216612 49.1433132 19054 2026-06-12 12:21:00.992399+00:00 2026-11-07 23:49:20.832221+00:00 9527 9527 hn-neckarufer-01 core
9 ba71377f hn-soleo-01 9.2173925 49.1449327 20654 2026-06-12 11:52:36.889419+00:00 2026-11-07 23:32:32.859699+00:00 10327 10327 hn-soleo-01 core
10 f4928534 hn-stadtgalerie-01 9.2186987 49.1402425 17964 2026-06-12 12:11:38.733449+00:00 2026-11-07 23:26:29.605846+00:00 8982 8982 hn-stadtgalerie-01 core
11 d3304b39 hn-bollwerksturm-01 9.2180336 49.1456643 10 2026-06-12 11:19:46.253032+00:00 2026-08-13 16:01:25.495505+00:00 5 5 hn-bollwerksturm-01 episodic
12 2aadbeed maker-fair 9.769265797482834 52.37609756665297 146 2026-08-13 16:07:26.589683+00:00 2026-08-16 15:06:01.464353+00:00 73 73 maker-fair episodic
13 062fe216 niklas-01 9.1497052 49.2239178 16 2026-05-01 17:19:43.603332+00:00 2026-05-01 19:06:51.407202+00:00 8 8 niklas-01 episodic
+10
View File
@@ -0,0 +1,10 @@
sensor_id,name,spikes_ratio5,max_ratio,spike_times
00091a5f,hn-goetzentuem-01,3,10.0,"2026-08-01 08:09, 2026-10-07 20:16, 2026-11-02 06:42"
0f8519cf,hn-hagenbuchersee-02,7,22.000000000000004,"2026-08-22 17:23, 2026-09-05 19:45, 2026-09-19 21:29"
863f75b0,density-01,4,10.0,"2025-10-20 13:21, 2025-11-22 02:43, 2025-11-28 14:47"
8d7eef22,hn-marahaus-01,1,5.1000000000000005,2026-10-17 10:01
ba71377f,hn-soleo-01,7,9.333333333333336,"2026-06-19 09:25, 2026-06-29 23:40, 2026-07-19 10:26"
d4cce74b,hn-neckarufer-01,6,11.0,"2026-08-14 16:57, 2026-08-18 18:54, 2026-08-20 18:36"
e52c55f3,hn-hagenbuchersee-01,4,8.0,"2026-08-14 19:19, 2026-10-17 10:04, 2026-10-21 11:48"
f4928534,hn-stadtgalerie-01,6,10.5,"2026-06-29 23:38, 2026-08-11 10:43, 2026-08-13 11:16"
f8c19730,density-01,1,16.0,2026-10-09 02:19
1 sensor_id name spikes_ratio5 max_ratio spike_times
2 00091a5f hn-goetzentuem-01 3 10.0 2026-08-01 08:09, 2026-10-07 20:16, 2026-11-02 06:42
3 0f8519cf hn-hagenbuchersee-02 7 22.000000000000004 2026-08-22 17:23, 2026-09-05 19:45, 2026-09-19 21:29
4 863f75b0 density-01 4 10.0 2025-10-20 13:21, 2025-11-22 02:43, 2025-11-28 14:47
5 8d7eef22 hn-marahaus-01 1 5.1000000000000005 2026-10-17 10:01
6 ba71377f hn-soleo-01 7 9.333333333333336 2026-06-19 09:25, 2026-06-29 23:40, 2026-07-19 10:26
7 d4cce74b hn-neckarufer-01 6 11.0 2026-08-14 16:57, 2026-08-18 18:54, 2026-08-20 18:36
8 e52c55f3 hn-hagenbuchersee-01 4 8.0 2026-08-14 19:19, 2026-10-17 10:04, 2026-10-21 11:48
9 f4928534 hn-stadtgalerie-01 6 10.5 2026-06-29 23:38, 2026-08-11 10:43, 2026-08-13 11:16
10 f8c19730 density-01 1 16.0 2026-10-09 02:19
+171
View File
@@ -0,0 +1,171 @@
{
"rows": 192464,
"density_rows": 96232,
"battery_rows": 96232,
"sensors_total": 12,
"sensors_core": 9,
"locations_core": 8,
"sensors_episodic": 3,
"time_min": "2025-10-16T21:17:59.316153+00:00",
"time_max": "2026-11-07T23:51:57.342194+00:00",
"panel_start": "2026-06-18T00:00:00+00:00",
"panel_end": "2026-11-08T00:00:00+00:00",
"panel_days": 143,
"panel_slots_total": 13728,
"all8_slots": 3876,
"all8_pct_panel": 28.23,
"all8_pct_any": 28.38,
"all8_expected_independence": 1281.5,
"all8_pct_30min": 63.37412587412587,
"all8_pct_60min": 69.72610722610723,
"all8_pct_120min": 70.97902097902097,
"any_slots": 13659,
"locations_per_slot_hist": {
"0": 69,
"1": 594,
"2": 1133,
"3": 721,
"4": 615,
"5": 769,
"6": 1834,
"7": 4117,
"8": 3876
},
"monthly_coverage_min_pct": 23.32,
"monthly_coverage_median_pct": 86.08500000000001,
"all_zero_pct": 5.18,
"all_lt1_pct": 10.36,
"all_lt5_pct": 39.57,
"share_multiple_0_7": 0.9458,
"quantization_summary": {
"devices_on_0_7_grid": 11,
"devices_integer_only": 1,
"devices_with_neither": 0,
"pct_neither_overall": 0.0,
"pct_integer_overall": 17.6344667054618
},
"density01_old": {
"series": "legacy 863f75b0",
"n": 8075.0,
"first": "2025-10-16T21:17:59.316153+00:00",
"last": "2026-03-12T18:28:06.032195+00:00",
"min": 0.0,
"max": 28.0,
"mean": 3.0080495356037154,
"median": 2.0,
"p95": 11.0,
"p99": 16.26000000000022,
"zero_pct": 29.77089783281734,
"n_unique": 27.0,
"effective_quantum": 1.0
},
"density01_new": {
"series": "new f8c19730",
"n": 17585.0,
"first": "2026-04-16T23:25:31.573918+00:00",
"last": "2026-11-07T23:48:49.162433+00:00",
"min": 0.0,
"max": 63.0,
"mean": 9.740608473130509,
"median": 7.7,
"p95": 28.0,
"p99": 38.5,
"zero_pct": 5.8686380437873185,
"n_unique": 84.0,
"effective_quantum": 0.7
},
"density01_gap_days": 35.20654562179398,
"cadence_legacy_median_s": 910.859239,
"cadence_new_density_median_s": 968.515136,
"corr_pearson_mean_offdiag": 0.5820317643141066,
"corr_pearson_min_offdiag": 0.13049968301306178,
"corr_pearson_max_offdiag": 0.9035816465731278,
"corr_spearman_mean_offdiag": 0.6782661930066668,
"corr_log1p_mean_offdiag": 0.6701882554258124,
"corr_detrended_mean_offdiag": 0.3848365272415827,
"corr_daily_mean_offdiag": 0.4774750913197265,
"daily_corr_days": 69,
"cv_median": 0.6614114460889224,
"cv_p90": 1.0540925533894598,
"cv_p95": 1.1837878154736543,
"share_cv_median": 0.21604098026935642,
"share_cv_p95": 0.46300905467601366,
"hourly_shape_mean_r": 0.7935830114859298,
"peak_hour_local": {
"density-01": 19,
"hn-goetzentuem-01": 19,
"hn-hagenbuchersee-01": 20,
"hn-hagenbuchersee-02": 21,
"hn-marahaus-01": 19,
"hn-neckarufer-01": 20,
"hn-soleo-01": 19,
"hn-stadtgalerie-01": 16
},
"weekend_weekday_ratio": {
"density-01": 1.182,
"hn-goetzentuem-01": 1.118,
"hn-hagenbuchersee-01": 1.237,
"hn-hagenbuchersee-02": 1.194,
"hn-marahaus-01": 1.166,
"hn-neckarufer-01": 1.139,
"hn-soleo-01": 0.977,
"hn-stadtgalerie-01": 0.812
},
"ratio_stability": {
"good_days": 69,
"panel_days": 143,
"median_daily_share_cv": 0.21604098026935642,
"max_daily_share_cv": 0.5080935272392274,
"median_pairwise_log_ratio_sd": 0.39852755944675217,
"p90_pairwise_log_ratio_sd": 0.7106193041695669,
"share_cv": {
"density-01": 0.166,
"hn-goetzentuem-01": 0.2215,
"hn-hagenbuchersee-01": 0.3616,
"hn-hagenbuchersee-02": 0.2071,
"hn-marahaus-01": 0.2106,
"hn-neckarufer-01": 0.1592,
"hn-soleo-01": 0.5081,
"hn-stadtgalerie-01": 0.3793
}
},
"coverage_bias_ratio_median": 0.6876158913031751,
"coverage_bias_ratio_min": 0.37987408192540045,
"coverage_bias_ratio_max": 1.4938524590163935,
"august_coverage_battery_r": 0.8439269706949696,
"duplicates": {
"rows_total": 192464,
"exact_duplicate_rows": 0,
"duplicate_time_sensor_type": 0,
"duplicate_time_sensor": 0,
"duplicate_sensor_name": 1,
"density_negative": 0,
"density_nan": 0,
"battery_negative": 0,
"battery_over_100": 0,
"battery_nan": 0,
"intervals_lt_60s_core": 113
},
"battery_structural": {
"dens_rows": 96232,
"battery_rows": 96232,
"density_rows_with_battery_60s": 96232,
"density_rows_without_battery_60s": 0,
"core_panel_density_rows": 82170,
"core_panel_active_devices": 8,
"core_panel_expected_rows": 109824,
"core_panel_battery_coverage_pct": 74.81971153846153
},
"battery_missing_pct": 0.0,
"battery_median_all": 17.0,
"battery_share_lt20": 51.3,
"battery_share_zero": 1.6,
"battery_offset_median_s": 0.0326995,
"battery_offset_p99_s": 5.818018220000006,
"near_duplicate_intervals": 113,
"max_flatline_run": 20,
"max_flatline_run_hours": 5.0,
"max_value": 292.6,
"max_value_sensor": "hn-hagenbuchersee-01",
"max_value_time": "2026-08-14T19:19:06.811528+00:00"
}
+16
View File
@@ -0,0 +1,16 @@
sensor_id,name,time,value,prev,next,ratio_to_local_median
0f8519cf,hn-hagenbuchersee-02,2026-08-22 17:23,46.2,1.4,0.7,22.000000000000004
f8c19730,density-01,2026-10-09 02:19,11.2,0.0,0.0,16.0
0f8519cf,hn-hagenbuchersee-02,2026-09-19 21:29,67.2,19.6,18.9,12.0
d4cce74b,hn-neckarufer-01,2026-09-08 17:07,77.0,11.2,69.3,11.0
f4928534,hn-stadtgalerie-01,2026-08-13 11:16,44.1,9.1,3.5,10.5
ba71377f,hn-soleo-01,2026-08-22 16:19,19.6,7.7,0.7,9.333333333333336
863f75b0,density-01,2025-11-22 02:43,24.0,7.0,1.0,8.888888888888888
f4928534,hn-stadtgalerie-01,2026-08-11 10:43,77.7,8.4,44.1,8.538461538461538
d4cce74b,hn-neckarufer-01,2026-09-08 19:09,69.3,77.0,4.9,8.25
f4928534,hn-stadtgalerie-01,2026-08-15 13:19,67.2,58.1,8.4,7.384615384615385
f4928534,hn-stadtgalerie-01,2026-06-29 23:38,15.4,0.7,1.4,7.333333333333335
f4928534,hn-stadtgalerie-01,2026-08-15 12:48,64.4,4.9,58.1,7.0769230769230775
ba71377f,hn-soleo-01,2026-06-29 23:40,14.0,1.4,1.4,6.666666666666668
ba71377f,hn-soleo-01,2026-09-08 18:34,31.5,19.6,2.8,6.428571428571428
f4928534,hn-stadtgalerie-01,2026-08-15 13:03,58.1,64.4,67.2,6.384615384615385
1 sensor_id name time value prev next ratio_to_local_median
2 0f8519cf hn-hagenbuchersee-02 2026-08-22 17:23 46.2 1.4 0.7 22.000000000000004
3 f8c19730 density-01 2026-10-09 02:19 11.2 0.0 0.0 16.0
4 0f8519cf hn-hagenbuchersee-02 2026-09-19 21:29 67.2 19.6 18.9 12.0
5 d4cce74b hn-neckarufer-01 2026-09-08 17:07 77.0 11.2 69.3 11.0
6 f4928534 hn-stadtgalerie-01 2026-08-13 11:16 44.1 9.1 3.5 10.5
7 ba71377f hn-soleo-01 2026-08-22 16:19 19.6 7.7 0.7 9.333333333333336
8 863f75b0 density-01 2025-11-22 02:43 24.0 7.0 1.0 8.888888888888888
9 f4928534 hn-stadtgalerie-01 2026-08-11 10:43 77.7 8.4 44.1 8.538461538461538
10 d4cce74b hn-neckarufer-01 2026-09-08 19:09 69.3 77.0 4.9 8.25
11 f4928534 hn-stadtgalerie-01 2026-08-15 13:19 67.2 58.1 8.4 7.384615384615385
12 f4928534 hn-stadtgalerie-01 2026-06-29 23:38 15.4 0.7 1.4 7.333333333333335
13 f4928534 hn-stadtgalerie-01 2026-08-15 12:48 64.4 4.9 58.1 7.0769230769230775
14 ba71377f hn-soleo-01 2026-06-29 23:40 14.0 1.4 1.4 6.666666666666668
15 ba71377f hn-soleo-01 2026-09-08 18:34 31.5 19.6 2.8 6.428571428571428
16 f4928534 hn-stadtgalerie-01 2026-08-15 13:03 58.1 64.4 67.2 6.384615384615385
+8
View File
@@ -0,0 +1,8 @@
,density-01,hn-goetzentuem-01,hn-hagenbuchersee-01,hn-hagenbuchersee-02,hn-marahaus-01,hn-neckarufer-01,hn-soleo-01,hn-stadtgalerie-01
0,7.857891637220259,8.401079414032383,9.301931464174455,15.997960725075528,11.844484958979033,14.975151515151515,9.262416604892513,21.050334448160537
1,9.177270029673592,10.431618195376585,11.008578008059875,17.123521320495186,13.029757243539546,17.150283171521036,10.73209219858156,21.143724023275148
2,9.468356643356644,10.440466392318244,11.819415807560137,17.48748344370861,12.862742382271467,19.640643491124262,10.81826484018265,22.055756578947367
3,9.363338973491258,11.80184331797235,13.134139637163276,17.57578914141414,12.144641638225258,17.98394495412844,10.550917141049968,23.25315870570108
4,11.038733183856502,12.309121621621621,14.787117903930131,22.408823529411766,15.407436260623228,21.418745692625773,11.854551539491299,24.620918754745635
5,12.708600337268127,15.63704215618521,18.155184135977336,24.897647897362795,17.521148036253777,24.843715469613258,11.801159618008185,29.773125463994063
6,9.475880281690142,8.241690751445086,11.559130434782608,18.377734375,12.92240663900415,16.701957831325302,8.999710564399422,6.646423357664234
1 density-01 hn-goetzentuem-01 hn-hagenbuchersee-01 hn-hagenbuchersee-02 hn-marahaus-01 hn-neckarufer-01 hn-soleo-01 hn-stadtgalerie-01
2 0 7.857891637220259 8.401079414032383 9.301931464174455 15.997960725075528 11.844484958979033 14.975151515151515 9.262416604892513 21.050334448160537
3 1 9.177270029673592 10.431618195376585 11.008578008059875 17.123521320495186 13.029757243539546 17.150283171521036 10.73209219858156 21.143724023275148
4 2 9.468356643356644 10.440466392318244 11.819415807560137 17.48748344370861 12.862742382271467 19.640643491124262 10.81826484018265 22.055756578947367
5 3 9.363338973491258 11.80184331797235 13.134139637163276 17.57578914141414 12.144641638225258 17.98394495412844 10.550917141049968 23.25315870570108
6 4 11.038733183856502 12.309121621621621 14.787117903930131 22.408823529411766 15.407436260623228 21.418745692625773 11.854551539491299 24.620918754745635
7 5 12.708600337268127 15.63704215618521 18.155184135977336 24.897647897362795 17.521148036253777 24.843715469613258 11.801159618008185 29.773125463994063
8 6 9.475880281690142 8.241690751445086 11.559130434782608 18.377734375 12.92240663900415 16.701957831325302 8.999710564399422 6.646423357664234
+13
View File
@@ -0,0 +1,13 @@
sensor_id,name,n,zeros,zero_pct,lt1_pct,lt5_pct,median_nonzero,mean_nonzero,negative,nan
f8c19730,density-01,17585,1032,5.9,13.2,39.7,8.4,10.347888600253729,0,0
e52c55f3,hn-hagenbuchersee-01,12345,686,5.6,14.3,45.7,7.0,13.737027189295823,0,0
ba71377f,hn-soleo-01,10327,493,4.8,9.7,32.1,9.1,11.110738255033556,0,0
0f8519cf,hn-hagenbuchersee-02,10099,70,0.7,3.0,30.4,11.9,19.41685113171802,0,0
00091a5f,hn-goetzentuem-01,9952,296,3.0,11.3,37.8,8.4,11.46503728251864,0,0
d4cce74b,hn-neckarufer-01,9527,7,0.1,0.5,23.2,11.9,19.214558823529412,0,0
8d7eef22,hn-marahaus-01,9254,1,0.0,1.8,33.9,9.1,13.727201988544255,0,0
f4928534,hn-stadtgalerie-01,8982,0,0.0,9.3,40.8,8.4,21.13513694054776,0,0
863f75b0,density-01,8075,2404,29.8,29.8,77.8,3.0,4.283195203667783,0,0
2aadbeed,maker-fair,73,0,0.0,0.0,0.0,127.4,117.83013698630137,0,0
062fe216,niklas-01,8,0,0.0,0.0,75.0,4.9,4.55,0,0
d3304b39,hn-bollwerksturm-01,5,0,0.0,0.0,0.0,9.1,10.78,0,0
1 sensor_id name n zeros zero_pct lt1_pct lt5_pct median_nonzero mean_nonzero negative nan
2 f8c19730 density-01 17585 1032 5.9 13.2 39.7 8.4 10.347888600253729 0 0
3 e52c55f3 hn-hagenbuchersee-01 12345 686 5.6 14.3 45.7 7.0 13.737027189295823 0 0
4 ba71377f hn-soleo-01 10327 493 4.8 9.7 32.1 9.1 11.110738255033556 0 0
5 0f8519cf hn-hagenbuchersee-02 10099 70 0.7 3.0 30.4 11.9 19.41685113171802 0 0
6 00091a5f hn-goetzentuem-01 9952 296 3.0 11.3 37.8 8.4 11.46503728251864 0 0
7 d4cce74b hn-neckarufer-01 9527 7 0.1 0.5 23.2 11.9 19.214558823529412 0 0
8 8d7eef22 hn-marahaus-01 9254 1 0.0 1.8 33.9 9.1 13.727201988544255 0 0
9 f4928534 hn-stadtgalerie-01 8982 0 0.0 9.3 40.8 8.4 21.13513694054776 0 0
10 863f75b0 density-01 8075 2404 29.8 29.8 77.8 3.0 4.283195203667783 0 0
11 2aadbeed maker-fair 73 0 0.0 0.0 0.0 127.4 117.83013698630137 0 0
12 062fe216 niklas-01 8 0 0.0 0.0 75.0 4.9 4.55 0 0
13 d3304b39 hn-bollwerksturm-01 5 0 0.0 0.0 0.0 9.1 10.78 0 0
File diff suppressed because it is too large. Load diff
+240
View File
@@ -0,0 +1,240 @@
"""Generate the paper figures from the delivered CSV and the trained model.
Run from the ML worktree so the `pulsecast` package is importable:
cd ML-py.git/feat-ml-bootstrap
PYTHONPATH=src .venv/bin/python \
/home/noel/Documents/Hackathon_2026/paper/research/scripts/02_paper_figures.py
Writes to `paper/figures/`.
"""
from __future__ import annotations
import json
from pathlib import Path
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from matplotlib.patches import FancyArrowPatch, FancyBboxPatch
ROOT = Path("/home/noel/Documents/Hackathon_2026")
PAPER = ROOT / "paper"
FIG = PAPER / "figures"
FIG.mkdir(parents=True, exist_ok=True)
CSV = ROOT / "data" / "raw" / "schwarz-pax.csv"
SENSORS = [
"hn-stadtgalerie-01",
"hn-neckarufer-01",
"hn-hagenbuchersee-02",
"hn-marahaus-01",
"hn-hagenbuchersee-01",
"hn-goetzentuem-01",
"hn-soleo-01",
"density-01",
]
plt.rcParams.update(
{
"figure.dpi": 160,
"savefig.dpi": 160,
"font.size": 8,
"axes.titlesize": 9,
"axes.labelsize": 8,
"legend.fontsize": 7,
"xtick.labelsize": 7,
"ytick.labelsize": 7,
}
)
def sensor_map() -> None:
df = pd.read_csv(CSV)
d = df[(df["type"] == "densityData") & (df["name"].isin(SENSORS))]
coords = (
d.groupby("name")[["longitude", "latitude"]]
.agg(["median"])
.droplevel(1, axis=1)
.reindex(SENSORS)
)
labels = {
"hn-stadtgalerie-01": "Stadtgalerie",
"hn-neckarufer-01": "Neckarufer",
"hn-hagenbuchersee-02": "Hagenbuchersee 02",
"hn-marahaus-01": "Marahaus",
"hn-hagenbuchersee-01": "Hagenbuchersee 01",
"hn-goetzentuem-01": "Götzenturm",
"hn-soleo-01": "Soleo",
"density-01": "density-01",
}
fig, ax = plt.subplots(figsize=(7.0, 3.5))
for i, (name, row) in enumerate(coords.iterrows(), start=1):
ax.scatter(row["longitude"], row["latitude"], s=46, c="#0b6e99", zorder=3)
ax.annotate(
str(i),
(row["longitude"], row["latitude"]),
xytext=(4, 4),
textcoords="offset points",
fontsize=7,
color="#0b3d4f",
fontweight="bold",
)
ax.set_xlabel("Longitude")
ax.set_ylabel("Latitude")
ax.set_title("Eight PAX counter locations, Heilbronn inner city")
ax.grid(True, color="#e5e5e5", linewidth=0.5)
ax.set_aspect("equal", adjustable="datalim")
handles = [
plt.Line2D(
[0],
[0],
marker="o",
color="w",
markerfacecolor="#0b6e99",
markersize=5,
label=f"{i} {labels[name]}",
)
for i, name in enumerate(coords.index, start=1)
]
ax.legend(
handles=handles,
loc="center left",
bbox_to_anchor=(1.01, 0.5),
frameon=False,
fontsize=6.5,
handletextpad=0.4,
)
fig.tight_layout()
fig.savefig(FIG / "fig_sensor_map.png", bbox_inches="tight")
plt.close(fig)
def forecast_plot() -> None:
from pulsecast import config
from pulsecast.model import fit, metrics
from pulsecast.train import _load_inputs
hourly, features, archive, forecast, meta, audit = _load_inputs()
data_end = hourly["t"].max()
test_start = data_end.floor("h") - pd.Timedelta(days=config.TEST_DAYS)
train = features[features["t"] < test_start].copy()
test = features[features["t"] >= test_start].copy()
eval_frame = test[test["lag_168"].notna() & test["y"].notna()].copy()
model = fit(train, config.FEATURE_WEEK_WEATHER)
pred = np.clip(model.predict(eval_frame[config.FEATURE_WEEK_WEATHER]), 0.0, None)
eval_frame["pred"] = pred
sensor = "hn-stadtgalerie-01"
g = eval_frame[eval_frame["sensor"] == sensor].sort_values("t").copy()
if g.empty:
sensor = sorted(eval_frame["sensor"].unique())[0]
g = eval_frame[eval_frame["sensor"] == sensor].sort_values("t").copy()
residual = g["y"].to_numpy() - g["pred"].to_numpy()
sd = float(np.std(residual, ddof=1))
t = pd.to_datetime(g["t"])
fig, ax = plt.subplots(figsize=(6.4, 3.0))
ax.plot(t, g["y"], color="#222222", linewidth=1.0, label="actual")
ax.plot(t, g["pred"], color="#0b6e99", linewidth=1.4, label="forecast")
ax.fill_between(
t,
np.clip(g["pred"] - 1.96 * sd, 0, None),
g["pred"] + 1.96 * sd,
color="#0b6e99",
alpha=0.16,
linewidth=0,
label="±1.96 residual SD",
)
ax.set_title(f"Forecast vs actual, {sensor}, 28-day holdout")
ax.set_ylabel("published index")
ax.grid(True, color="#e5e5e5", linewidth=0.5)
ax.legend(loc="upper left", frameon=False, ncol=3)
fig.autofmt_xdate(rotation=0, ha="center")
fig.tight_layout()
fig.savefig(FIG / "fig_forecast.png", bbox_inches="tight")
plt.close(fig)
# Save the numbers used in the plot for reproducibility.
(FIG / "fig_forecast_meta.json").write_text(
json.dumps(
{
"sensor": sensor,
"n": int(len(g)),
"residual_sd": sd,
"metrics": metrics(g["y"], g["pred"]),
},
indent=2,
)
+ "\n"
)
def architecture() -> None:
fig, ax = plt.subplots(figsize=(7.8, 4.2))
ax.set_xlim(0, 11)
ax.set_ylim(0, 6.5)
ax.axis("off")
def box(x, y, w, h, text, fc="#eef5f9", ec="#0b6e99"):
p = FancyBboxPatch(
(x, y),
w,
h,
boxstyle="round,pad=0.05,rounding_size=0.12",
linewidth=1.0,
edgecolor=ec,
facecolor=fc,
)
ax.add_patch(p)
ax.text(x + w / 2, y + h / 2, text, ha="center", va="center", fontsize=7)
def arrow(x1, y1, x2, y2, ls="-"):
ax.add_patch(
FancyArrowPatch(
(x1, y1),
(x2, y2),
arrowstyle="-|>",
mutation_scale=9,
linewidth=0.9,
color="#444444",
linestyle=ls,
)
)
box(0.2, 4.9, 1.8, 0.8, "phones / laptop\nNuxt SSR + Onyx")
box(2.5, 4.9, 1.8, 0.8, "Cloudflare\nedge + Tunnel")
box(4.8, 4.9, 1.5, 0.8, "cloudflared\npod")
box(7.0, 5.2, 1.7, 0.8, "Nuxt SSR\nweb pod")
box(7.0, 3.6, 1.7, 0.8, "Go API\nSSE + cache")
box(9.3, 3.6, 1.5, 0.8, "FastAPI ML\npredict / simulate")
box(1.2, 1.2, 2.3, 0.8, "build-time batch\nCSV → model + JSON")
box(4.2, 1.2, 2.3, 0.8, "baked images\nmodel + fallback JSON")
arrow(2.0, 5.3, 2.5, 5.3)
arrow(4.3, 5.3, 4.8, 5.3)
arrow(5.55, 5.3, 7.0, 5.6)
arrow(5.55, 5.15, 7.0, 4.0)
arrow(7.85, 5.2, 7.85, 4.4)
arrow(8.7, 4.0, 9.3, 4.0)
arrow(3.5, 1.6, 4.2, 1.6)
arrow(5.35, 2.0, 7.2, 3.6, ls="--")
arrow(5.9, 2.0, 9.6, 3.6, ls="--")
ax.text(8.0, 4.55, "internal HTTP", fontsize=6, color="#555555")
ax.text(7.05, 3.42, "cached fallback", fontsize=6, color="#555555")
ax.text(6.1, 2.35, "baked at build time", fontsize=6, color="#555555")
ax.set_title("PulseCast deployment: three services, Cloudflare Tunnel, build-time batch")
fig.tight_layout()
fig.savefig(FIG / "fig_architecture.png", bbox_inches="tight")
plt.close(fig)
if __name__ == "__main__":
sensor_map()
forecast_plot()
architecture()
print("wrote figures to", FIG)