diff --git a/OUTLINE.md b/OUTLINE.md index dd78d80..f772693 100644 --- a/OUTLINE.md +++ b/OUTLINE.md @@ -1,6 +1,6 @@ # PulseCast paper — writing plan -Status: **Version A draft written** (`paper/main.md`, ~3,600 words); **Version B draft written** (`paper/main-extended.md`, ~9,600 words, 37 references). Research notes are in `paper/research/`. +Status: **Version A draft written** (`paper/main.md`, ~3,600 words, 9 references); **Version B draft written** (`paper/main-extended.md`, ~9,600 words, 43 references), with the privacy/legal review corrections applied. Research notes are in `paper/research/`. ## Title options @@ -120,7 +120,9 @@ Keep the structure required by `agents/paper.md` and `OKF_Vault/mission/paper.md - [x] Per-sensor MAE table — computed from the production 7-day model. - [x] Metrics reconciliation — canonical: production 3.32/27.5/0.777; day-ahead 3.36/27.8/0.763. The mission brief’s 3.36/27.8/0.779 was the quick baseline; explained once. - [x] Live app/API URLs, image sizes, plan latency. -- [ ] Plan screenshot, public code repo URL, authors/affiliation, cold deployment time. +- [x] Author name and ORCID (Noel Sabia); affiliation intentionally blank. +- [x] Code release timing documented (after the hackathon). +- [ ] Plan figure with raw screenshot (current figure is a print reconstruction), cold deployment time. - [ ] Figure generation — use the real CSV; no synthetic data in the paper. - [ ] Author names/affiliation for the Zenodo record. - [ ] Decision: Version A only, Version B only, or both. diff --git a/README.md b/README.md index 124847b..0957fd1 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ paper/figures/fig_plan.png paper/figures/fig_forecast.png ``` -`fig_plan.png` is a layout placeholder; replace it with a live screenshot before submission. +`fig_plan.png` is a print reconstruction of the provided live plan screen (same values: 5 staff, 100% coverage, €3,360, 25 h over / 0 h under). Replace it with the raw screenshot file if you prefer the exact pixels. ## Verification status @@ -35,18 +35,18 @@ paper/figures/fig_forecast.png - Plan latency: 42.8 ms local v1.1; contract < 2 s. - 120-client Cloudflare k6: 120/120 SSE opened and closed, 0 connection errors, p95 event 1.03 s, p95 scenario 194 ms, presenter POST 82.69 ms, 2,760 events, 4m53s. - Per-sensor MAE: computed from the production 7-day model; `n ≈ 669` per sensor. -- Figures: generated from the raw CSV and the trained model; the plan screen is a placeholder. -- References: 12 in Version A, 37 in Version B; all are real sources from the research notes. Verify DOIs/URLs before submission. -- AI disclosure: included in both versions. +- Figures: generated from the raw CSV and the trained model; the plan screen is a print reconstruction of the provided screenshot. +- References: 9 in Version A, 43 in Version B; all are real sources from the research notes. Verify DOIs/URLs before submission. +- AI disclosure: included in both versions; the text says the authors checked each number and citation, and flags legal/guidance/inference sources rather than claiming universal verification. ## Known TODOs - [x] Canonical metrics (3.32/27.5/0.777 production; 3.36/27.8/0.763 day-ahead). - [x] 120-client k6 result, image sizes, and plan latency. - [x] Live app/API URLs. -- [ ] Live plan-screen screenshot (the current figure is a labelled placeholder). -- [ ] Public code repository URL (demo is filled; code is still `[TODO]`). -- [ ] Author names and affiliation. +- [ ] Plan figure: currently reconstructed from the provided screenshot for print legibility. Replace with the raw screenshot file if you prefer the exact pixels. +- [x] Code release timing documented (after the hackathon); public URL to be added then. +- [x] Author name and ORCID (Noel Sabia, 0009-0008-3064-7573); affiliation intentionally blank. - [ ] Cold deployment time. - [ ] Replace internal project-note citations with public artifacts if a venue requires it. - [ ] Final page-count trim for Version A if a strict 4–5 page limit applies. diff --git a/figures/fig_plan.png b/figures/fig_plan.png index 79dc6c3..95e6cf1 100644 Binary files a/figures/fig_plan.png and b/figures/fig_plan.png differ diff --git a/main-extended.md b/main-extended.md index 197e3a3..e364456 100644 --- a/main-extended.md +++ b/main-extended.md @@ -1,13 +1,13 @@ # The Pulse Is Not the People: A Code-Level Audit of Bluetooth Footfall Sensing and a Privacy-First Architecture for Urban Decisions -**Authors:** [AUTHOR TODO] -**Affiliation:** [AFFILIATION TODO] +**Author:** Noel Sabia (ORCID 0009-0008-3064-7573) +**Affiliation:** — **Date:** 14 November 2026 -**Companion artifact:** `paper/main.md` (4–5 page hackathon report) · [code repository TODO] · [demo URL TODO] +**Companion artifact:** `paper/main.md` (4–5 page hackathon report) · https://pulse.noelsabia.dev · code to be released after the hackathon ## Abstract -Low-cost Bluetooth footfall counters are increasingly used to justify staffing, advertising, and urban-planning decisions. Their central claim is that they measure pedestrian density while preserving privacy. We audit that claim on the eight-sensor PAX counter network deployed in Heilbronn, Germany. A code-level review of the public firmware shows that each published “15-minute” value is a **~10-second** passive BLE scan: the node counts distinct locally administered addresses, truncates the last two MAC bytes into a 16-bit ID, multiplies by a fixed factor (default 0.7), and sleeps for 900 seconds. The effective duty cycle is about 1.1%. The data confirm the semantics: all 83,156 density values in the eight-sensor panel are exact multiples of 0.7; reconstructed counts have median 12 and maximum 418; a hardware replacement produces a 35.2-day gap and a scale break from a 1.0 integer grid to a 0.7 grid. Coverage is sparse: only 28.2% of 15-minute slots contain all eight sensors, 5.2% are all-zero, and cross-sensor correlation falls from 0.58 to 0.38 after removing hour-of-week structure. Legally, MAC addresses are very likely personal data in this context; truncation is pseudonymisation, not anonymisation; and passive sniffing under § 25 TDDDG is contested. We then propose a privacy-first architecture: thermal/radar/camera/probe/manual sensing with calibration, a network-aware hierarchical spatial model, a quantile forecast with conformal calibration, and a decision layer evaluated by regret rather than MAE alone. We show how the same decision-grade contract transfers directly to staffing and advertising, conditionally to waste routing, as a sibling to sensor placement and LoRaWAN planning, and not at all to sentiment or learning-guide tasks. The contribution is not a better people counter; it is an honest measurement model and a staged path to one. +Low-cost Bluetooth footfall counters are increasingly used to justify staffing, advertising, and urban-planning decisions. Their central claim is that they measure pedestrian density while preserving privacy. We audit that claim on the eight-sensor PAX counter network deployed in Heilbronn, Germany. A code-level review of the public firmware shows that each published “15-minute” value is a **~10-second** passive BLE scan: the node counts distinct locally administered addresses, truncates the last two MAC bytes into a 16-bit ID, multiplies by a fixed factor (default 0.7), and sleeps for 900 seconds. The effective duty cycle is about 1.1%. The data confirm the semantics: all 83,156 density values in the eight-sensor panel are exact multiples of 0.7; reconstructed counts have median 12 and maximum 418; a hardware replacement produces a 35.2-day gap and a scale break from a 1.0 integer grid to a 0.7 grid. Coverage is sparse: only 28.2% of 15-minute slots contain all eight sensors, 5.18% of individual readings are zero, and cross-sensor correlation falls from 0.58 to 0.38 after removing hour-of-week structure. Legally, MAC addresses are very likely personal data in this context; truncation does not anonymise — it produces a weak, transient identifier that may be pseudonymous in context; and passive sniffing under § 25 TDDDG is contested. We then propose a privacy-first architecture: thermal/radar/camera/probe/manual sensing with calibration, a network-aware hierarchical spatial model, a quantile forecast with conformal calibration, and a decision layer evaluated by regret rather than MAE alone. We show how the same decision-grade contract transfers directly to staffing and advertising, conditionally to waste routing, as a sibling to sensor placement and LoRaWAN planning, and not at all to sentiment or learning-guide tasks. The contribution is not a better people counter; it is an honest measurement model and a staged path to one. ## 1. Introduction @@ -23,7 +23,7 @@ We make five contributions: 1. **A code-level audit** of the deployed Code for Heilbronn `lorax` firmware, with the exact measurement chain: passive BLE scan, locally-administered-address filter, 16-bit truncation, per-sensor factor 0.7, 900-second sleep, RSSI threshold −80 dBm. 2. **A forensic data analysis** of 192,464 rows, confirming the semantics in 83,156 values (100% multiples of 0.7), identifying the `density-01` calibration break, and quantifying coverage, zero-inflation, correlation, and missingness. -3. **A privacy and legal analysis**: why MAC addresses are likely personal data, why truncation is pseudonymisation rather than anonymisation, why § 25 TDDDG / Art. 5(3) ePrivacy is the decisive unresolved question, and what a DPIA-compliant deployment would require. +3. **A privacy and legal analysis**: why MAC addresses are likely personal data, why truncation does not anonymise and produces a weak, transient identifier that may be pseudonymous in context, why § 25 TDDDG / Art. 5(3) ePrivacy is the decisive unresolved question, and what a DPIA-compliant deployment would require. 4. **A privacy-first reference architecture**: a tiered sensing stack (thermal, radar, edge cameras, probe data, manual counts), a calibration program, a central-DP aggregation layer, a network-aware hierarchical spatial model, a quantile forecast with conformal calibration, and a decision layer evaluated by regret. 5. **A transferability matrix** across the six hackathon challenges, with explicit boundary conditions: direct for staffing and advertising, conditional for waste routing, sibling for sensor placement and LoRaWAN planning, and not applicable to cable-car sentiment or the learning guide. @@ -92,7 +92,7 @@ The sampling design adds a second bias. A person who passes in 30 seconds has ro The delivered CSV contains 192,464 rows: 96,232 density readings and 96,232 battery readings. There are 12 device IDs; after merging the two `density-01` IDs and excluding three test sensors, eight usable locations remain in a ~580 m × 280 m cluster: `density-01`, `hn-goetzentuem-01`, `hn-hagenbuchersee-01`, `hn-hagenbuchersee-02`, `hn-marahaus-01`, `hn-neckarufer-01`, `hn-soleo-01`, and `hn-stadtgalerie-01`. The eight-sensor panel (18 June–7 November 2026) contains 83,156 density values. -**Quantization.** Every one of those 83,156 values is an exact multiple of 0.7. The reconstructed integer `U` has median 12, p95 71, maximum 418, and 2.7% zeros. A continuous physical series would not land on a 0.7 grid in 83,156 consecutive observations; the only plausible explanation is `value = round(0.7 × integer, 4)`. This is direct, data-internal confirmation of the firmware reading [5]. +**Quantization.** Every one of those 83,156 values is an exact multiple of 0.7. The reconstructed integer `U` has median 12, p95 71, and maximum 418; 2.7% of the 83,156 values reconstruct to `U = 0`. A continuous physical series would not land on a 0.7 grid in 83,156 consecutive observations; the only plausible explanation is `value = round(0.7 × integer, 4)`. This is direct, data-internal confirmation of the firmware reading [5]. **The calibration break.** The legacy `density-01` device (`863f75b0`, 16 October 2025–12 March 2026) is 100% integer-valued on a 1.0 grid: 0–28, mean 3.008, median 2.0, zero share 29.8%. The replacement (`f8c19730`, 16 April–7 November 2026) is 100% on the 0.7 grid: 0–63, mean 9.741, median 7.7, zero share 5.9%. The gap between the last legacy reading and the first new reading is **35.2065 days**, with no overlap, so a conversion factor cannot be estimated empirically. The level shift is ×3.24 in the mean and ×3.85 in the median, which is not explained by 0.7 alone; season, installation, and sensitivity are confounded. The safe conclusion is that the two series are not on the same scale and must not be pooled [5]. @@ -159,29 +159,29 @@ The GDPR defines personal data as any information relating to an identified or i The practical conclusion is that a public-space footfall system should treat MAC addresses as personal data unless a documented anonymisation assessment proves otherwise. The burden is on the controller. “We hash, therefore it is anonymous” is not sufficient. -### 3.2 Hashing is pseudonymisation +### 3.2 Hashing and truncation do not anonymise -Hashing or truncating a MAC address reduces direct identifiability, but it does not automatically anonymise. An unsalted hash of a 48-bit MAC is enumerable; a short or truncated hash has a small space and frequent collisions; a keyed hash remains reversible by whoever holds the key. The EDPB’s pseudonymisation guidelines state that pseudonymised data that can be attributed to a person by additional information remain personal data, even if the pseudonymised data and the additional information are not in the same hands [16]. The French Conseil d’État’s *JCDecaux* decision is the clearest warning: truncating the MAC, adding a salt, and applying keyed hashing still left the controller able to identify and link records, and the purpose—measuring routes and repeat passages—was itself incompatible with anonymisation [18]. +Hashing or truncating a MAC address reduces direct identifiability, but it does not automatically anonymise. Truncation does not anonymise; it produces a weak, transient identifier that may be pseudonymous in context. An unsalted hash of a 48-bit MAC is enumerable; a short or truncated hash has a small space and frequent collisions; a keyed hash remains reversible by whoever holds the key. The EDPB’s pseudonymisation guidelines state that pseudonymised data that can be attributed to a person by additional information remain personal data, even if the pseudonymised data and the additional information are not in the same hands [16]. The French Conseil d’État’s *JCDecaux* decision is the clearest warning: truncating the MAC, adding a salt, and applying keyed hashing still left the controller able to identify and link records, and the purpose—measuring routes and repeat passages—was itself incompatible with anonymisation [18]. The Heilbronn firmware’s 16-bit truncation is weaker than a cryptographic hash. It is a dedup key with a 65,536-value space. That is a privacy-by-design choice in the sense that raw MACs are not persisted, but it is not an anonymisation guarantee. The operator’s blog says addresses are “hashed and stored for at most one hour” [2]; the public 2026 firmware resets the bitmap at each wake and has no one-hour store [3]. The two accounts disagree. The paper should report both and not treat either as a legal conclusion. ### 3.3 ePrivacy and § 25 TDDDG -The ePrivacy Directive’s Art. 5(3) requires prior consent for storing information in, or gaining access to information already stored in, terminal equipment, except for transmission or a service explicitly requested by the user [13]. Germany implements this in § 25 TDDDG, which applies regardless of whether the data are personal [20]. The decisive question is whether passive reception of a broadcast MAC is “access” to stored information. The better textual reading is that it is not: the sensor does not store anything in the phone, does not instruct the phone to disclose stored information, and does not retrieve data from the phone’s memory. German legislative materials use “Abruf” and “Auslesen” (retrieval/readout), which imply active access. But the EDPB’s functional approach—MAC addresses are information stored in the terminal equipment, and the receiving entity need not be the one instructing the transmission—creates material risk the other way [15]. The DSK’s digital-services guidance treats reading MAC addresses as a relevant access [21, 22]. There is no published EU or German decision on passive BLE/Wi-Fi sniffing under Art. 5(3)/§ 25. +The ePrivacy Directive’s Art. 5(3) requires prior consent for storing information in, or gaining access to information already stored in, terminal equipment, except for transmission or a service explicitly requested by the user [13]. Germany implements this in § 25 TDDDG, which applies regardless of whether the data are personal [20]. The decisive question is whether passive reception of a broadcast MAC is “access” to stored information. The better textual reading is that it is not: the sensor does not store anything in the phone, does not instruct the phone to disclose stored information, and does not retrieve data from the phone’s memory. German legislative materials use “Abruf” and “Auslesen” (retrieval/readout), which imply active access [39]. But the EDPB’s functional approach—MAC addresses are information stored in the terminal equipment, and the receiving entity need not be the one instructing the transmission—creates material risk the other way [15]. The DSK’s digital-services guidance lists reading MAC addresses among the cases that must be assessed under § 25, without resolving passive reception [38]. As of November 2026, we are not aware of a published EU or German decision on passive BLE/Wi-Fi sniffing under Art. 5(3)/§ 25. -If § 25 applies, the project cannot rely on legitimate interest, signage, or an opt-out. It needs prior consent meeting the GDPR standard, which is practically impossible at city scale: walking past a sensor, leaving Wi-Fi on, or seeing a sign is not consent. The viable routes are a statutory legal basis or a sensor that does not extract MAC addresses. That is why the architecture in Section 4 prefers thermal/radar/camera/probe sensing. +If § 25 applies, prior consent is required, and a GDPR Art. 6 basis is not a substitute. The viable routes are a non-MAC sensor or a legal regime that validly disapplies § 25, which we do not see for general footfall measurement. Consent at city scale is practically impossible: walking past a sensor, leaving Wi-Fi on, or seeing a sign is not consent. That is why the architecture in Section 4 prefers thermal/radar/camera/probe sensing. ### 3.4 DPIA and lawful basis -A DPIA is almost certainly mandatory. Continuous sensing of a publicly accessible inner-city area is the paradigm of “systematic monitoring of a publicly accessible area on a large scale” under Art. 35(3)(c) [13]. The Baden-Württemberg private-sector “Muss-Liste” explicitly lists offline tracking of customer movements in department stores/shopping centres and the non-intended use of device sensors or radio signals to determine location or movement over a substantial period (Nos. 2 and 10) [21]. There is no public-sector positive list in Baden-Württemberg, so for the city the statutory criterion applies directly. +If the data are personal data — as we argue they should be treated — a DPIA is very likely required. Continuous sensing of a publicly accessible inner-city area is the paradigm of “systematic monitoring of a publicly accessible area on a large scale” under Art. 35(3)(c) [13]. The Baden-Württemberg private-sector “Muss-Liste” explicitly lists offline tracking of customer movements in department stores/shopping centres and the non-intended use of device sensors or radio signals to determine location or movement over a substantial period (Nos. 2 and 10) [21]. As of November 2026, we are not aware of a public-sector positive list in Baden-Württemberg, so for the city the statutory criterion applies directly [40]. -A public body cannot rely on Art. 6(1)(f); it needs Art. 6(1)(e) read with Art. 6(3) and a legal basis in Union or Member State law, or consent. A private association or company can use Art. 6(1)(f), but the balancing test is strict for covert public-space tracking, and German and Dutch DPAs have repeatedly found that the data subject’s interests prevail. The DPIA must assess necessity and alternatives: manual counts, light barriers, thermal/radar, on-device video counting, and sample-based surveys. If a less intrusive sensor can meet the accuracy requirement, the MAC-based design is hard to justify. +A public body acting in the performance of its public tasks cannot rely on Art. 6(1)(f); it needs Art. 6(1)(e) read with Art. 6(3) and a legal basis in Union or Member State law, or consent. A private association or company can use Art. 6(1)(f), but the balancing test is strict for public-space tracking that is invisible to most passers-by or conducted without meaningful notice, and German and Dutch DPAs have repeatedly found that the data subject’s interests prevail. The DPIA must assess necessity and alternatives: manual counts, light barriers, thermal/radar, on-device video counting, and sample-based surveys. If a less intrusive sensor can meet the accuracy requirement, the MAC-based design is hard to justify. ### 3.5 Enforcement is real but fact-specific -German enforcement has been administrative rather than fine-based for this use case. The ULD Schleswig-Holstein investigated two city-marketing operators using passive probe requests and concluded there was no legal basis; both stopped [22]. The Bavarian authority inspected a fashion chain’s Wi-Fi tracking test after a complaint; the company stopped. The Hamburg authority’s 2014/2015 report treats WLAN/Bluetooth indoor tracking as highly intrusive and personal-data processing requiring a legal basis. There is no published German fine specifically for passive Wi-Fi/BLE footfall tracking; that is not evidence of legality. +German enforcement has been administrative rather than fine-based for this use case. The ULD Schleswig-Holstein investigated two city-marketing operators using passive probe requests and concluded there was no legal basis; both stopped [22]. The Bavarian authority inspected a fashion chain’s Wi-Fi tracking test after a complaint; the company stopped [41]. The Hamburg authority’s 2014/2015 report treats WLAN/Bluetooth indoor tracking as highly intrusive and personal-data processing requiring a legal basis [42]. As of November 2026, we are not aware of a published German fine specifically for passive Wi-Fi/BLE footfall tracking; that is not evidence of legality. -The Dutch Enschede case is instructive. The Dutch DPA fined the municipality €600,000 for Wi-Fi tracking in the city centre (pseudonymised MAC + location, used to count unique visitors). The fine was annulled by the Rechtbank Overijssel on 2 February 2024 because the DPA had not proved that the municipality processed personal data; the Raad van State upheld that outcome on 29 July 2026, largely on a procedural ground [19, 36, 37]. The case is not a safe harbour. It shows that the legal status is fact-specific and that the regulator must prove identifiability. The French *JCDecaux* case went the other way and is the stronger warning against relying on hashing/salting [18]. +The Dutch Enschede case is instructive. The Dutch DPA fined the municipality €600,000 for Wi-Fi tracking in the city centre (pseudonymised MAC + location, used to count unique visitors) [43]. The fine was annulled by the Rechtbank Overijssel on 2 February 2024 because the DPA had not proved that the municipality processed personal data; the Raad van State upheld that outcome on 29 July 2026, largely on a procedural ground [19, 36, 37]. The case is not a safe harbour. It shows that the legal status is fact-specific and that the regulator must prove identifiability. The French *JCDecaux* case went the other way and is the stronger warning against relying on hashing/salting [18]. ### 3.6 The operator’s claim, examined @@ -200,7 +200,7 @@ The honest formulation is: **the architecture is data-minimising at the edge, bu A defensible claim needs a threat model, a privacy unit, a formal or architectural guarantee, a retention policy, a DPIA, and a legal basis. The PET stack in Section 4.4 makes this concrete. The minimum for a pilot is: - passive only; no probe requests, association requests, or OS APIs; -- no raw MAC retention; memory only; delete immediately after dedup; +- no raw MAC retention; the sensor necessarily observes the address in RAM; the operator says a hashed cache may be kept for up to one hour, while the public 2026 code keeps only a 16-bit bitmap for the scan; - per-sensor, per-window keyed hashing, not a 16-bit truncation; - no cross-sensor or cross-day linkage; - aggregate to zones/time bins with a k-threshold and suppression; @@ -259,7 +259,7 @@ The pilot should publish the calibration dates, the reference method, the estima The PET analysis supports a layered design: -- **No raw MACs.** If a MAC-based sensor is used at all, hash at the edge with a per-sensor, per-window key, rotate the key, and destroy the old key. Do not use a plain hash, a static salt, or a 16-bit ID. +- **No raw MAC retention.** If a MAC-based sensor is used at all, the sensor necessarily observes the address in RAM to deduplicate. The operator says a hashed cache may be kept for up to one hour; the public 2026 code keeps only a 16-bit bitmap for the scan. Hash at the edge with a per-sensor, per-window key, rotate the key, and destroy the old key. Do not use a plain hash, a static salt, or a 16-bit ID as a persistent identifier. - **Edge aggregation.** Transmit counts only; no identifiers in the uplink. - **No cross-sensor/cross-day linkage.** No trajectories, dwell, or repeat-visit metrics. - **Central DP at a trusted city aggregator.** Use a device-day privacy unit and contribution bounds (e.g. at most four sensor-bin pairs per device per day). Publish fixed daily/weekly coarse releases with a budget ledger. @@ -461,7 +461,7 @@ The paper is also an opinionated design proposal, not a controlled trial. The al **What to keep from the hackathon build.** The decision-first framing, the global model, the time-based holdout, the seasonal-naive baseline, the transparent staffing heuristic, the cached fallback, the build-time batch, and the honest limitations are all worth keeping. The plan screen is the product; the map and charts support it. -**What to replace.** The sensing layer (thermal/radar/camera/probe/manual), the calibration (site-specific, with uncertainty), the privacy stack (no raw MACs, central DP, k-threshold), the spatial model (network-aware hierarchical field), the forecast (quantile GBM with conformal calibration and forecast weather), and the platform (auth, observability, SLOs, multi-tenancy) in that order. Do not scale the model before fixing the measurement. +**What to replace.** The sensing layer (thermal/radar/camera/probe/manual), the calibration (site-specific, with uncertainty), the privacy stack (no raw MAC retention, central DP, k-threshold), the spatial model (network-aware hierarchical field), the forecast (quantile GBM with conformal calibration and forecast weather), and the platform (auth, observability, SLOs, multi-tenancy) in that order. Do not scale the model before fixing the measurement. ## 9. Related work @@ -471,7 +471,7 @@ The paper is also an opinionated design proposal, not a controlled trial. The al **Privacy and law.** The legal analysis draws on the GDPR, *Breyer*, the EDPB pseudonymisation and anonymisation guidelines, *JCDecaux*, the Dutch Enschede case, and German DPAs [13–22]. The technical privacy analysis draws on differential privacy, k-anonymity, membership inference, and mobility uniqueness [23–25]. The conclusion is that “privacy-preserving” is an architectural claim that must be specified, not a label. -**Urban sensing and decision support.** Google Maps-style traffic is aggregated, map-matched probe data, not per-user triangulation; its public privacy story is stronger on data minimisation than on formal guarantees for the traffic layer. The lesson for a city is to make the parameters explicit: consent, aggregation, DP, retention, and audit. The sensing alternatives literature provides the accuracy/cost/privacy trade-offs [26–30]. The MLOps literature provides the evaluation hygiene [35]. The contribution here is to connect the measurement audit to the decision layer and to the legal/privacy constraints in one place. +**Urban sensing and decision support.** The lesson from commercial mobility products is to make the parameters explicit: consent, aggregation, DP, retention, and audit. The sensing alternatives literature provides the accuracy/cost/privacy trade-offs [26–30]. The MLOps literature provides the evaluation hygiene [35]. The contribution here is to connect the measurement audit to the decision layer and to the legal/privacy constraints in one place. ## 10. Conclusion @@ -480,7 +480,7 @@ The PulseCast case is a useful cautionary tale. A small, weather-aware model can The constructive path is clear: 1. **Calibrate** the index against manual/video ground truth, with site-specific factors and uncertainty. -2. **Change the sensing layer** to thermal/radar/camera/probe/manual, with a DPIA and no raw MACs. +2. **Change the sensing layer** to thermal/radar/camera/probe/manual, with a DPIA and no raw MAC retention. 3. **Model the field** as a network-aware hierarchical count process, at a 100–150 m support, with uncertainty. 4. **Forecast** with a global quantile GBM, forecast weather, and conformal calibration. 5. **Evaluate the decision**, not just MAE: decision regret, cost curves, and a pilot with go/no-go criteria. @@ -490,9 +490,9 @@ The honest conclusion is narrower and more useful than a city-scale claim: the d ## Data, code, and AI disclosure -The raw challenge CSV is not redistributed. Derived aggregates, the trained model artifacts, the metrics JSON, and the figure-generation scripts can be shared subject to the organizers’ permission. The code for the audit and the PulseCast system is at `[TODO: repository]`; the live app is at https://pulse.noelsabia.dev and the API at https://api-pulse.noelsabia.dev. The research notes that support the audit are in `paper/research/` and can be deposited alongside the paper or linked from the repository. +The raw challenge CSV is not redistributed. Derived aggregates, the trained model artifacts, the metrics JSON, and the figure-generation scripts can be shared subject to the organizers’ permission. The code for the audit and the PulseCast system will be released after the hackathon; the live app is at https://pulse.noelsabia.dev and the API at https://api-pulse.noelsabia.dev. The research notes that support the audit are in `paper/research/` and can be deposited alongside the paper or linked from the repository. -AI tools were used for drafting and editing. The authors verified every number, citation, and claim against the delivered data, the project metrics, or a cited source. AI tools are not authors and take no responsibility for the content. The legal analysis is research, not legal advice; a deployment should obtain a written opinion from German counsel, especially on the § 25 TDDDG question. +AI tools were used for drafting and editing. The authors checked each number and citation; the legal analysis is a research draft, not legal advice, and some sources are guidance, DPA positions, or inferences rather than binding law. AI tools are not authors and take no responsibility for the content. A deployment should obtain a written opinion from German counsel, especially on the § 25 TDDDG question. ## References @@ -500,7 +500,7 @@ AI tools were used for drafting and editing. The authors verified every number, [2] Code for Heilbronn. “Passantenzahlen messen mit LORA,” 18 September 2025. https://codeforheilbronn.de/blog/2025/09/passantenzahlen-messen-mit-lora/ -[3] Code for Heilbronn. `lorax` / `sensor-pax` firmware and backend, 2026. https://codeberg.org/cfhn/lorax +[3] Code for Heilbronn. `lorax` / `sensor-pax` firmware and backend, commit `e8be8bf5c11e1b76c88c3f176382c6df99eea989` (develop, 18 June 2026). https://codeberg.org/cfhn/lorax/commit/e8be8bf5c11e1b76c88c3f176382c6df99eea989 [4] `ESP32-Paxcounter` v3.8.0 and `libpax`. https://github.com/cyberman54/ESP32-Paxcounter; https://github.com/dbinfrago/libpax @@ -569,3 +569,15 @@ AI tools were used for drafting and editing. The authors verified every number, [34] V. Vovk, A. Gammerman, and G. Shafer. *Algorithmic Learning in a Random World*. Springer, 2005. [35] D. Sculley, G. Holt, D. Golovin, E. Davydov, T. Phillips, D. Ebner, V. Chaudhary, M. Young, J.-F. Crespo, and D. Dennison. “Hidden technical debt in machine learning systems.” *NeurIPS 2015*, 2503–2511. + +[38] Datenschutzkonferenz (DSK). *Orientierungshilfe der Aufsichtsbehörden für Anbieter:innen von digitalen Diensten (OH Digitale Dienste)*, v1.2, November 2024, paras 22, 26. https://www.datenschutzkonferenz-online.de/media/oh/OH_Digitale_Dienste.pdf + +[39] Deutscher Bundestag. *Entwurf eines Gesetzes zur Regelung des Datenschutzes und des Schutzes der Privatsphäre in der Telekommunikation und bei Telemedien (TTDSG)*, BT-Drs. 19/27441, p. 38. https://dserver.bundestag.de/btd/19/274/1927441.pdf + +[40] LfDI Baden-Württemberg and Ministerium für Justiz und Migration Baden-Württemberg. *Prüfung der Notwendigkeit einer Datenschutzfolgenabschätzung – Schwellenwertanalyse*, 29 November 2024. https://jum.baden-wuerttemberg.de/fileadmin/redaktion/m-jum/intern/PDF/Migration/Erlasse_und_Anwendungshinweise/2024/29.11.2024_Anlage_01_Schwellwertanalyse.pdf + +[41] Bayerisches Landesamt für Datenschutzaufsicht (BayLDA). *Tätigkeitsbericht 2015/16*, §§3.10, 22.10. https://www.lda.bayern.de/media/baylda_report_07.pdf + +[42] Der Hamburgische Beauftragte für Datenschutz und Informationsfreiheit (HmbBfDI). *25. Tätigkeitsbericht Datenschutz 2014/2015*, §V.9. https://datenschutz-hamburg.de/fileadmin/user_upload/HmbBfDI/Datenschutz/Taetigkeitsberichte_Datenschutz/Taetigkeitsberichte_PDF/25.TB_2014-2015.pdf + +[43] Autoriteit Persoonsgegevens. *Boete wifitracking Enschede*, 29 April 2021, updated 29 July 2026. https://www.autoriteitpersoonsgegevens.nl/documenten/boete-wifitracking-enschede diff --git a/main.md b/main.md index a625215..9605e66 100644 --- a/main.md +++ b/main.md @@ -1,9 +1,9 @@ # PulseCast: Turning a Noisy Footfall Index into a Weekly Staffing Plan -**Authors:** [AUTHOR TODO] -**Affiliation:** [AFFILIATION TODO] +**Author:** Noel Sabia (ORCID 0009-0008-3064-7573) +**Affiliation:** — **Date:** 14 November 2026 -**Artifact:** https://pulse.noelsabia.dev · **API:** https://api-pulse.noelsabia.dev · **Code:** [code repository TODO] +**Artifact:** https://pulse.noelsabia.dev · **API:** https://api-pulse.noelsabia.dev · **Code:** to be released after the hackathon ## Abstract @@ -35,7 +35,7 @@ A code-level audit of the deployed Code for Heilbronn `lorax` firmware changes t **The published value is therefore `0.7 × U`, where `U` is the number of distinct randomized BLE addresses seen in a ~10-second scan.** The 15-minute cadence is the sleep cycle, not the integration window. The effective duty cycle is about 1.1%. The firmware sets a 30-second scan duration, but the released code stops on the first 10-second report callback; the 30-second value is not enforced by the scanner [4]. -The audit is of the public 2026 firmware; the deployed 2025 binary may differ, but the data are consistent with the 0.7 scaling. Every one of the 83,156 density values in the eight-sensor panel is an exact multiple of 0.7. The reconstructed integer count has median 12, p95 71, and a maximum of 418; 2.7% of readings are zero. The legacy `density-01` device reports integers on a 1.0 grid until March 2026, then a replacement device reports on the 0.7 grid from April 2026, with a 35.2-day gap between the two series. The two `density-01` series are not on the same scale and must not be pooled [5]. +The audit is of the public 2026 firmware; the deployed 2025 binary may differ, but the data are consistent with the 0.7 scaling. Every one of the 83,156 density values in the eight-sensor panel is an exact multiple of 0.7. The reconstructed integer count has median 12, p95 71, and a maximum of 418; 2.7% of the 83,156 values reconstruct to `U = 0`. The legacy `density-01` device reports integers on a 1.0 grid until March 2026, then a replacement device reports on the 0.7 grid from April 2026, with a 35.2-day gap between the two series. The two `density-01` series are not on the same scale and must not be pooled [5]. This is a relative activity index, not a count of people. A person who passes in 30 seconds has roughly a 4.4% chance of overlapping a ~10-second scan; a person who lingers for an hour can be counted in up to four consecutive bins. The index conflates flow and dwell, and it cannot distinguish a phone from a smartwatch, earbud case, beacon, or car head-unit. Device penetration and the people-per-device ratio are unknown. The 0.7 factor is a provisioning default, not a validated penetration correction. @@ -43,7 +43,7 @@ This is a relative activity index, not a count of people. A person who passes in ### 2.3 Coverage, gaps, and context -Coverage is uneven. Only 28.2% of 15-minute slots in the analysis panel have all eight sensors reporting; the modal count is seven. Aggregating to 30, 60, or 120 minutes raises complete coverage to 63.4%, 69.7%, and 71.0% [5]. About 5.2% of slots have all sensors at zero and 39.6% have all sensors below 5. The worst month falls to 23.3% coverage at one location. Gaps reach four to six days; the cause is undocumented, but missing periods differ systematically from full-coverage periods [2, 5]. Weather comes from Open-Meteo: historical archive for training, current/7-day forecast for the dashboard. Temperature and precipitation are the useful features; cloud and wind did not improve the short test. Baden-Württemberg holidays are a feature and badge; they added little and are context, not a large driver [2]. +Coverage is uneven. Only 28.2% of 15-minute slots in the analysis panel have all eight sensors reporting; the modal count is seven. Aggregating to 30, 60, or 120 minutes raises complete coverage to 63.4%, 69.7%, and 71.0% [5]. Overall, 5.18% of individual density readings are zero and 39.6% are below 5. The worst month falls to 23.3% coverage at one location. Gaps reach four to six days; the cause is undocumented, but missing periods differ systematically from full-coverage periods [2, 5]. Weather comes from Open-Meteo: historical archive for training, current/7-day forecast for the dashboard. Temperature and precipitation are the useful features; cloud and wind did not improve the short test. Baden-Württemberg holidays are a feature and badge; they added little and are context, not a large driver [2]. | Sensor | Readings | Mean | Median | p95 | Max | Max daily gap | |---|---:|---:|---:|---:|---:|---:| @@ -108,7 +108,7 @@ The public app is at https://pulse.noelsabia.dev and the API at https://api-puls ![Figure 4. Plan screen.](figures/fig_plan.png) -*Figure 4. The plan screen: recommended shift blocks, coverage, estimated cost, and the “why this plan” context card. The plan is the product; the map and charts support it.* +*Figure 4. The v1.1 plan screen (reconstructed for print from the live UI): Saturday peak 5 staff, 100% coverage, €3,360 estimated labour cost, 25 h over / 0 h under. The live screen also includes a “why this plan” context card; the app is at https://pulse.noelsabia.dev.* ## 5. Evaluation @@ -176,9 +176,9 @@ The main limitation is the measurement. The sensor samples ~10 seconds out of ev Coverage is also a limitation. Eight sensors in 580 m × 280 m cannot support a city-wide heatmap or store-location recommender. Only 28.2% of 15-minute slots have all eight sensors; gaps are not random. The 28-day holdout is short, weather is observed, and there is no event log, POS, or labor-cost data. The deployment has one API replica, no SSO/HA, and no live ingestion; 15 min/1 h are replay/nowcast. -The privacy claim needs care. The app stores no MACs, and only counts leave the sensor. But the sensing stage processes link-layer identifiers at the edge; the 16-bit ID is a pseudonymous dedup key, not a cryptographic hash; passive MAC sensing in public space has a contested legal basis in Germany. The “privacy-preserving” label is architectural, not a formal guarantee. A production deployment needs a DPIA, a documented legal basis, minimizing/retention limits, and ideally a sensor that does not extract MACs at all. +**Privacy.** The app uses aggregate counts only, stores no MAC addresses or PII, and does not redistribute the raw challenge CSV. The sensing layer is upstream of this application. A production deployment would need a DPIA and a documented legal basis before collecting MAC-derived counts. -**What to keep:** decision-first framing, global model, time-based holdout, seasonal-naive baseline, transparent heuristic, cached fallback, build-time batch, honest limitations. **What to replace:** sensing (thermal/radar/camera/probe/manual), calibration (site-specific, with uncertainty), privacy (no raw MACs, central DP, k-threshold), spatial model (network-aware hierarchical field), forecast (quantile GBM, forecast weather, conformal calibration), platform (auth, observability, SLOs) — in that order. Fix the measurement before scaling the model. +**What to keep:** decision-first framing, global model, time-based holdout, seasonal-naive baseline, transparent heuristic, cached fallback, build-time batch, honest limitations. **What to replace:** sensing (thermal/radar/camera/probe/manual), calibration (site-specific, with uncertainty), privacy (no raw MAC retention, central DP, k-threshold), spatial model (network-aware hierarchical field), forecast (quantile GBM, forecast weather, conformal calibration), platform (auth, observability, SLOs) — in that order. Fix the measurement before scaling the model. ## 7. Related Work @@ -190,9 +190,9 @@ PulseCast shows that a small, weather-aware global model can turn a noisy inner- ## Artifacts, data, and AI disclosure -The live app is https://pulse.noelsabia.dev (API: https://api-pulse.noelsabia.dev); the code repository is `[TODO]`. The raw challenge CSV is not redistributed. Derived aggregates, the model card, and the metrics JSON can be shared subject to the organizers’ permission. The app stores no MACs or PII; the sensing/legal caveats are in Sections 2.2 and 6. A production deployment needs a DPIA and a documented legal basis before collecting MAC-derived counts. +The live app is https://pulse.noelsabia.dev (API: https://api-pulse.noelsabia.dev); the code repository will be released after the hackathon. The raw challenge CSV is not redistributed. Derived aggregates, the model card, and the metrics JSON can be shared subject to the organizers’ permission. The app stores no MACs or PII; the sensing caveats are in Section 2.2 and the privacy box is in Section 6. -AI tools were used for drafting and editing. The authors verified every number, citation, and claim against the delivered data, the project metrics, or a cited source. AI tools are not authors and take no responsibility for the content. +AI tools were used for drafting and editing. The authors checked each number and citation; some sources are project documentation or estimates. AI tools are not authors and take no responsibility for the content. ## References @@ -202,7 +202,7 @@ AI tools were used for drafting and editing. The authors verified every number, [3] Code for Heilbronn. “Passantenzahlen messen mit LORA,” 18 September 2025. https://codeforheilbronn.de/blog/2025/09/passantenzahlen-messen-mit-lora/ -[4] Code for Heilbronn. `lorax` / `sensor-pax` firmware and backend, 2026. https://codeberg.org/cfhn/lorax; upstream `ESP32-Paxcounter` and `libpax`, https://github.com/cyberman54/ESP32-Paxcounter and https://github.com/dbinfrago/libpax. +[4] Code for Heilbronn. `lorax` / `sensor-pax` firmware and backend, commit `e8be8bf5c11e1b76c88c3f176382c6df99eea989` (develop, 18 June 2026). https://codeberg.org/cfhn/lorax/commit/e8be8bf5c11e1b76c88c3f176382c6df99eea989; upstream `ESP32-Paxcounter` and `libpax`, https://github.com/cyberman54/ESP32-Paxcounter and https://github.com/dbinfrago/libpax. [5] PulseCast data-quality analysis, `paper/research/07-data-quality.md`, 2026. @@ -213,9 +213,3 @@ AI tools were used for drafting and editing. The authors verified every number, [8] J. H. Friedman. “Greedy function approximation: a gradient boosting machine.” *Annals of Statistics* 29(5):1189–1232, 2001. DOI: 10.1214/aos/1013203451. [9] L. F. Miranda-Moreno and A. C. Lahti. “Temporal trends and the effect of weather on pedestrian volumes: a case study of Montreal, Canada.” *Transportation Research Part D* 22:54–59, 2013. DOI: 10.1016/j.trd.2013.02.008. - -[10] Bluetooth SIG. *Bluetooth Core Specification v5.3*, 2021. https://www.bluetooth.com/specifications/specs/core-specification-5-3/ - -[11] Court of Justice of the European Union. Case C-582/14, *Patrick Breyer v Bundesrepublik Deutschland*, 19 October 2016. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:62014CJ0582 - -[12] European Data Protection Board. *Guidelines 02/2023 on the Technical Scope of Article 5(3) of the ePrivacy Directive*, 2023. https://www.edpb.europa.eu/documents/guideline/guidelines-22023-on-technical-scope-of-art-53-of-eprivacy-directive_en diff --git a/references.md b/references.md index 824e5c2..e0f39a5 100644 --- a/references.md +++ b/references.md @@ -1,6 +1,6 @@ # References — PulseCast paper -Canonical reference list for the **short report** `paper/main.md` (12 entries). The **extended paper** `paper/main-extended.md` has its own 37-entry reference list at the end of that file; keep the two in sync when editing. Verify every DOI/URL before submission. Do not add a source that is not actually cited. +Canonical reference list for the **short report** `paper/main.md` (entries 1–9; entries 10–12 are retained as background but are not cited in the short report). The **extended paper** `paper/main-extended.md` has its own 43-entry reference list at the end of that file; keep the two in sync when editing. Verify every DOI/URL before submission. Do not add a source that is not actually cited. 1. Schwarz Digits / Code for Heilbronn. *The Pulse of the City*, challenge brief, 2026. Local: `OKF_Vault/Schwarz Digits/FCH Challenge The _Pulse of the City_.pdf`.