7.0 KiB
Data findings — PAX counters, Heilbronn
This is based on the actual challenge download, not the brief. Downloaded 2026-11-13 from the Schwarz Digits download link with the published password. The file is not committed to this repo; keep it in data/raw/ and add that folder to .gitignore, or reuse the cache copy.
File
| Item | Value |
|---|---|
| File | data-1794095549194.csv |
| Size | 18.49 MB, 192,464 rows |
| Columns | time (UTC ISO-8601), sensor_id, name, longitude, latitude, type, value |
| Types | densityData (96,232 rows), batteryData (96,232 rows) |
| Time range | 2025-10-16 21:17 UTC → 2026-11-07 23:51 UTC |
| Sensors | 12 device IDs; 8 usable locations (see below) |
| Cadence | ~15 minutes (median 15.3 min; some sensors 30 min) |
What is actually in it
- 8 usable sensors, all in a ~580 m × 280 m cluster around Hagenbuchersee, Neckarufer, Stadtgalerie, Marahaus and Götzenturm. This is an inner-city micro-area, not a city-wide network.
- 5 months of multi-sensor data: 2026-06-12 → 2026-11-07. The named sensors start then; the reason is not documented. The one sensor
density-01has 13 months of history, but it is a single location and is not enough for annual seasonality. - 3 sensors must be excluded:
maker-fair(73 readings, located in Hannover),niklas-01(8 readings, outside the inner city),hn-bollwerksturm-01(5 readings). density-01has two device IDs (863f75b0andf8c19730) at the same location, 2025-10-16 → 2026-? and 2026-? → 2026-11-07. Merge them as one logical sensor for analysis; keep the device ID for battery/health.- Battery data exists for every density reading. Useful for a sensor-health panel, not for forecasting.
Sensor profile (multi-sensor period, 2026-06-12 → 2026-11-07)
value is the mean of 15-minute density readings. Higher = more devices detected.
| Sensor | Readings | Mean | Median | p95 | Max | Max daily gap |
|---|---|---|---|---|---|---|
hn-stadtgalerie-01 |
8,982 | 21.1 | 8.4 | 67.2 | 137.2 | 6 days |
hn-neckarufer-01 |
9,527 | 19.2 | 11.9 | 61.6 | 255.5 | 5 days |
hn-hagenbuchersee-02 |
10,099 | 19.3 | 11.2 | 64.4 | 200.9 | 4 days |
hn-marahaus-01 |
9,254 | 13.7 | 9.1 | 38.5 | 194.6 | 6 days |
hn-hagenbuchersee-01 |
12,345 | 13.0 | 6.3 | 49.7 | 292.6 | 4 days |
hn-goetzentuem-01 |
9,952 | 11.1 | 8.4 | 29.4 | 210.7 | 5 days |
hn-soleo-01 |
10,327 | 10.6 | 8.4 | 28.0 | 111.3 | 4 days |
density-01 |
12,670 | 9.9 | 7.7 | 28.0 | 63.0 | 35 days (over its longer history) |
Across the multi-sensor period the weighted mean is ~14.4. Over the full file the distribution is right-skewed: median 7.7, p95 46.9, p99 78.4, max 292.6 — most readings are low, a few hours are very busy.
Patterns that matter for the product
Hourly mean (local time, all 8 sensors):
| Hour | 00 | 03 | 06 | 09 | 12 | 15 | 17 | 18 | 19 | 20 | 21 | 23 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean | 9.7 | 2.4 | 3.1 | 8.0 | 17.2 | 18.8 | 24.5 | 28.4 | 31.5 | 31.3 | 28.6 | 16.8 |
- The inner city peaks 18:00–21:00 local, with the top at 19:00. Morning is quiet.
- Saturday is the busiest day (mean 19.1), then Friday (16.3). Sunday is quieter (11.4) than Monday (11.9).
- The three busiest locations are Stadtgalerie, Neckarufer and Hagenbuchersee-02. That is a retail/leisure cluster, which fits the staffing story.
- Weather is a real signal: the quick model below improves by ~20% when temperature/rain/cloud/wind are added.
What the data does not support
- A city-wide heatmap. Eight points across 580 m are not a city-wide measurement. An interpolated surface is an estimate; label it as such. A "pulse map" with measured points plus a clearly marked interpolation is honest.
- A store-location recommender. Eight sensors cannot rank commercial zones across Heilbronn. This would be a vision slide, not a feature.
- A census of people. PAX counters detect discoverable Bluetooth devices. Device penetration is unknown. The values are an index of estimated devices, not a count of people. Show a calibration assumption (e.g. "if 40% of visitors carry a discoverable device, multiply by 2.5") rather than pretending the index is exact.
- Annual seasonality. The 8-sensor network only covers June–November. Do not claim a full-year model. The one long sensor can illustrate a year, but not the network.
Quick forecasting baseline
A deliberately small, reproducible baseline — not a tuned model. Hourly target = mean of 15-minute readings. Train on 2026-06-12 → 2026-10-10; test on the last 28 days (2026-10-10 → 2026-11-07). Global model across sensors, HistGradientBoostingRegressor, features: hour, weekday, month, weekend, lags 24/48/168, rolling means, sensor ID. Weather from Open-Meteo (historical, Heilbronn).
| Model | MAE | WAPE | R² |
|---|---|---|---|
| Seasonal naive (same hour last week) | 5.40 | 44.7% | 0.462 |
| Day-ahead, no weather | 4.20 | 34.8% | 0.678 |
| Day-ahead, + temperature/rain | 3.36 | 27.8% | 0.779 |
| 7-day-ahead, no weather | 4.37 | 36.2% | 0.672 |
The honest read: temperature + rain cuts day-ahead error by ~20% versus the no-weather model, and ~38% versus seasonal naive. Adding cloud/wind did not improve the short test. That is a credible differentiator, not a breakthrough. The test period is 4 weeks, weather is observed rather than forecast, and events are not modeled. Do not claim more than this.
Why this matters for the product: the forecast is good enough to drive a staffing recommendation, not good enough to promise exact headcounts. Show the uncertainty band and the baseline comparison. Judges reward understanding, not decimal places.
Recommended pipeline
- Parse CSV as UTC; convert to
Europe/Berlinfor features; handle DST explicitly. - Merge
density-01device IDs into one logical sensor. - Exclude
maker-fair,niklas-01,hn-bollwerksturm-01. - Aggregate to hourly means per sensor; keep 15-minute data for the drill-down view.
- Join historical/forecast weather from Open-Meteo; join Baden-Württemberg holidays.
- Train the global model; validate on a time-based holdout; export
forecasts.json,metrics.json,sensors.json. - Bake the JSON into the Go binary (or serve from a volume). No runtime Python dependency.
- Show data coverage and model metrics in the app's "About the data" panel.
Open data questions for the mentors
- What is the unit of
densityData— devices per 15-minute interval, concurrent devices, or something else? - Where exactly are the sensors mounted, and what is their catchment area?
- What device penetration should we assume, and is there a correction factor?
- Why do some sensors have 4–6 day gaps? Maintenance, power, LoRaWAN coverage?
- Is there an event log (markets, festivals, holidays) for the period?
- Are there more sensors or a longer multi-sensor history available?
- How often is the dataset refreshed? Is there a live API, or only this CSV?
Ask these early. The answers change the UI copy, not the build.