feat: add OKF knowledge bundle for hackathon challenges

Capture all six Future City Hackathon 2026 challenges as OKF v0.2 concepts, with dataset and organizer concepts, indexes, and a log.

Configure release-please with the simple strategy so the first release PR bumps the repository from 0.0.0 to 0.1.0.
This commit is contained in:
Noel Sabia committed 2026-11-13 18:23:00 +01:00
commit 7b9893a89e
45 files changed
+45531

No files matched your search

+139
View File
@@ -0,0 +1,139 @@
---
type: Dataset
title: HNVG LoRaWAN reception data
description: Real LoRaWAN uplink receptions from water meters in Heilbronn, with gateway metadata, transmitter metadata, an address table, and candidate gateway locations.
resource: ../../HNVG/transmission_data.csv
tags: [dataset, hnvg, lorawan, signal-quality, geospatial, water-meters, smart-city]
status: stable
generated: { by: opencode/deepseek-v4.1-flash, at: 2026-11-13T17:13:56Z }
sources:
- id: transmission-data
resource: ../../HNVG/transmission_data.csv
title: "transmission_data.csv — LoRaWAN uplink receptions"
- id: transmitter-meta
resource: ../../HNVG/transmitter%20meta%20data.csv
title: "transmitter meta data.csv — water meter metadata"
- id: gateway-meta
resource: ../../HNVG/gateway%20meta%20data.csv
title: "gateway meta data.csv — gateway locations"
- id: transmitter-mapping
resource: ../../HNVG/mapping%20transmitter%20id.csv
title: "mapping transmitter id.csv — device to meter mapping"
- id: hn-addresses
resource: ../../HNVG/HN_Adresses.csv
title: "HN_Adresses.csv — Heilbronn address list"
- id: potential-gw
resource: ../../HNVG/potential_gw_location.csv
title: "potential_gw_location.csv — candidate gateway sites"
- id: hnvg-readme
resource: ../../HNVG/readme%20LoRaWAN_Network_Planning_Challenge.docx
title: "LoRaWAN Network Planning Challenge — README"
author: team:hnvg
---
# Overview
This package supports the [HNVG LoRaWAN network planning challenge](/challenges/hnvg-lorawan-network-planning.md). It combines real uplink receptions from water meters with the metadata needed to join transmissions to meters and gateways, plus address and candidate-site tables for out-of-sample planning.
All transmission and metadata files use a semicolon (`;`) delimiter. Identifiers should be imported as text to preserve leading zeros.[^hnvg-readme]
# Files
| File | Rows | Contents |
|---|---:|---|
| [`transmission_data.csv`](../../HNVG/transmission_data.csv) | 8,009 | One row per received uplink message per gateway reception, with 31 columns. |
| [`transmitter meta data.csv`](../../HNVG/transmitter%20meta%20data.csv) | 593 | One row per water meter, including size, installation date, calibration year, address, and GPS coordinates. |
| [`gateway meta data.csv`](../../HNVG/gateway%20meta%20data.csv) | 8 | One row per gateway, with display name, latitude, and longitude. |
| [`mapping transmitter id.csv`](../../HNVG/mapping%20transmitter%20id.csv) | 593 | Links `device_id` and device name to `zaehlernummer_id`. |
| [`HN_Adresses.csv`](../../HNVG/HN_Adresses.csv) | 23,127 | Addresses in Heilbronn with postal code, city, and street with house number. |
| [`potential_gw_location.csv`](../../HNVG/potential_gw_location.csv) | 71 | Candidate gateway locations with an ID, latitude, and longitude. |
# Schema
## `transmission_data.csv`
31 columns, grouped by meaning. The README is the authoritative data dictionary.[^hnvg-readme]
| Group | Fields | Meaning |
|---|---|---|
| Identifiers | `id`, `device_id`, `interface_id` | Message, transmitting device, and interface identifiers. |
| Reception time | `transceived_at`, `gateways.transceived_at` | When the message was received. Timestamps are ISO 8601 in UTC (`Z`). |
| Gateway | `gateways.gateway_id`, `gateway_anzahl`, `gateway_nr` | Receiving gateway, reported gateway count, and gateway entry number within the exported message. |
| Signal quality | `gateways.rssi`, `gateways.snr` | Received signal strength (dBm) and signal-to-noise ratio (dB). |
| Radio settings | `meta.frequency`, `meta.codr`, `meta.data_rate`, `meta.datr`, `meta.region_meta.spreadingfactor`, `meta.region_meta.bandwidth` | Frequency, coding rate, data rate, spreading factor, and bandwidth. |
| Message | `meta.frame_count_up`, `meta.frame_port`, `meta.size`, `meta.lorawan_toa_ms` | Uplink frame counter, frame port, message size, and time on air. |
| Protocol flags | `meta.ack`, `meta.confirm`, `meta.adr`, `meta.adr_ack_req` | Acknowledgement, confirmation, and adaptive data rate flags. |
| Duplicated stats | `meta.gateway_stats.rssi`, `meta.gateway_stats.snr`, `meta.gateway_stats.transceived_at`, `meta.lns_packet_uuid` | Gateway statistics repeated in the export; check equivalence before dropping. |
| Other | `meta.region_meta.datarate`, `meta.stat` | Additional export metadata. |
## `transmitter meta data.csv`
`zaehlernummer_id`, `zaehlergroesse`, `einbaudatum`, `eichjahr`, `house_number`, `location`, `postal_code`, `street`, `meter_state`, `is_garden_meter`, `tags`, `latitude`, `longitude`.
Coordinates are presumably WGS 84, but the README flags uncertainty about their quality.[^hnvg-readme] This file is Latin-1 encoded in the delivered copy, not UTF-8.
## `gateway meta data.csv`
`gateways.gateway_id`, `Anzeigename`, `latitude`, `longitude`.
Coordinates use a comma as the decimal separator in the delivered file, for example `49,1254725`.
## `mapping transmitter id.csv`
`device_id`, `name`, `zaehlernummer_id`.
## `HN_Adresses.csv`
`Postleitzahl`, `Ort`, `Straße`.
## `potential_gw_location.csv`
`potential_gateway_location`, `latitude`, `longitude`.
Coordinates also use a comma as the decimal separator in the delivered file.
# Coverage snapshot
- **Transmissions:** 8,009 rows from 2026-11-04T00:01:29Z to 2026-11-10T23:59:07Z.
- **Devices observed:** 448 distinct `device_id` values. All of them appear in the 593-row device-to-meter mapping; 145 mapped meters did not transmit during the six-day window.
- **Gateways observed:** 9 distinct `gateways.gateway_id` values, while gateway metadata contains 8 rows. One observed gateway, `00007076FF03010C`, has no metadata row; the other eight observed gateways all match the metadata.
- **Signal range:** RSSI from -127 to -70 dBm; SNR from -21.5 to 12 dB.
- **Spreading factors:** 7 (3,494 rows), 8 (372), 9 (691), 10 (733), 11 (609), and 12 (2,110).
- **Metadata:** 593 meters and 593 device-to-meter mappings; 71 candidate gateway sites; 23,127 addresses.
# Data quality notes
- **Count mismatch.** The README says roughly 600 transmitters and 8 gateways. The transmission file contains 448 devices and 9 gateway IDs. Check whether the extra gateway is real, a test device, or an export artifact, and remember that not every meter necessarily transmitted during the six-day window.
- **Decimal separators differ.** `transmission_data.csv` uses a period; the coordinate columns in the metadata files use a comma. Normalise before joins or spatial calculations.
- **Encoding differs.** `transmitter meta data.csv` does not decode as UTF-8 in the delivered copy. Read it as Latin-1 or detect the encoding.
- **Packet loss is not directly observable.** Received messages alone do not establish packet-loss rates; these require a reliable estimate of expected transmissions.[^hnvg-readme]
- **Keep gateway redundancy.** The same message can be received by multiple gateways. Preserve separate gateway receptions when analysing redundancy.[^hnvg-readme]
- **GPS uncertainty.** Transmitter coordinates are of uncertain quality. Do not treat them as survey-grade.
- **Timestamp duplicates.** Several identifiers and timestamps appear in multiple fields. Check equivalence before removing duplicates.
# Example: load the transmission data
```python
import pandas as pd
df = pd.read_csv(
"HNVG/transmission_data.csv",
sep=";",
dtype=str, # preserve leading zeros in identifiers
parse_dates=["transceived_at", "gateways.transceived_at"],
)
```
For the metadata coordinates, replace the decimal comma before casting:
```python
meta = pd.read_csv("HNVG/gateway meta data.csv", sep=";", encoding="utf-8-sig")
meta["latitude"] = meta["latitude"].str.replace(",", ".", regex=False).astype(float)
meta["longitude"] = meta["longitude"].str.replace(",", ".", regex=False).astype(float)
```
# Sources
The README describes the fields, preparation, and intended use.[^hnvg-readme] All row counts and ranges above were computed from the delivered files.
[^hnvg-readme]: HNVG, "LoRaWAN Network Planning Challenge — README", challenge brief and data documentation, 2026.
+7
View File
@@ -0,0 +1,7 @@
# Datasets
* [HNVG LoRaWAN reception data](hnvg-lorawan-reception.md) - Real uplink receptions from roughly 600 water meters and 8 gateways, plus transmitter, gateway, address, and candidate-site tables.
* [PreZero glass fill levels](prezero-glass-fill-levels.md) - 11,701 fill-level readings for 449 anonymized glass containers over almost one year.
* [Stadt Heilbronn media mentions](stadt-heilbronn-media-mentions.md) - 39,361 keyword-filtered social media and news records covering cable-car and IPAI discussions from October 2024 to October 2026.
The Schwarz Digits PAX counter dataset and the AI TRAQC satellite data are described in their challenge concepts but are not stored in this repository. The aim challenge uses public institution websites rather than a packaged dataset.
@@ -0,0 +1,79 @@
---
type: Dataset
title: PreZero glass fill levels
description: 11,701 anonymized fill-level readings for 449 glass recycling containers in Heilbronn, covering almost one year across white, green, and brown glass.
resource: ../../PreZero/glass_fill_levels_anonymized_f.csv
tags: [dataset, prezero, waste-management, fill-level, time-series, glass-recycling]
status: stable
generated: { by: opencode/deepseek-v4.1-flash, at: 2026-11-13T17:13:56Z }
sources:
- id: fill-levels
resource: ../../PreZero/glass_fill_levels_anonymized_f.csv
title: "glass_fill_levels_anonymized_f.csv — glass container fill levels"
- id: prezero-readme
resource: ../../PreZero/20261113_Readme%20Challenge%20PZ.docx
title: "WasteRouting.AI — PreZero challenge README"
author: team:prezero
---
# Overview
This dataset supports the [PreZero WasteRouting.AI challenge](/challenges/prezero-waste-routing.md). It contains fill-level readings for glass recycling containers, intended as the starting point for pattern analysis and fill-level-driven route optimization.[^prezero-readme]
# Schema
| Column | Type | Description |
|---|---|---|
| `Location_id` | string (UUID) | Anonymized, stable container identifier. No coordinates are included. |
| `material_type` | enum | Glass colour: `white`, `green`, or `brown`. |
| `date` | ISO 8601 date (`YYYY-MM-DD`) | Date of the reading. |
| `fill_level_perc` | integer or string | Fill level in percent, usually 0–100. The value `overfull` appears as a separate state. |
# Coverage
| Metric | Value |
|---|---|
| Readings | 11,701 |
| Distinct containers | 449 |
| Date range | 2025-10-17 to 2026-09-30 |
| White glass readings | 4,432 |
| Green glass readings | 4,169 |
| Brown glass readings | 3,100 |
| `overfull` readings | 1,192 |
| Numeric fill-level range | 0 to 100 |
# Data quality notes
- **No coordinates.** `Location_id` is anonymized and there is no address, latitude, or longitude column. Per-container history and grouping work, but route geometry, inter-point distances, and depot distances require an explicit assumption or external geodata.
- **No container specifications.** The file does not include container type, capacity, or maximum volume. These are variables the challenge asks teams to model.
- **No fleet or depot data.** Vehicle capacities, weight limits, shift constraints, and depot locations are not part of the dataset.
- **Irregular sampling.** Readings are event-like, not a complete daily grid. Each container has a different number and spacing of observations.
- **Discrete fill levels.** Values cluster on round percentages (for example 20, 30, 40, 60, 80, 90, 100), suggesting reported or estimated levels rather than continuous sensor measurements.
- **`overfull` is not numeric.** Encode it explicitly, for example as `100+` or a separate ordinal category, rather than dropping those rows or coercing them to `NaN` silently.
- **Anonymized IDs.** IDs are stable within the file, so joins across rows are safe. Do not assume they map to public container IDs.
# Example: load and normalize
```python
import pandas as pd
df = pd.read_csv("PreZero/glass_fill_levels_anonymized_f.csv", encoding="utf-8-sig")
df["date"] = pd.to_datetime(df["date"])
df["is_overfull"] = df["fill_level_perc"].eq("overfull")
df["fill_numeric"] = pd.to_numeric(df["fill_level_perc"], errors="coerce")
print(df.groupby("material_type")["fill_numeric"].describe())
```
# Questions the data can answer
- Which containers share similar fill cycles, and can they be grouped into service patterns?
- How do school vacations, public holidays, and seasons shift fill rates?
- Which containers are consistently full or overfull, and which are rarely used?
- How irregular is the reading cadence, and how does that limit prediction?
# Sources
The README describes the challenge context and intended use.[^prezero-readme] All counts and ranges above were computed from the delivered CSV.
[^prezero-readme]: PreZero, "The Challenge — WasteRouting.AI", challenge README, 2026.
@@ -0,0 +1,113 @@
---
type: Dataset
title: Stadt Heilbronn media mentions
description: 39,361 keyword-filtered social media and news records from October 2024 to October 2026 covering Heilbronn's cable-car debate, the IPAI project, and comparable cable-car projects.
resource: ../../Stadt%20Heilbronn/FSI_Heilbronn_Medienbeitr%C3%A4ge%20(1).json
tags: [dataset, stadt-heilbronn, media-monitoring, sentiment, nlp, cable-car, ipai]
status: stable
generated: { by: opencode/deepseek-v4.1-flash, at: 2026-11-13T17:13:56Z }
sources:
- id: media-export
resource: ../../Stadt%20Heilbronn/FSI_Heilbronn_Medienbeitr%C3%A4ge%20(1).json
title: "FSI_Heilbronn_Medienbeiträge (1).json — media mention export"
- id: seilbahn-readme
resource: ../../Stadt%20Heilbronn/readme%20Seilbahn_Challenge.docx
title: "Riding the Sentiment — Seilbahn challenge README"
author: team:stadt-heilbronn
---
# Overview
This dataset supports the [Stadt Heilbronn cable-car sentiment challenge](/challenges/stadt-heilbronn-cable-car-sentiment.md). It is a media-monitoring export of publicly accessible social media and news content, collected without login and without circumventing technical access restrictions. Direct personal identifiers are not included.[^seilbahn-readme]
The export was pre-filtered with the keywords `IPAI`, `Seilbahn`, and `téléphérique`. That filter is broad: it also captures tourist cable cars, toys, and unrelated projects, so relevance filtering is part of the challenge.[^seilbahn-readme]
The export is about 210 MB. It is kept in the local workspace but is **not tracked in git** because it exceeds common repository and hosting limits. The `resource` path above points to its expected local location.
# Schema
The file is a JSON array. Each record has the following fields:
| Field | Type | Description |
|---|---|---|
| `mention_id` | string | Provider-specific identifier for the mention. |
| `media_type_id` | integer | Numeric media type code. |
| `media_type_name` | string | Human-readable media type, for example `News`, `YouTube Video`, `Forums`, `Blog`, `Social Network`, `Microblog`. |
| `published_at` | string | Publication timestamp with UTC offset, for example `2025-05-10T18:05:00+0000`. |
| `source_name` | string | Domain or source name. |
| `url` | string | URL of the original content. |
| `country_code` | string or null | Two-letter country code where available. |
| `language_code` | string | Language code, for example `de`, `fr`, `en`. |
| `sentiment` | integer | Provider sentiment value: `1`, `0`, `-1`, or `-99`. |
| `reach` | integer or null | Estimated reach where available. |
| `engagements` | integer | Engagement count. |
| `authors` | array of strings | Author names or handles. |
| `title` | string | Title or headline. |
| `content` | string | Text content of the mention. |
# Coverage
| Metric | Value |
|---|---|
| Records | 39,361 |
| Date range | 2024-10-19T23:34:44Z to 2026-10-27T12:29:23Z |
| Records with a non-null reach | 28,805 |
| Total engagements | 1,308,843 |
| German-language records | 30,812 |
| French-language records | 7,999 |
| News records | 15,594 |
| Other media type | 9,061 |
| YouTube videos | 4,458 |
| Pictures | 3,799 |
| YouTube comments | 2,335 |
| Forums | 1,425 |
| Blogs | 977 |
| Comments | 892 |
| Social network posts | 454 |
| Microblog posts | 332 |
Sentiment value counts: `1` (20,712), `0` (9,928), `-99` (6,797), `-1` (1,924).
# Data quality notes
- **Not a survey.** The export reflects what media monitoring captured under a keyword filter, not a representative sample of the population. Report observed conversation, not population opinion.
- **Keyword noise.** `Seilbahn` and `téléphérique` capture mountain cable cars, toys, and unrelated content. `IPAI` may appear in unrelated contexts. Build and document a relevance classifier.
- **Sentiment sentinel.** `-99` is a distinct value and is most plausibly an unknown or unclassified marker. Do not average it as an extreme negative score without verifying its meaning.
- **Mixed languages.** German and French dominate, with smaller English, Hungarian, Spanish, and other groups. Normalise or stratify before comparing.
- **Media-type imbalance.** News dominates by volume, while social platforms may dominate engagement. Weight or separate the channels.
- **Missing reach.** About 10,500 records have no reach value. Do not treat missing reach as zero.
- **Possible duplicates.** The same article or post can appear through multiple sources or be reposted. Deduplicate by URL, title, and content similarity.
- **Content length varies.** Some records contain full articles, others only snippets. Text models should handle short and long inputs differently.
- **Author fields are lists.** A record can have zero, one, or several authors.
- **Privacy.** Although direct personal identifiers were removed, quoted content can still contain names. Avoid exposing personal information in a dashboard.[^seilbahn-readme]
# Example: load and profile
The file is about 210 MB. Loading it with the standard library is fine on a normal laptop, but stream it if memory is tight.
```python
import json
from collections import Counter
with open("Stadt Heilbronn/FSI_Heilbronn_Medienbeiträge (1).json", encoding="utf-8") as f:
records = json.load(f)
print(len(records))
print(Counter(r["media_type_name"] for r in records).most_common())
print(Counter(r["language_code"] for r in records).most_common(10))
print(Counter(r["sentiment"] for r in records))
```
# Questions the data can answer
- Which topics and arguments dominate the Heilbronn cable-car debate?
- How does stance differ between social media and news coverage?
- When did attention spike, and which real-world events coincide with the spikes?
- How does Heilbronn compare with Paris and Bonn?
- Is the cable car discussed differently from the IPAI campus?
# Sources
The README describes the collection rules, keyword filter, and intended analysis.[^seilbahn-readme] All counts and ranges above were computed from the delivered JSON export.
[^seilbahn-readme]: Stadt Heilbronn, "Riding the Sentiment: Exploring Public Opinion on Urban Cable Cars", challenge README, 2026.