Build artifacts/prezero_priority.json from the anonymized glass fill-level export (11,701 rows, 449 containers, 1,338 location/material series). The contract tier rule yields high 88, medium 248, low 113 with 1,192 overfull readings.
Add /prezero/summary, /prezero/priority, and /prezero/containers/{id} to the internal FastAPI service, with startup validation, filtering/pagination, recent reading history, and 503 isolation when the artifact is missing. Raw CSV stays gitignored; the derived 1.1 MB artifact is the source of truth. This is a priority proof, not a route: the dataset has no coordinates.
180 lines
6.2 KiB
Markdown
180 lines
6.2 KiB
Markdown
# PreZero container priority
|
||
|
||
A transparent collection-priority snapshot built from the anonymized glass
|
||
fill-level export. The committed artifact is
|
||
`artifacts/prezero_priority.json`; the raw CSV is **not** committed.
|
||
|
||
**This is a priority proof, not a route.** The dataset has no coordinates, no
|
||
container capacity, and no depot data. `overfull` is a reported state, not a
|
||
measured volume.
|
||
|
||
## Provenance
|
||
|
||
| Field | Value |
|
||
|---|---:|
|
||
| Input | `glass_fill_levels_anonymized_f.csv` (UTF-8 with BOM) |
|
||
| Rows | 11,701 |
|
||
| Distinct `Location_id` | 449 |
|
||
| `Location_id` + `material_type` series | 1,338 |
|
||
| Date range | 2025-10-17 → 2026-09-30 |
|
||
| `overfull` readings | 1,192 |
|
||
| Material readings | white 4,432 · green 4,169 · brown 3,100 |
|
||
| Material overfull | white 731 · green 357 · brown 104 |
|
||
|
||
The export is read with `utf-8-sig`; header names and cells are stripped. Every
|
||
row must have `Location_id`, `material_type`, a `YYYY-MM-DD` date, and a
|
||
`fill_level_perc` of either `overfull` or an integer 0–100.
|
||
|
||
The raw file stays git-ignored under `data/raw/`. Put it there or set
|
||
`PULSECAST_PREZERO_CSV`; the generated artifact is committed so the API and
|
||
Docker image do not need the CSV.
|
||
|
||
## Rule
|
||
|
||
Each `Location_id` is one container location. For all rows at that location:
|
||
|
||
- `readings`, `overfull_count`, `overfull_rate = overfull_count / readings`
|
||
- `numeric_count` = rows with a numeric 0–100 fill (excludes `overfull`)
|
||
- `p95_fill` = 95th percentile of numeric readings, `null` if none
|
||
- `max_fill`, `mean_fill` = numeric readings only
|
||
- `last_fill`, `last_is_overfull`, `last_date` = most recent row
|
||
- `days_since_last` = `reference_date - last_date`; reference is the dataset
|
||
max date (2026-09-30), not today
|
||
- `trend` = least-squares slope over the last up to 3 numeric readings
|
||
(percentage points per reading); `null` with fewer than 2
|
||
- `priority_score = round(overfull_count + 2 * overfull_rate, 4)` — a display
|
||
and sorting convenience, not the tier
|
||
- `materials` = per material: readings, overfull count, last fill/state/date
|
||
|
||
Tier:
|
||
|
||
- **high:** `overfull_count >= 5` OR (`overfull_count >= 2` AND
|
||
`overfull_rate >= 0.25`)
|
||
- **medium:** `overfull_count >= 1` and not high
|
||
- **low:** `overfull_count == 0`
|
||
|
||
Result on the attached CSV: **high 88, medium 248, low 113**.
|
||
|
||
Order: tier (`high` first), then overfull count desc, overfull rate desc,
|
||
`days_since_last` asc, `location_id` asc.
|
||
|
||
## Artifact shape
|
||
|
||
```json
|
||
{
|
||
"meta": {
|
||
"source": "glass_fill_levels_anonymized_f.csv",
|
||
"generated_at": "2026-11-14T17:00:00Z",
|
||
"rows": 11701,
|
||
"containers": 449,
|
||
"material_series": 1338,
|
||
"date_from": "2025-10-17",
|
||
"date_to": "2026-09-30",
|
||
"reference_date": "2026-09-30",
|
||
"overfull_readings": 1192,
|
||
"priority_rule": "high: ...; medium: ...; low: 0",
|
||
"limitations": ["No coordinates ...", "..."]
|
||
},
|
||
"summary": {
|
||
"priority_counts": {"high": 88, "medium": 248, "low": 113},
|
||
"by_material": {
|
||
"white": {"readings": 4432, "overfull": 731},
|
||
"green": {"readings": 4169, "overfull": 357},
|
||
"brown": {"readings": 3100, "overfull": 104}
|
||
}
|
||
},
|
||
"containers": [
|
||
{
|
||
"location_id": "...",
|
||
"priority": "high",
|
||
"priority_score": 21.525,
|
||
"readings": 80,
|
||
"overfull_count": 21,
|
||
"overfull_rate": 0.2625,
|
||
"numeric_count": 59,
|
||
"p95_fill": 100.0,
|
||
"max_fill": 100,
|
||
"mean_fill": 64.24,
|
||
"last_fill": null,
|
||
"last_is_overfull": true,
|
||
"last_date": "2026-09-29",
|
||
"days_since_last": 1,
|
||
"trend": 30.0,
|
||
"materials": {"white": {"readings": 29, "...": "..."}},
|
||
"recent_readings": [
|
||
{"date": "2026-09-29", "material_type": "white",
|
||
"fill_level_perc": "overfull", "is_overfull": true}
|
||
]
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
The artifact holds at most 10 recent readings per container, not the full
|
||
11,701-row table. `last_fill` is `null` when the newest reading is `overfull`.
|
||
`p95_fill`, `max_fill`, `mean_fill`, and `trend` are `null` when the location
|
||
has no numeric readings or too few for a slope.
|
||
|
||
## Generation
|
||
|
||
```bash
|
||
# Raw CSV is git-ignored; copy it or set PULSECAST_PREZERO_CSV.
|
||
cp glass_fill_levels_anonymized_f.csv data/raw/
|
||
make prezero
|
||
|
||
# Equivalent explicit command:
|
||
uv run python scripts/build_prezero.py \
|
||
--csv data/raw/glass_fill_levels_anonymized_f.csv \
|
||
--out artifacts/prezero_priority.json
|
||
```
|
||
|
||
The script prints rows, containers, material series, overfull readings, date
|
||
range, and the three tier counts. It fails loudly on missing columns, blank
|
||
cells, bad dates, or invalid fill values instead of guessing.
|
||
|
||
## Internal API
|
||
|
||
The ML service loads and validates the artifact at startup. If it is missing or
|
||
malformed, only these routes return 503; `/healthz`, `/readyz`, `/bootstrap`,
|
||
`/predict`, `/simulate`, and `/meta` keep working.
|
||
|
||
| Method | Path | Behavior |
|
||
|---|---|---|
|
||
| GET | `/prezero/summary` | `meta` + `summary` |
|
||
| GET | `/prezero/priority` | Ranked containers; `priority` (`high\|medium\|low`), `limit` 1–500 (default 50), `offset` >= 0 |
|
||
| GET | `/prezero/containers/{location_id}` | One container + 10 newest readings; unknown → 404 |
|
||
|
||
Invalid `priority`, `limit`, or `offset` → 400 in the existing error envelope.
|
||
The priority list omits `recent_readings`; the container route returns it as
|
||
`readings`. `/prezero/priority` reports `total` before pagination and `count`
|
||
after it.
|
||
|
||
```bash
|
||
curl -s localhost:8000/prezero/summary
|
||
curl -s 'localhost:8000/prezero/priority?priority=high&limit=3'
|
||
curl -s localhost:8000/prezero/containers/<known-id>
|
||
```
|
||
|
||
## Verification
|
||
|
||
```bash
|
||
uv run pytest -q tests/test_prezero.py
|
||
make check
|
||
```
|
||
|
||
The tests assert the dataset counts above (11,701 rows, 449 containers, 1,338
|
||
series, 1,192 overfull, 88/248/113 tiers), filtering/pagination, known and
|
||
unknown containers, invalid queries, artifact schema and tier consistency, and
|
||
the missing-artifact 503 path.
|
||
|
||
## Limits
|
||
|
||
- No coordinates: cannot compute distance, travel time, or a route.
|
||
- No capacity, depot, vehicle, or shift data.
|
||
- `overfull` is a reported state; fill percentages are coarse, user-reported
|
||
values (0/10/.../100), not continuous sensor volumes.
|
||
- The reference date is the dataset max date, so staleness is relative to the
|
||
snapshot, not to when the API is called.
|
||
- Counts and rates describe this export only; they are not a forecast of future
|
||
fill levels.
|