Add staffing timing to the integration harness and commit the verified run: small train 171ms, forecast train 3.6s, SSE propagation 15ms, 120/120 clients for 300s with 0 drops and p95 5.6ms, 20 heartbeats at 15s, FastAPI-down fallback from the snapshot, and recovery in 1.6s.
86 lines
3.4 KiB
Markdown
86 lines
3.4 KiB
Markdown
# api-go
|
|
|
|
Golang API that is the backbone for daily usage, while python works in batches and is heavy in compute while Golang handles concurrent usage cheap
|
|
|
|
## Checks
|
|
|
|
```sh
|
|
make check
|
|
```
|
|
|
|
Runs `go vet ./...`, `go test -count=1 ./...`, and `go build ./...`. The same
|
|
target runs in Gitea Actions on every push and pull request
|
|
(`.gitea/workflows/ci.yml`).
|
|
|
|
## Runtime data
|
|
|
|
The API image contains **no baked data**. At startup it fetches
|
|
`GET {FASTAPI_URL}/bootstrap` and keeps `sensors`, `series`, `context` and
|
|
`meta` in memory. A background loop refreshes the snapshot; on failure it keeps
|
|
the last good snapshot and retries with exponential backoff
|
|
**1, 2, 4, 8, 16, 30 s (cap)**, resetting on success.
|
|
|
|
- `GET /api/readyz` returns `503` until the first valid snapshot is loaded.
|
|
After that, FastAPI being down does not fail readiness.
|
|
- `GET /api/meta` reports the cache state under `cache`:
|
|
`source`, `last_success_at`, `last_error`, `next_retry_at`, `failures`.
|
|
- `GET /api/forecast` calls FastAPI `/predict`, remembers the last good points
|
|
per `sensor|horizon|scenario`, and serves them with `"cached": true` when
|
|
FastAPI is unavailable.
|
|
- `testdata/` is used by tests only and is excluded from the image by
|
|
`.dockerignore`.
|
|
|
|
| Env | Default | Purpose |
|
|
|---|---|---|
|
|
| `FASTAPI_URL` | `http://localhost:8000` | FastAPI base URL |
|
|
| `BOOTSTRAP_REFRESH` | `60s` | Healthy snapshot refresh interval |
|
|
| `BOOTSTRAP_TIMEOUT` | `20s` | Per-request `/bootstrap` timeout |
|
|
| `ML_TIMEOUT` | `1500ms` | Per-request `/predict` timeout |
|
|
| `CONTROL_USER` / `CONTROL_PASS` | unset | Basic auth for `POST /api/scenario` and `/api/train*` |
|
|
| `CONTROL_TOKEN` | unset | Optional Bearer token forwarded to FastAPI `/train*` |
|
|
|
|
## Integration and load test
|
|
|
|
`make integration` runs the real stack (it refuses to start if `:18000` or
|
|
`:18080` is busy):
|
|
|
|
1. starts FastAPI from the ML checkout on `:18000` and the Go API on `:18080`;
|
|
2. uploads a 24k-row CSV subset to `POST /api/train` and polls the job
|
|
(`small`, then `forecast` by default);
|
|
3. verifies the live forecast still answers, scenario propagation, the 15 s
|
|
heartbeat and reconnect behaviour;
|
|
4. stops FastAPI, verifies `/api/meta.cache`, snapshot-served sensors, the
|
|
last-good forecast (`cached: true`) and staffing fallback;
|
|
5. restarts FastAPI and waits for the cache to recover.
|
|
|
|
`make integration-load` additionally runs `scripts/sse-load`: 120 concurrent
|
|
SSE clients for 5 minutes with a scenario change every 30 s. It records peak
|
|
connections, drops, per-change and overall p95 latency, and heartbeat cadence.
|
|
The tool exits non-zero when a client drops, an event is missing, or p95
|
|
scenario latency exceeds the contract's 2 s threshold.
|
|
|
|
Results land in `/tmp/pulsecast-integration/results` by default. Override with
|
|
`WORKDIR=/path` or `RESULTS_DIR=/path`. `make load` runs only the load test
|
|
against an API that is already up. The verified run is written up in
|
|
[`docs/integration.md`](docs/integration.md) with raw evidence under
|
|
`docs/results/`.
|
|
|
|
|
|
## Container
|
|
|
|
The image is a multi-stage build. The runtime is
|
|
`gcr.io/distroless/static-debian12:nonroot` and runs as numeric
|
|
**UID/GID `65532:65532`** (`USER 65532:65532` in the `Dockerfile`).
|
|
|
|
Use the same numbers in Kubernetes:
|
|
|
|
```yaml
|
|
securityContext:
|
|
runAsUser: 65532
|
|
runAsGroup: 65532
|
|
runAsNonRoot: true
|
|
```
|
|
|
|
Do not use named users such as `nonroot` in a `securityContext`: the kubelet
|
|
cannot verify a non-numeric image user.
|