feat: harden Gate 1 manifests and runbook

- add read-only make gate1-check: permissions, guardrails, dry-run, Secret keys
- set numeric runAsUser/runAsGroup (SKE non-numeric user caveat), pin cloudflared and use HTTP/2
- align ML env with PULSECAST_* and wire CONTROL_TOKEN for API/ML training proxy
- fix service worktree paths and smoke.sh executable bit
- pin k6 v2.0.0 for xk6-sse, fix closed-connection metric and initial scenario latency
- rewrite docs/deploy.md as an exact copy-paste Gate 1 checklist
This commit is contained in:
Noel Sabia committed 2026-11-14 05:10:41 +01:00
1 parent 55dfe8b616
commit 5b199f8fe3
13 files changed
+1231 -235

No files matched your search

+30 -11
View File
@@ -7,17 +7,22 @@ and reports connection count plus p95 event latency.
Native k6 buffers a complete HTTP response, so it cannot parse a stream that
never ends. The script imports the community `k6/x/sse` extension
(`github.com/phymbert/xk6-sse`). Newer k6 releases resolve community extensions
automatically. If the stock `grafana/k6` image rejects the import, build a
binary with the extension:
(`github.com/phymbert/xk6-sse`), which a stock k6 binary does not include.
Build the pinned binary once:
```bash
make k6-build # creates k6/k6-sse (git-ignored)
K6=./k6/k6-sse make load
./k6/k6-sse version # expect k6 v2.0.0 + xk6-sse v0.2.0
```
`make load` uses, in order: `k6` on `PATH`, `k6/k6-sse`, then
`docker run --rm -i grafana/k6`.
The build pins **k6 v2.0.0**. On k6 v2.4.0 the same `xk6-sse` v0.2.0 build
fails at runtime with `TypeError: Object has no member 'on'` (the extension's
client methods are not exposed); v2.0.0 was verified locally on 2026-11-14. If
you bump `K6_BUILD_VERSION`, rerun the script against a test stream first.
`make load` uses, in order: `K6` if set, `k6/k6-sse`, then `k6` on `PATH`.
A stock `grafana/k6` image is **not** a fallback: it lacks the SSE extension.
## Procedure
@@ -35,7 +40,7 @@ K6=./k6/k6-sse make load
```bash
make load
# or: API_URL=https://api.pulse.noelsabia.dev k6 run k6/sse.js
# or: API_URL=https://api.pulse.noelsabia.dev ./k6/k6-sse run k6/sse.js
```
`make load` writes `k6/results/summary-<timestamp>.json` (git-ignored).
@@ -56,25 +61,35 @@ K6=./k6/k6-sse make load
4. Record these numbers in the paper/README from the saved summary:
- `sse_connections_opened` count (must be 120)
- `sse_connections_active` max (must reach 120)
- `sse_connections_closed` count (must equal connections opened)
- `sse_open_success` rate (must be >= 99%)
- `sse_event_latency` p95 (target < 2000 ms)
- `sse_connection_errors` (target 0)
- `sse_events_received` total
## Local validation before the cluster run
The script was validated locally on 2026-11-14 with the pinned k6 v2.0.0 build
against a mock SSE stream and the real API image (3 VUs, 1 heartbeat each):
connections opened and closed cleanly, open success 100%. The real 120-client
run still requires the deployed API, because localhost results say nothing
about Cloudflare buffering or SKE scheduling.
## Pass criteria
| Metric | Target |
|---|---|
| Connections opened | 120 |
| Peak active connections | 120 |
| Connections closed (stream ended) | 120 |
| Open success rate | >= 99% |
| p95 event latency | < 2000 ms |
| Unexpected connection errors | 0 |
| Duration | 5 minutes |
The k6 thresholds enforce the first, third, and fourth; the others are read
from the summary. A p95 above 2 s means Cloudflare, the Go SSE hub, or the
The k6 thresholds enforce connections opened, open success, and p95 latency;
the others are read from the summary. A final `sse_connections_closed` below
120 means at least one opened VU did not finish; inspect
`sse_connection_errors` for unexpected errors. A p95 above 2 s means Cloudflare, the Go SSE hub, or the
client is buffering/delaying events; check `X-Accel-Buffering: no`, the 15 s
heartbeat, and whether the connection is HTTP/2-buffered at the edge.
@@ -83,6 +98,10 @@ heartbeat, and whether the connection is HTTP/2-buffered at the edge.
- `sse_event_latency` uses the server timestamp in `data.t` (ping) or `data.at`
(scenario). It includes clock skew between the k6 host and the pod; keep both
on NTP. Negative or > 1 h values are ignored.
- The initial `scenario` event on connect replays the current state and carries
the original state-change time, not the send time. The script excludes that
first scenario event per VU; only scenario changes broadcast during the run
are sampled by `sse_scenario_latency`. Heartbeat pings always count.
- Each VU closes after 20 heartbeats (20 × 15 s = 5 min), so the run ends
cleanly instead of waiting for the 7-minute request timeout.
- One API replica means all clients share one SSE hub; a pod restart drops