docs: record v1.1 load test and failure drill results
v1.1 Cloudflare run: 120/120 connections, 0 errors, p95 event 331 ms, p95 scenario 646 ms, presenter 49.60 ms, 2,760 events, 4m57.9s. Failure drill: ML-down cache/readyz/web pass; API pod restart pass. Also record v1.1 deployment, stage/live/control verification and hello removal.
This commit is contained in:
1 parent
1171a15311
commit
bd3176bef0
4 files changed
+54
-39
No files matched your search
+21
-26
@@ -24,22 +24,25 @@ Gate 1 passes only when **all** of these are true:
|
||||
Do not claim Gate 1 from a local `kubectl port-forward`, a green manifest
|
||||
dry-run, or a container that only runs on your laptop.
|
||||
|
||||
## Current external status (2026-11-14 ~08:00 CET)
|
||||
## Current external status (2026-11-14 ~08:30 CET)
|
||||
|
||||
Re-check before trusting this snapshot:
|
||||
|
||||
- **Verified:** cluster access, scheduling, Service DNS/HTTP, probes, Secrets,
|
||||
non-root + read-only-root + `emptyDir`, and egress to Cloudflare,
|
||||
Open-Meteo, and `git.noelsabia.dev`.
|
||||
- **Tunnel active:** `pulsecast-gate1` has routes for `pulse.noelsabia.dev`
|
||||
and `api-pulse.noelsabia.dev`; `cloudflared` is `1/1` with four HTTP/2
|
||||
connections. `pulse` already serves the v1.0 Nuxt app
|
||||
(`x-powered-by: Nuxt`), so **no Cloudflare route switch remains**.
|
||||
`/stage` and `/live` return 404 on v1.0; v1.1 adds them. `api-pulse` serves
|
||||
the deployed API.
|
||||
- **120-VU baseline passed:** 120/120 SSE connections opened/closed, 0
|
||||
connection errors, p95 event latency 1.03 s, p95 scenario latency 194 ms;
|
||||
see `docs/load-test.md`. This was against v1.0.
|
||||
- **v1.1 is deployed and Ready:** web/api/ml `:v1.1`, cloudflared
|
||||
`2026.11.0`, all four Deployments `1/1`. Numeric UIDs are
|
||||
`10001:10001` for web/ml and `65532:65532` for API/cloudflared.
|
||||
- **Tunnel active:** `pulsecast-gate1` serves `pulse.noelsabia.dev` through
|
||||
Nuxt (`x-powered-by: Nuxt`) and `api-pulse.noelsabia.dev` through the API;
|
||||
four HTTP/2 connections. `/stage` and `/live` return 200; `/control` is
|
||||
401 anonymous and 200 with control auth. The temporary hello is deleted.
|
||||
- **v1.1 120-VU Cloudflare run passed:** 120/120 SSE connections opened and
|
||||
closed, 0 errors, p95 event 331 ms, p95 scenario 646 ms, presenter POST
|
||||
49.60 ms, 361/361 checks. See `docs/load-test.md`.
|
||||
- **Failure drill passed:** ML scaled to 0 while API readiness/forecast cache
|
||||
and web stayed available; ML restored and API pod restarted cleanly. See
|
||||
`docs/failure-drills.md`.
|
||||
- **API/product checks:** 8 sensors, `/api/meta` `mae=3.32`, `wape=27.5`,
|
||||
`r2=0.777`, cache `source=fastapi`, staffing plan under 2 s.
|
||||
- **DNS works:** public resolvers return Cloudflare IPs for
|
||||
`pulse.noelsabia.dev` and `api-pulse.noelsabia.dev`. A local
|
||||
`systemd-resolved` negative cache can be cleared with
|
||||
@@ -47,30 +50,22 @@ Re-check before trusting this snapshot:
|
||||
- **API hostname is `api-pulse.noelsabia.dev`:** Universal SSL covers only
|
||||
`*.noelsabia.dev`; a two-level `api.pulse.noelsabia.dev` is not covered by
|
||||
that certificate.
|
||||
- **Gitea pull is proven:** `gitea-regcred` exists; hello and the v1.0 service
|
||||
images pulled from Gitea. The Gitea owner is `noel`.
|
||||
- **v1.1 images pushed:** web/api/ml `:v1.1` were built from the service
|
||||
`main` worktrees, locally exercised under numeric UIDs/read-only root, and
|
||||
pushed. The cluster still runs the previous v1.0 images until the v1.1
|
||||
branch is reviewed/merged and explicitly applied.
|
||||
- **All three Secrets exist:** `gitea-regcred`, `cloudflared`, `control-auth`.
|
||||
- **QUIC/UDP 7844 is untested.** TCP/443 to Cloudflare works, so the
|
||||
`cloudflared` manifest uses `--protocol http2` instead of relying on
|
||||
unverified UDP.
|
||||
- **Numeric UIDs are required:** a non-numeric image user breaks
|
||||
`runAsNonRoot` on SKE (reproduced with `curlimages/curl`). Every pod in
|
||||
`deploy/k8s/` sets explicit numeric `runAsUser`/`runAsGroup`/`fsGroup`.
|
||||
|
||||
## What is possible now vs what is blocked
|
||||
|
||||
| Step | Who | Status |
|
||||
|---|---|---|
|
||||
| Read-only cluster/manifest check | agent or human | **ready now** (`make gate1-check`) |
|
||||
| Gitea push login + hello-world push | human | **done**; hello pull and Service verified |
|
||||
| Cluster/manifest check | agent or human | **done**; static and live checks pass |
|
||||
| Gitea pull + hello proof | human | **done**; hello deleted after cutover |
|
||||
| `gitea-regcred`, `cloudflared`, `control-auth` Secrets | human | **done** |
|
||||
| Cloudflare Tunnel + public hostnames/DNS | human | **done**; tunnel active; `pulse` serves Nuxt |
|
||||
| `api`, `ml`, `web` images | human | **v1.1 pushed**; local manifest-equivalent checks pass |
|
||||
| v1.1 apply + smoke + presenter k6 + drills | human | **blocked on branch review/merge** |
|
||||
| `api`, `ml`, `web` v1.1 images | human | **done**; pushed and deployed |
|
||||
| v1.1 apply + smoke + presenter k6 + drills | human | **done** |
|
||||
| Results docs | agent | **pending new PR review/merge** |
|
||||
|
||||
`make build` and `make push` cover all three services; no Dockerfile is
|
||||
missing now. Do not deploy placeholder images under the real service names.
|
||||
|
||||
Reference in new issue
Block a user