Files
infra-k8/README.md
Noel Sabia bd3176bef0
infra-ci / manifests (push) Failing after 3s
infra-ci / manifests (pull_request) Failing after 4s
docs: record v1.1 load test and failure drill results
v1.1 Cloudflare run: 120/120 connections, 0 errors, p95 event 331 ms,
p95 scenario 646 ms, presenter 49.60 ms, 2,760 events, 4m57.9s.
Failure drill: ML-down cache/readyz/web pass; API pod restart pass.
Also record v1.1 deployment, stage/live/control verification and hello
removal.
2026-11-14 08:32:12 +01:00

96 lines
4.5 KiB
Markdown

# PulseCast infra/QA
Namespace-only Kubernetes manifests, smoke tests, and the k6 SSE load test for
PulseCast. This repo never applies cluster changes by itself: the human runs the
commands in `docs/deploy.md`.
## Layout
```text
deploy/k8s/ web, api, ml + cloudflared Deployments and Services, ConfigMaps
deploy/examples/ example Secret shapes (placeholders only, never applied)
scripts/smoke.sh public + in-cluster smoke checks
scripts/gate1-check.sh read-only permissions/manifest/secret readiness check
scripts/check_manifests.py cluster-free manifest guardrails used by CI
.gitea/workflows/ci.yml manifest validation; no secrets, no cluster writes
k6/sse.js 120 concurrent SSE clients for 5 minutes
docs/deploy.md Gate 1 runbook and exact human checklist
docs/load-test.md load-test procedure and how to record results
docs/failure-drills.md FastAPI-down, tunnel-down, pod-restart drills
Makefile validate, build, push, deploy, smoke, load helpers
```
## Quick start
```bash
export KUBECONFIG=/path/to/fch26-onemanarmy-sa-kubeconfig.yaml # chmod 600
make gate1-check # read-only: permissions, guardrails, dry-run, Secrets
make validate # client-side dry run only, never applies
make build # all three images
make push # after docker login to Gitea
make deploy CONFIRM=yes # human only; requires the three Secrets first
make smoke # public HTTPS + API + ML health checks
make k6-build # once: pinned k6 + SSE extension
make load # 120 SSE clients for 5 minutes
```
`make gate1-check` exits 0 when ready, 1 on a hard permission/manifest failure,
and 2 when only human prerequisites (Secrets, Deployments) are pending. It
never mutates the cluster. DNS and public HTTPS are checked separately in the
runbook and by `make smoke`.
Read `docs/deploy.md` before the first deploy. It contains the Gitea token,
`docker login`, hello-world push, `gitea-regcred`, Cloudflare Tunnel, public
hostname, and verification steps.
## Gate 1 status
- Static `make gate1-check STATIC_ONLY=1`/CI passes. The live check correctly
flags the cluster's v1.0 images until the reviewed v1.1 branch is applied.
- All three v1.1 images build and pass local manifest-equivalent runtime checks
(read-only root, `/tmp` tmpfs, numeric UID). There is no Dockerfile blocker.
- Gitea pull is proven: `gitea-regcred` exists and the hello pod pulled from
`git.noelsabia.dev/noel/hello:gate1`; the Service returned HTTP 200.
- Tunnel `pulsecast-gate1` is active with `pulse` and `api-pulse` routes.
`pulse` serves the v1.1 Nuxt app; `/stage` and `/live` return 200, and
`/control` requires auth. The API hostname is
`api-pulse.noelsabia.dev` because Universal SSL only covers
`*.noelsabia.dev`.
- v1.1 is deployed: web/api/ml `:v1.1`, cloudflared `2026.11.0`, all four
Deployments Ready.
- The v1.1 120-VU Cloudflare run passed: 120/120 connections, 0 errors, p95
event 331 ms, p95 scenario 646 ms, presenter 49.60 ms. The v1.0 baseline is
retained for comparison in `docs/load-test.md`.
- The ML-down/API-restart failure drill passed; details are in
`docs/failure-drills.md`.
- The results documentation still needs a new PR review/merge; the v1.1
runtime code is already merged in `main`.
## CI
`.gitea/workflows/ci.yml` runs `make gate1-check STATIC_ONLY=1` and
`make validate`. It uses no Secrets, pushes no images, and never mutates a
cluster. If the Gitea runner is unavailable, run the same two commands
locally; `STATIC_ONLY=1` needs no kubeconfig.
## Git workflow
`main` is integration. New work happens on a `feat/<topic>` branch in its own
worktree. Before every push: `git fetch origin && git merge origin/main`.
Never force-push `main`. The initial scaffold is the one bootstrap exception
committed directly to `main`.
## Safety rules
- No Ingress, cert-manager, external-dns, Jobs, CronJobs, CRDs, Roles,
RoleBindings, NetworkPolicies, or any cluster-scoped resource.
- Exposure only through the `cloudflared` Deployment.
- No secrets, tokens, kubeconfig, or raw CSV in git.
- The human applies cluster mutations. `make deploy` refuses to run without
`CONFIRM=yes`.
- Every pod sets explicit numeric `runAsUser`/`runAsGroup`/`fsGroup`: web/ml
`10001`, api/cloudflared `65532`. SKE rejects a non-numeric image user under
`runAsNonRoot`.
- The tunnel uses HTTP/2 (`--protocol http2`) because TCP/443 to Cloudflare is
verified and UDP/7844 is not.