Noel Sabia bd3176bef0
infra-ci / manifests (push) Failing after 3s
infra-ci / manifests (pull_request) Failing after 4s
docs: record v1.1 load test and failure drill results
v1.1 Cloudflare run: 120/120 connections, 0 errors, p95 event 331 ms,
p95 scenario 646 ms, presenter 49.60 ms, 2,760 events, 4m57.9s.
Failure drill: ML-down cache/readyz/web pass; API pod restart pass.
Also record v1.1 deployment, stage/live/control verification and hello
removal.
2026-11-14 08:32:12 +01:00

PulseCast infra/QA

Namespace-only Kubernetes manifests, smoke tests, and the k6 SSE load test for PulseCast. This repo never applies cluster changes by itself: the human runs the commands in docs/deploy.md.

Layout

deploy/k8s/           web, api, ml + cloudflared Deployments and Services, ConfigMaps
deploy/examples/      example Secret shapes (placeholders only, never applied)
scripts/smoke.sh      public + in-cluster smoke checks
scripts/gate1-check.sh read-only permissions/manifest/secret readiness check
scripts/check_manifests.py cluster-free manifest guardrails used by CI
.gitea/workflows/ci.yml manifest validation; no secrets, no cluster writes
k6/sse.js             120 concurrent SSE clients for 5 minutes
docs/deploy.md        Gate 1 runbook and exact human checklist
docs/load-test.md     load-test procedure and how to record results
docs/failure-drills.md FastAPI-down, tunnel-down, pod-restart drills
Makefile              validate, build, push, deploy, smoke, load helpers

Quick start

export KUBECONFIG=/path/to/fch26-onemanarmy-sa-kubeconfig.yaml   # chmod 600
make gate1-check              # read-only: permissions, guardrails, dry-run, Secrets
make validate                 # client-side dry run only, never applies
make build                    # all three images
make push                     # after docker login to Gitea
make deploy CONFIRM=yes       # human only; requires the three Secrets first
make smoke                    # public HTTPS + API + ML health checks
make k6-build                 # once: pinned k6 + SSE extension
make load                     # 120 SSE clients for 5 minutes

make gate1-check exits 0 when ready, 1 on a hard permission/manifest failure, and 2 when only human prerequisites (Secrets, Deployments) are pending. It never mutates the cluster. DNS and public HTTPS are checked separately in the runbook and by make smoke.

Read docs/deploy.md before the first deploy. It contains the Gitea token, docker login, hello-world push, gitea-regcred, Cloudflare Tunnel, public hostname, and verification steps.

Gate 1 status

  • Static make gate1-check STATIC_ONLY=1/CI passes. The live check correctly flags the cluster's v1.0 images until the reviewed v1.1 branch is applied.
  • All three v1.1 images build and pass local manifest-equivalent runtime checks (read-only root, /tmp tmpfs, numeric UID). There is no Dockerfile blocker.
  • Gitea pull is proven: gitea-regcred exists and the hello pod pulled from git.noelsabia.dev/noel/hello:gate1; the Service returned HTTP 200.
  • Tunnel pulsecast-gate1 is active with pulse and api-pulse routes. pulse serves the v1.1 Nuxt app; /stage and /live return 200, and /control requires auth. The API hostname is api-pulse.noelsabia.dev because Universal SSL only covers *.noelsabia.dev.
  • v1.1 is deployed: web/api/ml :v1.1, cloudflared 2026.11.0, all four Deployments Ready.
  • The v1.1 120-VU Cloudflare run passed: 120/120 connections, 0 errors, p95 event 331 ms, p95 scenario 646 ms, presenter 49.60 ms. The v1.0 baseline is retained for comparison in docs/load-test.md.
  • The ML-down/API-restart failure drill passed; details are in docs/failure-drills.md.
  • The results documentation still needs a new PR review/merge; the v1.1 runtime code is already merged in main.

CI

.gitea/workflows/ci.yml runs make gate1-check STATIC_ONLY=1 and make validate. It uses no Secrets, pushes no images, and never mutates a cluster. If the Gitea runner is unavailable, run the same two commands locally; STATIC_ONLY=1 needs no kubeconfig.

Git workflow

main is integration. New work happens on a feat/<topic> branch in its own worktree. Before every push: git fetch origin && git merge origin/main. Never force-push main. The initial scaffold is the one bootstrap exception committed directly to main.

Safety rules

  • No Ingress, cert-manager, external-dns, Jobs, CronJobs, CRDs, Roles, RoleBindings, NetworkPolicies, or any cluster-scoped resource.
  • Exposure only through the cloudflared Deployment.
  • No secrets, tokens, kubeconfig, or raw CSV in git.
  • The human applies cluster mutations. make deploy refuses to run without CONFIRM=yes.
  • Every pod sets explicit numeric runAsUser/runAsGroup/fsGroup: web/ml 10001, api/cloudflared 65532. SKE rejects a non-numeric image user under runAsNonRoot.
  • The tunnel uses HTTP/2 (--protocol http2) because TCP/443 to Cloudflare is verified and UDP/7844 is not.
S
Description
Everything around k8, maybe k6 in here to build, check and verify the infrastructure
Readme
356 KiB
0 Stars 1 Watchers 0 Forks
Languages
Shell 29.5%
Python 29.2%
Makefile 23%
JavaScript 18.3%