215 lines
13 KiB
Markdown
215 lines
13 KiB
Markdown
# Architecture — namespace-only, three services, Cloudflare Tunnel
|
||
|
||
## Recommendation in one line
|
||
|
||
Run **three namespace-scoped Deployments** — Nuxt SSR web, Go API/SSE, Python FastAPI ML — behind a **Cloudflare Tunnel**, with the model and data baked into images, no database, no Ingress, no cert-manager, no external-dns, and a cached fallback in Go if FastAPI is unavailable.
|
||
|
||
## What the kubeconfig actually allows
|
||
|
||
Verified against `fch26-onemanarmy-sa-kubeconfig.yaml` on 2026-11-13:
|
||
|
||
| Can create | Cannot create |
|
||
|---|---|
|
||
| Deployments, Services, ConfigMaps, Secrets, ServiceAccounts, Pods, PVCs | **Ingress**, **Jobs**, **CronJobs**, NetworkPolicies, Roles, RoleBindings |
|
||
|
||
- No cert-manager CRDs, no Gateway API, no `IngressClass` visibility.
|
||
- No ResourceQuota or LimitRange — three small pods fit.
|
||
- Public image pull works; a non-root `nginx-unprivileged` pod ran successfully.
|
||
- **Exposure:** Cloudflare Tunnel, because Ingress is not permitted.
|
||
- **Batch:** build-time, because Jobs/CronJobs are not permitted.
|
||
- **TLS:** terminated at Cloudflare; no cert-manager needed.
|
||
- **DNS:** Cloudflare Tunnel public hostnames; no external-dns needed.
|
||
|
||
## System
|
||
|
||
```text
|
||
┌────────────────────────────────────────────┐
|
||
phones / laptop │ Nuxt 4 SSR (Onyx, Chart.js, MapLibre) │
|
||
pulse.noelsabia │ Node server, server-rendered shell │
|
||
└───────────────┬────────────────────────────┘
|
||
│ HTTPS
|
||
┌───────────────▼────────────────────────────┐
|
||
│ Cloudflare edge + Tunnel │
|
||
│ pulse.noelsabia.dev -> web:3000 │
|
||
│ api.pulse.noelsabia.dev -> api:8080 │
|
||
└───────────────┬────────────────────────────┘
|
||
│ outbound from cloudflared pod
|
||
┌───────────────▼────────────────────────────┐
|
||
│ cloudflared Deployment (namespace-scoped) │
|
||
└───────┬───────────────────────┬────────────┘
|
||
│ │
|
||
┌─────────────▼────────────┐ ┌───────▼────────────────┐
|
||
│ Go API + SSE + cache │ │ Nuxt SSR server │
|
||
│ scenario state (1 pod) │ │ calls api:8080 │
|
||
└─────────────┬────────────┘ └────────────────────────┘
|
||
│ internal HTTP
|
||
┌─────────────▼────────────┐
|
||
│ FastAPI ML service │
|
||
│ /predict /simulate │
|
||
│ /healthz /meta │
|
||
└──────────────────────────┘
|
||
```
|
||
|
||
**Build-time:**
|
||
|
||
```text
|
||
CSV ──► Python batch (pandas, sklearn) ──► model + JSON forecasts
|
||
│
|
||
┌─────────────────────────┴──────────────────┐
|
||
│ baked into ml image │
|
||
│ copied into Go image as fallback JSON │
|
||
└────────────────────────────────────────────┘
|
||
```
|
||
|
||
## Why this shape
|
||
|
||
| Choice | Why | Cost |
|
||
|---|---|---|
|
||
| Nuxt SSR | Fast first paint on phones; you know Nuxt; Onyx SSR was tested and works. | A Node Deployment; hydration and SSE care. |
|
||
| Go API | Bonus stack; small binary; SSE fan-out; scenario state; cache. | One more service to deploy. |
|
||
| FastAPI | Your requested Python service; model inference and scenario queries. | Third Deployment; must not be a single point of failure. |
|
||
| Cloudflare Tunnel | The only exposure path the namespace allows; free TLS; no Ingress, no cert-manager. | One `cloudflared` pod and a tunnel token. |
|
||
| Baked model/data | No PVC, no database, no runtime data fetch. | Rebuild image to update data/model. Fine for 24 h. |
|
||
| Cached fallback in Go | If FastAPI is down, the core screens still work from precomputed JSON. | Slight duplication of data. Worth it. |
|
||
| One API replica | SSE state is in memory; 120 clients fit. | No HA. Documented. |
|
||
|
||
## Stack
|
||
|
||
- **Frontend:** Nuxt 4.6.4, `ssr: true`, `@sit-onyx/nuxt` 1.2.0, `sit-onyx` 1.22.0, `@sit-onyx/chartjs-plugin` 1.0.1, Chart.js, MapLibre GL JS (static basemap fallback).
|
||
- **Backend:** Go 1.27+, `net/http`; `chi` optional. SSE hub, scenario state, cache, health/ready.
|
||
- **ML:** Python 3.12, pandas, scikit-learn (`HistGradientBoostingRegressor`), FastAPI/Uvicorn. Horizons: 15 min, 1 h, 48 h, 7 d.
|
||
- **Registry:** Gitea at `git.noelsabia.dev`, Docker Registry v2, Let's Encrypt certificate. `imagePullSecret` in the namespace. Public Docker Hub pull also works, so `cloudflare/cloudflared` can be used directly.
|
||
- **K8s:** three Deployments + `cloudflared`, four Services, ConfigMaps/Secrets, no PVC, no CRDs, no cluster-scoped resources.
|
||
- **TLS/DNS:** Cloudflare Tunnel public hostnames. No cert-manager, no external-dns, no Ingress.
|
||
|
||
## Cloudflare Tunnel setup
|
||
|
||
1. Create a tunnel in the Cloudflare dashboard (Zero Trust → Networks → Tunnels).
|
||
2. Add two public hostnames:
|
||
- `pulse.noelsabia.dev` → `http://web:3000`
|
||
- `api.pulse.noelsabia.dev` → `http://api:8080`
|
||
(If the tunnel UI supports path routing, `pulse.noelsabia.dev/api` → `http://api:8080` is also possible; verify. Two hostnames are the reliable default.)
|
||
3. Store the tunnel token as a Secret:
|
||
```bash
|
||
kubectl -n fch26-onemanarmy create secret generic cloudflared \
|
||
--from-literal=TUNNEL_TOKEN='<token>'
|
||
```
|
||
4. Deploy `cloudflare/cloudflared:latest` with `tunnel --no-autoupdate run --token $(TUNNEL_TOKEN)`.
|
||
5. Verify `https://pulse.noelsabia.dev` and `https://api.pulse.noelsabia.dev/api/healthz`.
|
||
6. Test SSE through Cloudflare; heartbeat every 15 s. If buffering appears, fall back to polling.
|
||
|
||
**CORS:** if you use two hostnames, Go must allow `Origin: https://pulse.noelsabia.dev` and the `Authorization` header for `/control`. The browser uses `https://api.pulse.noelsabia.dev` as the API base. If you use path routing, same-origin and no CORS.
|
||
|
||
## Registry
|
||
|
||
```bash
|
||
docker login git.noelsabia.dev -u <user>
|
||
docker tag <image> git.noelsabia.dev/<owner>/<package>:<tag>
|
||
docker push git.noelsabia.dev/<owner>/<package>:<tag>
|
||
|
||
kubectl -n fch26-onemanarmy create secret docker-registry gitea-regcred \
|
||
--docker-server=git.noelsabia.dev \
|
||
--docker-username=<user> \
|
||
--docker-password=<read-only-package-token>
|
||
```
|
||
|
||
Reference `gitea-regcred` in each Deployment or the default ServiceAccount. Verify with a hello-world before building the real images. Public Docker Hub pull works as a fallback for `cloudflared`.
|
||
|
||
## Quotas and Pod Security
|
||
|
||
- No ResourceQuota or LimitRange was visible; three small pods fit.
|
||
- Still set requests/limits: web 100m/256Mi, api 100m/128Mi, ml 200m/512Mi, cloudflared 50m/64Mi.
|
||
- Run as non-root; drop all capabilities; `allowPrivilegeEscalation: false`; read-only root filesystem with an `emptyDir` at `/tmp`.
|
||
- The namespace has no explicit PSA labels; a non-root `nginx-unprivileged` pod ran. Design for restricted anyway.
|
||
- No NetworkPolicy permission; assume default allow. Test web → api and api → ml after deployment. If a default-deny policy blocks traffic, escalate; you cannot create NetworkPolicies.
|
||
|
||
## API contract
|
||
|
||
### Go API
|
||
|
||
```text
|
||
GET /api/healthz -> 200 {"ok":true}
|
||
GET /api/readyz -> 200 if data loaded
|
||
GET /api/meta -> data/model versions, coverage, metrics
|
||
GET /api/sensors -> [{id,name,lat,lon,mean,p95,battery}]
|
||
GET /api/series?sensor=&from=&to= -> [{t,v}]
|
||
GET /api/forecast?sensor=&horizon=15m|1h|48h|7d&scenario= -> [{t,v,lo,hi}]
|
||
GET /api/staffing?sensor=&scenario=&standard= -> {hours:[...],summary:{...}}
|
||
POST /api/scenario -> 202 {version, scenario}
|
||
GET /api/stream -> text/event-stream; events: scenario, pulse
|
||
```
|
||
|
||
- `scenario` enum: `base`, `sunny-saturday-market`, `rainy-saturday`, `weekday-event`; `standard`: `relaxed|standard|premium`.
|
||
- Go never computes the model. It calls FastAPI or serves the cached JSON.
|
||
- Every response carries `data_version` and `model_version`.
|
||
- SSE: `event: scenario\ndata: {...}\n\n`, heartbeat 15 s, `retry: 3000`.
|
||
- **Plan generation:** `GET /api/staffing` returns a complete next-week plan in under 2 seconds; the heuristic is deterministic and cached by `sensor|scenario|standard`.
|
||
|
||
### FastAPI ML service
|
||
|
||
```text
|
||
GET /healthz -> 200
|
||
GET /readyz -> 200 if model loaded
|
||
GET /meta -> model version, metrics, training window
|
||
POST /predict -> {sensor, horizon, scenario, weather} -> forecast
|
||
POST /simulate -> {scenario} -> forecasts for all sensors/horizons
|
||
```
|
||
|
||
- Go calls `/simulate` on a scenario change; caches by `scenario|standard`.
|
||
- If FastAPI times out (> 1.5 s) or is unhealthy, Go serves the cached JSON and marks the UI "cached forecast."
|
||
- The batch job is build-time; there is no Job/CronJob permission. A production path can use a CronJob in a less restricted namespace; document it, do not depend on it.
|
||
|
||
## Horizon semantics
|
||
|
||
| Horizon | Use | Data reality |
|
||
|---|---|---|
|
||
| 15 min | "Now" / replay pulse | One-step nowcast; needs a live/replay clock. |
|
||
| 1 h | "Now" / short operational | 4 steps; useful for immediate staffing. |
|
||
| 48 h | Plan | Day-ahead; weather-sensitive. |
|
||
| 7 d | Plan | Weekly shift plan; the hero. |
|
||
|
||
The CSV is static. Without a live feed, 15 min/1 h are replay/nowcast features, not live sensor data. Label them accordingly.
|
||
|
||
## Failure modes
|
||
|
||
| Failure | Detection | Fallback |
|
||
|---|---|---|
|
||
| No SKE access | No kubeconfig by Sat 09:00 | Escalate to contact; build locally; manifests ready. |
|
||
| Gitea pull fails | `ImagePullBackOff` | Check `gitea-regcred`, DNS, NetworkPolicy; public Docker Hub pull works as fallback. |
|
||
| Cloudflare Tunnel down | URL 502 / no tunnel | Restart `cloudflared`; use a temporary `kubectl port-forward` for the demo? No — use a backup tunnel or local fallback. |
|
||
| SSE buffered | No events on phone | Poll `/api/scenario` every 5 s; keep UI. |
|
||
| FastAPI down | `/readyz` fails | Go serves cached JSON; banner "cached forecast." |
|
||
| Nuxt SSR down | 502 on `/` | Restart pod; local fallback; recorded video. |
|
||
| No Jobs/CronJobs | Batch cannot run in-cluster | Build-time pipeline; document production path. |
|
||
| No NetworkPolicy permission | Cannot fix a default-deny | Test early; escalate to the platform. |
|
||
| Venue network | Phones cannot load | Mobile data; laptop local build; video. |
|
||
| Single API pod restarts | Brief 502 | Probes, small image, `Recreate`; SSE clients reconnect. |
|
||
|
||
## Security and privacy
|
||
|
||
- No PII, no MAC addresses, no geolocation permission; audience view uses a random session ID.
|
||
- `/control` uses **basic auth** (Go middleware) or a token; public dashboard and `/live` stay open. No SSO.
|
||
- All images run as non-root. No privileged containers.
|
||
- Secrets: `gitea-regcred`, `cloudflared` token, control credentials. Never in git.
|
||
- Same-origin or explicit CORS; no third-party trackers; bundle fonts/assets.
|
||
- UI states the data limitation: "estimated devices, not people; 8 sensors; 5 months."
|
||
- **Cluster credential:** the provided kubeconfig is a secret. It is git-ignored and chmod 600. Rotate the token after the event; it was shared in chat.
|
||
|
||
## Degradation ladder
|
||
|
||
1. **Full:** Nuxt SSR + Go + FastAPI + `cloudflared`.
|
||
2. **Two services:** Nuxt SPA served by Go + FastAPI; drop SSR if resources/time are tight.
|
||
3. **One service:** Go serves SPA, API, SSE and cached forecasts; Python is build-time only.
|
||
4. **No tunnel:** temporary `kubectl port-forward` for local verification only; not a public demo.
|
||
5. **No SKE:** local kind + recorded demo; escalate; do not pretend it meets the requirement.
|
||
|
||
## Open decisions
|
||
|
||
1. Q1: **settled** — staffing copilot.
|
||
2. SSR: **settled** — yes.
|
||
3. Replay vs static for 15 min/1 h — default: replay P1, static P0.
|
||
4. Paper: **settled** — hackathon deliverable, 4–5 pages, AI draft + human verification.
|
||
5. Herdr: **settled** — four workstreams/worktrees.
|
||
6. Basic auth scope: `/control` only (default) vs whole app (would hurt the 120-phone demo).
|
||
7. Weather: use Open-Meteo/holidays (default) or only the provided CSV. See paper/data implications.
|