Files
Noel Sabia e022c30527 feat(okf): run on full Kubernetes with FreeLens for admin
k3s is out. The cluster is upstream Kubernetes installed with kubeadm:
containerd, single control-plane etcd, Calico for NetworkPolicy.

- add the cluster decision and the FreeLens decision
- move Terraform from the k3s service to kubeadm, Calico, metrics-server
  and the local-path provisioner
- budget ~900 MB for the control plane and plan a bigger machine before 0.3.0
- note kubelet failSwapOn and containerd hosts.toml
- rename 0.3.0 and commit the 0.2.0 done-when to the new stack
- update the German learning plan
2026-10-24 19:02:17 +02:00

69 lines
3.2 KiB
Markdown

# Server provisioning
Terraform owns everything outside the cluster: packages, swap, containerd,
the kubeadm control plane, the reverse proxy, and DNS records. Rebuilding the
server from nothing is two commands.
## Scope
| Layer | Managed by | Examples |
|---|---|---|
| Server, packages, swap, runtime, kubeadm install | Terraform (SSH provisioner, cloud-init or shell) | `fallocate`, `mkswap`, containerd, `kubeadm init`, Calico |
| Cluster addons | Terraform (`kubectl` provisioner on the kubeadm output) | metrics-server, local-path provisioner |
| DNS records | Terraform provider for the DNS zone | A/AAAA record for the apex domain |
| Reverse proxy | Terraform-managed Caddy, Caddyfile templated | reverse_proxy block to the gateway |
| Workloads inside the cluster | kustomize, applied by CI or hand | Namespace `cv`, Deployments, Jobs, Services, quota |
Terraform does not manage anything inside the cluster. Kubernetes already
describes its own state; Terraform and kubectl would fight over it. The one
exception is bootstrap: the namespace, the quota and the addons are created
once, after `kubeadm init`, and never reconciled again.
## Files
```
terraform/
main.tf server: packages, swap, containerd, kubeadm, addons, Caddy
dns.tf A record for the apex domain
variables.tf server IP, SSH key, domain
outputs.tf server IP, kubeconfig path, proxy URL
```
Plan output for one run:
```
+ swapfile 2 GB
+ containerd 2.x, kubeadm, kubelet, kubectl
+ control plane on 127.0.0.1:6443 (etcd, apiserver, scheduler, controller-manager)
+ Calico (NetworkPolicy)
+ metrics-server and local-path provisioner
+ Caddy reverse_proxy block -> http://127.0.0.1:8080 (gateway NodePort)
+ certificate for noelsabia.dev
```
## Why
- The server became disposable: `terraform destroy && terraform apply` plus
`kubectl apply -k deploy/` returns the whole system. That is the fastest
possible disaster recovery and the clearest possible demo.
- kubeadm is the upstream installer. The same steps on a bigger machine or a
cloud VM behave identically, which is what the [cluster decision](/decisions/full-kubernetes.md) asks for.
- Proxmox and cloud names change; code does not. The same Terraform targets the
home server now and a VM later.
- Home IP changes: the DNS provider has an A record the server can update.
Terraform manages the record; a small script re-sets it if the IP moves.
## Constraints
- **Terraform is new tooling** for Noel and needs approval under the stack
rule. Worth it: infrastructure as code is a core work skill.
- **Control plane memory:** allow about 900 MB to 1.2 GB for etcd, apiserver,
scheduler, controller-manager and kubelet. On a 4 GB server this is tight;
the machine is enlarged or replaced before 0.3.0. See [server budget](/architecture/server-budget.md).
- **No bundled addons:** full Kubernetes ships no ingress, load balancer,
metrics-server or storage provisioner. Caddy, metrics-server and the
local-path provisioner are explicit steps.
- State file: local, not in Git. Nothing secret in the config itself; the
kubeconfig it produces stays on Noel's machine.
- Changing a completed chapter or a Job is a delete plus apply, documented.