k3s is out. The cluster is upstream Kubernetes installed with kubeadm: containerd, single control-plane etcd, Calico for NetworkPolicy. - add the cluster decision and the FreeLens decision - move Terraform from the k3s service to kubeadm, Calico, metrics-server and the local-path provisioner - budget ~900 MB for the control plane and plan a bigger machine before 0.3.0 - note kubelet failSwapOn and containerd hosts.toml - rename 0.3.0 and commit the 0.2.0 done-when to the new stack - update the German learning plan
3.2 KiB
3.2 KiB
Server provisioning
Terraform owns everything outside the cluster: packages, swap, containerd, the kubeadm control plane, the reverse proxy, and DNS records. Rebuilding the server from nothing is two commands.
Scope
| Layer | Managed by | Examples |
|---|---|---|
| Server, packages, swap, runtime, kubeadm install | Terraform (SSH provisioner, cloud-init or shell) | fallocate, mkswap, containerd, kubeadm init, Calico |
| Cluster addons | Terraform (kubectl provisioner on the kubeadm output) |
metrics-server, local-path provisioner |
| DNS records | Terraform provider for the DNS zone | A/AAAA record for the apex domain |
| Reverse proxy | Terraform-managed Caddy, Caddyfile templated | reverse_proxy block to the gateway |
| Workloads inside the cluster | kustomize, applied by CI or hand | Namespace cv, Deployments, Jobs, Services, quota |
Terraform does not manage anything inside the cluster. Kubernetes already
describes its own state; Terraform and kubectl would fight over it. The one
exception is bootstrap: the namespace, the quota and the addons are created
once, after kubeadm init, and never reconciled again.
Files
terraform/
main.tf server: packages, swap, containerd, kubeadm, addons, Caddy
dns.tf A record for the apex domain
variables.tf server IP, SSH key, domain
outputs.tf server IP, kubeconfig path, proxy URL
Plan output for one run:
+ swapfile 2 GB
+ containerd 2.x, kubeadm, kubelet, kubectl
+ control plane on 127.0.0.1:6443 (etcd, apiserver, scheduler, controller-manager)
+ Calico (NetworkPolicy)
+ metrics-server and local-path provisioner
+ Caddy reverse_proxy block -> http://127.0.0.1:8080 (gateway NodePort)
+ certificate for noelsabia.dev
Why
- The server became disposable:
terraform destroy && terraform applypluskubectl apply -k deploy/returns the whole system. That is the fastest possible disaster recovery and the clearest possible demo. - kubeadm is the upstream installer. The same steps on a bigger machine or a cloud VM behave identically, which is what the cluster decision asks for.
- Proxmox and cloud names change; code does not. The same Terraform targets the home server now and a VM later.
- Home IP changes: the DNS provider has an A record the server can update. Terraform manages the record; a small script re-sets it if the IP moves.
Constraints
- Terraform is new tooling for Noel and needs approval under the stack rule. Worth it: infrastructure as code is a core work skill.
- Control plane memory: allow about 900 MB to 1.2 GB for etcd, apiserver, scheduler, controller-manager and kubelet. On a 4 GB server this is tight; the machine is enlarged or replaced before 0.3.0. See server budget.
- No bundled addons: full Kubernetes ships no ingress, load balancer, metrics-server or storage provisioner. Caddy, metrics-server and the local-path provisioner are explicit steps.
- State file: local, not in Git. Nothing secret in the config itself; the kubeconfig it produces stays on Noel's machine.
- Changing a completed chapter or a Job is a delete plus apply, documented.