k3s is out. The cluster is upstream Kubernetes installed with kubeadm: containerd, single control-plane etcd, Calico for NetworkPolicy. - add the cluster decision and the FreeLens decision - move Terraform from the k3s service to kubeadm, Calico, metrics-server and the local-path provisioner - budget ~900 MB for the control plane and plan a bigger machine before 0.3.0 - note kubelet failSwapOn and containerd hosts.toml - rename 0.3.0 and commit the 0.2.0 done-when to the new stack - update the German learning plan
69 lines
3.2 KiB
Markdown
69 lines
3.2 KiB
Markdown
# Server provisioning
|
|
|
|
Terraform owns everything outside the cluster: packages, swap, containerd,
|
|
the kubeadm control plane, the reverse proxy, and DNS records. Rebuilding the
|
|
server from nothing is two commands.
|
|
|
|
## Scope
|
|
|
|
| Layer | Managed by | Examples |
|
|
|---|---|---|
|
|
| Server, packages, swap, runtime, kubeadm install | Terraform (SSH provisioner, cloud-init or shell) | `fallocate`, `mkswap`, containerd, `kubeadm init`, Calico |
|
|
| Cluster addons | Terraform (`kubectl` provisioner on the kubeadm output) | metrics-server, local-path provisioner |
|
|
| DNS records | Terraform provider for the DNS zone | A/AAAA record for the apex domain |
|
|
| Reverse proxy | Terraform-managed Caddy, Caddyfile templated | reverse_proxy block to the gateway |
|
|
| Workloads inside the cluster | kustomize, applied by CI or hand | Namespace `cv`, Deployments, Jobs, Services, quota |
|
|
|
|
Terraform does not manage anything inside the cluster. Kubernetes already
|
|
describes its own state; Terraform and kubectl would fight over it. The one
|
|
exception is bootstrap: the namespace, the quota and the addons are created
|
|
once, after `kubeadm init`, and never reconciled again.
|
|
|
|
## Files
|
|
|
|
```
|
|
terraform/
|
|
main.tf server: packages, swap, containerd, kubeadm, addons, Caddy
|
|
dns.tf A record for the apex domain
|
|
variables.tf server IP, SSH key, domain
|
|
outputs.tf server IP, kubeconfig path, proxy URL
|
|
```
|
|
|
|
Plan output for one run:
|
|
|
|
```
|
|
+ swapfile 2 GB
|
|
+ containerd 2.x, kubeadm, kubelet, kubectl
|
|
+ control plane on 127.0.0.1:6443 (etcd, apiserver, scheduler, controller-manager)
|
|
+ Calico (NetworkPolicy)
|
|
+ metrics-server and local-path provisioner
|
|
+ Caddy reverse_proxy block -> http://127.0.0.1:8080 (gateway NodePort)
|
|
+ certificate for noelsabia.dev
|
|
```
|
|
|
|
## Why
|
|
|
|
- The server became disposable: `terraform destroy && terraform apply` plus
|
|
`kubectl apply -k deploy/` returns the whole system. That is the fastest
|
|
possible disaster recovery and the clearest possible demo.
|
|
- kubeadm is the upstream installer. The same steps on a bigger machine or a
|
|
cloud VM behave identically, which is what the [cluster decision](/decisions/full-kubernetes.md) asks for.
|
|
- Proxmox and cloud names change; code does not. The same Terraform targets the
|
|
home server now and a VM later.
|
|
- Home IP changes: the DNS provider has an A record the server can update.
|
|
Terraform manages the record; a small script re-sets it if the IP moves.
|
|
|
|
## Constraints
|
|
|
|
- **Terraform is new tooling** for Noel and needs approval under the stack
|
|
rule. Worth it: infrastructure as code is a core work skill.
|
|
- **Control plane memory:** allow about 900 MB to 1.2 GB for etcd, apiserver,
|
|
scheduler, controller-manager and kubelet. On a 4 GB server this is tight;
|
|
the machine is enlarged or replaced before 0.3.0. See [server budget](/architecture/server-budget.md).
|
|
- **No bundled addons:** full Kubernetes ships no ingress, load balancer,
|
|
metrics-server or storage provisioner. Caddy, metrics-server and the
|
|
local-path provisioner are explicit steps.
|
|
- State file: local, not in Git. Nothing secret in the config itself; the
|
|
kubeconfig it produces stays on Noel's machine.
|
|
- Changing a completed chapter or a Job is a delete plus apply, documented.
|