feat(okf): run on full Kubernetes with FreeLens for admin
k3s is out. The cluster is upstream Kubernetes installed with kubeadm: containerd, single control-plane etcd, Calico for NetworkPolicy. - add the cluster decision and the FreeLens decision - move Terraform from the k3s service to kubeadm, Calico, metrics-server and the local-path provisioner - budget ~900 MB for the control plane and plan a bigger machine before 0.3.0 - note kubelet failSwapOn and containerd hosts.toml - rename 0.3.0 and commit the 0.2.0 done-when to the new stack - update the German learning plan
This commit is contained in:
1 parent
6ab76c2a36
commit
e022c30527
17 files changed
+214
-64
No files matched your search
+25
-10
@@ -1,36 +1,42 @@
|
||||
# Server provisioning
|
||||
|
||||
Terraform owns everything outside the cluster: packages, swap, k3s itself,
|
||||
the reverse proxy, and DNS records. Rebuilding the server from nothing is two
|
||||
commands.
|
||||
Terraform owns everything outside the cluster: packages, swap, containerd,
|
||||
the kubeadm control plane, the reverse proxy, and DNS records. Rebuilding the
|
||||
server from nothing is two commands.
|
||||
|
||||
## Scope
|
||||
|
||||
| Layer | Managed by | Examples |
|
||||
|---|---|---|
|
||||
| Server, packages, swap, k3s install | Terraform (SSH provisioner, cloud-init or shell) | `fallocate`, `mkswap`, k3s service with trimmed config |
|
||||
| Server, packages, swap, runtime, kubeadm install | Terraform (SSH provisioner, cloud-init or shell) | `fallocate`, `mkswap`, containerd, `kubeadm init`, Calico |
|
||||
| Cluster addons | Terraform (`kubectl` provisioner on the kubeadm output) | metrics-server, local-path provisioner |
|
||||
| DNS records | Terraform provider for the DNS zone | A/AAAA record for the apex domain |
|
||||
| Reverse proxy | Terraform-managed Caddy, Caddyfile templated | reverse_proxy block to the gateway |
|
||||
| Workloads inside k3s | kustomize, applied by CI or hand | Namespace `cv`, Deployments, Jobs, Services, quota |
|
||||
| Workloads inside the cluster | kustomize, applied by CI or hand | Namespace `cv`, Deployments, Jobs, Services, quota |
|
||||
|
||||
Terraform does not manage anything inside the cluster. Kubernetes already
|
||||
describes its own state; Terraform and kubectl would fight over it.
|
||||
describes its own state; Terraform and kubectl would fight over it. The one
|
||||
exception is bootstrap: the namespace, the quota and the addons are created
|
||||
once, after `kubeadm init`, and never reconciled again.
|
||||
|
||||
## Files
|
||||
|
||||
```
|
||||
terraform/
|
||||
main.tf server: packages, swap, k3s, Caddy
|
||||
main.tf server: packages, swap, containerd, kubeadm, addons, Caddy
|
||||
dns.tf A record for the apex domain
|
||||
variables.tf server IP, SSH key, domain
|
||||
outputs.tf server IP, proxy URL
|
||||
outputs.tf server IP, kubeconfig path, proxy URL
|
||||
```
|
||||
|
||||
Plan output for one run:
|
||||
|
||||
```
|
||||
+ swapfile 2 GB
|
||||
+ k3s service (traefik, servicelb disabled)
|
||||
+ containerd 2.x, kubeadm, kubelet, kubectl
|
||||
+ control plane on 127.0.0.1:6443 (etcd, apiserver, scheduler, controller-manager)
|
||||
+ Calico (NetworkPolicy)
|
||||
+ metrics-server and local-path provisioner
|
||||
+ Caddy reverse_proxy block -> http://127.0.0.1:8080 (gateway NodePort)
|
||||
+ certificate for noelsabia.dev
|
||||
```
|
||||
@@ -40,6 +46,8 @@ Plan output for one run:
|
||||
- The server became disposable: `terraform destroy && terraform apply` plus
|
||||
`kubectl apply -k deploy/` returns the whole system. That is the fastest
|
||||
possible disaster recovery and the clearest possible demo.
|
||||
- kubeadm is the upstream installer. The same steps on a bigger machine or a
|
||||
cloud VM behave identically, which is what the [cluster decision](/decisions/full-kubernetes.md) asks for.
|
||||
- Proxmox and cloud names change; code does not. The same Terraform targets the
|
||||
home server now and a VM later.
|
||||
- Home IP changes: the DNS provider has an A record the server can update.
|
||||
@@ -49,5 +57,12 @@ Plan output for one run:
|
||||
|
||||
- **Terraform is new tooling** for Noel and needs approval under the stack
|
||||
rule. Worth it: infrastructure as code is a core work skill.
|
||||
- State file: local, not in Git. Nothing secret in the config itself.
|
||||
- **Control plane memory:** allow about 900 MB to 1.2 GB for etcd, apiserver,
|
||||
scheduler, controller-manager and kubelet. On a 4 GB server this is tight;
|
||||
the machine is enlarged or replaced before 0.3.0. See [server budget](/architecture/server-budget.md).
|
||||
- **No bundled addons:** full Kubernetes ships no ingress, load balancer,
|
||||
metrics-server or storage provisioner. Caddy, metrics-server and the
|
||||
local-path provisioner are explicit steps.
|
||||
- State file: local, not in Git. Nothing secret in the config itself; the
|
||||
kubeconfig it produces stays on Noel's machine.
|
||||
- Changing a completed chapter or a Job is a delete plus apply, documented.
|
||||
Reference in new issue
Block a user