k3s is out. The cluster is upstream Kubernetes installed with kubeadm: containerd, single control-plane etcd, Calico for NetworkPolicy. - add the cluster decision and the FreeLens decision - move Terraform from the k3s service to kubeadm, Calico, metrics-server and the local-path provisioner - budget ~900 MB for the control plane and plan a bigger machine before 0.3.0 - note kubelet failSwapOn and containerd hosts.toml - rename 0.3.0 and commit the 0.2.0 done-when to the new stack - update the German learning plan
3.7 KiB
type, title, description, tags, status, generated, sources
| type | title | description | tags | status | generated | sources | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Reference | Server and memory budget | The CV shares a small home server with Gitea, so every workload gets a hard memory cap; a kubeadm control plane needs more room than k3s did. |
|
revised |
|
|
Server
server0: 2 vCPU, 3805 MB RAM, 2 GB swap. Shared with Gitea, its Actions
runner, and a small webapp with one user.
A control plane with etcd costs about twice what k3s' single binary cost. The
plan is a stronger machine (8 GB or more, or a second server that carries
Gitea) before release 0.3.0. The numbers below are the floor that must fit
until then; the cv quota can rise once the bigger machine exists.
Budget
| What | Memory | Basis |
|---|---|---|
| OS, Gitea, runner (idle), webapp | 1143 MB | Measured1 |
| etcd, kube-apiserver, scheduler, controller-manager, kubelet | ~900 MB | Estimate, measure after install |
| Calico, Caddy, metrics-server | ~150 MB | Estimate |
Namespace cv, hard quota |
384 MB | ResourceQuota |
| Left for CI builds and page cache | ~1.2 GB (4 GB) / ~5 GB (8 GB) | |
| Swap | 2 GB | Safety net for build spikes |
Per workload inside cv:
| Workload | Memory limit | CPU limit |
|---|---|---|
| Gateway, including the tracker | 64Mi, GOMEMLIMIT=50MiB |
100m |
| Each chapter (~10) | 16Mi, GOMEMLIMIT=12MiB |
10m |
Completed Jobs do not count against the quota.
Namespace caps
apiVersion: v1
kind: ResourceQuota
metadata: { name: cv-budget, namespace: cv }
spec:
hard: { limits.memory: 384Mi }
---
apiVersion: v1
kind: LimitRange
metadata: { name: cv-defaults, namespace: cv }
spec:
limits:
- type: Container
default: { memory: 16Mi, cpu: 50m }
defaultRequest: { memory: 8Mi, cpu: 5m }
Kubelet config
kubeadm writes /var/lib/kubelet/config.yaml from the
KubeletConfiguration in its config file. The values that matter:
failSwapOn: false # node has swap; pods still never swap
systemReserved: { memory: 1300Mi } # measured 1143 MB plus margin
evictionHard: { memory.available: 200Mi }
Full Kubernetes installs no addons. Two are required and are applied by the
Terraform module after kubeadm init: metrics-server for the HPA demo in
1.9.0, and a local-path provisioner for the tracker's SQLite volume.
Caddy replaces the ingress controller. Calico provides NetworkPolicy.
Do not use --pod-network-cidr values that collide with the home LAN; Calico
defaults to 192.168.0.0/16, which on a typical 192.168.x.x home network is
a conflict. Pick 10.244.0.0/16.
Swap
sudo fallocate -l 2G /swapfile && sudo chmod 600 /swapfile
sudo mkswap /swapfile && sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
kubeadm and the kubelet refuse to start on a swapped host unless
failSwapOn: false is set. With that flag the node runs, pods still get no
swap, and host processes such as Gitea and builds can use it. Enabling swap
for pods needs the NodeSwap feature gate and is not worth it here.
CI builds
The biggest memory risk. A Nuxt build can take over 1 GB.
- Runner capacity 1: one job at a time.
NODE_OPTIONS=--max-old-space-size=512for the Nuxt build.- Swap catches the rest.
What does not run here
- Language model training: on Noel's own computer. The server only serves the weights file.
- Large load tests: k6 and the HPA demo stay tiny (max 3 replicas, a few virtual users), or are cut.
-
free -mon server0 ↩︎