Skip to content

k3s

Recipe card from the charly-infrastructure plugin (Images — the deployable catalog).

Property Value
Install files charly.yml (binary fetch + symlinks via a run: plan step)
Pinned version v1.37.0+k3s1 (edit K3S_VERSION in charly.yml vars to cut over)

Downloads the verified-checksum k3s binary (plus sha256sum from the release manifest), installs it to /usr/local/bin/k3s, and creates symlinks for kubectl, crictl, ctr (k3s is multi-call). Installs runtime dependencies (iptables, conntrack, socat, ethtool, ca-certificates) via the distro package manager — not via the upstream curl | sh installer. Deliberate, per R9.

No service is started by this candy. Role selection happens in the dependent candies /charly-infrastructure:k3s-server and /charly-infrastructure:k3s-agent, which emit systemd units that wrap this binary with the right CLI verb (k3s server vs k3s agent).

Typically not used directly — compose /charly-infrastructure:k3s-server or /charly-infrastructure:k3s-agent (both depend on this candy).

# For a bare binary-only image (rare) — a box composes the candy via an
# inline candy: list in its body:
k3s-base:
candy:
base: fedora
candy:
- k3s

k3s-shared-containerd — k3s on the ONE shared containerd

Section titled “k3s-shared-containerd — k3s on the ONE shared containerd”

k3s-shared-containerd composes the k3s candy and points k3s at the shared rootful daemon (K3S.md §1, §2, §4; CONTAINERD-IMAGES.md §1.7). It starts no daemon and owns no part of the shared containerd’s config.

Artifact Value
k3s config /etc/rancher/k3s/config.yaml → container-runtime-endpoint: unix:///run/containerd/containerd.sock
CNI drop-in /etc/containerd/conf.d/50-k3s-cni.toml (partial): bin_dirs = ["/var/lib/rancher/k3s/data/current/bin"], conf_dir = "/var/lib/rancher/k3s/agent/etc/cni/net.d"
RuntimeClasses /var/lib/rancher/k3s/server/manifests/{krun,nvidia}-runtimeclass.yaml (node.k8s.io/v1, handler: krun / handler: nvidia)
Handler guard /usr/local/bin/k3s-runtime-handler-guard (from files/) + a run: step that records the baseline when the shared socket exists and restarts a running k3s only on a change
Service / port none (k3s-server / the lease image owns the unit)

Namespaces. k3s uses k8s.io; nerdctl uses default, on the same socket.

The CNI drop-in is required. k3s writes its flannel conflist and CNI binaries under /var/lib/rancher/k3s; stock containerd does not read them. Without the two keys, pods start but have no network. The drop-in is a PARTIAL config: it adds only these keys to the shared daemon’s version = 4 config, which imports /etc/containerd/conf.d/*.toml. Never rewrite the shared daemon’s full config from this layer.

The restart-after-handler-change rule. k3s reads the CRI runtime set (and the cgroup driver) once, at start; a handler added later is invisible to kubelet until k3s restarts (K3S.md §2.3 step 3). The guard fingerprints the MERGED containerd runtime map (containerd config dump, which includes conf.d) and behaves as follows:

  • no shared-daemon socket (image build, or a host that has not started containerd): exit 0 without recording a baseline — there is nothing to fingerprint;
  • first application with the socket present: record the baseline. On a fresh deploy this runs before k3s starts, so the baseline is the set k3s latches; if k3s was already running, restart it once because its latch predates this layer;
  • later application, fingerprint unchanged: exit 0, no restart;
  • later application, fingerprint changed while k3s runs: restart k3s;
  • later application, fingerprint changed while k3s is not running: update the baseline; the next start latches the new set (no restart is needed);
  • a handler added outside charly: run sudo /usr/local/bin/k3s-runtime-handler-guard or sudo systemctl restart k3s explicitly.

The guard takes K3S_HANDLER_GUARD_STATE_DIR and K3S_HANDLER_GUARD_SOCKET overrides for its regression test (test/k3s-runtime-handler-guard-test.sh), which covers every branch above.

The discriminating proof. nerdctl -n k8s.io ps on /run/containerd/containerd.sock must show k3s’s pods. An embedded containerd would hold them on its own socket, leaving that namespace empty.

Bed. check-k3s-shared-containerd is the absence → perform → presence proof. It is a host: local bed and must run inside a disposable containerd-host venue (charly --host <containerd-host-vm> check run check-k3s-shared-containerd), never on an operator workstation.

  • rpm: (Fedora) — conntrack-tools, iptables, ethtool, socat, ca-certificates
  • pac: (Arch) — conntrack-tools, iptables-nft, ethtool, socat, ca-certificates
  • deb: (Debian/Ubuntu) — conntrack, iptables, ethtool, socat, ca-certificates
  • k3s --version matches pinned version.
  • /usr/local/bin/k3s is mode 0755.
  • /usr/local/bin/kubectl exists as a symlink.
  • Each runtime package is installed (per-distro package_map handles Debian’s conntrack rename).
  • k3s-shared-containerd: the endpoint, both CNI keys, both RuntimeClass manifests, and the executable guard are present.
  • The bed proves an absence, then a Ready node on the shared containerd with k3s’s pods in nerdctl -n k8s.io ps, then both RuntimeClasses, then that the installed guard has a baseline and does not restart k3s on an unchanged set.
  • test/k3s-runtime-handler-guard-test.sh covers the guard’s socket-absent, baseline, unchanged, changed-while-running, and changed-while-inactive branches.