K3s: Lightweight Kubernetes for Edge and Small Clusters

K3s

K3s is a lightweight, fully conformant Kubernetes distribution packaged as a single binary. Originally created by Rancher Labs (now part of SUSE) and a CNCF project, it bundles everything a small cluster needs — container runtime, networking, DNS, ingress, a load balancer, storage provisioner, and metrics — and starts on machines with as little as half a gigabyte of memory. It runs on x86 and ARM, from Raspberry Pis and factory gateways to development laptops, CI environments, and modest production clusters.

K3s is still Kubernetes: the same APIs, manifests, Helm charts, and tooling work. What changes is how much there is to operate. The trade is that defaults suited to getting started — a single server with SQLite, bundled ingress, open firewall assumptions — need deliberate choices before production: a highly available datastore, backups, hardening, and an upgrade process.

TL;DR

One binary bundles containerd, Flannel, CoreDNS, Traefik, ServiceLB, local-path storage, and metrics-server; disable what you replace.
Servers run the control plane; agents run workloads; nodes join with a token.
Datastores: embedded SQLite (single server), embedded etcd (HA with three or more servers), or external MySQL, PostgreSQL, or etcd.
Configure with /etc/rancher/k3s/config.yaml instead of long command lines; pin versions or channels.
Harden with the CIS guide, secrets encryption, and network policy; plan air-gapped installs for disconnected sites.
Take etcd snapshots (scheduled and to S3), test restores, and upgrade with the system upgrade controller.

Quick Example

Configuration for the first server of a three-server, highly available cluster with embedded etcd:

Installing servers and agents, pinned to the stable channel:

An automated upgrade plan for server nodes, applied by the system upgrade controller:

Core Concepts

What Is Bundled

Files placed in /var/lib/rancher/k3s/server/manifests/ are applied automatically, which is convenient for bootstrapping add-ons on edge devices. Disable bundled components you replace so they do not conflict.

Servers, Agents, and Datastores

Put a load balancer or DNS name in front of the servers for agents and kubectl, and include it in tls-san. Keep the cluster token safe: it is needed to join nodes and to restore from snapshots.

Edge and Constrained Environments

K3s suits sites with limited hardware and unreliable connectivity: retail stores, factories, vehicles, and remote equipment. Consider resource reservations for the system, image pre-loading and air-gapped installs (images tarball plus binary), private registry mirrors (registries.yaml), and fleet management tools such as Rancher Fleet or GitOps controllers to manage many small clusters. Single-node clusters at the edge trade high availability for simplicity; design applications to tolerate restarts and local data loss accordingly. See Edge Computing.

Security

Follow the K3s CIS hardening guide: protected kernel defaults, Pod Security admission, audit logging, and restrictive kubelet settings.
Enable secrets encryption at rest, and rotate encryption keys and certificates on a schedule.
Restrict access to the API server and node ports; K3s does not configure host firewalls for you.
Add network policies (Flannel needs the bundled policy controller, which K3s includes, or a CNI such as Cilium).

For regulated environments, RKE2 — a sibling distribution with a security-focused default configuration — may be a better fit.

Backup, Restore, and Upgrades

With embedded etcd, K3s takes scheduled snapshots (and can upload them to S3-compatible storage); restore with k3s server --cluster-reset --cluster-reset-restore-path=<snapshot> on one server, then rejoin the others. With SQLite, back up the database directory and token. Cluster state is only part of recovery: persistent volumes on local-path storage live on individual nodes and need their own backups. See Stateful Kubernetes Backup and Restore.

Upgrade one minor version at a time, servers before agents. The system upgrade controller applies upgrade plans node by node with cordon and drain; for small clusters, rerunning the installer with a pinned version works too. Snapshot first, read release notes for removed APIs, and test in a staging cluster. See Kubernetes Cluster Upgrades and Recovery.

Best Practices

Choose the Datastore Deliberately

Use embedded etcd with three servers for production control planes, and SQLite only where a single server is acceptable.

Keep Configuration in Files and Git

Use config.yaml, manifests, and Helm values in version control instead of ad hoc command-line flags.

Snapshot Off-Node and Test Restores

Send snapshots to remote storage and rehearse restores, including the token.

Pin Versions and Upgrade Regularly

Pin a channel or version, upgrade one minor release at a time, and do not fall behind supported releases.

Common Mistakes

Single-Server SQLite in Production

Bundled and Custom Components Fighting

Unpinned Installs at the Edge

FAQ

Is K3s real Kubernetes?

Yes. K3s is a CNCF-certified conformant Kubernetes distribution; standard manifests, Helm charts, and tools work, with some components packaged differently and lighter defaults.

When should I use K3s instead of a managed Kubernetes service?

For edge and on-premises sites, small self-hosted clusters, development and CI environments, and devices with limited resources. Managed services usually suit cloud workloads better.

How do I make K3s highly available?

Run three or more server nodes with embedded etcd (or two or more with an external datastore), put a load balancer or DNS name in front of the API servers, and back up regularly.

How do I back up K3s?

With embedded etcd, use scheduled snapshots stored off-node and keep the cluster token. With SQLite, back up the database directory. Back up persistent volumes separately.

What is the difference between K3s and RKE2?

Both are SUSE Rancher Kubernetes distributions. K3s is optimized for lightweight and edge use; RKE2 is focused on security and compliance with hardened defaults, at a higher resource cost.

K3s