Setup lightweight Kubernetes with K3s

Peter Gillich
6 min readDec 19, 2020


K3s is a lightweight Kubernetes distribution, which looks like easy to install for devops, but not for dummies. This article gives answers for questions: How to solve install error? How to configure ingress? How to use dashboards? How to force Docker? How to use already installed kubectl?

K3s uses a built-in containerd as the container runtime, by default. This default setup has more advantages:

  • Docker is not needed.
  • If Docker is installed, the K3s containers and Docker containers are running in separated environment and cannot influence each other

Below description forces to use Docker to give opportunity for Docker experts to see what K3s makes under the hood. Of course, below example works without Docker, too (noticed at right place).

The default K3s ingress controller is Traefik, it’s not changed.

Used versions:

Tested on:

  • Ubuntu 16.04, bare metal (Intel Celeron N2830, 4 GB RAM)
  • Ubuntu 18.04, VM (12 GB RAM), running in Windows 10 Hyper-V

Warning: Below example is not secure and must be hardened before using it in production.

If you are interested in a more complex K8s deployment, you can read my next article Setup On-premise Kubernetes with Kubeadm, MetalLB, Traefik and Vagrant . These two K8s clusters can run same time on a Linux bare metal machine, so it’s good opportunity to compare a lightweight K8s development environment and a VM-based multi-node on-premise K8s deployment.


If you want to force Docker, Docker CE must be installed, for example:

The TCP port 80 must be free (for ingress).

For homeworkers: if you are using VPN, a few K3s subnets ( and shouldn’t be in the VPN-routed address space. See more details at .

Install K3s

If you don’t want to force Docker, you should skip --docker option below.

If you don’t have installed kubectl, then (makes link to built-in K3s kubectl, by default):

curl -sfL | sh -s - --docker --write-kubeconfig-mode 644


curl -sfL | INSTALL_K3S_SYMLINK=skip sh -s - --docker --write-kubeconfig-mode 644# Execute more steps: , for example:
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Note: the --write-kubeconfig-mode 644 option is needed to avoid kubectl permission denied error later on.

Configure kubectl autocomplete

If kubectl autocomplete is not set yet:

echo 'source <(kubectl completion bash)' >>~/.bashrcsource <(kubectl completion bash)

Check K3s install

k3s check-configkubectl cluster-infokubectl get nodes -o wide# Wait until all pods and deployments aren't Running or Completed,
# see READY and STATUS columns
kubectl get all -A -o wide
kubectl get endpoints -Asudo k3s crictl ps -akubectl top pod --containers -A# Below command works only if Docker has already been installed
docker ps


Dashboards are not needed, but easy for getting information about Kubernetes resources.

Dashboards can be accessed by several ways. Below chapters describe 2 ways, one of them is enough:

  • kubectl proxy, trough http://localhost:8001/api/v1/namespaces/<NAMESPACE>/services/<HTTP_PROTO>:<SERVICE>:<PORT>/proxy/ pattern
  • Trough ingress controller

If ingress controller is configured, dashboards can be accessed trough a dedicated hostname oam.internal. Open hosts file:

sudo nano /etc/hosts

Add below line and save+exit:       oam.internal

Install Traefik Dashboard


Traefik Dashboard is not enabled by default. It can be enabled in Helm file of Traefik.

Open Helm file:

sudo nano /var/lib/rancher/k3s/server/manifests/traefik.yaml

Append dashboard lines to spec.valuesContent, like:

chart: https://%{KUBERNETES_API}%/static/charts/traefik-1.81.0.tgz
valuesContent: |-
enabled: true
domain: "oam.internal"

enabled: true

After save+exit, wait for an existing endpoint:

watch kubectl get endpoints traefik-dashboard -n kube-system

Note: after reboot the Helm file and the ingress config are recovered to the original content.

Access Traefik Dashboard trough kubectl proxy

Start kubectl proxy in a new terminal:

kubectl proxy

Open http://localhost:8001/api/v1/namespaces/kube-system/services/http:traefik-dashboard:80/proxy/dashboard/#/ in a browser.

Access Traefik Dashboard trough ingress

Ingress has already been configured by changing Helm file. The Helm chart does not support to set path in ingress config, so it’s bound to the root.

Open http://oam.internal/dashboard/ in a browser.

Install Kubernetes Dashboard


Deploying the Kubernetes Dashboard:

GITHUB_URL=$(curl -w '%{url_effective}' -I -L -s -S ${GITHUB_URL}/latest -o /dev/null | sed -e 's|.*/||')kubectl create -f${VERSION_KUBE_DASHBOARD}/aio/deploy/alternative.yaml

Dashboard RBAC configuration:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard
kind: ClusterRoleBinding
name: admin-user
kind: ClusterRole
name: cluster-admin
- kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard

Wait for an existing endpoint:

watch kubectl get endpoints kubernetes-dashboard -n kubernetes-dashboard

Obtain the Bearer Token (output will be used for login):

kubectl -n kubernetes-dashboard describe secret admin-user-token | grep ^token

Access Kubernetes Dashboard trough kubectl proxy

Start kubectl proxy in a new terminal:

kubectl proxy

Open http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ in a browser.
Use the bearer token printed out at earlier step.

Access Kubernetes Dashboard trough ingress

Configure ingress controller:

cat <<EOF | kubectl apply -f -
kind: Ingress
name: kubernetes-dashboard-ingress
namespace: kubernetes-dashboard
annotations: "traefik" "PathPrefixStrip"
- host: oam.internal
- path: /kubernetes
pathType: Prefix
name: kubernetes-dashboard
number: 80

Open https://oam.internal/kubernetes/ in a browser.
Use the bearer token printed out at an earlier step.

Setup a simple web server

Based on: , but shorter.

Deploy and expose web server

Below 2 commands supplements the deprecated/removed deployment generator in kubectl run :

kubectl create deployment --image nginx:latest my-nginxkubectl expose deployment my-nginx --port=80

Note: all generators are deprecated/removed in kubectl run command (depending on K8s version), see more details at and

Check web server

kubectl get all -o wide
# Wait until deployment.apps/my-nginx READY is not 1/1
# Use pod NAME from above output in below command
kubectl get pod/my-nginx-b7d7bc74d-7mhdf -o yaml
kubectl get deployment.apps/my-nginx -o yamlkubectl get service/my-nginx -o wide# Use CLUSTER-IP and PORT from above output in below command

Checking service DNS:

kubectl run cluster-tester -it --rm --restart=Never --image=busybox:1.28
#Or: kubectl run cluster-tester -it --rm --restart=Never
nslookup kubernetes.defaultnslookup my-nginx.default.svc.cluster.localwget -qSO- my-nginx.default.svc.cluster.local
#For dnsutils: wget -qO- my-nginx.default.svc.cluster.local

Custom configuration


If you would like to deploy and expose web server by kubeclt apply, as it written in the base documentation, you can start it with output of below harmless commands:

kubectl create deployment --image nginx:latest my-nginx --dry-run=client -o yamlkubectl expose deployment my-nginx --port=80 --dry-run=client -o yaml

The outputs can be compared to kubectl run output (without deployment generator):

kubectl run my-nginx --image=nginx --port=80 --expose=true --dry-run=client -o yaml

Publish web server

Below commands expose a HTTP route from outside to Kubernetes my-nginx service:

cat <<EOT >> /tmp/mysite.yaml
kind: Ingress
name: mysite-nginx-ingress
annotations: "traefik" "PathPrefixStrip"
- http:
- path: /my-nginx
pathType: Prefix
name: my-nginx
number: 80
kubectl apply -f /tmp/mysite.yamlcurl $(hostname --fqdn)/my-nginx

Open hostname --fqdn output + /my-nginx URL in a browser.

Scaling web server


Scaling simple web server to 3 instances:

kubectl scale --replicas=3 deployment/my-nginxkubectl get deployment/my-nginx -o wide
# Wait until deployment.apps/my-nginx READY is not 3/3

Cleanup web server


All resources are created by kubectl apply -f can be deleted by below command:

kubectl delete -f /tmp/mysite.yaml

The rest resources can be deleted by below commands (if not deleted by kubectl delete -f earlier):

kubectl delete ingress/mysite-nginx-ingresskubectl delete service/my-nginxkubectl delete deployment/my-nginx

Uninstall K3s


/usr/local/bin/k3s-uninstall.shsudo rm -rf /var/lib/rancher/k3s/ /etc/rancher/k3s


More info about K3d: