k3s
看同事写的k3s的总结，还以为看错了，不应该是k8s么。查了下，还真的是有k3s的。
我对k8s也是了解一些主要组件的作用，更多的是用kubectl定位问题。干技术的就是不仅工作上的知识要学习，也要扩展知识面的宽度。
下面的转自：https://www.jianshu.com/p/dbc8d9a8374e
k3s是rancher®出品的一个简化、轻量的k8s，本篇博客记录k3s的安装及踩的部分坑。
从名字上也能看出，k3s比k8s少了些东西，详情可见其官网k3s.io，本地试验可参考官网离线安装教程
k3s官网
安装步骤
准备工作
首先去其github的releases页下载主可执行文件k3s、离线安装包k3s-airgap-images-amd64.tar和安装脚本
我用的是v1.18.6+k3s1版本，其于2020年7月16日发布。
增加可执行文件和脚本的可执行权限
wget https://get.k3s.io -O install-k3s.sh
chmod +x install-k3s.sh
需要有/usr/local/bin/k3s，可考虑软连接
sudo ln -s /home/dev/program/k3s /usr/local/bin/k3s
复制tar文件到/var/lib/rancher/k3s/agent/images
sudo mkdir -p /var/lib/rancher/k3s/agent/images
sudo cp k3s-airgap-images-amd64.tar /var/lib/rancher/k3s/agent/images
定制一些变量
先设置变量如下：
export INSTALL_K3S_SKIP_DOWNLOAD=true
export INSTALL_K3S_EXEC="--docker --write-kubeconfig ~/.kube/config --write-kubeconfig-mode 666"
逐个解释一下：
INSTALL_K3S_SKIP_DOWNLOAD=true效果为不去下载k3s可执行文件INSTALL_K3S_EXEC="（略）"效果为启动k3s服务时使用的额外参数--docker效果为使用docker而不是默认的containerd--write-kubeconfig-mode 666效果为将配置文件权限改为非所有者也可读可写，进而使kubectl命令无需root或sudo--write-kubeconfig ~/.kube/config效果为将配置文件写到k8s默认会用的位置，而不是k3s默认的位置/etc/rancher/k3s/k3s.yaml。后者会导致istio、helm需要额外设置或无法运行。
官网教程-安装选项中还有其他可用的选项
执行安装脚本
$ ./install-k3s.sh
[INFO] Skipping k3s download and verify
[INFO] Skipping /usr/local/bin/kubectl symlink to k3s, already exists
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3s
执行k3s命令看看效果
$ k3s
NAME:
k3s - Kubernetes, but small and simple
USAGE:
k3s [global options] command [command options] [arguments...]
VERSION:
v0.9.1 (755bd1c6)
COMMANDS:
server Run management server
agent Run node agent
kubectl Run kubectl
crictl Run crictl
ctr Run ctr
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--debug Turn on debug logs
--help, -h show help
--version, -v print the version
还有k3s kubectl和kubectl
$ k3s kubectl get all --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-66f496764-mkwjv 1/1 Running 0 5m9s
kube-system pod/helm-install-traefik-t4xlj 1/1 Running 0 5m8s
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-system service/kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 5m27s
default service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 5m25s
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system deployment.apps/coredns 1/1 1 1 5m27s
NAMESPACE NAME DESIRED CURRENT READY AGE
kube-system replicaset.apps/coredns-66f496764 1 1 1 5m9s
NAMESPACE NAME COMPLETIONS DURATION AGE
kube-system job.batch/helm-install-traefik 0/1 5m8s 5m25s
访问kubernetes服务
由于k3s默认没有提供dashboard作为web ui，先访问k8s的rest
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP
会要求输入用户名密码，在~/.kube/config中有访问其的用户名密码，内容类似如下：
users:
- name: default
user:
password: ec2fb0ab4401d7f2525d480fd08e908d
username: admin
文件位置默认为/etc/rancher/k3s/k3s.yaml，但是前述步骤中通过--write-kubeconfig ~/.kube/config修改
认证似乎是www basic（对k8s还没了解到这种程度，此处存疑）
也可kubectl version或随便kubectl run测试一下
若干问题
如何卸载
见install.sh的回显，其中有uninstall-script：
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
离线安装包
如果没有复制k3s-airgap-images-amd64.tar，会卡着
$ k3s kubectl get all --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/helm-install-traefik-t4xlj 0/1 ContainerCreating 0 4m42s
kube-system pod/coredns-66f496764-mkwjv 0/1 ContainerCreating 0 4m43s
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-system service/kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 5m1s
default service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 4m59s
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system deployment.apps/coredns 0/1 1 0 5m1s
NAMESPACE NAME DESIRED CURRENT READY AGE
kube-system replicaset.apps/coredns-66f496764 1 1 0 4m43s
NAMESPACE NAME COMPLETIONS DURATION AGE
kube-system job.batch/helm-install-traefik 0/1 4m42s 4m59s
复制后，安装过程继续
拉不下镜像
可能因为啥拉镜像失败，可通过kubectl describe pod coredns-57d8bbb86-mndrr -n kube-system查看events：
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling <unknown> default-scheduler 0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.
Normal Scheduled <unknown> default-scheduler Successfully assigned kube-system/coredns-57d8bbb86-mndrr to dk-aspire-5943g
Warning FailedCreatePodSandBox 3s (x4 over 89s) kubelet, dk-aspire-5943g Failed create pod sandbox: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
可见是由于拉不下镜像k8s.gcr.io/pause:3.1，于是从阿里云拉下镜像，再tag
$ docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1
3.1: Pulling from google_containers/pause
cf9202429979: Pull complete
Digest: sha256:759c3f0f6493093a9043cc813092290af69029699ade0e3dbe024e968fcb7cca
Status: Downloaded newer image for registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1
registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.cn-hangzhou.aliyuncs.com/google_containers/pause 3.1 da86e6ba6ca1 22 months ago 742kB
$ docker tag da86e6ba6ca1 k8s.gcr.io/pause:3.1
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
k8s.gcr.io/pause 3.1 da86e6ba6ca1 22 months ago 742kB
registry.cn-hangzhou.aliyuncs.com/google_containers/pause 3.1 da86e6ba6ca1 22 months ago 742kB
kubectl需要root权限
前已述及，在安装前设置若干变量，其中有针对这个问题的
$ kubectl get all
WARN[2019-10-20T22:58:52.068331383+08:00] Unable to read /etc/rancher/k3s/k3s.yaml, please start server with --write-kubeconfig-mode to modify kube config permissions
error: Error loading config file "/etc/rancher/k3s/k3s.yaml": open /etc/rancher/k3s/k3s.yaml: permission denied
/etc/rancher/k3s/k3s.yaml的默认权限为-rw-------即600，所有者root root
根据提示，在启动时需要带有--write-kubeconfig-mode *新权限*，经试验，666可以起到让kubectl无需root权限的效果
此外，v1.17.0+k3s.1的文档中提到一个选项：
--rootless (experimental) Run rootless
但是试验不成功，service k3s启动失败
定制环境变量如下：
export INSTALL_K3S_SKIP_DOWNLOAD=true
export INSTALL_K3S_EXEC="--docker --write-kubeconfig ~/.kube/config --write-kubeconfig-mode 666"
启动失败日志片段如下：
$ ./install-k3s.sh
（略）
[INFO] systemd: Starting k3s
Job for k3s.service failed because the control process exited with error code.
See "systemctl status k3s.service" and "journalctl -xe" for details.
$ journalctl -xe
（略）
Jan 09 15:31:40 dk-mi13 k3s[4490]: time="2020-01-09T15:31:40.488024565+08:00" level=fatal msg="resolving : determining current user: $HOME is not defined"
Jan 09 15:31:40 dk-mi13 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
-- Subject: Unit process exited
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- An ExecStart= process belonging to unit k3s.service has exited.
--
-- The process' exit code is 'exited' and its exit status is 1.
Jan 09 15:31:40 dk-mi13 systemd[1]: k3s.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- The unit k3s.service has entered the 'failed' state with result 'exit-code'.
Jan 09 15:31:40 dk-mi13 systemd[1]: Failed to start Lightweight Kubernetes.
-- Subject: A start job for unit k3s.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A start job for unit k3s.service has finished with a failure.
--
-- The job identifier is 16669 and the job result is failed.
//TODO
这个问题暂未解决，报错中似乎比较关键的
level=fatal msg="resolving : determining current user: $HOME is not defined"
也有问题，无论是自己的账户还是su用root账户，查看$HOME变量均可得到值
$ echo $HOME
/home/dk
$ env | grep HOME
HOME=/home/dk
（略）
$ su
Password:
# echo $HOME
/root
# env | grep HOME
HOME=/root
（略）
KUBECONFIG位置
配置文件默认位置给其他处带来一些不便，例如使用helm需要如下额外参数以指定配置文件的位置
--kubeconfig /etc/rancher/k3s/k3s.yaml
想改为更为通用的~/.kube/config，使用参数--write-kubeconfig ~/.kube/config
此外，在v1.17.0+k3s.1版本中，使用kubectl -v 6可见其对配置文件的处理：
$ kubectl get all -v 6
I0109 11:27:11.815808 20876 loader.go:375] Config loaded from file: /etc/rancher/k3s/k3s.yaml
依然读取/etc/rancher/k3s/k3s.yaml，但这个文件实际上链接到了~/.kube/config：
$ ll /etc/rancher/k3s/k3s.yaml
lrwxrwxrwx 1 root root 21 Jan 9 15:48 /etc/rancher/k3s/k3s.yaml -> /home/dk/.kube/config
$ ll ~/.kube/config
-rw-rw-rw- 1 root root 1052 Jan 9 15:48 /home/dk/.kube/config
kubectl get all 耗时长
用v1.17.0+k3s.1执行kubectl get all耗时较长（v1.18.6+k3s1中问题依旧），但是kubectl get pod等查看一种资源的命令耗时并不较长，增加-v 6查看更详细日志：
$ kubectl get all -v 6
（略）
I0109 11:27:11.824426 20876 round_trippers.go:443] GET https://127.0.0.1:6443/api?timeout=32s 200 OK in 8 milliseconds
I0109 11:27:11.824977 20876 round_trippers.go:443] GET https://127.0.0.1:6443/apis?timeout=32s 200 OK in 0 milliseconds
I0109 11:27:11.825346 20876 cached_discovery.go:130] failed to write cache to /home/dk/.kube/cache/discovery/127.0.0.1_6443/servergroups.json due to mkdir /home/dk/.kube/cache: permission denied
I0109 11:27:11.828528 20876 round_trippers.go:443] GET https://127.0.0.1:6443/api/v1?timeout=32s 200 OK in 2 milliseconds
I0109 11:27:11.829574 20876 cached_discovery.go:87] failed to write cache to /home/dk/.kube/cache/discovery/127.0.0.1_6443/v1/serverresources.json due to mkdir /home/dk/.kube/cache: permission denied
（略）
可知，原因是向~/.kube/cache文件夹下写时无权限，处理大量错误耗费了时间。默认无此文件夹，上层.kube文件夹所有者root root，权限755
$ ll ~ | grep .kube
drwxr-xr-x 2 root root 4096 Jan 9 11:30 .kube/
若使用sudo kubectl get all没有此耗时问题。
修正方法，将此文件夹权限改为其他用户可写；或者新建cache和http-cache两文件夹，并更改所有者为当前用户。后一种方法例：
$ sudo mkdir cache http-cache
$ sudo chown dk:dk cache http-cache
至此，解决了kubectl get all等命令耗时太长问题
作者：dracula337435
链接：https://www.jianshu.com/p/dbc8d9a8374e
来源：简书
著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。
