Mac下multipass+k3s环境搭建

下载multipass

用brew下载太慢，用迅雷下载pkg包，1.11.0+mac版本
https://github.com/canonical/multipass/releases/
安装一直点就可以了

创建3个虚拟机
-n：名称
-c：cpu核数
-d：磁盘大小
-m：内存

multipass launch -n node-1 -c 1 -d 10G  -m 1G
multipass launch -n node-2 -c 1 -d 10G  -m 1G
multipass launch -n node-2 -c 1 -d 10G  -m 1G

为了方便传文件，挂载本机目录home目录到虚拟机

multipass mount $HOME node-1:/home/ubuntu/Home
multipass mount $HOME node-2:/home/ubuntu/Home
multipass mount $HOME node-3:/home/ubuntu/Home

进入虚拟机
```
multipass shell node-1
```
将每台虚拟机的id_rsa.pub拷贝到每台虚拟机的authorized_keys

安装autok3s

方便创建k3s集群，使用rancher公司的autok3s插件，0.8.62版本
在node-1安装

curl -sS https://rancher-mirror.rancher.cn/autok3s/install.sh  | INSTALL_AUTOK3S_MIRROR=cn sh

创建集群
默认会从github下载k3s，会有网络问题导致安装失败，所以通过
```
--k3s-install-script  https://rancher-mirror.oss-cn-beijing.aliyuncs.com/k3s/k3s-install.sh \
--k3s-install-mirror  INSTALL_K3S_MIRROR=cn \
```
来指定k3s镜像源
```
autok3s -d create \
  --provider native \
  --name imac \
  --ssh-user ubuntu \
  --ssh-key-path $HOME/.ssh/id_rsa \
  --master-ips 192.168.64.2 \
  --worker-ips 192.168.64.3,192.168.64.4 \
  --k3s-install-script  https://rancher-mirror.oss-cn-beijing.aliyuncs.com/k3s/k3s-install.sh \
  --k3s-install-mirror  INSTALL_K3S_MIRROR=cn \
```
autok3s支持本地集群和云服务器集群，这里用--provider native表示创建本地集群
--name：集群名称
--ssh-user：ssh登录服务器时的用户名，（3台节点用户名要一致）
--ssh-key-path：ssh登录密钥
之前有把node-1的公钥拷贝到node-2,node-3上，也可以直接用node-1的密钥登录node-2,node-3节点
--master-ips：master节点ip
--worker-ips：从节点ip

卸载集群

autok3s -d delete --provider native --name imac

在node-1上安装arkade

arkade可以很方便的下载k8s软件，如kubelet，docker-register

安装ssl模块

sudo apt-get install openssl
sudo apt-get install libssl-dev

下载arkade
https://github.com/alexellis/arkade/releases
ubuntu下载arm64版本，苹果下darwin-arm64

拷贝k3s config到～目录

cp ~/.autok3s/.kube/config ~/.kube/config

arkade下载portainer 管理面板

udo arkade install portainer --kubeconfig ~/.kube/config

如果/tmp权限拒绝

    chmod -R 777 /tmp

安装成功，可以看到pod有了

转发端口让外部访问（这样试过不行）

sudo kubectl -n default port-forward svc/portainer 9000:9000

解决办法：将portainer service从ClusterIp改成NodePort，让宿主机访问

kubectl edit svc portainer

将type：改成NodePort，保存后退出

删除pod实例重启

访问30466端口，成功

k3s使用Docker作为运行时容器

k3s默认使用containerd作为运行时容器

ubuntu@node-1:~$ sudo kubectl get node -o wide
NAME     STATUS   ROLES                  AGE     VERSION        INTERNAL-IP    EXTERNAL-IP    OS-IMAGE       KERNEL-VERSION      CONTAINER-RUNTIME
node-1   Ready    control-plane,master   7m26s   v1.25.6+k3s1   192.168.64.2   192.168.64.2   Ubuntu 22.10   5.19.0-29-generic   containerd://1.6.15-k3s1

CONTAINER-RUNTIME：运行时容器

安装docker

使用官方安装脚本自动安装

curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun

在集群内部安装docker私服仓库

ubuntu@node-1:~$ sudo arkade install docker-registry --kubeconfig ~/.autok3s/.kube/config -n default
Using Kubeconfig: /home/ubuntu/.autok3s/.kube/config
Client: aarch64, Linux
2023/02/04 11:40:16 User dir established as: /root/.arkade/
"twuni" already exists with the same configuration, skipping

Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "twuni" chart repository
...Successfully got an update from the "portainer" chart repository
Update Complete. ⎈Happy Helming!⎈

Node architecture: "arm64"
Chart path:  /tmp/charts
VALUES values.yaml
Command: /root/.arkade/bin/helm [upgrade --install docker-registry twuni/docker-registry --namespace default --values /tmp/charts/docker-registry/values.yaml --set persistence.enabled=false --set secrets.htpasswd=admin:$2a$10$OCRH9nOdGyH7p2LYtSjHRuwOnGnMd/Q7zIlFuBQgEuHEObidaBnUG
]
Release "docker-registry" does not exist. Installing it now.
E0204 11:40:23.545892   12210 memcache.go:255] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0204 11:40:23.557059   12210 memcache.go:106] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
NAME: docker-registry
LAST DEPLOYED: Sat Feb  4 11:40:23 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace default -l "app=docker-registry,release=docker-registry" -o jsonpath="{.items[0].metadata.name}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl -n default port-forward $POD_NAME 8080:5000
2023/02/04 11:40:23 stderr: E0204 11:40:23.545892   12210 memcache.go:255] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0204 11:40:23.557059   12210 memcache.go:106] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request

=======================================================================
= docker-registry has been installed.                                 =
=======================================================================

# Your docker-registry has been configured

kubectl logs deploy/docker-registry

export IP="192.168.0.11" # Set to WiFI/ethernet adapter
export PASSWORD="" # See below
kubectl port-forward svc/docker-registry --address 0.0.0.0 5000 &

docker login $IP:5000 --username admin --password $PASSWORD
docker tag alpine:3.11 $IP:5000/alpine:3.11
docker push $IP:5000/alpine:3.11

# This chart is community maintained.
# Find out more at:
# https://github.com/twuni/docker-registry.helm
# https://github.com/distribution/distribution

🐳 arkade needs your support: https://github.com/sponsors/alexellis
Registry credentials: admin 1yj7k4w8Vgcy24E8666R
export PASSWORD=1yj7k4w8Vgcy24E8666R

仓库账号密码：

admin

1yj7k4w8Vgcy24E8666R

用同样的方式修改service，让外部可以访问docker-register访问

sudo kebectl edit svc docker-registry

修改type: NodePort

ubuntu@node-1:~$ sudo kubectl get svc
NAME              TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                                        AGE
kubernetes        ClusterIP   10.43.0.1     <none>        443/TCP                                        9h
portainer         NodePort    10.43.91.83   <none>        9000:30466/TCP,9443:30999/TCP,8000:32156/TCP   156m
docker-registry   NodePort    10.43.5.226   <none>        5000:32560/TCP                                 6m41s

修改k3s配置

sudo vim /etc/rancher/k3s/registries.yaml

mirrors:
  docker.io:
    endpoint:
      - "http://192.168.64.2:31929"
configs:
  "192.168.64.2:31929":
    auth:
       username: admin
       password: 1yj7k4w8Vgcy24E8666R
    tls:
      insecure_skip_verify: true

修改k3s使用docker作为运行时容器

修改ExecStart，在后面添加--docker

重启k3s服务

sudo systemctl daemon-reload
sudo service k3s restart

再次查看，发现node-1的容器运行时已经变成docker了

ubuntu@node-1:~$ sudo kubectl get node -o wide
NAME     STATUS   ROLES                  AGE   VERSION        INTERNAL-IP    EXTERNAL-IP    OS-IMAGE       KERNEL-VERSION      CONTAINER-RUNTIME
node-1   Ready    control-plane,master   20m   v1.25.6+k3s1   192.168.64.2   192.168.64.2   Ubuntu 22.10   5.19.0-29-generic   docker://23.0.0

登录docker私服

sudo docker login -u admin -p 1yj7k4w8Vgcy24E8666R 192.168.64.2:31929
```
Error response from daemon: Get "https://192.168.64.2:31929/v2/": http: server gave HTTP response to HTTPS client
```
解决：

sudo vim /lib/systemd/system/docker.service

修改ExecStart，后面加上--insecure-registry=192.168.64.2:31929

重启docker服务
```
ubuntu@node-1:~$ sudo systemctl daemon-reload
ubuntu@node-1:~$ sudo systemctl restart docker
```
登录成功

docker镜像部署k3s

写一个简单的页面来演示如何部署服务。

mkdir test
cd test

# 这是最后要验证的结果
cat >> index.html << EOF
<p>Hello from Test App.</p>
EOF

# 基于 nginx 镜像，把这个文件拷进去
cat >> Dockerfile << EOF
FROM nginx

COPY index.html /usr/share/nginx/html/
EOF

# 构建镜像
sudo docker build . -t 192.168.64.2:31929/test-app:0.1
# push 到我们的私有 registry 中
sudo docker push 192.168.64.2:31929/test-app:0.1

部署的时候，发现镜像拉取失败：

sudo kubectl apply -f app.yaml

no basic auth credentials：验证失败

Events:
  Type     Reason     Age                 From               Message
  ----     ------     ----                ----               -------
  Normal   Scheduled  15m                 default-scheduler  Successfully assigned default/test-app-5dbbdc56cd-dp9sb to node-1
  Normal   Pulling    14m (x4 over 15m)   kubelet            Pulling image "192.168.64.2:31929/test-app"
  Warning  Failed     14m (x4 over 15m)   kubelet            Failed to pull image "192.168.64.2:31929/test-app": rpc error: code = Unknown desc = Error response from daemon: Head "http://192.168.64.2:31929/v2/test-app/manifests/latest": no basic auth credentials
  Warning  Failed     14m (x4 over 15m)   kubelet            Error: ErrImagePull
  Warning  Failed     14m (x6 over 15m)   kubelet            Error: ImagePullBackOff
  Normal   BackOff    50s (x62 over 15m)  kubelet            Back-off pulling image "192.168.64.2:31929/test-app"

解决：

可以查看创建的secret：kubectl get secrets

ubuntu@node-1:~$ sudo kubectl get secrets
NAME                                    TYPE                 DATA   AGE
docker-registry-secret                  Opaque               4      139m
sh.helm.release.v1.docker-registry.v1   helm.sh/release.v1   1      139m

可以看到docker-registry-secret，将这个配置到app.yaml中

修改app.yaml：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-app
spec:
  selector:
    matchLabels:
      app: test-app
  template:
    metadata:
      labels:
        app: test-app
    spec:
      containers:
      - name: test-app
        # 使用刚刚打包的镜像
        image: 192.168.64.2:31929/test-app:0.1
        ports:
        - containerPort: 80
      imagePullSecrets:
      - name: docker-registry-secret
---
apiVersion: v1
kind: Service
metadata:
  name: test-service
spec:
  selector:
    app: test-app
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
    nodePort: 30001
  type: NodePort

加了这行配置

spec.template.spec.imagePullSecrets.name: docker-registry-secret

测试

curl -k https://test-app.imac.local

# <p>Hello from Test App.</p>

参考

使用 multipass+autok3s 快速搭建本地 k3s 集群

使用autok3s创建高可用集群失败 - K3s - Rancher 中文论坛

将ClusterIP类型改为NodePort类型的一种方式_wang725的博客-CSDN博客_clusterip改成nodeport

容器与云|如何在 Ubuntu 22.04 LTS 中安装 Docker 和 Docker Compose

轻量级Kubernetes：K3S修改docker为运行环境

docker login报错：Error response from daemon: Get "https://192.168.64.2/v2/": x509: cannot validate certificate for 192.168.64.2 because it doesn't contain any IP SANs

解决Error response from daemon: Get https://: http: server gave HTTP response to HTTPS client

K8S部署java项目，解决拉取私有镜像仓库报错no basic auth credentials_大哥你玩摇滚的博客-CSDN博客

posted @ 2023-02-02 21:55 dagger9527 阅读(1212) 评论(0) 编辑收藏举报

刷新页面返回顶部

dagger