13、K8S实战布署Nginx、Tomcat、config.toml
1、准备镜像
1.1、Nginx镜像
# 1、编写DockerFile mkdir /opt/my_nginx_dockerfile cat >/opt/my_nginx_dockerfile/Dockerfile << 'EOF' FROM nginx RUN echo "nginx v1 version" >/usr/share/nginx/html/index.html EOF cd /opt/my_nginx_dockerfile # 2、编译镜像 docker build -t 192.168.10.33:80/k8s/my_nginx:v1 . # 3、登陆镜像 docker login 192.168.10.33:80 # 4、推送至仓库 docker push 192.168.10.33:80/k8s/my_nginx:v1
1.2、Tomcat镜像
# 编写DockerFile mkdir -p /opt/my_tomcat_dockerfile cat >/opt/my_tomcat_dockerfile/Dockerfile << 'EOF' FROM tomcat:latest RUN mkdir webapps/ROOT/tomcat -p && echo "My Tomcat v1 version">webapps/ROOT/tomcat/index.html EOF cd /opt/my_tomcat_dockerfile # 编译镜像 docker build -t 192.168.10.33:80/k8s/my_tomcat:v1 . # 登陆镜像 docker login 192.168.10.33:80 # 推送至仓库 docker push 192.168.10.33:80/k8s/my_tomcat:v1
2、Nginx yaml文件
2.1、nginx.yml
cat >nginx-proxy.yml<<'EOF' --- apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: 192.168.10.33:80/k8s/my_nginx:v1 ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: nginx-web-service labels: app: nginx-web-service spec: type: NodePort selector: app: nginx ports: - protocol: TCP name: http port: 80 targetPort: 80 nodePort: 30086 EOF
2.2、应用yaml
[root@master1 deplay]# kubectl apply -f nginx-proxy.yml deployment.apps/nginx-deployment created service/nginx-web-service unchanged # 创建两个deployment、service
2.3、注意
注意: NodePort的端口值范围,必须处于 30000-32767 之间,否则就会发生报错。
3、Tomcat yaml文件
3.1、tomcat.yaml
cat >tomcat-proxy.yml<<'EOF' apiVersion: apps/v1 kind: Deployment metadata: name: tomcat-deployment labels: app: tomcat spec: replicas: 1 selector: matchLabels: app: tomcat template: metadata: labels: app: tomcat spec: containers: - name: tomcat image: 192.168.10.33:80/k8s/my_tomcat:v1 ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: tomcat-web-service labels: app: tomcat-web-service spec: type: NodePort selector: app: tomcat ports: - protocol: TCP name: http port: 80 targetPort: 8080 nodePort: 30087 EOF
2.3、应用yaml
kubectl apply -f tomcat-proxy.yml
4、查询布署的结果
4.1、查询service
[root@master1 deplay]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 18h nginx-web-service NodePort 10.106.76.157 <none> 80:30086/TCP 3h5m tomcat-web-service NodePort 10.98.112.76 <none> 80:30087/TCP 3m14s
4.2、查询pods
[root@master1 deplay]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-deployment-5b47ccdd5c-b9pnc 1/1 Running 0 3h6m tomcat-deployment-774f9fdf7d-kw9x4 1/1 Running 0 3m40s
4.3、测试访问
# 集群的 IP 地址 [root@master1 deplay]# curl 10.106.76.157 nginx v1版本 [root@master1 deplay]# curl 10.98.112.76 My Tomcat v1 version # Node IP 地址 [root@master1 deplay]# curl 192.168.10.29:30086 nginx v1版本 [root@master1 deplay]# curl 192.168.10.29:30087 My Tomcat v1 version # Pods IP 地址 [root@master1 deplay]# curl 10.244.3.25 nginx v1版本 [root@master1 deplay]# curl 10.244.3.26:8080 My Tomcat v1 version
5、错误排查
5.1、拉取镜像失败-config.toml
5.1.1、手动拉取镜试试
[root@node1 ~]# crictl -D pull 192.168.10.33:80/k8s/my_nginx:v1 &PullImageRequest{Image:&ImageSpec{Image:192.168.10.33:80/k8s/my_nginx:v1,Annotations:map[string]string{},},Auth:nil,SandboxConfig:nil,} E0317 12:16:25.853996 47848 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to
pull and unpack image \"192.168.10.33:80/k8s/my_nginx:v1\": failed to resolve reference \"192.168.10.33:80/k8s/my_nginx:v1\": failed to do
request: Head \"https://192.168.10.33:80/v2/k8s/my_nginx/manifests/v1\": http: server gave HTTP response to HTTPS client" image="192.168.10.33:80/k8s/my_nginx:v1" FATA[0000] pulling image: rpc error: code = Unknown desc = failed to pull and unpack image "192.168.10.33:80/k8s/my_nginx:v1":
failed to resolve reference "192.168.10.33:80/k8s/my_nginx:v1": failed to do request: Head "https://192.168.10.33:80/v2/k8s/my_nginx/manifests/v1":
http: server gave HTTP response to HTTPS client
5.1.2、原因
K8S新版本经过containerd 调用 dokcer去拉镜像,所以containerd需要配置
5.1.3、解决方法
[root@ ~]# vi /etc/containerd/config.toml [plugins."io.containerd.grpc.v1.cri".registry.mirrors] [plugins."io.containerd.grpc.v1.cri".registry.mirrors."192.168.10.33:80"] endpoint = ["http://192.168.10.33:80"] [root@ ~]# systemctl restart containerd
6、项目改造需求
6.1、需求
1、nginx需要实现反向代理的功能 2、tomcat应用不对外暴露端口
6.2、改造的必须知识点
6.2.1、查看service
[root@master2 ~]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 18h nginx-web-service NodePort 10.106.76.157 <none> 80:30086/TCP 3h19m tomcat-web-service NodePort 10.98.112.76 <none> 80:30087/TCP 17m
6.2.2、查看pod
[root@master1 deplay]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-deployment-5b47ccdd5c-b9pnc 1/1 Running 0 3h16m tomcat-deployment-774f9fdf7d-kw9x4 1/1 Running 0 13m
6.2.3、进入nginx pod里面安装调试所需的工具
[root@master1 deplay]# kubectl exec -it nginx-deployment-5b47ccdd5c-b9pnc bash root@nginx-deployment-5b47ccdd5c-b9pnc:/# apt update root@nginx-deployment-5b47ccdd5c-b9pnc:/# apt install vim net-tools iputils-ping dnsutils curl -
6.2.4、nginx pods DNS查询service名字是否解析正常
root@nginx-deployment-5b47ccdd5c-b9pnc:/# nslookup tomcat-web-service Server: 10.96.0.10 Address: 10.96.0.10#53 Name: tomcat-web-service.default.svc.cluster.local Address: 10.98.112.76 # service名字的IP地址
6.2.5、使用服务名做为域名访问
root@nginx-deployment-5b47ccdd5c-b9pnc:/# curl tomcat-web-service
My Tomcat v1 version
6.2.6、总结
在pod内部,可以基于服务的service的名称来进行通信,因为在k8s集群内部有coredns来进行域名的解析功能
6.3、Nginx重新打包
6.3.1、 default.conf
cat >default.conf<<'EOF' server { listen 80; listen [::]:80; server_name localhost; #access_log /var/log/nginx/host.access.log main; location / { root /usr/share/nginx/html; index index.html index.htm; } location /tomcat/ { proxy_pass http://tomcat-web-service; } #error_page 404 /404.html; # redirect server error pages to the static page /50x.html # error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html; } # proxy the PHP scripts to Apache listening on 127.0.0.1:80 # #location ~ \.php$ { # proxy_pass http://127.0.0.1; #} # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000 # #location ~ \.php$ { # root html; # fastcgi_pass 127.0.0.1:9000; # fastcgi_index index.php; # fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name; # include fastcgi_params; #} # deny access to .htaccess files, if Apache's document root # concurs with nginx's one # #location ~ /\.ht { # deny all; #} } EOF
6.3.2、编译镜像并且上传
docker build -t 192.168.10.33:80/k8s/my_nginx:v2 . docker push 192.168.10.33:80/k8s/my_nginx:v2
6.3.3、修改yaml配置文件重新应用
vi nginx-proxy ... - name: nginx image: 192.168.10.33:80/k8s/my_nginx:v2 ports: - containerPort: 80 ... kubectl apply -f nginx-proxy.yml
6.3.4、测试访问
[root@master1 deplay]# curl http://192.168.10.30:30086/tomcat/ My Tomcat v1 version