kubernetes 安装 ingress controller
ingress-nginx
ingress 官方网站
ingress 仓库地址
ingress-nginx v1.0 最新版本 v1.0
适用于 Kubernetes 版本 v1.19+ （包括 v1.19 ）
Kubernetes-v1.22+ 需要使用 ingress-nginx>=1.0，因为 networking.k8s.io/v1beta 已经移除
直接部署 ingress-nginx
直接部署比较简单，直接拉去 girhub 的文件就可以了，如果遇到长时间无响应，可以终止任务从新拉取。
拉取镜像部分，可以修改为一下的镜像地址
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.0/deploy/static/provider/baremetal/deploy.yaml
sed -i 's@k8s.gcr.io/ingress-nginx/controller:v1.0.0\(.*\)@willdockerhub/ingress-nginx-controller:v1.0.0@' deploy.yaml
sed -i 's@k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0\(.*\)$@hzde0128/kube-webhook-certgen:v1.0@' deploy.yaml
kubectl apply -f ingress-nginx.yaml
检查安装
Completed 状态的是正常的，可以忽略。
[root@master ~]# kubectl get po -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-pm6sw 0/1 Completed 0 22m
ingress-nginx-admission-patch-m8w94 0/1 Completed 0 22m
ingress-nginx-controller-7d4df87d89-272ft 1/1 Running 0 22m
[root@master ~]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.96.88.139 <none> 80:30497/TCP,443:32581/TCP 22m
ingress-nginx-controller-admission ClusterIP 10.96.193.26 <none> 443/TCP 22m
创建应用yaml
vim tomcat.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deployment
labels:
app: tomcat
spec:
replicas: 2
selector:
matchLabels:
app: tomcat
minReadySeconds: 1
progressDeadlineSeconds: 60
revisionHistoryLimit: 2
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
template:
metadata:
labels:
app: tomcat
spec:
containers:
- name: tomcat
image: wenlongxue/tomcat:tomcat-demo-62-8fe6052
imagePullPolicy: Always
ports:
- containerPort: 8080
resources:
requests:
memory: "2Gi"
cpu: "80m"
limits:
memory: "2Gi"
cpu: "80m"
readinessProbe:
httpGet:
path: /
port: 8080
initialDelaySeconds: 180
periodSeconds: 5
timeoutSeconds: 3
successThreshold: 1
failureThreshold: 30
---
apiVersion: v1
kind: Service
metadata:
name: tomcat-service
labels:
app: tomcat
spec:
selector:
app: tomcat
ports:
- name: tomcat-port
protocol: TCP
port: 8080
targetPort: 8080
type: ClusterIP
部署 tomcat 应用
kubectl apply -f tomcat.yaml
创建 ingress yaml
vim tomcat-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: tomcat
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: tomcat.cnsre.cn
http:
paths:
- path: "/"
pathType: Prefix
backend:
service:
name: tomcat-service
port:
number: 8080
部署 tomcat ingress yaml
kubectl apply -f tomcat-ingress.yaml
查看 ingress 对应节点的端口
kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.96.88.139 <none> 80:30497/TCP,443:32581/TCP 54m
ingress-nginx-controller-admission ClusterIP 10.96.193.26 <none> 443/TCP 54m
添加 hosts
在 hosts 文件最后追加 ingress 节点的 IP 地址
54.xxx.xxx.xxx tomcat.cnsre.cn
然后在浏览器中访问 tomcat.cnsre.cn:30497。
使用 hostNetwork 的方式部署 ingress-nginx
每次部署 ingres-nginx 都随机一个 nodePort ，而使用 ingres-nginx 访问的时候也要以 域名:端口 的形式去访问如何直接使用域名去访问呢？下面介绍另外一种安装方式。
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.0/deploy/static/provider/baremetal/deploy.yaml
sed -i 's@k8s.gcr.io/ingress-nginx/controller:v1.0.0\(.*\)@willdockerhub/ingress-nginx-controller:v1.0.0@' deploy.yaml
sed -i 's@k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0\(.*\)$@hzde0128/kube-webhook-certgen:v1.0@' deploy.yaml
优化 ingress-nginx
使用 hostNetwork
默认 ingress-nginx 随机提供 nodeport 端口，开启 hostNetwork 启用80、443端口。
修改 Deployment 下面的 spec
参数如下：
...
spec:
hostNetwork: true # 新增
dnsPolicy: ClusterFirst
containers:
- name: controller
image: willdockerhub/ingress-nginx-controller:v1.0.0 # 更换镜像地址
imagePullPolicy: IfNotPresent
lifecycle:
...
修改负载均衡问题
把 kind: Deployment 改为 kind: DaemonSet 模式，这样每台 node 上都有 ingress-nginx-controller pod 副本。
参数如下：
...
# Source: ingress-nginx/templates/controller-deployment.yaml
apiVersion: apps/v1
#kind: Deployment # 注释
kind: DaemonSet # 新增
metadata:
labels:
helm.sh/chart: ingress-nginx-4.0.1
...
修改 ingressClass 问题
如果不关心 ingressClass 或者很多没有 ingressClass 配置的 ingress 对象，
添加参数 ingress-controller --watch-ingress-without-class=true 。
...
args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-dev-v1-test-controller
- --election-id=ingress-controller-leader
- --controller-class=k8s.io/ingress-nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-dev-v1-test-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
- --watch-ingress-without-class=true # 新增
...
部署检查 ingress
# 部署
kubectl apply -f ingress-nginx.yaml
# 检查 pod
[root@master ~]# kubectl get pods -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ingress-nginx-admission-create-gmnmp 0/1 Completed 0 84m 10.100.219.105 master <none> <none>
ingress-nginx-admission-patch-f5sgc 0/1 Completed 0 84m 10.100.219.106 master <none> <none>
ingress-nginx-controller-b62w7 1/1 Running 0 84m 10.0.10.51 master <none> <none>
ingress-nginx-controller-lsn7h 1/1 Running 0 84m 10.0.20.222 node1 <none> <none>
# 检查端口
[root@master ~]# netstat -pntl |grep 443
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 31248/nginx: master
[root@master ~]# netstat -pntl |grep 80
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 31248/nginx: master
创建应用yaml
vim tomcat.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deployment
labels:
app: tomcat
spec:
replicas: 2
selector:
matchLabels:
app: tomcat
minReadySeconds: 1
progressDeadlineSeconds: 60
revisionHistoryLimit: 2
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
template:
metadata:
labels:
app: tomcat
spec:
containers:
- name: tomcat
image: wenlongxue/tomcat:tomcat-demo-62-8fe6052
imagePullPolicy: Always
ports:
- containerPort: 8080
resources:
requests:
memory: "2Gi"
cpu: "80m"
limits:
memory: "2Gi"
cpu: "80m"
readinessProbe:
httpGet:
path: /
port: 8080
initialDelaySeconds: 180
periodSeconds: 5
timeoutSeconds: 3
successThreshold: 1
failureThreshold: 30
---
apiVersion: v1
kind: Service
metadata:
name: tomcat-service
labels:
app: tomcat
spec:
selector:
app: tomcat
ports:
- name: tomcat-port
protocol: TCP
port: 8080
targetPort: 8080
type: ClusterIP
部署 tomcat 应用
kubectl apply -f tomcat.yaml
创建 ingress yaml
vim tomcat-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: tomcat
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: tomcat.cnsre.cn
http:
paths:
- path: "/"
pathType: Prefix
backend:
service:
name: tomcat-service
port:
number: 8080
部署 tomcat ingress yaml
kubectl apply -f tomcat-ingress.yaml
添加 hosts
在 hosts 文件最后追加 ingress 节点的 IP 地址
54.xxx.xxx.xxx tomcat.cnsre.cn
然后在浏览器中访问 tomcat.cnsre.cn:30497。
给 ingress-nginx 配置 HTTPS 访问
创建自签证书文件
openssl req -x509 -nodes -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginx/O=nginx"
创建后会生成两个文件
ll tls.*
-rw-r--r--. 1 root root 1127 9月 2 13:04 tls.crt
-rw-r--r--. 1 root root 1708 9月 2 13:04 tls.key
创建 secret
kubectl create secret tls tls-secret --key tls.key --cert tls.crt
修改 tomcat-ingress yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: tomcat
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls: # 新增
- hosts: # 新增
- tomcat.cnsre.cn # 新增
secretName: tls-secret # 新增
rules:
- host: tomcat.cnsre.cn
http:
paths:
- path: "/"
pathType: Prefix
backend:
service:
name: tomcat-service
port:
number: 8080
修改完重新部署下
kubectl apply -f tomcat-ingress.yaml
