详解 helm 部署 ingress-nginx

使用Helm安装

参考文档：https://kubernetes.github.io/ingress-nginx/deploy/

添加ingress-nginx官方helm仓库

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

下载Chart包

#查找所有的版本
helm search repo ingress-nginx/ingress-nginx -l
#下载
helm fetch ingress-nginx/ingress-nginx --version 4.11.3
#解压缩
tar -zxvf ingress-nginx-4.11.3.tgz
cd ingress-nginx

执行下载命令时helm fetch ingress-nginx/ingress-nginx --version 4.11.3，网络不同时会报以下错误

Error: Get "https://github.com/kubernetes/ingress-nginx/releases/download/helm-chart-4.11.3/ingress-nginx-4.11.3.tgz": read tcp 10.0.2.11:47602->20.205.243.166:443: read: connection reset by peer

直接使用以下命令下载即可

wget https://github.com/kubernetes/ingress-nginx/releases/download/helm-chart-4.11.3/ingress-nginx-4.11.3.tgz

拉取镜像

官方提供的registry.k8s.io/ingress-nginx/controller无法直接拉取，需要使用替代的镜像。以下操作需要在一个节点操作下载进行后，在把进行拷贝到其他节点。

docker pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/registry.k8s.io/ingress-nginx/controller:v1.11.3
docker tag swr.cn-north-4.myhuaweicloud.com/ddn-k8s/registry.k8s.io/ingress-nginx/controller:v1.11.3 registry.k8s.io/ingress-nginx/controller:v1.11.3
docker save -o ncontroller.tar registry.k8s.io/ingress-nginx/controller:v1.11.3
scp ncontroller.tar root@10.0.2.12:/root
docker load -i ncontroller.tar 


docker pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.4
docker tag swr.cn-north-4.myhuaweicloud.com/ddn-k8s/registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.4 registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.4
docker save -o certgen.tar registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.4
scp certgen.tar root@10.0.2.12:/root
docker load -i certgen.tar 

修改values.yaml文件

修改ingress-nginx-contorller，注释掉digest

controller:
  name: controller
  enableAnnotationValidations: false
  image:
    ## Keep false as default for now!
    chroot: false
    registry: registry.k8s.io
    image: ingress-nginx/controller
    ## for backwards compatibility consider setting the full image url via the repository value below
    ## use *either* current default registry/image or repository format or installing chart by providing the values.yaml will fail
    ## repository:
    tag: "v1.11.3"
	# 修改这里，注释
    #digest: sha256:d56f135b6462cfc476447cfe564b83a45e8bb7da2774963b00d12161112270b7
    #digestChroot: sha256:22701f0fc0f2dd209ef782f4e281bfe2d8cccd50ededa00aec88e0cdbe7edd14

修改 hostNetwork 的值为 true

  # false 改成 true
  hostNetwork: true

修改 dnsPolicy 的值为 ClusterFirstWithHostNet

  # ClusterFirst 改成 
  dnsPolicy: ClusterFirstWithHostNet

nodeSelector添加标签: ingress: "true"，用于部署ingress-controller到指定节点

  nodeSelector:
    kubernetes.io/os: linux
	# 增加
    ingress: "true"

修改 kind 类型为 DaemonSet

  # -- Use a `DaemonSet` or `Deployment`
  # Deployment 改成 DaemonSet
  kind: DaemonSet

修改 kube-webhook-certgen 的镜像，注释掉digest

    patch:
      enabled: true
      image:
        registry: registry.k8s.io
        image: ingress-nginx/kube-webhook-certgen
        ## for backwards compatibility consider setting the full image url via the repository value below
        ## use *either* current default registry/image or repository format or installing chart by providing the values.yaml will fail
        ## repository:
        tag: v1.4.4
		# 修改这里，注释
        #digest: sha256:a9f03b34a3cbfbb26d103a14046ab2c5130a80c3d69d526ff8063d2b37b9fd3f

修改 service 类型为 NodePort

注意不要改成 service 对应的配置

controller:
  service:
    # LoadBalancer 改成 NodePort
    type: NodePort
    nodePorts:
      # -- Node port allocated for the external HTTP listener. If left empty, the service controller allocates one from the configured node port range.
      http: "30080"
      # -- Node port allocated for the external HTTPS listener. If left empty, the service controller allocates one from the configured node port range.
      https: "30443"

安装

# 创建命名空间
kubectl create ns ingress-nginx
# helm安装
helm install ingress-nginx -n ingress-nginx .

执行helm install ingress-nginx -n ingress-nginx .会出现以下错误

Error: INSTALLATION FAILED: template: ingress-nginx/templates/controller-role.yaml:48:9: executing "ingress-nginx/templates/controller-role.yaml" at <ne (index .Values.controller.extraArgs "update-status") "false">: error calling ne: incompatible types for comparison

修改values.yaml文件的配置参数controller.extraArgs

原

extraArgs: {}

修改后

controller:
  extraArgs:
    update-status: "false"

参考：

https://blog.csdn.net/qq_63344556/article/details/138470376
https://blog.csdn.net/qq_65380630/article/details/135620045

出现如下错误，直接把ingress-nginx的Charts删除后，解压ingress-nginx的Charts包，重新修改values.yaml；在安装即可

Error: INSTALLATION FAILED: create: failed to create: Request entity too large: limit is 3145728
helm.go:88: [debug] Request entity too large: limit is 3145728

安装完成后，需要给节点打上刚刚设置的标签ingress=true，让Pod调度到指定的节点

# 查看节点
kubectl get nodes
# 设置标签
kubectl label node k8s-worker01 ingress=true
kubectl label node k8s-worker02 ingress=true

k8s默认集群中，出于安全考虑，默认配置下 Kubernetes 不会将 Pod 调度到 Master 节点。测试环境无所谓，所以执行下面命令去除master的污点：

kubectl taint node k8s-master01 node-role.kubernetes.io/master-

执行完成之后，就可以看到 ingress-nginx 部署到了master节点了

kubectl get all -n ingress-nginx -o wide

其他配置

设置 TCP 反代

在 tcp 节点下添加对应的规则

然后更新资源

helm upgrade ingress-nginx -n ingress-nginx .

修改端口

kubectl edit ds ingress-nginx-controller -n ingress-nginx

指定http和https端口

spec:
  template:
    spec:  
      containers:
      - args:
        - /nginx-ingress-controller
        - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
        - --election-id=ingress-nginx-leader
        - --controller-class=k8s.io/ingress-nginx
        - --ingress-class=nginx
        - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
        - --validating-webhook=:8443
        - --validating-webhook-certificate=/usr/local/certificates/cert
        - --validating-webhook-key=/usr/local/certificates/key
        - --enable-metrics=false
        - --update-status=false
        - --http-port=8880
        - --https-port=8881

参考：

https://www.cnblogs.com/tangxuliang/p/16922807.html