Kubernetes 部署 Nginx Ingress Controller 之 nginxinc/kubernetes-ingress
更新：这里用的是 nginxinc/kubernetes-ingress ，还有个 kubernetes/ingress-nginx ，它们的区别见 Differences Between nginxinc/kubernetes-ingress and kubernetes/ingress-nginx Ingress Controllers ，后来我们选用了 kubernetes/ingress-nginx ，详见博文。
开始天真地以为只要写一个 ingress 配置文件并部署好就行了。
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cnblogs-ingress
spec:
rules:
- host: q.cnblogs.com
http:
paths:
- backend:
serviceName: q-web
servicePort: 80
# kubectl apply -f cnblogs-ingress.yaml
# kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
cnblogs-ingress q.cnblogs.com 80 6h18
但部署后发现所有 node 服务器上没有任何进程监听 80 端口，显然不对。
从 k8s 帮助文档中知道了答案：
You must have an ingress controller to satisfy an Ingress. Only creating an Ingress resource has no effect.
In order for the Ingress resource to work, the cluster must have an ingress controller running.
Unlike other types of controllers which run as part of the kube-controller-manager binary, Ingress controllers are not started automatically with a cluster. Use this page to choose the ingress controller implementation that best fits your cluster.
原来 k8s 没有内置 ingress controller ，需要安装第三方的 ingress controller ，比如 nginx ingress controller ，上面通过 cnblogs-ingress.yaml 只是创建了 ingress 资源。那为什么通过 deployment.yaml 创建了 deployment 资源就能正常部署 pod ？那是因为 kube-controller-manager 中内置了 deployment controller 。
我们选用 nginx ingress controller ，部署操作步骤如下（参考文档）：
1）从 github 上签出 kubernetes-ingress 仓库
$ git clone https://github.com/nginxinc/kubernetes-ingress/
$ cd kubernetes-ingress
$ git checkout v1.6.1 -f
$ cd deployments
2）创建 namespace 与 ServiceAccount ，都叫 nginx-ingress
kubectl apply -f common/ns-and-sa.yaml
3）创建 cluster role 与 cluster role binding
kubectl apply -f rbac/rbac.yaml
4）创建 secret
使用自己的证书文件创建 secret
kubectl create secret tls default-server-secret --cert=path/to/cert.pem --key=path/to/key.pem
或者使用 nginx-ingress 自带的证书创建 sescret
kubectl apply -f common/default-server-secret.yaml
5）创建 ConfigMap
kubectl apply -f common/nginx-config.yaml
6）创建 custom resource definitions
kubectl apply -f common/custom-resource-definitions.yaml
7）创建 DaemonSet
kubectl apply -f daemon-set/nginx-ingress.yaml
8）查看 pod 是否部署成功
$ kubectl get pods --namespace=nginx-ingress 1 ↵
NAME READY STATUS RESTARTS AGE
nginx-ingress-7xdzp 1/1 Running 5 12m
nginx-ingress-rs4th 1/1 Running 0 114s
nginx-ingress-w2fnh 1/1 Running 0 12m
nginx-ingress-z54r6 1/1 Running 5 12m
9）创建监听 31080 端口的 NodePort 类型的 service
配置文件 nodeport.yaml （去掉了443端口）
注0：nodePort 只能使用 30000-32767 范围的端口。
注1：去掉了443端口，我们在最前端使用了阿里云负载均衡，请求都通过 http 转发。
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress
namespace: nginx-ingress
spec:
type: NodePort
ports:
- nodePort: 31080
port: 80
targetPort: 80
protocol: TCP
name: http
selector:
app: nginx-ingress
部署命令
kubectl apply -f service/nodeport.yaml
10）检查 nginx-ingress 部署成功
进入 nginx-ingress 容器
kubectl exec -it daemonset/nginx-ingress -n nginx-ingress /bin/bash
查看 nginx 配置
cat /etc/nginx/conf.d/production-cnblogs-ingress.conf
确认 ingress 中添加的转发规则已被导入
upstream production-cnblogs-ingress-q.cnblogs.com-q-web-80 {
zone production-cnblogs-ingress-q.cnblogs.com-q-web-80 256k;
random two least_conn;
server 192.168.107.211:80 max_fails=1 fail_timeout=10s max_conns=0;
server 192.168.186.72:80 max_fails=1 fail_timeout=10s max_conns=0;
}
server {
listen 80;
server_tokens on;
server_name q.cnblogs.com;
location / {
proxy_http_version 1.1;
proxy_connect_timeout 60s;
proxy_read_timeout 60s;
proxy_send_timeout 60s;
client_max_body_size 1m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering on;
proxy_pass http://production-cnblogs-ingress-q.cnblogs.com-q-web-80;
}
}
至此 nginx-ingress 部署成功。
解决转发 X-Forwarded-Proto 请求头问题
解决方法：在 ingress 配置文件中添加 nginx.org/redirect-to-https: "true"，详见博问 K8s Nginx Ingress Controller 转发 X-Forwarded-Proto 请求头的问题
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cnblogs-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.org/redirect-to-https: "true"
添加 proxy_set_header 配置
通过 ingress 的 nginx.org/location-snippets 注解添加如下的配置：
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: cnblogs-ingress
annotations:
nginx.org/location-snippets: |
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
七层负载均衡对 nginx-ingress 进行健康检查问题
详见博问 阿里云负载均衡对 K8s Nginx Ingress 的健康检查问题
基于二级域名自动转发到 service 的实现方法
详见博问 K8s Ingress 如何自动根据主机名中的二级域名匹配 service
