Ingress 应用

一 、Ingress测试示例

定义一个deployment

nginx-test.yaml

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

--- apiVersion: v1 kind: Service metadata:   name: ingress-test   labels:     app: ingress-test spec:   selector:     app: ingress-test   type: ClusterIP   ports:   - name: web     port: 80     protocol: TCP --- apiVersion: apps/v1 kind: Deployment metadata:   name: ingress-test   labels:     app: ingress-test spec:   replicas: 1   selector:     matchLabels:       app: ingress-test   template:     metadata:       labels:         app: ingress-test     spec:       containers:       - name: nginx         image: nginx:1.15.2         imagePullPolicy: IfNotPresent         volumeMounts:         - name: tz-config           mountPath: /etc/localtime           readOnly: true       volumes:         - name: tz-config           hostPath:             path: /usr/share/zoneinfo/Asia/Shanghai

　　

定义一个ingress

ingress-web.yaml

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata:   name: ingress-test   annotations:     kubernetes.io/ingressClass: "nginx" spec:   rules:   - host: ingress.test.com     http:       paths:       - path: /         backend:           serviceName: ingress-test           servicePort: 80

　　

创建完成后修改host文件，将ingress.test.com指向ingress-controller所在的node节点

访问ingress.test.com

 

二、Redirect

只需要添加一个annotation，就能将访问指向重定向的网址

地址：https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#permanent-redirect

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata:   name: ingress-test   annotations:     kubernetes.io/ingressClass: "nginx"     nginx.ingress.kubernetes.io/permanent-redirect: "https://www.baidu.com" spec:   rules:   - host: ingress.test.com     http:       paths:       - path: /         backend:           serviceName: ingress-test           servicePort: 80

• nginx.ingress.kubernetes.io/permanent-redirect：301跳转

这样访问页面会直接跳转到https://www.baidu.com

查看ingress-controller配置

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

cat nginx.conf | grep "end server ingress.test.com"  -B 20             proxy_request_buffering                 on;             proxy_http_version                      1.1;                           proxy_cookie_domain                     off;             proxy_cookie_path                       off;                           # In case of errors try the next upstream server before returning an error             proxy_next_upstream                     error timeout;             proxy_next_upstream_timeout             0;             proxy_next_upstream_tries               3;                           return 301 https://www.baidu.com;                           proxy_pass http://upstream_balancer;                           proxy_redirect                          off;                       }               }     ## end server ingress.test.com

• 发现配置中有 return 301，跳转到百度去了

三、Rewrite

地址：https://kubernetes.github.io/ingress-nginx/examples/rewrite/

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata:   name: ingress-test   annotations:     kubernetes.io/ingressClass: "nginx"      nginx.ingress.kubernetes.io/rewrite-target: /$2 spec:   rules:   - host: ingress.test.com     http:       paths:       - path: /something(/|$)(.*)         backend:           serviceName: ingress-test           servicePort: 80

• 将xxx.com/something/xxx 重定向到 xxx.com/xxx

　　

 

　　该处其实还是重定向到 ingress.test.com根目录下

 

 

四、TLS/HTTPS

创建证书（自己的测试证书）

生成证书

?

1 2 3 4 5 6 7

# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.cert -subj "/CN=test-tls.test.com/O=test-tls   Generating a RSA private key .+++++ ......................................................................................+++++ writing new private key to 'tls.key' -----

创建secret

# kubectl create secret tls ca-cert --key tls.key --cert tls.cert
secret/ca-cert created

查看域名证书

?

1 2 3

# kubectl get secret NAME                  TYPE                                  DATA   AGE ca-cert               kubernetes.io/tls                     2      5s

创建ingress请求域名

ingress-tls.yaml

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata:   name: ingress-tls   annotations:     kubernetes.io/ingressClass: "nginx" spec:   rules:   - host: ingresstls.test.com     http:       paths:       - path: /         backend:           serviceName: ingress-test           servicePort: 80   tls:   - hosts:     - ingresstls.test.com     secretName: ca-cert

 

?

1	# kubectl create -f ingress-tls.yaml

　　

• 默认强制跳转到https

• nginx.ingress.kubernetes.io/ssl-redirect: "false" ： 禁用强制跳转，在annotations中添加

 

五、配置通用域名证书

　　如果要使得某个域名下的网址都使用同一个证书，可以配置一个默认的域名证书

　　创建secret

?

1 2	# kubectl create secret tls default-tls --key xxx.com.key --cert xxx.com.pem secret/default-tls created

　　修改ingress-controller

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

# kubectl get daemonset -n ingress-nginx   # kubectl edit daemonset ingress-nginx-controller  -n ingress-nginx .... - args:         - /nginx-ingress-controller         - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller         - --election-id=ingress-controller-leader         - --ingress-class=nginx         - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services         - --udp-services-configmap=$(POD_NAMESPACE)/udp-services         - --configmap=$(POD_NAMESPACE)/nginx-configuration         - --validating-webhook=:8443         - --validating-webhook-certificate=/usr/local/certificates/cert         - --validating-webhook-key=/usr/local/certificates/key         - --default-ssl-certificate=default/default-tls ....

• 新增--default-ssl-certificate=default/default-tls
• default ：namespace的名称
• default-tls：域名证书生成的secret名称

创建ingress请求域名

　ingress-default-tls.yaml

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata:   name: ingress-default-tls   annotations:     kubernetes.io/ingressClass: "nginx" spec:   rules:   - host: ingresstls.xxxx.com     http:       paths:       - path: /         backend:           serviceName: ingress-test           servicePort: 80   tls:   - hosts:     - ingresstls.xxxxx.com

 

• 不用设置证书secret，会自动使用默认的证书
• tls 整个部分都能省略

　　访问

　　

 

六、Kubernetes Dashboard使用证书

　　kubernetes dashboard会使用其自己生成的证书，但是浏览器等并不认可。可以使用机构颁发的证书替换其自带的证书

　　生成证书的secret　　

?

1	# kubectl create secret tls default-tls --key xxx.com.key --cert xxx.com.pem -n kubernetes-dashboard

　　查看

# kubectl get secret -n kubernetes-dashboard
NAME                               TYPE                                  DATA   AGE
dashboard-tls                      kubernetes.io/tls                     2      81m


# 查看生成的证书名称，在data中
# kubectl get secret dashboard-tls -n kubernetes-dashboard -oyaml

 

　　修改原本dashboard的deployment

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

# kubectl get deploy kubernetes-dashboard -n kubernetes-dashboard -o yaml ...         - args:         - --auto-generate-certificates=false         - --tls-key-file=tls.key         - --tls-cert-file=tls.crt         - --token-ttl=21600 ...         volumeMounts:         - mountPath: /certs           name: kubernetes-dashboard-certs-new   ...     volumes:       - name: kubernetes-dashboard-certs-new         secret:           defaultMode: 420           secretName: dashboard-tls

　　

• --auto-generate-certificates=false ：禁止使用dashboard自己生成的证书

• 自己的证书挂载在 /certs目录下

• 新的证书就是生成dashboard-tls这个secret　

　　最终的kubernetes-dashboard.yaml　

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

apiVersion: apps/v1 kind: Deployment metadata:   labels:     k8s-app: kubernetes-dashboard   name: kubernetes-dashboard   namespace: kubernetes-dashboard spec:   replicas: 1   selector:     matchLabels:       k8s-app: kubernetes-dashboard   strategy:     rollingUpdate:       maxSurge: 25%       maxUnavailable: 25%     type: RollingUpdate   template:     metadata:       labels:         k8s-app: kubernetes-dashboard     spec:       containers:       - args:         - --auto-generate-certificates=false         - --tls-key-file=tls.key         - --tls-cert-file=tls.crt         - --token-ttl=21600         - --namespace=kubernetes-dashboard         image: kubernetesui/dashboard:v2.0.3         imagePullPolicy: IfNotPresent         livenessProbe:           failureThreshold: 3           httpGet:             path: /             port: 8443             scheme: HTTPS           initialDelaySeconds: 30           periodSeconds: 10           successThreshold: 1           timeoutSeconds: 30         name: kubernetes-dashboard         ports:         - containerPort: 8443           protocol: TCP         resources: {}         securityContext:           allowPrivilegeEscalation: false           readOnlyRootFilesystem: true           runAsGroup: 2001           runAsUser: 1001         volumeMounts:         - mountPath: /certs           name: kubernetes-dashboard-certs-new         - mountPath: /tmp           name: tmp-volume       serviceAccount: kubernetes-dashboard       serviceAccountName: kubernetes-dashboard       volumes:       - name: kubernetes-dashboard-certs-new         secret:           defaultMode: 420           secretName: dashboard-tls       - emptyDir: {}         name: tmp-volume

　　

　　修改dashboard 的service，去掉原本的nodeport，type改为ClusterIP

?

1 2 3 4 5 6 7 8 9 10

spec:   ports:   - name: dashboard     port: 443     protocol: TCP     targetPort: 8443   selector:     k8s-app: kubernetes-dashboard   sessionAffinity: None   type: ClusterIP

　　最终的kubernetes-dashboard 的service

?

1 2	# kubectl get svc -n kubernetes-dashboard | grep kubernetes-dashboard kubernetes-dashboard        ClusterIP   10.104.46.152   <none>        443/TCP    34d

　　为dashboard配置ingress域名

　　ingress-dashboar.yaml

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata:   name: ingress-dashboard   annotations:     kubernetes.io/ingressClass: "nginx"     nginx.ingress.kubernetes.io/ssl-passthrough: "true"     nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" spec:   rules:   - host: dashboard.xxxxxx.com     http:       paths:       - path: /         backend:           serviceName: kubernetes-dashboard           servicePort: 443   tls:   - hosts:     - dashboard.xxxx.com     secretName: dashboard-tls

　　

• nginx.ingress.kubernetes.io/ssl-passthrough ：https不通过ingress解析，传给后端解析

• nginx.ingress.kubernetes.io/backend-protocol：后端使用https通信

• 如果在ingress-controller中配置过默认证书，即--default-ssl-certificate，就不用配置tls

　　创建　

?

1	# kubectl create -f ingress-dashboard.yaml -n kubernetes-dashboard

　　

　　访问: https://dashboard.xxx.com