Ingress
Ingress概念
Ingress为Kubernetes集群中的服务提供了入口，可以提供负载均衡、SSL终止和基于名称的虚拟主机，在生产环境中常用的Ingress有Treafik、Nginx、HAProxy、Istio等，几种常用的ingress功能对比和选型可以参考这里。
Ingress原理
Ingress 工作原理
- ingress-controller通过和 kubernetes APIServer 交互，动态的去感知集群中ingress规则变化，
- 然后读取它，按照自定义的规则，规则就是写明了哪个域名对应哪个service，生成一段nginx配置，
- 再写到nginx-ingress-controller的pod里，这个ingress-controller的pod里运行着一个Nginx服务，控制器会把生成的 nginx配置写入 /etc/nginx.conf文件中，
- 然后reload一下使配置生效。以此达到域名区分配置和动态更新的作用。
在使用普通的Service时，集群中每个节点的kube-proxy在监听到Service和Endpoints的变化时，会动态的修改相关的iptables的转发规则。 客户端在访问时通过iptables设置的规则进行路由转发达到访问服务的目的。
而Ingress则跳过了kube-proxy这一层，通过Ingress Controller中的代理配置进行路由转发达到访问目标服务的目的。
部署使用Ingress
官网下载地址：https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.25.0/deploy/static/mandatory.yaml
国内gitee下载地址：https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml
修改配置
[root@master ingress]# grep -n5 nodeSelector mandatory.yaml
213- spec:
214- hostNetwork: true #添加为host模式
215- # wait up to five minutes for the drain of connections
216- terminationGracePeriodSeconds: 300
217- serviceAccountName: nginx-ingress-serviceaccount
218: nodeSelector:
219- ingress: "true" #替换此处，来决定将ingress部署在哪些机器
220- containers:
221- - name: nginx-ingress-controller
222- image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
223- args:
创建ingress
#给node1节点添加label
[root@master ingress]# kubectl label nodes node1 ingress=true
node/node1 labeled
[root@master ingress]# kubectl get nodes --show-labels
NAME STATUS ROLES AGE VERSION LABELS
master Ready control-plane,master 41d v1.20.0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=master,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=
node1 Ready <none> 41d v1.20.0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=node1,kubernetes.io/os=linux
node2 Ready <none> 41d v1.20.0 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=node2,kubernetes.io/os=linux
[root@master ingress]# kubectl create -f mandatory.yaml
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRole
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
Warning: rbac.authorization.k8s.io/v1beta1 Role is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 Role
role.rbac.authorization.k8s.io/nginx-ingress-role created
Warning: rbac.authorization.k8s.io/v1beta1 RoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 RoleBinding
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRoleBinding
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
limitrange/ingress-nginx created
[root@master ingress]# kubectl -n ingress-nginx get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ingress-controller-66bff489bb-zvqr8 1/1 Running 0 3m23s 10.0.0.81 node1 <none> <none>
[root@master ingress]# kubectl -n ingress-nginx get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-ingress-controller 1/1 1 1 4m7s
[root@master ingress]# kubectl -n ingress-nginx get cm
NAME DATA AGE
ingress-controller-leader-nginx 0 3m11s
kube-root-ca.crt 1 4m15s
nginx-configuration 0 4m15s
tcp-services 0 4m15s
udp-services 0 4m15s
在node1查看
[root@node1 ~]# netstat -lntp|grep nginx
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 22644/nginx: master
tcp 0 0 0.0.0.0:8181 0.0.0.0:* LISTEN 22644/nginx: master
tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 22644/nginx: master
tcp 0 0 127.0.0.1:10245 0.0.0.0:* LISTEN 22604/nginx-ingress
tcp 0 0 127.0.0.1:10246 0.0.0.0:* LISTEN 22644/nginx: master
tcp 0 0 127.0.0.1:10247 0.0.0.0:* LISTEN 22644/nginx: master
tcp6 0 0 :::10254 :::* LISTEN 22604/nginx-ingress
tcp6 0 0 :::80 :::* LISTEN 22644/nginx: master
tcp6 0 0 :::8181 :::* LISTEN 22644/nginx: master
tcp6 0 0 :::443 :::* LISTEN 22644/nginx: master
由于配置了 hostnetwork，nginx 已经在 node 主机本地监听 80/443/8181 端口。其中 8181 是 nginx-controller 默认配置的一个 default backend（Ingress 资源没有匹配的 rule 对象时，流量就会被导向这个 default backend）。
这样，只要访问 node 主机有公网 IP，就可以直接映射域名来对外网暴露服务了。如果要 nginx 高可用的话，可以在多个 node上部署，并在前面再搭建一套 LVS+keepalived 做负载均衡
演示
创建deployment和server
[root@master ingress]# cat service-nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-app
spec:
replicas: 2
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-app-svc
spec:
type: ClusterIP
ports:
- protocol: TCP
port: 80
targetPort:
selector:
app: nginx
[root@master ingress]# kubectl create -f service-nginx.yaml
deployment.apps/nginx-app created
service/nginx-app-svc created
[root@master ingress]# kubectl get deploy nginx-app -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx-app 2/2 2 2 69s nginx nginx app=nginx
[root@master ingress]# kubectl get svc nginx-app-svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
nginx-app-svc ClusterIP 10.97.44.108 <none> 80/TCP 79s app=nginx
[root@master ingress]# kubectl describe svc nginx-app-svc
Name: nginx-app-svc
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=nginx
Type: ClusterIP
IP Families: <none>
IP: 10.97.44.108
IPs: 10.97.44.108
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints: 10.244.1.81:80,10.244.1.82:80
Session Affinity: None
Events: <none>
创建ingress的方法
方法一：（extensions/v1beta1 Ingress 在1.22版本即将弃用）
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-app-ingress
spec:
rules:
- host: www.test.com
http:
paths:
- path: /
backend:
serviceName: nginx-app-svc
servicePort: 80
方法二：
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-app-ingress
spec:
rules:
- host: www.test.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-app-svc
port:
number: 80
创建ingress
[root@master ingress]# cat ingress-app.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-app-ingress
spec:
rules:
- host: www.test.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-app-svc
port:
number: 80
[root@master ingress]# kubectl create -f ingress-app.yaml
ingress.networking.k8s.io/nginx-app-ingress created
[root@master ingress]# kubectl get ingress nginx-app-ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-app-ingress <none> www.test.com 80 73s
[root@master ingress]# kubectl describe ingress nginx-app-ingress
Name: nginx-app-ingress
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
www.test.com
/ nginx-app-svc:80 (10.244.1.81:80,10.244.1.82:80)
Annotations: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 2m38s nginx-ingress-controller Ingress default/nginx-app-ingress
ingress支持的path类型：
- ImplementationSpecific：对于这种path类型，匹配取决于IngressClass。可以将其视为一个单独的pathType或者将其认为和Prefix或者Exact路径类型一样。
- Exact：精确匹配URL路径，并且区分大小写
- Prefix: 根据URL中的，被/分割的前缀进行匹配。匹配区分大小写并且按照元素对路径进行匹配。path元素指的是路径中由/分隔符分隔的标签列表。
注意：如果路径的最后一个元素是请求路径中最后一个元素的子字符串，那么这个是不匹配的。【举例：/foo/bar匹配/foo/bar/baz，但是不匹配/foo/barbaz】参考
测试
添加hosts
测试访问
查看ingress-nginx动态生成upstream配置
[root@master ingress]# kubectl -n ingress-nginx get pod
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-66bff489bb-zvqr8 1/1 Running 0 37m
[root@master ingress]# kubectl -n ingress-nginx exec -it nginx-ingress-controller-66bff489bb-zvqr8 -- /bin/bash
bash-5.0$ vi /etc/nginx/nginx.conf
...
## start server www.test.com
server {
server_name www.test.com ;
listen 80 ;
listen [::]:80 ;
listen 443 ssl http2 ;
listen [::]:443 ssl http2 ;
set $proxy_upstream_name "-";
ssl_certificate_by_lua_block {
certificate.call()
}
location / {
set $namespace "default";
set $ingress_name "nginx-app-ingress";
set $service_name "nginx-app-svc";
set $service_port "80";
set $location_path "/";
...
