kubernetes之十二---Ingress控制器详解
1、认识Ingress
1.1 什么是Ingress?
通常情况下,service和pod仅可在集群内部网络中通过IP地址访问。所有到达边界路由器的流量或被丢弃或被转发到其他地方。从概念上讲,可能像下面这样:
1
2
3
4
|
internet | ------------ [ Services ] |
Ingress是授权入站连接到达集群服务的规则集合。
1
2
3
4
5
|
internet | [ Ingress ] --|-----|-- [ Services ] |
可以给Ingress配置提供外部可访问的URL、负载均衡、SSL、基于名称的虚拟主机等。用户通过POST Ingress资源到API server的方式来请求ingress。 Ingress controller负责实现Ingress,通常使用负载平衡器,它还可以配置边界路由和其他前端,这有助于以HA方式处理流量。
可以将 Ingress 配置为提供服务外部可访问的 URL、负载均衡流量、终止 SSL / TLS 并提供基于名称的虚拟主机。Ingress 控制器通常负责通过负载均衡器来实现 Ingress,尽管它也可以配置边缘路由器或其他前端来帮助处理流量。
Ingress 不会公开任意端口或协议。 将 HTTP 和 HTTPS 以外的服务公开到 Internet 时,通常使用 Service.Type=NodePort 或者 Service.Type=LoadBalancer 类型的服务。
1.2 Ingress工作示意图
1.3先决条件
在使用Ingress resource之前,有必要先了解下面几件事情。Ingress是beta版本的resource,在kubernetes1.1之前还没有。需要一个Ingress Controller来实现Ingress,单纯的创建一个Ingress没有任何意义。
GCE/GKE会在master节点上部署一个ingress controller。你可以在一个pod中部署任意个自定义的ingress controller。你必须正确地annotate每个ingress,比如 运行多个ingress controller 和 关闭glbc.
确定你已经阅读了Ingress controller的beta版本限制。在非GCE/GKE的环境中,你需要在pod中 ingress-nginx。
1.4 Ingress定义资源清单几个字段
- apiVersion: v1 版本
- kind: Ingress 类型
- metadata 元数据
- spec 期望状态
- backend: 默认后端,能够处理与任何规则不匹配的请求
- rules:用于配置Ingress的主机规则列表
- tls:目前Ingress仅支持单个TLS端口443
- status 当前状态
2、基于ingress-nginx控制器实现公网访问
1、在kubernetes官网上下载ingerss-nginx控制器
(1)在githab上下载yaml文件,并创建部署
githab ingress-nginx项目:https://github.com/kubernetes/ingress-nginx
K8s官方部署文档:
ingress安装指南:https://kubernetes.github.io/ingress-nginx/deploy/
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml # 基于官网下载ingress-nginx控制器
第二种方式拉取ingress-nginx控制器,里边包含了service使用NodePort提供的端口号,可以对外提供公网地址,访问内部服务,目前还没有研究透,不通过FQ怎么下载镜像。
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.34.1/deploy/static/provider/baremetal/deploy.yaml
2、由于是需要在国外下载镜像,我们可以将ingress-nginx里的yaml文件的下载镜像路径位置改到国内即可
[root@master data]# vim mandatory.yaml # 修改下载下来的yaml文件,镜像路径指向阿里云路径 image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:0.30.0
3、执行下载的mandatory.yaml文件。
kubectl apply -f mandatory.yaml # 执行完的yaml文件默认在ingress-nginx名称空间中
4、查看创建在ingress-nginx名称空间的控制器
[root@master data]# kubectl get pods -n ingress-nginx # 查看此时的控制器已经创建完成 NAME READY STATUS RESTARTS AGE nginx-ingress-controller-fbf967dd5-r7dfk 1/1 Running 3 5h58m
2、创建一个NodePort的service文件
1、创建一个service,使用NodePort模式将端口暴露到公网上,方便客户端访问到ingress调度到后端服务器pod的网页上。
[root@master data]# cat my-svc.yaml # 创建一个负责暴露端口的service服务,使用NodePort网络模式 apiVersion: v1 kind: Service metadata: name: ingress namespace: ingress-nginx # 名称空间要与ingress-nginx一致,否则无法关联 spec: selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx type: NodePort ports: - name: http port: 80 nodePort: 30080 # 指定暴露到外网的端口号 - name: https port: 443 nodePort: 30443
2、通过my-svc.yaml文件创建一个暴露公网端口的Service
[root@master data]# kubectl apply -f my-svc.yaml service/ingress changed
3、验证,查看此时对外服务的端口号是30080
[root@master data]# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress NodePort 10.107.61.112 <none> 80:30080/TCP,443:30443/TCP 6h30m
3、准备后端pod和service
(1)编写yaml文件,并创建
创建3个nginx服务的pod副本,并创建一个service绑定
[root@master data]# cat deploy-damo.yaml apiVersion: v1 kind: Service metadata: name: myapp namespace: default spec: selector: app: myapp release: canary ports: - name: http targetPort: 80 port: 80 --- apiVersion: apps/v1 kind: Deployment metadata: name: myapp-deploy namespace: default spec: replicas: 3 selector: matchLabels: app: myapp release: canary template: metadata: labels: app: myapp release: canary spec: containers: - name: myapp image: ikubernetes/myapp:v2 ports: - name: http containerPort: 80
(2)基于deploy-damo.yaml文件创建pod和绑定service
[root@master data]# kubectl apply -f deploy-damo.yaml service/myapp unchanged deployment.apps/myapp-deploy unchanged
(3)查询验证
[root@master data]# kubectl get svc # 查看创建的服务IP地址 NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17h myapp ClusterIP 10.97.192.121 <none> 80/TCP 7h18m [root@master data]# kubectl get pods -o wide # 查看pod的IP地址 NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp-deploy-798dc9b584-97qb9 1/1 Running 3 7h18m 10.244.1.45 node1 <none> <none> myapp-deploy-798dc9b584-qctz4 1/1 Running 3 7h18m 10.244.2.47 node2 <none> <none> myapp-deploy-798dc9b584-tww7g 1/1 Running 3 7h18m 10.244.2.46 node2 <none> <none> [root@master data]# curl 10.244.1.45 # 访问service可以访问 Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a> [root@master data]# curl 10.97.192.121 # 访问pod也可以访问 Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
4、 创建ingress,绑定后端nginx服务
查看ingress支持的apiVersion版本
[root@master manifests]# kubectl explain ingress KIND: Ingress VERSION: extensions/v1beta1 # 此时版本属于此群组 DESCRIPTION: Ingress is a collection of rules that allow inbound connections to reach the endpoints defined by a backend. An Ingress can be configured to give services externally-reachable urls, load balance traffic, terminate SSL, offer name based virtual hosting etc. DEPRECATED - This group version of Ingress is deprecated by networking.k8s.io/v1beta1 Ingress. See the release notes for more information.
(1)编写yaml文件,并创建
[root@master data]# cat ingress-myapp.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: myapp namespace: default # 定义名称空间,与前面的nginx的名称空间一致 annotations: # 此项非必须,需不需要做URL的/重定向什么地方 nginx.ingress.kubernetes.io/rewrite-target: / # 此项非必须 kubernetes.io/ingress.class: "nginx" # 定义ingress类别,如果有多个标志,就需要进行区分 spec: rules: - host: www.peng.com # 不写此主机名,默认就是虚拟主机的名称,一般不写 http: paths: - path: / backend: serviceName: myapp # 与service名称一致 servicePort: 80 # 与servie的端口号一致
(2)创建并查询验证
[root@master data]# kubectl apply -f ingress-myapp.yaml # 创建Ingress ingress.extensions/ingress-myapp unchanged [root@master data]# kubectl get ingress # 查看此时的ingress信息 NAME HOSTS ADDRESS PORTS AGE ingress-myapp www.peng.com 80 7h21m [root@master ~]# kubectl get pods -n ingress-nginx # 查看pod名称 NAME READY STATUS RESTARTS AGE nginx-ingress-controller-fbf967dd5-r7dfk 1/1 Running 4 10h [root@master ~]# kubectl exec -it nginx-ingress-controller-fbf967dd5-r7dfk -n ingress-nginx -- /bin/sh # 进入到pod中查看定义nginx的配置信息 /etc/nginx $ more nginx.conf
(3)在集群外,查询服务验证
① 可以先修改一下Linux主机和windows的hosts文件,因为不是公网域名
192.168.7.101 www.peng.com
② 访问业务成功
通过Linux进行访问,实现了轮询调度到后端服务器的效果:
[root@master data]# while :;do curl http://www.peng.com:30080/hostname.html;sleep 2; done myapp-deploy-798dc9b584-97qb9 myapp-deploy-798dc9b584-97qb9 myapp-deploy-798dc9b584-qctz4 myapp-deploy-798dc9b584-tww7g
4、创建Ingress,代理到后端tomcat服务
4.1 准备后端pod和service
(1)编写yaml文件,并创建
创建3个tomcat服务的pod,并创建一个service绑定
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
|
[root@master ingress] # vim tomcat-deploy.yaml apiVersion: v1 kind: Service metadata: name: tomcat namespace: default spec: selector: app: tomcat release: canary ports: - name: http targetPort: 8080 port: 8080 - name: ajp targetPort: 8009 port: 8009 --- apiVersion: apps /v1 kind: Deployment metadata: name: tomcat-deploy namespace: default spec: replicas: 3 selector: matchLabels: app: tomcat release: canary template: metadata: labels: app: tomcat release: canary spec: containers: - name: tomcat image: tomcat:8.5.37-jre8-alpine ports: - name: http containerPort: 8080 - name: ajp containerPort: 8009 [root@master ingress] # kubectl apply -f tomcat-deploy.yaml service /tomcat created deployment.apps /tomcat-deploy created |
(2)查询验证
1
2
3
4
5
6
7
8
9
|
[root@master ~] # kubectl get pods NAME READY STATUS RESTARTS AGE tomcat-deploy-97d6458c5-hrmrw 1 /1 Running 0 1m tomcat-deploy-97d6458c5-ngxxx 1 /1 Running 0 1m tomcat-deploy-97d6458c5-xchgn 1 /1 Running 0 1m [root@master ~] # kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443 /TCP 146d tomcat ClusterIP 10.98.193.252 <none> 8080 /TCP ,8009 /TCP 1m |
4.2 创建ingress,绑定后端tomcat服务
(1)编写yaml文件,并创建
[root@master ingress]# vim ingress-tomcat.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: ingress-tomcat namespace: default annotations: # 此项非必须,需不需要做URL重定向到什么地方 nginx.ingress.kubernetes.io/rewrite-target: / # 此项非必须 kubernetes.io/ingress.class: "nginx" # 定义ingress类别,只对关键的nginx字样的服务进行调度 spec: rules: - host: net.peng.com http: paths: - path: backend: serviceName: tomcat servicePort: 8080 [root@master ingress]# kubectl apply -f ingress-tomcat.yaml ingress.extensions/ingress-tomcat created
(2)查询验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
[root@master ~] # kubectl get ingress NAME HOSTS ADDRESS PORTS AGE ingress-myapp www.peng.com 80 17m ingress-tomcat net.peng.com 80 6s [root@master ~] # kubectl describe ingress ingress-tomcat Name: ingress-tomcat Namespace: default Address: Default backend: default-http-backend:80 (<none>) Rules: Host Path Backends ---- ---- -------- net.peng.com tomcat:8080 (<none>) Annotations: kubectl.kubernetes.io /last-applied-configuration : { "apiVersion" : "extensions/v1beta1" , "kind" : "Ingress" , "metadata" :{ "annotations" :{}, "name" : "ingress-tomcat" , "namespace" : "default" }, "spec" :{ "rules" :[{ "host" : "net.peng.com" , "http" :{ "paths" :[{ "backend" :{ "serviceName" : "tomcat" , "servicePort" :8080}, "path" :null}]}}]}} Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 17s nginx-ingress-controller Ingress default /ingress-tomcat |
(3)在集群外,查询服务验证
① 可以先修改一下windows和Linux主机的hosts,因为不是公网域名
192.168.7.101 net.peng.com
② 访问业务成功
4.3 使用https协议访问服务
4.3.1 创建证书、私钥和secret
(1)创建私钥
1
2
3
4
5
6
7
|
[root@master ingress] # openssl genrsa -out tls.key 2048 Generating RSA private key, 2048 bit long modulus .............................................+++ ...............+++ e is 65537 (0x10001) [root@master ingress] # ls *key tls.key |
(2)创建证书
1
2
3
|
[root@master ingress] # openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=myapp.peng.com [root@master ingress] # ls tls.* tls.crt tls.key |
(3)创建secret
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
[root@master ingress] # kubectl create secret tls myapp-ingress-secret --cert=tls.crt --key=tls.key # 证书起名叫myapp-ingress-secret secret /tomcat-ingress-secret created [root@master ingress] # kubectl get secret NAME TYPE DATA AGE myapp-ingress-secret kubernetes.io /tls 2 8s [root@master ingress] # kubectl describe secret myapp-ingress-secret Name: myapp-ingress-secret Namespace: default Labels: <none> Annotations: <none> Type: kubernetes.io /tls Data ==== tls.key: 1675 bytes tls.crt: 1294 bytes |
4.3.2 重新创建ingress,使用https协议绑定后端tomcat服务
(1)编写yaml文件,并创建
[root@master ingress]# vim ingress-tls.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: ingress-tls namespace: default annotations: # 此项非必须,需不需要做URL重定向到什么地方 nginx.ingress.kubernetes.io/rewrite-target: / # 此项非必须 kubernetes.io/ingress.class: "nginx" # 定义ingress类别,只对关键的nginx字样的服务进行调度 spec: tls: - hosts: - myapp.peng.com # 与自签名的域名一致 secretName: myapp-ingress-secret # 与创建的secret名称一致 rules: - host: myapp.peng.com # 与自签名域名一致 http: paths: - path: backend: serviceName: myapp # 与后端servie名称一致 servicePort: 80 # 与后端端口一致
(2)查询验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
[root@master ~] # kubectl get ingress NAME HOSTS ADDRESS PORTS AGE ingress-myapp myapp.peng.com 80 34m ingress-myapp www.peng.com 80 16m ingress-tls net.peng.com 80, 443 8s [root@master ~] # kubectl describe ingress ingress-tls Name: ingress-tls Namespace: default Address: Default backend: default-http-backend:80 (<none>) TLS: tomcat-ingress-secret terminates myapp.peng.com Rules: Host Path Backends ---- ---- -------- myapp.peng.com nginx:80 (<none>) Annotations: kubectl.kubernetes.io /last-applied-configuration : { "apiVersion" : "extensions/v1beta1" , "kind" : "Ingress" , "metadata" :{ "annotations" :{}, "name" : "ingress-tls" , "namespace" : "default" }, "spec" :{ "rules" :[{ "host" : "net.peng.com" , "http" :{ "paths" :[{ "backend" :{ "serviceName" : "tomcat" , "servicePort" :80}, "path" :null}]}}], "tls" :[{ "hosts" :[ "net.peng.com" ], "secretName" :myapp -ingress-secret" }]}} Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 14s nginx-ingress-controller Ingress default /ingress-tomcat-tls |
(3)在集群外,查询服务验证
使用https协议,访问业务成功