Kubernetes进阶实战读书笔记：Ingress和Ingress-controller

一、如何实现https负载均衡 Ingress和Ingress-controller

1、存在问题

kubernetes中、service资源和pod资源的IP地址仅能用于集群网络内部的通信

所有的网络流量都无法穿透边界路由器以实现集群内外通信、尽管可以为service使用NodePortand LoadBalancer类型通过节点引入外部流量

但它依然是4层部分流量转发、可用的负载均衡器也为传输层负载均衡机制

2、解决方案及工作原理

Ingress是kubernetes API的标准资源类型之一、它其实就是一组基于DNS名称或URL路径把请求转发至指定的service资源的规则

用于将集群外部的请求流量转发至集群内部完成服务发布、然而Ingress资源自身并不能进行"流量穿透"、它仅是一组路由规则的集合

这些规则要想真正发挥作用还需要其他功能的辅助、如监听套接字、然后根据这些规则的匹配机制路由请求流量、这种为够为Ingress资源监听套接字并转发流量的组件成为Ingress控制器

Ingress控制器并不直接运行为kube-controller-manager的一部分、它是kubernetes集群的一个重要附件、类似于CoreDNS、需要在集群上单独部署

3、Ingress控制器的实现

Nginx、Envoy、HAProxy、Vulcand和Traefik等

Ingress控制器自身也是运行于集群中的pod资源对象、它于被代理的运行为pod资源的应用运行于同一网络中

4、优点和优势

使用Ingress资源进行流量分发是、Ingress控制器可基于某Ingress资源定义的规则将客户端请求流量直接转发至于service对应的后端pod资源上

这种转发机制会绕过service资源、从而省去了由kube-proxy实现的端口代理开销

Ingress规则需要由一个service资源对象辅助识别相关的所有pod对象、但Ingress-nginx控制器可经由api.ilinux.io规则的定义直接将请求流量

调度至pod3或pod4、而无需经由service对象API的再次转发、WAP相关规则的作用方式与此类同

 

二、安装ingress-nginx

1、Ingress.spec核心资源

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

[root@master chapter6]# kubectl explain Ingress.spec KIND:     Ingress VERSION:  extensions/v1beta1   RESOURCE: spec <Object>   DESCRIPTION:      Spec is the desired state of the Ingress. More info:      https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status        IngressSpec describes the Ingress the user wishes to exist.   FIELDS:    backend  <Object>      #默认的后端用于服务那些没有匹配到任何规则的请求；定义Ingress资源时、至少应该定义backend或rules两者之一；此字段用于让负责均衡器指定一个全局默认的后端      A default backend capable of servicing requests that don't match any rule.      At least one of 'backend' or 'rules' must be specified. This field is      optional to allow the loadbalancer controller or defaulting logic to      specify a global default.             #backend对象的定义由两个必选的内嵌字段组成：serviceName和servicePort、分别用于指定流量转发的后端目标serbice资源的名称和端口      ingressClassName <string>      IngressClassName is the name of the IngressClass cluster resource. The      associated IngressClass defines which controller will implement the      resource. This replaces the deprecated `kubernetes.io/ingress.class`      annotation. For backwards compatibility, when that annotation is set, it      must be given precedence over this field. The controller may emit a warning      if the field and annotation have different values. Implementations of this      API should ignore Ingresses without a class specified. An IngressClass      resource may be marked as default, which can be used to set a default value      for this field. For more information, refer to the IngressClass      documentation.      rules    <[]Object>        #用于定义当前Ingress资源的转发规则列表；未由rules定义规则、或者没有匹配到任何规则时、所有流量都会转发到由backend定义的默认后端      A list of host rules used to configure the Ingress. If unspecified, or no      rule matches, all traffic is sent to the default backend.      #对象由一系列配置Ingress资源的host规则组成、这些host规则用于将一个主机上的某个URL路径映射至相关的后端service对象    tls  <[]Object>    #TLS配置、目前仅支持通过默认端口443提供服务；如果要配置指定的列表成员指向了不同的主机、则必须通过SNI TLS扩展机制来支持      TLS configuration. Currently the Ingress only supports a single TLS port,      443. If multiple members of this list specify different hosts, they will be      multiplexed on the same port according to the hostname specified through      the SNI TLS extension, if the ingress controller fulfilling the ingress      supports SNI.

2、Ingress.spec.rules对象

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

[root@master chapter6]# kubectl explain Ingress.spec.rules KIND:     Ingress VERSION:  extensions/v1beta1   RESOURCE: rules <[]Object>   DESCRIPTION:      A list of host rules used to configure the Ingress. If unspecified, or no      rule matches, all traffic is sent to the default backend.        IngressRule represents the rules mapping the paths under a specified host      to the related backend services. Incoming requests are first evaluated for      a host match, then routed to the backend associated with the matching      IngressRuleValue.   FIELDS:    host <string>    #属性值目前不支持使用IP地址、也不支持后跟"PORT" 格式的端口号、且此字段值留空表示统配所有的主机名      Host is the fully qualified domain name of a network host, as defined by      RFC 3986. Note the following deviations from the "host" part of the URI as      defined in RFC 3986: 1. IPs are not allowed. Currently an IngressRuleValue      can only apply to the IP in the Spec of the parent Ingress. 2. The `:`      delimiter is not respected because ports are not allowed. Currently the      port of an Ingress is implicitly :80 for http and :443 for https. Both      these may change in the future. Incoming requests are matched against the      host before the IngressRuleValue. If the host is unspecified, the Ingress      routes all traffic based on the specified IngressRuleValue. Host can be      "precise" which is a domain name without the terminating dot of a network      host (e.g. "foo.bar.com") or "wildcard", which is a domain name prefixed      with a single wildcard label (e.g. "*.foo.com"). The wildcard character '*'      must appear by itself as the first DNS label and matches only a single      label. You cannot have a wildcard label by itself (e.g. Host == "*").      Requests will be matched against the Host field in the following way: 1. If      Host is precise, the request matches this rule if the http host header is      equal to Host. 2. If Host is a wildcard, then the request matches this rule      if the http host header is to equal to the suffix (removing the first      label) of the wildcard rule.      http <Object>

3、Ingress.spec.tls对象

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

[root@master data]# kubectl explain Ingress.spec.tls KIND:     Ingress VERSION:  extensions/v1beta1   RESOURCE: tls <[]Object>   DESCRIPTION:      TLS configuration. Currently the Ingress only supports a single TLS port,      443. If multiple members of this list specify different hosts, they will be      multiplexed on the same port according to the hostname specified through      the SNI TLS extension, if the ingress controller fulfilling the ingress      supports SNI.        IngressTLS describes the transport layer security associated with an      Ingress.   FIELDS:    hosts    <[]string>     #包含于使用的TLS证书之内的主机名称字符串列表、因此、此处使用的主机名必须匹配tlsSecret中的名称      Hosts are a list of hosts included in the TLS certificate. The values in      this list must match the name/s used in the tlsSecret. Defaults to the      wildcard host setting for the loadbalancer controller fulfilling this      Ingress, if left unspecified.      secretName   <string>    #用于引用SSL会话的secret对象名称、在基于SNI实现多主机路由的场景中、此字段为可选      SecretName is the name of the secret used to terminate SSL traffic on 443.      Field is left optional to allow SSL routing based on SNI hostname alone. If      the SNI host in a listener conflicts with the "Host" header field used by      an IngressRule, the SNI host is used for termination and value of the Host      header is used for routing.

三、Ingress资源类型

1、单service资源型Ingress

暴露单个服务的方法有很多种、如服务类型中的NodePort and LoadBalancer、不过一样可以考虑使用IngressL来暴露服务、此时只需要为Ingress指定
"default-backend" 即可、例如下面的示例

Ingress控制器会为其分配一个IP地址接入请求流量、并将它们转至示例中的my-svc后端

?

1 2 3 4 5 6 7 8

apiVersion: v1 kind: Ingress metadata:   name: my-ingress spec:        backend:      serviceName: test      servicePort: 80

2、基于URL路径进行流量分发

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata:   name: simple-fanout-example   annotations:     nginx.ingress.kubernetes.io/rewrite-target: / spec:   rules:   - host: foo.bar.com     http:       paths:       - path: /foo         backend:           serviceName: service1           servicePort: 4200       - path: /bar         backend:           serviceName: service2           servicePort: 8080

3、基于主机名称的虚拟机主机

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata:   name: name-virtual-host-ingress spec:   rules:   - host: foo.bar.com     http:       paths:       - backend:           serviceName: service1           servicePort: 80   - host: bar.foo.com     http:       paths:       - backend:           serviceName: service2           servicePort: 80

4、TLS类型的Ingress资源

?

1 2 3 4 5 6 7 8

apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata:   name: test-ingress spec:   backend:     serviceName: testsvc     servicePort: 80

四、部署Ingress控制器(Nginx)

部署Ingress-Nginx控制器的配置文件被切割存放在了多个不同的文件中、并集中存储于源码deploy子目录下、同时为了方便用户部署、它还将所有的资源全部集成为一个配置文件mandatory.yaml

?

1 2	wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/deploy.yaml 修改331行镜像为：image: siriuszg/nginx-ingress-controller:latest

1、创建运行

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

[root@master data]# kubectl apply -f deploy.yaml namespace/ingress-nginx unchanged serviceaccount/ingress-nginx unchanged configmap/ingress-nginx-controller configured clusterrole.rbac.authorization.k8s.io/ingress-nginx unchanged clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx unchanged role.rbac.authorization.k8s.io/ingress-nginx unchanged rolebinding.rbac.authorization.k8s.io/ingress-nginx unchanged service/ingress-nginx-controller-admission unchanged service/ingress-nginx-controller unchanged deployment.apps/ingress-nginx-controller configured validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission configured serviceaccount/ingress-nginx-admission unchanged clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission unchanged clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission unchanged role.rbac.authorization.k8s.io/ingress-nginx-admission unchanged rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission unchanged job.batch/ingress-nginx-admission-create unchanged job.batch/ingress-nginx-admission-patch unchanged

2、验证

?

1 2 3 4 5

[root@master chapter6]# kubectl get pods -n ingress-nginx -o wide NAME                                        READY   STATUS      RESTARTS   AGE   IP             NODE     NOMINATED NODE   READINESS GATES ingress-nginx-admission-create-6nwwj        0/1     Completed   0          15m   10.244.2.103   nodes2   <none>           <none> ingress-nginx-admission-patch-qndv7         0/1     Completed   3          15m   10.244.0.91    master   <none>           <none> ingress-nginx-controller-5d9498494d-97blq   1/1     Running     0          38s   10.244.0.92    master   <none>

在线的配置清单中采用了基于Deployment控制器部署ingress nginx的方式、因此接入外部流量之前还要手动为其创建相关的NodePort and LoadBalancer
类型的service资源对象、下面的配置清单示例中对类型定义了NodePort、并明确指定了易记的端口和IP地址

3、service资源清单

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

[root@master chapter6]# cat nginx-ingress-service.yaml apiVersion: v1 kind: Service metadata:   name: nginx-ingress-controller   namespace: ingress-nginx spec:        type: NodePort   clusterIP: 10.99.99.99   ports:     - port: 80       name: http       nodePort: 30080     - port: 443       name: https       nodePort: 30443   selector:     app.kubernetes.io/name: ingress-nginx

4、创建service资源

?

1 2	[root@master chapter6]# kubectl apply -f nginx-ingress-service.yaml service/nginx-ingress-controller created

5、验证

?

1 2 3 4 5

[root@master chapter6]# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx-controller NodePort 10.96.209.11 <none> 80:30724/TCP,443:32624/TCP 30m ingress-nginx-controller-admission ClusterIP 10.96.105.3 <none> 443/TCP 30m nginx-ingress-controller NodePort 10.99.99.99 <none> 80:30080/TCP,443:30443/TCP 13m

如果读者的集群运行支持LBaaS的IaaS云环境、则可以将其类型指定为LoadBalancer这样直接就有了可用的external-LB

6、测试代码和截图

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203

[root@master data]# curl tomcat.ikubernetes.io:30080       <!DOCTYPE html> <html lang="en">     <head>         <meta charset="UTF-8" />         <title>Apache Tomcat/8.0.50</title>         <link href="favicon.ico" rel="icon" type="image/x-icon" />         <link href="favicon.ico" rel="shortcut icon" type="image/x-icon" />         <link href="tomcat.css" rel="stylesheet" type="text/css" />     </head>       <body>         <div id="wrapper">             <div id="navigation" class="curved container">                 <span id="nav-home"><a href="http://tomcat.apache.org/">Home</a></span>                 <span id="nav-hosts"><a href="/docs/">Documentation</a></span>                 <span id="nav-config"><a href="/docs/config/">Configuration</a></span>                 <span id="nav-examples"><a href="/examples/">Examples</a></span>                 <span id="nav-wiki"><a href="http://wiki.apache.org/tomcat/FrontPage">Wiki</a></span>                 <span id="nav-lists"><a href="http://tomcat.apache.org/lists.html">Mailing Lists</a></span>                 <span id="nav-help"><a href="http://tomcat.apache.org/findhelp.html">Find Help</a></span>                 <br class="separator" />             </div>             <div id="asf-box">                 <h1>Apache Tomcat/8.0.50</h1>             </div>             <div id="upper" class="curved container">                 <div id="congrats" class="curved container">                     <h2>If you're seeing this, you've successfully installed Tomcat. Congratulations!</h2>                 </div>                 <div id="notice">                     <img src="tomcat.png" alt="[tomcat logo]" />                     <div id="tasks">                         <h3>Recommended Reading:</h3>                         <h4><a href="/docs/security-howto.html">Security Considerations HOW-TO</a></h4>                         <h4><a href="/docs/manager-howto.html">Manager Application HOW-TO</a></h4>                         <h4><a href="/docs/cluster-howto.html">Clustering/Session Replication HOW-TO</a></h4>                     </div>                 </div>                 <div id="actions">                     <div class="button">                         <a class="container shadow" href="/manager/status"><span>Server Status</span></a>                     </div>                     <div class="button">                         <a class="container shadow" href="/manager/html"><span>Manager App</span></a>                     </div>                     <div class="button">                         <a class="container shadow" href="/host-manager/html"><span>Host Manager</span></a>                     </div>                 </div>                 <!--                 <br class="separator" />                 -->                 <br class="separator" />             </div>             <div id="middle" class="curved container">                 <h3>Developer Quick Start</h3>                 <div class="col25">                     <div class="container">                         <p><a href="/docs/setup.html">Tomcat Setup</a></p>                         <p><a href="/docs/appdev/">First Web Application</a></p>                     </div>                 </div>                 <div class="col25">                     <div class="container">                         <p><a href="/docs/realm-howto.html">Realms & AAA</a></p>                         <p><a href="/docs/jndi-datasource-examples-howto.html">JDBC DataSources</a></p>                     </div>                 </div>                 <div class="col25">                     <div class="container">                         <p><a href="/examples/">Examples</a></p>                     </div>                 </div>                 <div class="col25">                     <div class="container">                         <p><a href="http://wiki.apache.org/tomcat/Specifications">Servlet Specifications</a></p>                         <p><a href="http://wiki.apache.org/tomcat/TomcatVersions">Tomcat Versions</a></p>                     </div>                 </div>                 <br class="separator" />             </div>             <div id="lower">                 <div id="low-manage" class="">                     <div class="curved container">                         <h3>Managing Tomcat</h3>                         <p>For security, access to the <a href="/manager/html">manager webapp</a> is restricted.                         Users are defined in:</p>                         <pre>$CATALINA_HOME/conf/tomcat-users.xml</pre>                         <p>In Tomcat 8.0 access to the manager application is split between                            different users.   <a href="/docs/manager-howto.html">Read more...</a></p>                         <br />                         <h4><a href="/docs/RELEASE-NOTES.txt">Release Notes</a></h4>                         <h4><a href="/docs/changelog.html">Changelog</a></h4>                         <h4><a href="http://tomcat.apache.org/migration.html">Migration Guide</a></h4>                         <h4><a href="http://tomcat.apache.org/security.html">Security Notices</a></h4>                     </div>                 </div>                 <div id="low-docs" class="">                     <div class="curved container">                         <h3>Documentation</h3>                         <h4><a href="/docs/">Tomcat 8.0 Documentation</a></h4>                         <h4><a href="/docs/config/">Tomcat 8.0 Configuration</a></h4>                         <h4><a href="http://wiki.apache.org/tomcat/FrontPage">Tomcat Wiki</a></h4>                         <p>Find additional important configuration information in:</p>                         <pre>$CATALINA_HOME/RUNNING.txt</pre>                         <p>Developers may be interested in:</p>                         <ul>                             <li><a href="http://tomcat.apache.org/bugreport.html">Tomcat 8.0 Bug Database</a></li>                             <li><a href="/docs/api/index.html">Tomcat 8.0 JavaDocs</a></li>                             <li><a href="http://svn.apache.org/repos/asf/tomcat/tc8.0.x/">Tomcat 8.0 SVN Repository</a></li>                         </ul>                     </div>                 </div>                 <div id="low-help" class="">                     <div class="curved container">                         <h3>Getting Help</h3>                         <h4><a href="http://tomcat.apache.org/faq/">FAQ</a> and <a href="http://tomcat.apache.org/lists.html">Mailing Lists</a></h4>                         <p>The following mailing lists are available:</p>                         <ul>                             <li id="list-announce"><strong><a href="http://tomcat.apache.org/lists.html#tomcat-announce">tomcat-announce</a><br />                                 Important announcements, releases, security vulnerability notifications. (Low volume).</strong>                             </li>                             <li><a href="http://tomcat.apache.org/lists.html#tomcat-users">tomcat-users</a><br />                                 User support and discussion                             </li>                             <li><a href="http://tomcat.apache.org/lists.html#taglibs-user">taglibs-user</a><br />                                 User support and discussion for <a href="http://tomcat.apache.org/taglibs/">Apache Taglibs</a>                             </li>                             <li><a href="http://tomcat.apache.org/lists.html#tomcat-dev">tomcat-dev</a><br />                                 Development mailing list, including commit messages                             </li>                         </ul>                     </div>                 </div>                 <br class="separator" />             </div>             <div id="footer" class="curved container">                 <div class="col20">                     <div class="container">                         <h4>Other Downloads</h4>                         <ul>                             <li><a href="http://tomcat.apache.org/download-connectors.cgi">Tomcat Connectors</a></li>                             <li><a href="http://tomcat.apache.org/download-native.cgi">Tomcat Native</a></li>                             <li><a href="http://tomcat.apache.org/taglibs/">Taglibs</a></li>                             <li><a href="/docs/deployer-howto.html">Deployer</a></li>                         </ul>                     </div>                 </div>                 <div class="col20">                     <div class="container">                         <h4>Other Documentation</h4>                         <ul>                             <li><a href="http://tomcat.apache.org/connectors-doc/">Tomcat Connectors</a></li>                             <li><a href="http://tomcat.apache.org/connectors-doc/">mod_jk Documentation</a></li>                             <li><a href="http://tomcat.apache.org/native-doc/">Tomcat Native</a></li>                             <li><a href="/docs/deployer-howto.html">Deployer</a></li>                         </ul>                     </div>                 </div>                 <div class="col20">                     <div class="container">                         <h4>Get Involved</h4>                         <ul>                             <li><a href="http://tomcat.apache.org/getinvolved.html">Overview</a></li>                             <li><a href="http://tomcat.apache.org/svn.html">SVN Repositories</a></li>                             <li><a href="http://tomcat.apache.org/lists.html">Mailing Lists</a></li>                             <li><a href="http://wiki.apache.org/tomcat/FrontPage">Wiki</a></li>                         </ul>                     </div>                 </div>                 <div class="col20">                     <div class="container">                         <h4>Miscellaneous</h4>                         <ul>                             <li><a href="http://tomcat.apache.org/contact.html">Contact</a></li>                             <li><a href="http://tomcat.apache.org/legal.html">Legal</a></li>                             <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>                             <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>                         </ul>                     </div>                 </div>                 <div class="col20">                     <div class="container">                         <h4>Apache Software Foundation</h4>                         <ul>                             <li><a href="http://tomcat.apache.org/whoweare.html">Who We Are</a></li>                             <li><a href="http://tomcat.apache.org/heritage.html">Heritage</a></li>                             <li><a href="http://www.apache.org">Apache Home</a></li>                             <li><a href="http://tomcat.apache.org/resources.html">Resources</a></li>                         </ul>                     </div>                 </div>                 <br class="separator" />             </div>             <p class="copyright">Copyright ©1999-2020 Apache Software Foundation.  All Rights Reserved</p>         </div>     </body>   </html>