Kubernetes——使用Ingress发布Java服务(以Jenkins为例）

使用Ingress发布Java服务（以Jenkins为例）

  假设有这样一套环境：Kubernetes 集群上的 java-deploy 控制器生成了两个运行于 Pod 资源中的 java 实例，java-svc 是将它们统一暴露于集群中的访问入口。现在需要通过 Ingress 资源将 java-svc 发布给集群外部的客户端访问。

  规划图如下：

一、准备名称空间

  假设本示例中创建的所有资源都位于新建的 java-testing 名称空间中，与其他的资源逻辑上进行隔离，便于管理。

  下面的配置信息保存于 java-testing-namespaces.yaml 资源清单文件中： 

?

1 2 3 4 5 6

kind: Namespace apiVersion: v1 metadata:   name: java-testing   labels:     env: java-testing

  而后运行创建命令完成资源的额创建，并确认资源的存在：

?

1 2 3 4 5 6

[root@mh-k8s-master-247-10 java-testing]# kubectl apply -f java-testing-namespaces.yaml namespace/java-testing created [root@mh-k8s-master-247-10 java-testing]# kubectl get namespaces java-testing NAME           STATUS   AGE java-testing   Active   34s [root@mh-k8s-master-247-10 java-testing]#

二、部署 java 实例（以jenkins为例子）

2.1、部署 NFS 服务

• 2.1.1 部署 NFS 服务端配置

  ?

  1 2 3 4 5 6 7

  mkdir -p /data/k8s chown -R nfsnobody.nfsnobody /data echo "/data/k8s 10.255.247.0/24(rw,no_root_squash,sync)" >/etc/exports systemctl enable rpcbind systemctl enable nfs systemctl start rpcbind systemctl start nfs

• 2.1.1 客户端挂载

  ?

  1 2 3 4

  systemctl start rpcbind systemctl enable rpcbind mkdir /data/k8s -p mount -t nfs 10.255.247.21:/mnt/data /data/k8s

2.2、创建 Jenkins 集群所需的 yaml 文件

• 2.2.1 为 jenkins 数据持久化存储创建一个pv

  ?

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

  apiVersion: v1 kind: PersistentVolume metadata:   name: java-testing-jenkins   namespace: java-testing spec:   capacity:     storage: 200Gi   accessModes:     - ReadWriteOnce   persistentVolumeReclaimPolicy: Delete   nfs:     server: 10.255.247.10     path: /data/k8s   --- kind: PersistentVolumeClaim apiVersion: v1 metadata:   name: java-testing-jenkins   namespace: java-testing spec:   accessModes:     - ReadWriteOnce   resources:     requests:       storage: 200Gi

• 2.2.2 为 jenkins 集群权限 serviceAccount 文件

  ?

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

  apiVersion: v1 kind: ServiceAccount metadata:   name: java-testing-jenkins   namespace: java-testing   --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata:   name: java-testing-jenkins rules:   - apiGroups: ["extensions", "apps"]     resources: ["deployments"]     verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]   - apiGroups: [""]     resources: ["services"]     verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]   - apiGroups: [""]     resources: ["pods"]     verbs: ["create","delete","get","list","patch","update","watch"]   - apiGroups: [""]     resources: ["pods/exec"]     verbs: ["create","delete","get","list","patch","update","watch"]   - apiGroups: [""]     resources: ["pods/log"]     verbs: ["get","list","watch"]   - apiGroups: [""]     resources: ["secrets"]     verbs: ["get"]   --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata:   name: java-testing-jenkins   namespace: java-testing roleRef:   apiGroup: rbac.authorization.k8s.io   kind: ClusterRole   name: java-testing-jenkins subjects:   - kind: ServiceAccount     name: java-testing-jenkins     namespace: java-testing

• 2.2.3 创建 jenkins Deployment

  ?

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84

  apiVersion: apps/v1 kind: Deployment metadata:   name: java-testing-jenkins   namespace: java-testing spec:   replicas: 1   selector:     matchLabels:         app: java-testing-jenkins   template:     metadata:       labels:         app: java-testing-jenkins     spec:       terminationGracePeriodSeconds: 10       serviceAccount: java-testing-jenkins       containers:       - name: jenkins         image: jenkins/jenkins:lts         imagePullPolicy: IfNotPresent         ports:         - containerPort: 8080           name: web           protocol: TCP         - containerPort: 50000           name: agent           protocol: TCP         resources:           limits:             cpu: 1000m             memory: 1Gi           requests:             cpu: 500m             memory: 512Mi         livenessProbe:           httpGet:             path: /login             port: 8080           initialDelaySeconds: 60           timeoutSeconds: 5           failureThreshold: 12         readinessProbe:           httpGet:             path: /login             port: 8080           initialDelaySeconds: 60           timeoutSeconds: 5           failureThreshold: 12         volumeMounts:         - name: jenkinshome           subPath: jenkins           mountPath: /var/jenkins_home         env:         - name: JAVA_OPTS           value: >-                 -Xms 256Mi -Xmx 256Mi  -XX:MaxRAMPercentage=75.0                 -XX:InitialRAMPercentage=75.0 -XX:MinRAMPercentage=75.0                 -Dhudson.slaves.NodeProvisioner.initialDelay=20                 -Dhudson.slaves.NodeProvisioner.MARGIN=50                 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85                 -Dhudson.model.LoaHeapDumpOnOutOfMemoryErrordStatistics.clock=5000                 -Dhudson.model.LoadStatistics.decay=0.2                 -Dhudson.slaves.NodeProvisioner.recurrencePeriod=5000                 -Duser.timezone=Asia/Shanghai                 -Dio.jenkins.plugins.casc.ConfigurationAsCode.initialDelay=10000                 -XX:+HeapDumpOnOutOfMemoryError                 -XX:HeapDumpPath=/var/jenkins_home/dump-%t.hprof -verbose:gc                 -Xloggc:/var/jenkins_home/gc-%t.log -XX:NumberOfGCLogFiles=15                 -XX:+UseGCLogFileRotation -XX:GCLogFileSize=100m -XX:+PrintGC                 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintHeapAtGC                 -XX:+PrintGCCause -XX:+PrintGCApplicationStoppedTime                 -XX:+PrintTenuringDistribution -XX:+PrintReferenceGC                 -XX:+PrintAdaptiveSizePolicy -XX:+UseG1GC                 -XX:+UseStringDeduplication -XX:+ParallelRefProcEnabled                 -XX:+DisableExplicitGC -XX:+UnlockDiagnosticVMOptions                 -XX:+UnlockExperimentalVMOptions                 -XX:+UseCGroupMemoryLimitForHeap            securityContext:         fsGroup: 1000       volumes:       - name: jenkinshome         persistentVolumeClaim:           claimName: java-testing-jenkins

• 2.2.4 为 jenkins 创建 Service 资源

  ?

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

  apiVersion: v1 kind: Service metadata:   name: java-testing-jenkins-svc   namespace: java-testing   labels:     app: java-testing-jenkins-svc spec:   selector:     app: java-testing-jenkins-svc   ports:   - name: web     port: 80     targetPort: 8080     protocol: TCP   - name: agent     port: 50000     targetPort: 50000     protocol: TCP

• 2.2.5 为 jenkins 创建 Ingress 资源

  ?

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

  apiVersion: extensions/v1beta1 kind: Ingress metadata:   name: java-testing-nfs   namespace: java-testing   annotations:     kubernetes.io/ingress.class: "nginx" spec:   rules:   - host: java.zuoyang.tech     http:       paths:       - path:         backend:           serviceName: java-testing-jenkins-svc           servicePort: 80

•  2.2.6 配置 TLS Ingress 实例

  在 Ingress 控制器上配置 HTTPS 主机时，不能直接使用私钥和证书文件，而是要使用 Secret 资源对象来传递相关的数据。所以，接下来要根据私钥和证书生成用于配置 TLS Ingress 的 Secret 资源，在创建 Ingress 规则时由其将用到的 Secret 资源中的信息注入 Ingress 控制器的 Pod 对象中，用于为配置的 HTTPS 虚拟主机提供相应的私钥和证书。

  下面的命令会创建一个 TLS 类型名为：java-ingress-secret 的 Secret 资源：

• • ?

    1	kubectl create secret tls java-ingress-secret --cert=tls.crt --key=tls.key -n java-testing

  可使用下面的命令确认 Secrets 资源 java-ingress-secret 的 Secret 资源创建成功完成：

• • ?

    1	kubectl get secrets java-ingress-secret -n java-testing

  而后去定义创建 TLS 类型 Ingress 资源的配置清单。下面的配置清单通过 spec.rules 定义了一组转发规则，并通过 .spec.tls 将此主机定义为了 HTTPS 类型的虚拟主机，用到的私钥和证书信息来自于 Secret 资源 java-ingress-secret:

• • ?

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

    apiVersion: extensions/v1beta1 kind: Ingress metadata:   name: java-testing-nfs   namespace: java-testing   annotations:     kubernetes.io/ingress.class: "nginx" spec:   tls:   - hosts:     - java.zuoyang.tech     secretName: java-ingress-secret   rules:   - host: java.zuoyang.tech     http:       paths:       - path: /         backend:           serviceName: java-testing-jenkins-svc           servicePort: 80