k8s平台集成kong ingress 布署konga集成ui

k8s平台集成kong ingress 布署konga集成ui

各ingress的对比,其他博客有详细对比

简单谈点体验

  • istio很强大,但过于复杂,个人用过一段时期,勉强算是熟悉了基本使用,但复杂是相对团队的,每个人都掌握不可能,团队大部分只是想简单的上线一个api,然后能快速cicd部署上线到k8s 公开访问罢了,istio的流程做的全透明化需要额外的工作,个人来不及搞,团队后续为了图方便,都不喜欢用,算是半废弃状态,目前服务治理的方案是consul

  • ambassador 也是很强大的方案,未深入使用,只用过一些rewrite规则,方案比较强大

  • nginx/traefik 这两个用过,只是很简单的使用,未使用过复杂功能

  • kong 本身相比其他ingress并没有太大的优点,支持一些常用的插件,ssl托管,账号认证token,ip白名单等,不过这些也不只是kong有,真正决定使用kong,主要是因为konga的存在

官方向导见

https://docs.konghq.com/kubernetes-ingress-controller/1.1.x/deployment/minikube/

个人的环境并不是minikube 而是线上的生产集群,只不过喜欢,先以minikube类的精简的方案为基准,再逐组件替换为线上的ha方案

相比deploy,个人目前倾向deploy,改为sts

  • 改动 变更postgres类型

    生产环境,建议使用postgres HA方案https://github.com/sorintlab/stolon 官方deploy为postgres,个人会替换为StatefulSet

    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: postgres
      namespace: kong
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: postgres
      serviceName: postgres
      template:
        metadata:
          labels:
            app: postgres
        spec:
          containers:
          - env:
            - name: POSTGRES_USER
              value: kong
            - name: POSTGRES_PASSWORD
              value: kong
            - name: POSTGRES_DB
              value: kong
            - name: PGDATA
              value: /var/lib/postgresql/data/pgdata
            image: postgres:9.5
            name: postgres
            ports:
            - containerPort: 5432
            volumeMounts:
            - mountPath: /var/lib/postgresql/data
              name: datadir
              subPath: pgdata
          terminationGracePeriodSeconds: 60
      volumeClaimTemplates:
      - metadata:
          name: datadir
        spec:
          accessModes:
          - ReadWriteOnce
            resources:
            requests:
              storage: 1Gi        
    
  • 官方只有kong ingress是不够的,需要布署konga,之所以用kong就是因为konga的存在

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: konga
      name: konga
      namespace: kong
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: konga
      template:
        metadata:
          labels:
            app: konga
        spec:
          containers:
          - env:
            - name: HOST
              value: 0.0.0.0
            - name: PORT
              value: '80'
            - name: NODE_ENV
              value: production
            - name: DB_ADAPTER
              value: postgres
            - name: DB_HOST
              value: kong
            - name: DB_PORT
              value: '5432'
            - name: DB_USER
              value: kong
            - name: DB_PASSWORD
              value: kong
            - name: DB_DATABASE
              value: kong
            - name: DB_PG_SCHEMA
              value: kong
            - name: NO_AUTH
              value: 'true'
            image: pantsel/konga:0.14.9
            name: konga
            ports:
            - containerPort: 80
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: konga
      namespace: kong
    spec:
      externalTrafficPolicy: Cluster
      ports:
      - name: konga
        port: 80
        protocol: TCP
        targetPort: 80
        selector:
        app: konga
    

初始化konga,默认konga不可执行,进入konga执行初始化db

./bin/konga.js -c prepare -a postgres -u postgresql://kong:kong@postgres:5432/konga


$ kb -n kong get pod
NAME                           READY   STATUS      RESTARTS   AGE
ingress-kong-6b9544969-2pxwl   2/2     Running     0          125m
kong-migrations-6rshd          0/1     Completed   0          170m
postgres-767c99c648-fgd97      1/1     Running     0          20m

$ kb -n kong get svc
NAME                      TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
kong-proxy                LoadBalancer   10.99.12.233   <pending>     80:30193/TCP,443:31473/TCP   170m
kong-validation-webhook   ClusterIP      10.102.217.8   <none>        443/TCP                      170m
postgres                  ClusterIP      10.105.201.5   <none>        5432/TCP                     170m

暂时把 kong-proxy                LoadBalancer  改为 NodePort

$ kb -n kong get svc
NAME                      TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
kong-proxy                NodePort    10.99.12.233   <none>        80:30193/TCP,443:31473/TCP   171m
kong-validation-webhook   ClusterIP   10.102.217.8   <none>        443/TCP                      171m
postgres                  ClusterIP   10.105.201.5   <none>        5432/TCP                     171m

验证访问

curl -i $PROXY_IP
HTTP/1.1 404 Not Found
Date: Tue, 30 Jun 2020 09:34:23 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Content-Length: 48
X-Kong-Response-Latency: 1
Server: kong/2.0.4
{"message":"no Route matched with those values"}

验证真实地址,确认kong 集成成功

$ curl -i $PROXY_IP/foo
HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Date: Tue, 30 Jun 2020 09:34:40 GMT
Server: echoserver
X-Kong-Upstream-Latency: 2
X-Kong-Proxy-Latency: 4
Via: kong/2.0.4
Hostname: echo-599d77c5c7-jv8jl
Pod Information:
	pod name:	echo-599d77c5c7-jv8jl
	pod namespace:	default
	pod IP:	192.168.63.51
Server values:
	server_version=nginx: 1.12.2 - lua: 10010
Request Information:
	client_address=192.168.111.254
	method=GET
	real path=/foo
	query=
	request_version=1.1
	request_scheme=http
Request Headers:
	accept=*/*
	connection=keep-alive
	user-agent=curl/7.29.0
	x-forwarded-for=192.168.75.0
	x-forwarded-port=8000
	x-forwarded-proto=http
	x-real-ip=192.168.75.0
Request Body:
	-no body in request-

通过konga注册绑定k8s的kong

试了 nodeport ingress host,api 都失败 查看konga日志得

KongProxyController request error undefined
Sending 500 ("Server Error") response:
 {
  error: Error: self signed certificate
      at TLSSocket.onConnectSecure (_tls_wrap.js:1474:34)
      at TLSSocket.emit (events.js:310:20)
      at TLSSocket.EventEmitter.emit (domain.js:482:12)
      at TLSSocket._finishInit (_tls_wrap.js:917:8)
      at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:687:12) {
    code: 'DEPTH_ZERO_SELF_SIGNED_CERT'
  }
}
error: unexpected EOF

是konga 访问kong-admin-api的证书认证失败

查看到官方kong ingress 默认的配置是

      spec:
        containers:
        - env:
          - name: KONG_PROXY_LISTEN
            value: 0.0.0.0:8000, 0.0.0.0:8443 ssl http2
          - name: KONG_ADMIN_LISTEN
            value: 127.0.0.1:8444 ssl
          - name: KONG_STATUS_LISTEN
            value: 0.0.0.0:8100
          - name: KONG_DATABASE
            value: postgres
          - name: KONG_PG_HOST
            value: stolon-proxy-service.default
          - name: KONG_PG_PASSWORD
            value: bia_miaozhen
          - name: KONG_NGINX_WORKER_PROCESSES
            value: "1"
          - name: KONG_ADMIN_ACCESS_LOG
            value: /dev/stdout
          - name: KONG_ADMIN_ERROR_LOG
            value: /dev/stderr
          - name: KONG_PROXY_ERROR_LOG
            value: /dev/stderr
          image: kong:2.0
          imagePullPolicy: IfNotPresent
      KONG_ADMIN_LISTEN:            127.0.0.1:8444 ssl

更改 KONG_ADMIN_LISTEN 为 0.0.0.0:8444 ssl ,可以外部访问,但证书认证不通过

两个思路

  • 1加证书认证

暂内网服务,为图简单,先不采用证书的方式

  • 2通过http访问

先公开http服务,看kong的官方示例和文档

https://hub.docker.com/_/kong

$ docker run -d --name kong \
    --link kong-database:kong-database \
    -e "KONG_DATABASE=postgres" \
    -e "KONG_PG_HOST=kong-database" \
    -e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" \
    -e "KONG_PROXY_ACCESS_LOG=/dev/stdout" \
    -e "KONG_ADMIN_ACCESS_LOG=/dev/stdout" \
    -e "KONG_PROXY_ERROR_LOG=/dev/stderr" \
    -e "KONG_ADMIN_ERROR_LOG=/dev/stderr" \
    -e "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl" \
    -p 8000:8000 \
    -p 8443:8443 \
    -p 8001:8001 \
    -p 8444:8444 \
    kong

KONG_ADMIN_LISTEN是允许公开多个地址的,带ssl后缀的为https,不带ssl的 为http

我们额外公开8001做为KONG_ADMIN_LISTEN http服务,重启生效

    - name: KONG_ADMIN_LISTEN
      value: 0.0.0.0:8001, 0.0.0.0:8444 ssl

外部通过ingress 可以访问kong-admin-api.bia.com

http://ingress-kong-admin:8001/

截图和配置略有不同,个人的db是 stolon

Screen Shot 2021-01-15 at 2.39.44 PM

Screen Shot 2021-01-15 at 2.57.57 PM

kong ingress+konga 配置完毕

k8s kong ingress有一些集成的功能可以通过参数配置好,例如rewrite,下篇会谈

也可以k8s kong ingress 只注册一个标准的ingress,之后konga会同步显示,在页面上为该ingress配置各种插件

End

posted @ 2021-02-28 18:47  cclient  阅读(2579)  评论(0编辑  收藏  举报