扩展kubernetes apiserver

apiserver实际负责对外提供kubernetes RESTful API的服务，同时提供 https（默认监听6443端口）和http（默认监听8080端口）

它是系统管理指令的统一接口，任何对资源的增删该查都要交给apiserver处理后再交给etcd。

客户端通过list-watch监听apiserver中资源的create、update、delete事件，并针对事件类型调用相应的事件处理函数。

可以使用一个聚合器去聚合k8s的api server与用户开发的其它api server（如metrics-server等）

API Aggregation允许在不修改Kubernetes核心代码的同时扩展Kubernetes API

开启API Aggregation，需要在kube-apiserver增加部分配置

 

k8s的api server会创建相应Local APIServices，不过总是可用的：

kind: APIService
metadata:
  creationTimestamp: "2020-04-10T09:21:43Z"
  labels:
    kube-aggregator.kubernetes.io/automanaged: onstart
  name: v1.apps
  resourceVersion: "4"
  uid: 95cb5138-9100-4e1e-9568-de7d0d21389b
spec:
  group: apps
  groupPriorityMinimum: 17800
  version: v1
  versionPriority: 15
status:
  conditions:
  - lastTransitionTime: "2020-04-10T09:21:43Z"
    message: Local APIServices are always available
    reason: Local
    status: "True"
    type: Available

 

扩展Kubernetes API需要创建APIService资源对象，配置将哪些客户端请求代理到用户开发的api server

例如要访问metrics-server，则由群组/apis/metrics.k8s.io/v1beta1来获取

apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    k8s-app: metrics-server
  name: v1beta1.metrics.k8s.io
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: metrics-server
    namespace: kube-system
    port: 443  # 默认为443
  version: v1beta1
  versionPriority: 100

一旦k8s的api server确定应将请求发送到用户开发的api server，就根据service部分配置的地址将请求转发给相应service

 

自heapster被废弃以后，所有的指标数据都从API接口中获取，由此kubernetes将资源指标分为了两种：

（1）Core metrics核心指标

部署metrics-server，提供API metrics.k8s.io，仅提供Node和Pod的CPU和内存使用情况。

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: paas
---
 
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
    rbac.authorization.k8s.io/aggregate-to-view: "true"
  name: system:aggregated-metrics-reader
rules:
- apiGroups:
  - metrics.k8s.io
  resources:
  - pods
  - nodes
  verbs:
  - get
  - list
  - watch
 
---
 
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - nodes
  - nodes/stats
  - namespaces
  - configmaps
  verbs:
  - get
  - list
  - watch
 
---
 
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server-auth-reader
  namespace: paas
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: paas
 
---
 
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server:system:auth-delegator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: paas
 
---
 
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:metrics-server
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: paas
 
---
 
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: paas
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    k8s-app: metrics-server
 
---
 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: paas
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  strategy:
    rollingUpdate:
      maxUnavailable: 0
  template:
    metadata:
      labels:
        k8s-app: metrics-server
    spec:
      containers:
      - args:
        - --cert-dir=/tmp
        - --secure-port=4443
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        image: reg.harbor.com/paas/metrics-server:v0.4.1
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /livez
            port: https
            scheme: HTTPS
          periodSeconds: 10
        name: metrics-server
        ports:
        - containerPort: 4443
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /readyz
            port: https
            scheme: HTTPS
          periodSeconds: 10
        securityContext:
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 1000
        volumeMounts:
        - mountPath: /tmp
          name: tmp-dir
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-cluster-critical
      serviceAccountName: metrics-server
      volumes:
      - emptyDir: {}
        name: tmp-dir
 
---
 
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    k8s-app: metrics-server
  name: v1beta1.metrics.k8s.io
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: metrics-server
    namespace: paas
  version: v1beta1
  versionPriority: 100

metrics-server内部组成如图所示

客户不再需要通过API层去访问metrics-server，而是通过apiserver访问API注册层，再到metrics-server。

查看node和pod指标：

kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes”
kubectl get --raw "/apis/metrics.k8s.io/v1beta1/pods”

metrics-server从每个节点上的Kubelet公开的Summary API 中采集指标信息。

（2）Custom Metrics自定义指标

由Prometheus Adapter提供API custom.metrics.k8s.io，由此可支持任意Prometheus采集到的指标。此后就可以让k8s一些核心组件（如HPA）获取核心指标以外的其它自定义指标