k8s(milvus operator)部署的milvus修改压缩参数(triggerInterval)

环境:
Os:Centos 7
milvus:v2.4.17
部署方式:milvus operator 在k8s部署

1.查看当前的配置
查看当前的配置,登录任何一个pod,每个pod的配置都是一样的

[root@master ~]# kubectl get pods -n middleware
NAME                                             READY   STATUS    RESTARTS        AGE
milvus-dev-milvus-datanode-79d69cb95d-hpxnm      1/1     Running   160 (40h ago)   4d22h
milvus-dev-milvus-indexnode-59d76648d8-mlhsk     1/1     Running   160 (40h ago)   4d22h
milvus-dev-milvus-mixcoord-5cbf5dcdbb-5x795      1/1     Running   159 (40h ago)   4d22h
milvus-dev-milvus-proxy-59bd97c596-tj2qw         1/1     Running   160 (15m ago)   4d22h
milvus-dev-milvus-querynode-0-86c97b9767-m4v5q   1/1     Running   161 (16m ago)   4d22h

登录其中一个 pod
[root@master ~]# kubectl exec -it milvus-dev-milvus-datanode-79d69cb95d-hpxnm -n middleware -- /bin/sh

# cd /milvus/configs
# ls
OWNERS advanced cert glog.conf hook.yaml milvus.yaml operator pgo user.yaml

说明:
milvus.yaml:默认的参数都在这个文件
user.yaml:用户在资源清单(milvus_cluster_external_dependencies.yaml)里配置的参数体现在这个文件

#more milvus.yaml
找到 triggerInterval的栏目

dataCoord:
  autoBalance: true
  autoUpgradeSegmentIndex: false
  brokerTimeout: 5000
  channel:
    balanceInterval: 360
    balanceSilentDuration: 300
    balanceWithRpc: true
    checkInterval: 1
    legacyVersionWithoutRPCWatch: 2.4.1
    notifyChannelOperationTimeout: 5
    watchTimeoutInterval: 300
  checkAutoBalanceConfigInterval: 10
  compaction:
    clustering:
      autoEnable: false
      enable: true
      maxCentroidsNum: 10240
      maxClusterSize: 5g
      maxClusterSizeRatio: 10
      maxInterval: 259200
      maxSegmentSizeRatio: 1
      maxTrainSizeRatio: 0.8
      minCentroidsNum: 16
      minClusterSizeRatio: 0.01
      minInterval: 3600
      newDataSizeThreshold: 512m
      preferSegmentSizeRatio: 0.8
      triggerInterval: 600

2.将triggerInterval参数修改为1200
[root@master my_yaml]# cd /root/my_yaml
vi milvus_cluster_external_dependencies.yaml
添加如下部分:

[root@master my_yaml]# more milvus_cluster_external_dependencies.yaml 
# This sample is used to deploy milvus with external dependencies
# change the <parameters> to match your environment
apiVersion: milvus.io/v1beta1
kind: Milvus
metadata:
  name: milvus-dev
  namespace: middleware
  labels:
    app: milvus
spec:
  mode: cluster
  components:
    enableRollingUpdate: true
    imageUpdateMode: rollingUpgrade
    proxy:
      replicas: 1
    dataNode:
      replicas: 1
    indexNode:
      replicas: 1
    queryNode:
      replicas: 1
    mixCoord:
      replicas: 1
    image: registry.cn-shenzhen.aliyuncs.com/hxlk8s/milvus:v2.4.15
    imagePullPolicy: IfNotPresent 
    resources:
      limits:
        cpu: '2'
        memory: 4Gi
      requests:
        cpu: '0.1'
        memory: 128Mi
  config:
    common:
      security:
        authorizationEnabled: true    
    queryNode:
      mmap:
        mmapEnabled: true
    mixCoord:
      activeStandby:
        enabled: true
    dataCoord:
      compaction:
        clustering:
          triggerInterval: 1200
    etcd:
      rootPath: milvus-dev
      #auth:
      #  enabled: true
      #  userName: milvus
      #  password: milvus
    msgChannel:
      chanNamePrefix:
        cluster: milvus-dev
    minio:
      bucketName: milvus-dev
      # config the prefix of the bucket milvus will use
      rootPath: milvus/milvus-dev
    kafka:
      # securityProtocol supports: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL 
      securityProtocol: PLAINTEXT
      # saslMechanisms supports: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512
      saslMechanisms: PLAIN
      saslUsername: ""
      saslPassword: ""
  dependencies: 
    etcd:
      external: true
      endpoints: ["192.168.1.106:2379"]
    storage:
      external: true
      endpoint: "192.168.1.106:9000"
      secretRef: "milvus-s3-secret"
      # if you want to use s3 check ./milvus_s3.yaml
      type: MinIO
    msgStreamType: "kafka"
    kafka:
      external: true
      brokerList: 
        - "192.168.1.106:9092"

3.应用
查看当前pod情况

[root@master my_yaml]# kubectl get pods -n middleware
NAME                                             READY   STATUS    RESTARTS        AGE
milvus-dev-milvus-datanode-79d69cb95d-hpxnm      1/1     Running   160 (40h ago)   4d23h
milvus-dev-milvus-indexnode-59d76648d8-mlhsk     1/1     Running   160 (40h ago)   4d23h
milvus-dev-milvus-mixcoord-5cbf5dcdbb-5x795      1/1     Running   159 (40h ago)   4d23h
milvus-dev-milvus-proxy-59bd97c596-tj2qw         1/1     Running   160 (40m ago)   4d23h
milvus-dev-milvus-querynode-0-86c97b9767-m4v5q   1/1     Running   161 (40m ago)   4d23h

应用
[root@master my_yaml]#cd /root/my_yaml
[root@master my_yaml]# kubectl apply -f milvus_cluster_external_dependencies.yaml
milvus.milvus.io/milvus-dev configured

再次查看pod情况

[root@master my_yaml]# kubectl get pods -n middleware
NAME                                            READY   STATUS    RESTARTS   AGE
milvus-dev-milvus-datanode-5f5c6cc76c-7b2x6     1/1     Running   0          77s
milvus-dev-milvus-indexnode-86784669c-6rmdw     1/1     Running   0          77s
milvus-dev-milvus-mixcoord-7b585bcd97-6sx7j     1/1     Running   0          77s
milvus-dev-milvus-proxy-86bfd744b7-tfqlx        1/1     Running   0          77s
milvus-dev-milvus-querynode-1-b64574b5f-s4rxg   1/1     Running   0          62s

可以看到RESTARTS栏目都变成了0,说明执行了 apply 命令集群重新部署了.

4.登录到pod查看配置是否生效

[root@master my_yaml]# kubectl get pods -n middleware
NAME                                            READY   STATUS    RESTARTS   AGE
milvus-dev-milvus-datanode-5f5c6cc76c-7b2x6     1/1     Running   0          2m48s
milvus-dev-milvus-indexnode-86784669c-6rmdw     1/1     Running   0          2m48s
milvus-dev-milvus-mixcoord-7b585bcd97-6sx7j     1/1     Running   0          2m48s
milvus-dev-milvus-proxy-86bfd744b7-tfqlx        1/1     Running   0          2m48s
milvus-dev-milvus-querynode-1-b64574b5f-s4rxg   1/1     Running   0          2m33s