milvus-backup安装部署

环境:
OS:Centos 7
milvus:2.4.6 standalone 部署
milvus-backup:0.4.19

 

1.官网:
https://milvus.io/docs/milvus_backup_cli.md

 

2.下载地址：
https://github.com/zilliztech/milvus-backup/releases

 

 

3.解压压缩包
[root@host135 milvus_backup]#mkdir -p /opt/milvus_backup ##软件部署目录
[root@host135 milvus_backup]#mv milvus-backup_Linux_x86_64.tar.gz /opt/milvus_backup/
[root@host135 milvus_backup]#cd /opt/milvus_backup
[root@host135 milvus_backup]#tar -zxvf milvus-backup_Linux_x86_64.tar.gz

 

4.准备配置文件
mkdir -p /opt/milvus_backup/conf
cd /opt/milvus_backup/conf
wget https://raw.githubusercontent.com/zilliztech/milvus-backup/master/configs/backup.yaml

若milvus启用了密码认证的,需要修改如下地方(红色标注,其他地方默认不需要修改):

[root@host135 backup]# more /opt/milvus_backup/conf/backup.yaml
# Configures the system log output.
log:
  level: info # Only supports debug, info, warn, error, panic, or fatal. Default 'info'.
  console: true # whether print log to console
  file:
    rootPath: "logs/backup.log"

http:
  simpleResponse: true

# milvus proxy address, compatible to milvus.yaml
milvus:
  address: localhost
  port: 19530
  authorizationEnabled: true
  # tls mode values [0, 1, 2]
  # 0 is close, 1 is one-way authentication, 2 is two-way authentication.
  tlsMode: 0
  user: "root"
  password: "Milvus"

# Related configuration of minio, which is responsible for data persistence for Milvus.
minio:
  # cloudProvider: "minio" # deprecated use storageType instead
  storageType: "minio" # support storage type: local, minio, s3, aws, gcp, ali(aliyun), azure, tc(tencent)
  
  address: localhost # Address of MinIO/S3
  port: 9000   # Port of MinIO/S3
  accessKeyID: minioadmin  # accessKeyID of MinIO/S3
  secretAccessKey: minioadmin # MinIO/S3 encryption string
  useSSL: false # Access to MinIO/S3 with SSL
  useIAM: false
  iamEndpoint: ""
  
  bucketName: "a-bucket" # Milvus Bucket name in MinIO/S3, make it the same as your milvus instance
  rootPath: "files" # Milvus storage root path in MinIO/S3, make it the same as your milvus instance

  # only for azure
  backupAccessKeyID: minioadmin  # accessKeyID of MinIO/S3
  backupSecretAccessKey: minioadmin # MinIO/S3 encryption string
  
  backupBucketName: "a-bucket" # Bucket name to store backup data. Backup data will store to backupBucketName/backupRo
otPath
  backupRootPath: "backup" # Rootpath to store backup data. Backup data will store to backupBucketName/backupRootPath

backup:
  maxSegmentGroupSize: 2G

  parallelism: 
    # collection level parallelism to backup
    backupCollection: 4
    # thread pool to copy data. reduce it if blocks your storage's network bandwidth
    copydata: 128
    # Collection level parallelism to restore
    restoreCollection: 2
  
  # keep temporary files during restore, only use to debug 
  keepTempFiles: false
  
  # Pause GC during backup through Milvus Http API. 
  gcPause:
    enable: true
    seconds: 7200
    address: http://localhost:9091

 

bucketName: "a-bucket" # Milvus Bucket name in MinIO/S3, make it the same as your milvus instance
rootPath: "files" # Milvus storage root path in MinIO/S3, make it the same as your milvus instance

 这两个参数是在minio中定义的,可以登录查看

http://192.168.1.135:9001/

账号密码:minioadmin/minioadmin

 

 

 

 

6.执行备份
备份整个库,这里备份db_test
/opt/milvus_backup/milvus-backup create -n bak_db_test_20240815 -d db_test --config=/opt/milvus_backup/conf/backup.yaml

备份文件存储路径为minio的路径下
[root@host135 backup]# pwd
/home/middle/milvus/volumes/minio/a-bucket/backup

 

7.恢复
恢复前可以删除目前的表
恢复整个库,需要带上 --restore_index=true 才能加载数据
/opt/milvus_backup/milvus-backup restore -n bak_db_test_20240815 -d db_test --restore_index=true --config=/opt/milvus_backup/conf/backup.yaml

 

说明:
1.针对某个库的备份,如db_test,-d会备份整个库下的表,但是恢复的时候也只能恢复整个库,指定-c,--database_collections 参数都不管用
2.针对default库的备份,-d default备份后,恢复的时候可以只指定-c参数恢复某个表;

3.api调用

执行如下命令

/opt/milvus_backup/milvus-backup server --config=/opt/milvus_backup/conf/backup.yaml

浏览器输入如下地址，在图形界面就可以调用提供的api

http://192.168.1.134:8080/api/v1/docs/index.html

 

备份多个库:

/opt/milvus_backup/milvus-backup create -n mytest_bak_20241206 -d mvsdb_ai,mydb_ai --config=/opt/milvus_backup/conf/backup.yaml