k8s(milvus-operator)部署milvusdb(带密码认证)
环境:
Os:Centos 7
前提条件,使用到的组件需要提前部署好:
kafka(单点,包含 zookeeper ):https://www.cnblogs.com/hxlasky/p/18615374
etcd(单点):https://www.cnblogs.com/hxlasky/p/18641819
minio(单点):https://www.cnblogs.com/hxlasky/p/18643317
1.创建namespace
kubectl create namespace middleware
2.安装 cert-manager(不需要修改镜像)
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yaml
Milvus Operator 需要一个 cert-manager 来管理安全通信的证书。请确保您安装的是 cert-manager 版本为 1.1.3 或以上版本。请运行以下命令进行安装：
配置文件 cert-manager.yaml 提前下载好
若已经下载好的可以执行如下命令:
[root@master soft]# cd /root/my_yaml
[root@master soft]# kubectl apply -f cert-manager.yaml
查看(其他节点执行也是一样的输出)
[root@master soft]# kubectl get pods -n cert-manager
NAME READY STATUS RESTARTS AGE
cert-manager-679895f5-qtrxk 1/1 Running 0 81s
cert-manager-cainjector-b78544fd4-jb2gm 1/1 Running 0 81s
cert-manager-webhook-cd8bbbd67-ll9r9 1/1 Running 0 81s
3. 安装 Milvus Operator
[root@yunwei-k8s-dev milvus]# kubectl get pod -n milvus-operator
配置文件可以提前下载到服务器上
cert-manager 启动并运行后，您可以安装 Milvus Operator。运行以下命令使用 kubectl 部署 Milvus Operator
kubectl apply -f https://github.com/zilliztech/milvus-operator/releases/latest/download/deployment.yaml
若已经下载的可以执行如下命令:
[root@master soft]# cd /root/my_yaml
[root@master soft]# kubectl apply -f deployment.yaml
您可以使用以下命令检查 Milvus Operator pod 是否正在运行:
[root@master soft]# kubectl get pods -n milvus-operator
NAME READY STATUS RESTARTS AGE
milvus-operator-545894474-dvdzr 0/1 ContainerCreating 0 34s
milvus-operator-checker-k8b98 0/1 ErrImagePull 0 34s
查看日志:
kubectl describe po milvus-operator-9598c4bb6-c2nbz -n milvus-operator
日志如下:
Warning Failed 31s kubelet Failed to pull image "milvusdb/milvus-operator:v1.1.4": rpc error: code = DeadlineExceeded desc = failed to pull and unpack image "docker.io/milvusdb/milvus-operator:v1.1.4": failed to resolve reference "docker.io/milvusdb/milvus-operator:v1.1.4": failed to do request: Head "https://registry-1.docker.io/v2/milvusdb/milvus-operator/manifests/v1.1.4": dial tcp 128.242.245.253:443: i/o timeout
Warning Failed 31s kubelet Error: ErrImagePull
Normal BackOff 30s kubelet Back-off pulling image "milvusdb/milvus-operator:v1.1.4"
Warning Failed 30s kubelet Error: ImagePullBackOff
Normal Pulling 16s (x2 over 61s) kubelet Pulling image "milvusdb/milvus-operator:v1.1.4"
解决办法：
删除应用
kubectl delete -f deployment.yaml
修改文件
vi deployment.yaml
找到如下语句:
image: 'milvusdb/milvus-operator:v1.1.4'
修改为自己下载的镜像
image: 'registry.cn-shenzhen.aliyuncs.com/hxlk8s/milvus-operator:v1.1.2'
再次创建应用
[root@master soft]# /root/my_yaml
[root@master soft]# kubectl apply -f deployment.yaml
[root@master soft]# /root/my_yaml
[root@master soft]# kubectl apply -f deployment.yaml
[root@master my_yaml]# kubectl get pods -n milvus-operator
NAME READY STATUS RESTARTS AGE
milvus-operator-554547495d-rx2lb 1/1 Running 0 3m57s
4.创建Secret
/root/my_yaml
vi milvus_secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: milvus-s3-secret
namespace: middleware
type: Opaque
stringData:
accesskey: LfUVNGzh4N1VJ2BQ
secretkey: YrHQBmXpE0OMrNH2PasUeRZutUYPvWlM
kubectl apply -f milvus_secret.yaml
[root@master my_yaml]# kubectl get Secret -n middleware
NAME TYPE DATA AGE
milvus-s3-secret Opaque 2 163m
这里的accesskey和secretkey 需要在minio控制台界面创建
4.安装milvusdb集群
准备yaml文件
内容如下:
more milvus_cluster_external_dependencies.yaml
[root@master my_yaml]# more milvus_cluster_external_dependencies.yaml
# This sample is used to deploy milvus with external dependencies
# change the <parameters> to match your environment
apiVersion: milvus.io/v1beta1
kind: Milvus
metadata:
name: milvus-dev
namespace: middleware
labels:
app: milvus
spec:
mode: cluster
components:
enableRollingUpdate: true
imageUpdateMode: rollingUpgrade
proxy:
replicas: 1
dataNode:
replicas: 1
indexNode:
replicas: 1
queryNode:
replicas: 1
mixCoord:
replicas: 1
image: registry.cn-shenzhen.aliyuncs.com/hxlk8s/milvus:v2.4.15
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: '2'
memory: 4Gi
requests:
cpu: '0.1'
memory: 128Mi
config:
common:
security:
authorizationEnabled: true
queryNode:
mmap:
mmapEnabled: true
etcd:
rootPath: milvus-dev
#auth:
# enabled: true
# userName: milvus
# password: milvus
msgChannel:
chanNamePrefix:
cluster: milvus-dev
minio:
bucketName: milvus-dev
# config the prefix of the bucket milvus will use
rootPath: milvus/milvus-dev
kafka:
# securityProtocol supports: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL
securityProtocol: PLAINTEXT
# saslMechanisms supports: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512
saslMechanisms: PLAIN
saslUsername: ""
saslPassword: ""
dependencies:
etcd:
external: true
endpoints: ["192.168.1.106:2379"]
storage:
external: true
endpoint: "192.168.1.106:9000"
secretRef: "milvus-s3-secret"
# if you want to use s3 check ./milvus_s3.yaml
type: MinIO
msgStreamType: "kafka"
kafka:
external: true
brokerList:
- "192.168.1.106:9092"
这里的milvus-dev Buckets需要提前创建好
修改自己的镜像
registry.cn-shenzhen.aliyuncs.com/hxlk8s/milvus:v2.4.15
应用:
[root@master my_yaml]# kubectl apply -f milvus_cluster_external_dependencies.yaml
milvus.milvus.io/milvus-dev created
删除命令
[root@master my_yaml]#kubectl delete -f milvus_cluster_external_dependencies.yaml
[root@yunwei-k8s-dev milvus]# kubectl get pod -n middleware
[root@master my_yaml]# kubectl get pods -n middleware
NAME READY STATUS RESTARTS AGE
milvus-dev-milvus-datanode-858f4784c9-k9gnh 1/1 Running 0 161m
milvus-dev-milvus-indexnode-5bfb846985-6tqxq 1/1 Running 0 161m
milvus-dev-milvus-mixcoord-885bbffb5-fvqx8 1/1 Running 0 161m
milvus-dev-milvus-proxy-54cd854d95-ftjs8 1/1 Running 0 161m
milvus-dev-milvus-querynode-0-5fd98f6cb6-dhgkh 1/1 Running 0 161m
