【docker】minio集群部署
minio 集群搭建
1. 创建三台虚拟机,本教程采用操作系统均为centos7版本:
操作系统 | 主机 | 节点 |
centos07 | 192.168.99.8 | minio-1 |
centos07 | 192.168.99.9 | minio-2 |
centos07 | 192.168.99.10 | minio-2 |
所有节点配置主机名解析:
cat >> /etc/hosts << EOF
192.168.99.8 minio-1
192.168.99.9 minio-2
192.168.99.10 minio-3
EOF
在minio-1 节点:
docker run -d --name minio01 \
--restart=always --net=host \
-e "MINIO_ROOT_USER=admin" \
-e "MINIO_ROOT_PASSWORD=admin123" \
-v /etc/localtime:/etc/localtime \
-v /data01/minio:/data1/minio \
-v /data02/minio:/data2/minio \
-v /data03/minio:/data3/minio \
-v /data04/minio:/data4/minio \
-v /data05/minio:/data5/minio \
-v /data06/minio:/data6/minio \
-v /data07/minio:/data7/minio \
-v /data08/minio:/data8/minio \
minio/minio:RELEASE.2021-11-24T23-19-33Z.fips server \
--address "192.168.99.8:49000" \
--console-address ":49001" \
http://minio-{1...3}/data{1...8}/minio
在minio2节点执行:
docker run -d --name minio02 \
--restart=always --net=host \
-e "MINIO_ROOT_USER=admin" \
-e "MINIO_ROOT_PASSWORD=admin123" \
-v /etc/localtime:/etc/localtime \
-v /data01/minio:/data1/minio \
-v /data02/minio:/data2/minio \
-v /data03/minio:/data3/minio \
-v /data04/minio:/data4/minio \
-v /data05/minio:/data5/minio \
-v /data06/minio:/data6/minio \
-v /data07/minio:/data7/minio \
-v /data08/minio:/data8/minio \
minio/minio:RELEASE.2021-11-24T23-19-33Z.fips server \
--address "192.168.99.9:49000" \
--console-address ":49001" \
http://minio-{1...3}/data{1...8}/minio
在minio3节点执行:
docker run -d --name minio03 \
--restart=always --net=host \
-e "MINIO_ROOT_USER=admin" \
-e "MINIO_ROOT_PASSWORD=admin123" \
-v /etc/localtime:/etc/localtime \
-v /data01/minio:/data1/minio \
-v /data02/minio:/data2/minio \
-v /data03/minio:/data3/minio \
-v /data04/minio:/data4/minio \
-v /data05/minio:/data5/minio \
-v /data06/minio:/data6/minio \
-v /data07/minio:/data7/minio \
-v /data08/minio:/data8/minio \
minio/minio:RELEASE.2021-11-24T23-19-33Z.fips server \
--address "192.168.99.10:49000" \
--console-address ":49001" \
http://minio-{1...3}/data{1...8}/minio
客户端查看minio状态,3个节点及每个节点上的Drivers全部在线:
$ mc config host add minio http://192.168.99.8:49000 admin admin123
$ mc admin info minio
● minio-1:49000
Uptime: 14 minutes
Version: 2021-11-24T23:19:33Z
Network: 3/3 OK
Drives: 4/4 OK
● minio-2:49000
Uptime: 12 minutes
Version: 2021-11-24T23:19:33Z
Network: 3/3 OK
Drives: 4/4 OK
● minio-3:49000
Uptime: 8 minutes
Version: 2021-11-24T23:19:33Z
Network: 3/3 OK
Drives: 4/4 OK
53 MiB Used, 1 Bucket, 4 Objects
12 drives online, 0 drives offline
nginx 实现负载均衡
1. 创建目录和文件
mkdir -p /cq-aiplatform/soft/nginx/conf
mkdir -p /cq-aiplatform/soft/nginx/html
创建配置文件,配置映射,要不然每次修改都要进入到容器内:
vim /cq-aiplatform/soft/nginx/conf/nginx.conf
复制以下内容到配置文件中:
events {
worker_connections 1024;
}
http {
upstream minio {
server 192.168.99.8:49000;
server 192.168.99.9:49000;
server 192.168.99.10:49000;
}
upstream console {
ip_hash;
server 192.168.99.8:49001;
server 192.168.99.9:49001;
server 192.168.99.10:49001;
}
server {
listen 6333;
server_name 192.168.99.11;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
server {
listen 6334;
server_name 192.168.99.11;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://console;
}
}
}
启动docker:
docker run -d --name nginx_minio \
-p 6333:6333 \
-p 6334:6334 \
-v /cq-aiplatform/soft/nginx/conf/nginx.conf:/etc/nginx/nginx.conf \
-v /cq-aiplatform/soft/nginx/logs:/var/log/nginx \
-v /cq-aiplatform/soft/nginx/html:/usr/share/nginx/html \
nginx