接下来是存储的部署, GlusterFS
主机: 192.168.1.201 / 192.168.1.202 / 192.168.1.203
系统: CentOS 7.4
CPU: 2 vCPU
Memory: 4 G
1. 安装 yum repo
# lvm $ ansible glusterfs -m lvg -a "vg=gluster-vg pvs=/dev/sdb,/dev/sdc,/dev/sdd " # 查看 vg 大小, 这里是 120G, 所以分为三个 lv ,每个给 35GB $ vgs VG #PV #LV #SN Attr VSize VFree gluster-vg 3 0 0 wz--n- <119.99g <119.99g # 划分 lv $ ansible glusterfs -v -m lvol -a "lv=ovirt size=35G vg=gluster-vg" $ ansible glusterfs -v -m lvol -a "lv=openshift size=35G vg=gluster-vg" $ ansible glusterfs -v -m lvol -a "lv=openstack size=35G vg=gluster-vg"
2. 准备 bricks
# lvm $ ansible glusterfs -m lvg -a "vg=gluster-vg pvs=/dev/sdb,/dev/sdc,/dev/sdd " # 查看 vg 大小, 这里是 120G, 所以分为三个 lv ,每个给 35GB $ vgs VG #PV #LV #SN Attr VSize VFree gluster-vg 3 0 0 wz--n- <119.99g <119.99g # 划分 lv $ ansible glusterfs -v -m lvol -a "lv=ovirt size=35G vg=gluster-vg" $ ansible glusterfs -v -m lvol -a "lv=openshift size=35G vg=gluster-vg" $ ansible glusterfs -v -m lvol -a "lv=openstack size=35G vg=gluster-vg" # 查看一下结果 $ lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert openshift gluster-vg -wi-a----- 35.00g openstack gluster-vg -wi-a----- 35.00g ovirt gluster-vg -wi-a----- 35.00g # 分区 $ ansible glusterfs -v -m filesystem -a "dev=/dev/gluster-vg/ovirt fstype=xfs opts='-i size=512'" $ ansible glusterfs -v -m filesystem -a "dev=/dev/gluster-vg/openshift fstype=xfs opts='-i size=512'" $ ansible glusterfs -v -m filesystem -a "dev=/dev/gluster-vg/openstack fstype=xfs opts='-i size=512'" # 创建挂载目录 $ ansible glusterfs -v -m file -a "state=directory path=/bricks/ovirt" $ ansible glusterfs -v -m file -a "state=directory path=/bricks/openshift" $ ansible glusterfs -v -m file -a "state=directory path=/bricks/openstack" # 修改 fstab $ ansible glusterfs -v -m lineinfile -a "path=/etc/fstab line='/dev/gluster-vg/ovirt /bricks/ovirt xfs defaults 1 2'" $ ansible glusterfs -v -m lineinfile -a "path=/etc/fstab line='/dev/gluster-vg/openshift /bricks/openshift xfs defaults 1 2'" $ ansible glusterfs -v -m lineinfile -a "path=/etc/fstab line='/dev/gluster-vg/openstack /bricks/openstack xfs defaults 1 2'" # 挂载 $ ansible glusterfs -v -m shell -a "mount -a" # 检查 $ ansible glusterfs -v -m shell -a "df -hT"
3. 安装 GlusterFS 软件包
$ ansible glusterfs -m yum -a "name=glusterfs-server" # 防火墙, 一般情况下加 glusterfs 的服务, 但是根据官网描述, 关掉会比较方便, 毕竟分布式存储是一个"只对内部提供访问"的服务 # "You can run gluster with iptables rules, but it's up to you to decide how you'll configure those rules. By default, glusterd will listen on tcp/24007 but opening that port isn't enough on the gluster nodes. Each time you add a brick, it will open a new port (that you'll be able to see with "gluster volume status")" # 2017-12-6 更新: 还是添加 glusterfs 的防火墙服务吧, 不然老是会连接不上, 要开启防火墙然后再关掉才能连接 $ ansible glusterfs -m firewalld -a "service=glusterfs state=enabled permanent=true" $ ansible glusterfs -m service -a "name=firewalld state=restarted enabled=yes" # 启动 GlusterFS 服务吧 $ ansible glusterfs -m service -a "name=glusterd state=started enabled=yes" # 检查服务状态 $ ansible glusterfs -a "systemctl status glusterd"
GlusterFS 就装好了, 接下来就可以创建 Volume 分享出去了
另外, 上面的过程可以直接跑个 playbook 来完成:
--- - hosts: glusterfs tasks: - name: pv / vg create lvg: vg: gluster-vg pvs: /dev/sdb,/dev/sdc,/dev/sdd - name: lv create lvol: lv={{ item }} size=35G vg=gluster-vg with_items: - ovirt - openshift - openstack - name: filesystem filesystem: dev=/dev/gluster-vg/{{ item }} fstype=xfs opts='-i size=512' with_items: - ovirt - openshift - openstack - name: create mount directories file: state=directory path=/bricks/{{ item }} with_items: - ovirt - openshift - openstack - name: modify fstab lineinfile: path=/etc/fstab line='/dev/gluster-vg/{{ item }} /bricks/{{ item }} xfs defaults 1 2' with_items: - ovirt - openshift - openstack - name: mount shell: mount -a - name: check mounts shell: df -hT - name: install glusterfs yum: name: glusterfs-server - name: disable firewalld service: name: firewalld state: stopped enabled: no - name: enable and start glusterfs service: name: glusterd state: started enabled: yes - name: check daemon status shell: systemctl status glusterd
