SUSE Ceph Cephfs - Storage6
（1）Policy 配置文件，添加MDS角色定义
# vim /srv/pillar/ceph/proposals/policy.cfg # MDS role-mds/cluster/mds*.sls role-mds/cluster/node00[234].example.com.sls
（2）执行stage2,stage4
salt-run state.orch ceph.stage.2 salt-run state.orch ceph.stage.4
（3）设置 文件系统元数据到SSD
# ceph osd pool set cephfs_metadata crush_rule ssd_replicated_rule
（4）设置 PG PGP 数量
总数据大小，6*12*4=288T ,大约 260TB数据
# ceph osd lspools 3 cephfs_data 4 cephfs_metadata
# ceph osd pool set cephfs_data pg_num 1024 # ceph osd pool set cephfs_metadata pg_num 128
（5）关闭scrub和deep scrub , pool级别关闭，cephe -s 不会显示
# ceph osd pool set cephfs_data noscrub 1 # ceph osd pool set cephfs_metadata nodeep-scrub 1
（6）设置多活
# ceph fs set cephfs max_mds 2 # cephfs 为fs_name 名字 # ceph mds stat cephfs-2/2/2 up {0=node003=up:active,1=node001=up:active}, 1 up:standby
（7）客户端挂载
# mkdir /mnt/cephfs_client/ # mount -t ceph 192.168.2.40,192.168.2.41,192.168.2.42:6789:/ \ /mnt/cephfs_client/ -o name=admin,\ secret=AQAfvWhdAAAAABAAIGnAtjOBdDLE8+t/u2zadQ==,rasize=16384 # read ahead 16MB
# df -TH Filesystem Type Size Used Avail Use% Mounted on 192.168.2.40,192.168.2.41,192.168.2.42:6789:/ ceph 20G 0 20G 0% /mnt/cephfs_client
NFS
（1）NFS
使用图形化界面配置，在Daemons选择节点主机名
编辑DeepSea安装配置文件
# vim /srv/pillar/ceph/proposals/policy.cfg
# NFS
role-ganesha/cluster/node003.example.com.sls
执行salt命令，stage2 和stage4
# salt-run state.orch ceph.stage.2
# salt 'node003*' pillar.items
# salt-run state.orch ceph.stage.4
# cat /etc/ganesha/ganesha.conf | grep -v ^# | grep -v ^$
RADOS_URLS {
# Path to a ceph.conf file for this cluster. 集群配置文件
Ceph_Conf = /etc/ceph/ceph.conf;
# RADOS_URLS use their own ceph client too. Authenticated access
# requires a cephx keyring file. 用户key ID
UserId = "ganesha.node003";
watch_url = "rados://cephfs_data/ganesha/conf-node003";
}
CACHEINODE {
# Size the dirent cache down as small as possible.
Dir_Chunk = 0;
# size the inode cache as small as possible
NParts = 1;
Cache_Size = 1; # 每个分区哈希表大小
}
NFS_KRB5 # 是否激活 Kerberos 5. 默认false
{
Active_krb5 = false;
}
%url rados://cephfs_data/ganesha/conf-node003
NFS 配置
# rados -p .rgw.root ls --namespace=ganesha
export-1
conf-node003
conf-node004
get <obj-name> <outfile> fetch object
put <obj-name> <infile> [--offset offset]
# rados -p .rgw.root get export-1 nfs --namespace=ganesha
CIFS
node002 节点
# zypper in samba-ceph samba-winbind
Admin节点
# cd /etc/ceph
# ceph auth get-or-create client.samba.gw mon 'allow r' \
osd 'allow *' mds 'allow *' -o ceph.client.samba.gw.keyring
# scp ceph.client.samba.gw.keyring node002:/etc/ceph/
# cp /etc/samba/smb.conf /etc/samba/smb.conf.bak
编辑配置文件
# vim /etc/samba/smb.conf
[global]
workgroup = WORKGROUP
passdb backend = tdbsam
printing = cups
printcap name = cups
printcap cache time = 750
cups options = raw
map to guest = Bad User
logon path = \\%L\profiles\.msprofile
logon home = \\%L\%U\.9xprofile
logon drive = P:
usershare allow guests = Yes
[suse]
path = /
vfs objects = ceph
ceph: config_file = /etc/ceph/ceph.conf
ceph: user_id = samba.gw
read only = no
oplocks = no
kernel share modes = no
systemctl restart smb.service
systemctl restart nmb.service
# systemctl start smb.service
# systemctl enable smb.service
# systemctl start nmb.service
# systemctl enable nmb.service
检查端口
# netstat -ntulp | grep mbd
tcp 0 0 0.0.0.0:139 0.0.0.0:* LISTEN 6457/smbd
tcp 0 0 0.0.0.0:445 0.0.0.0:* LISTEN 6457/smbd
tcp6 0 0 :::139 :::* LISTEN 6457/smbd
tcp6 0 0 :::445 :::* LISTEN 6457/smbd
udp 0 0 172.200.50.255:137 0.0.0.0:* 6483/nmbd
udp 0 0 172.200.50.41:137 0.0.0.0:* 6483/nmbd
udp 0 0 192.168.2.255:137 0.0.0.0:* 6483/nmbd
udp 0 0 192.168.2.41:137 0.0.0.0:* 6483/nmbd
udp 0 0 192.168.3.255:137 0.0.0.0:* 6483/nmbd
udp 0 0 192.168.3.41:137 0.0.0.0:* 6483/nmbd
udp 0 0 0.0.0.0:137 0.0.0.0:* 6483/nmbd
udp 0 0 172.200.50.255:138 0.0.0.0:* 6483/nmbd
udp 0 0 172.200.50.41:138 0.0.0.0:* 6483/nmbd
udp 0 0 192.168.2.255:138 0.0.0.0:* 6483/nmbd
udp 0 0 192.168.2.41:138 0.0.0.0:* 6483/nmbd
udp 0 0 192.168.3.255:138 0.0.0.0:* 6483/nmbd
udp 0 0 192.168.3.41:138 0.0.0.0:* 6483/nmbd
udp 0 0 0.0.0.0:138 0.0.0.0:* 6483/nmbd
验证配置文件
# testparm
重载配置文件
# smbcontrol all reload-config
设置samba密码
# smbpasswd -a root
New SMB password:
Retype new SMB password:
Added user root.
Linux 访问samba
# smbclient -L //172.200.50.41/
Enter WORKGROUP\root's password:
Sharename Type Comment
--------- ---- -------
suse Disk
IPC$ IPC IPC Service (Samba 4.9.5-git.176.375e1f057883.6.1-SUSE-oS15.0-x86_64)
Reconnecting with SMB1 for workgroup listing.
Server Comment
--------- -------
Workgroup Master
--------- -------
WORKGROUP NODE002
Windows 访问samba
1) 打开来宾登录策略
- 按住“Win”和“R”键，在弹出的对话框中输入“gpedit.msc”，点击“确定”。
- 在本地组策略编辑器对话框中，依次点击“计算机配置” --> “管理模板” --> “网络”。
- 点击“lanman工作站”。
- 双击“启用不安全的来宾登录”，点击“已启用”，点击“确定”即可。
windows 访问
\\172.200.50.41\suse
