linux下安装drbd详解

关于drbd理论方面的知识本文中不做介绍,本文章主要讲述如何搭建drbd。

 
一、系统环境
 
系统平台: Centos6.3(内核2.6.32-431.29.2.el6.x86_64)
节点一主机名: Cenos-HA1     IP:192.168.1.202
节点一主机名: Cenos-HA2     IP:192.168.1.202
 DRBD版本:DRBD-8.4.3
 
二、安装
 
1.准备
(1).关闭iptables和SELINUX,避免安装过程中报错。
节点1和节点2
#/etc/init.d/iptables stop
#/etc/init.d/ip6tables stop
#chkconfig iptables off
#chkconfig ip6tables off

# vi /etc/sysconfig/selinux

---------------

SELINUX=disabled

---------------

 
(2).设置 hosts文件(节点1和节点2)
 
# vi /etc/hosts

-----------------

192.168.1.202 Centos-HA1
192.168.1.203 Centos-HA2
 
-----------------
 
节点1和节点2(重启)
(3).在两台虚拟机分别添加一块2G硬盘sdb作为DRBD,分别分区为sdb1,大小2G,并在本地系统创建/data目录,不做挂载操作。(节点1和节点2)
节点1
[root@Centos-HA1 ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xd59786d2.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-261, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-261, default 261):
Using default value 261

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
 
 
[root@Centos-HA1 ~]#mkdir /data
 
节点2
[root@Centos-HA2 ~]# fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x9adc5d39.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
         switch off the mode (command 'c') and change display units to
         sectors (command 'u').

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-261, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-261, default 261):
Using default value 261

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
 

[root@Centos-HA2 ~]#mkdir /data
 
(4).时间同步

# ntpdate -u asia.pool.ntp.org

2、DRBD的安装配置
 
(1).安装依赖包:( 节点1和节点2)
 
#yum install gcc gcc-c++ make glibc flex kernel kernel-devel kernel-headers
 
(2)安装DRBD: (节点1和节点2)

# wget http://oss.linbit.com/drbd/8.4/drbd-8.4.3.tar.gz

# tar -zxvf drbd-8.4.3.tar.gz

# cd drbd-8.4.3

# ./configure --prefix=/usr/local/drbd --with-km

# make KDIR=/usr/src/kernels/2.6.32-431.29.2.el6.x86_64/

# make install

# mkdir -p /usr/local/drbd/var/run/drbd

# cp /usr/local/drbd/etc/rc.d/init.d/drbd /etc/init.d/

# chkconfig --add drbd 

# chkconfig --level 35 drbd on
 
(3)加载DRBD模块(节点1和节点2)
#modprobe drbd
 
如果出现如下错误
[root@Centos-HA1 drbd-8.4.3]# modprobe drbd
FATAL: Module drbd not found.
解决办法:
(注意:不要使用yum install kernel* 因为我做实验然后重启系统进不去了
可以使用yum install kernel kernel-devel kernel-headers)
 

查看DRBD模块是否加载到内核: (节点1和节点2)

# lsmod |grep drbd
 
节点1
 
节点2
 
 

3.参数配置:(节点1和节点2)

# vi /usr/local/drbd/etc/drbd.conf

清空文件内容,并添加如下配置:

---------------

resource r0{

protocol C;

 

startup { wfc-timeout 0; degr-wfc-timeout 120;}

disk { on-io-error detach;}

net{

  timeout 60;

  connect-int 10;

  ping-int 10;

  max-buffers 2048;

  max-epoch-size 2048;

}

syncer { rate 30M;}

 

on Centos-HA1{

  device /dev/drbd0;

  disk   /dev/sdb1;

  address 192.168.1.202:7788;

  meta-disk internal;

}

on Centos-HA2{

  device /dev/drbd0;

  disk   /dev/sdb1;

  address 192.168.1.203:7788;

  meta-disk internal;

}

}

---------------
 
(注:至于参数详解请参考 http://www.cnblogs.com/Elliot-wang/p/3994362.html )

(4).创建DRBD设备并激活ro资源:(节点1和节点2)

# mknod /dev/drbd0 b 147 0

# drbdadm create-md r0
 
 

等待片刻,显示success表示drbd块创建成功

----------------

Writing meta data...

initializing activity log

NOT initializing bitmap

New drbd meta data block successfully created.

 

               --== Creating metadata ==--

As with nodes, we count the total number of devices mirrored by DRBD

at http://usage.drbd.org.

 

The counter works anonymously. It creates a random number to identify

the device and sends that random number, along with the kernel and

DRBD version, to usage.drbd.org.

 

http://usage.drbd.org/cgi-bin/insert_usage.pl?

 

nu=716310175600466686&ru=15741444353112217792&rs=1085704704

 

* If you wish to opt out entirely, simply enter 'no'.

* To continue, just press [RETURN]

 

success

----------------

 

再次输入该命令:

# drbdadm create-md r0

成功激活r0

----------------

[need to type 'yes' to confirm] yes

 

Writing meta data...

initializing activity log

NOT initializing bitmap

New drbd meta data block successfully created.

----------------
 
 

5.启动DRBD服务:(节点1,节点2)

# service drbd start

 
6.查看状态:(node1,node2)
 
#service drbd status
# cat /proc/drbd
 
 
 
 

(7).将Centos-HA1主机配置为主节点:(节点1)

[root@Centos-HA1 ~]# drbdsetup /dev/drbd0 primary --force
 
查看主机状态
节点1
[root@Centos-HA1 ~]#service drbd status
 
 
节点2
[root@Centos-HA1 ~]#service drbd status
 
 
 

ro在主从服务器上分别显示 Primary/Secondary和Secondary/Primary

ds显示UpToDate/UpToDate

表示主从配置成功。
 

(8).挂载DRBD:(节点1)

 

从刚才的状态上看到mounted和fstype参数为空,所以我们这步开始挂载DRBD到系统目录

[root@Centos-HA1 ~]# mkfs.ext4 /dev/drbd0

[root@Centos-HA1 ~]# mount /dev/drbd0 /data     

注:Secondary节点上不允许对DRBD设备进行任何操作,包括只读,所有的读写操作只能在Primary节点上进行,只有当Primary节点挂掉时,Secondary节点才能提升为Primary节点继续工作。

(9).模拟DRBD1故障,DRBD2接管并提升为Primary

(节点1)

[root@Centos-HA1 ~]#cd /data

[root@Centos-HA1 ~]# touch a.txt b.txt c.txt d.txt e.txt

[root@Centos-HA1 ~]#cd ..

[root@Centos-HA1 ~]# umount /data

[root@Centos-HA1 ~]# drbdsetup /dev/drbd0 secondary
 
注:这里实际生产环境若DRBD1宕机,在DRBD2状态信息中ro的值会显示为Secondary/Unknown,只需要进行DRBD提权操作即可。
 
(节点2)
 
[root@Centos-HA2 ~]# drbdsetup /dev/drbd0 primary
 

[root@Centos-HA2 ~]#  mount  /dev/drbd0 /data

[root@Centos-HA2 ~]# cd /data

[root@Centos-HA2 ~]#  touch f.txt g.txt h.txt j.txt k.txt

[root@Centos-HA2 ~]# ls

--------------

a.txt  b.txt  c.txt  d.txt  e.txt  f.txt  g.txt  h.txt  j.txt  k.txt  lost+found
--------------
 
 
到此DRBD已经安装完成。
 

不过如何保证DRBD主从结构的智能切换,实现高可用,这里就需要Heartbeat来实现了。

Heartbeat会在DRBD主端挂掉的情况下,自动切换从端为主端并自动挂载/data分区。
 
 

注:(摘自酒哥的构建高可用LINUX服务器第2版)

假设你把Primary的eth0挡掉,然后直接在Secondary上进行主Primary主机的提升,并且mount上,你可能会发现在Primary上测试考入的文件确实同步过来了,之后把Primary的eth0恢复后,看看有没有自动恢复 主从关系。经过查看,发现DRBD检测出了Split-Brain的状况,也就是两个节点都处于standalone状态,故障描述如下:Split-Brain detected,dropping connection! 这就是传说中的“脑裂”。
 
 
下面本人做个测试
 
首先将刚才的主备环境切换回来,此步我不再叙说,安照第9步反过来操作即可
 
下面我来模拟测试
节点1
[root@Centos-HA1 ~]# cd /data
[root@Centos-HA1 ~]# ls
a.txt  b.txt  c.txt  d.txt  e.txt  f.txt  g.txt  h.txt  j.txt  k.txt  lost+found
[root@Centos-HA1 ~]# touch aa.txt bb.txt cc.txt
[root@Centos-HA1 ~]# cd ..
[root@Centos-HA1 ~]# umount data
[root@Centos-HA1 ~]# ifdown eth0
 
 
节点2
[root@Centos-HA2 ~]#drbdsetup /dev/drbd0 primary
[root@Centos-HA2 ~]# cd /data
[root@Centos-HA2 ~]#mount /dev/drbd0 /data
[root@Centos-HA2 ~]#cd /data
[root@Centos-HA2 ~]#ls
 
aa.txt  bb.txt  cc.txt  d.txt  f.txt  h.txt  k.txt
a.txt   b.txt   c.txt   e.txt  g.txt  j.txt  lost+found
 
 
 
 
节点1
[root@Centos-HA1 ~]# ifup eth0
 
 
[root@Centos-HA1 ~]#drbdadm secondary r0
 
[root@Centos-HA1 ~]#service drbd status
 
 
 
节点2
[root@Centos-HA2 ~]# cd /data
 
[root@Centos-HA2 ~]# touch  aaa.txt  bbb.txt ccc.txt

[root@Centos-HA2 ~]#ls
aaa.txt  a.txt    bb.txt  ccc.txt  c.txt  e.txt  g.txt  j.txt  lost+found
aa.txt   bbb.txt  b.txt   cc.txt   d.txt  f.txt  h.txt  k.txt
 
[root@Centos-HA2 ~]# cd ..
[root@Centos-HA2 ~]# umount /data
[root@Centos-HA2 ~]# drbdadm secondary r0
 
 
节点1
[root@Centos-HA1 ~]# drbdsetup /dev/drbd0 primary
 
 
[root@Centos-HA1 ~]#mount /dev/drbd0 /data
[root@Centos-HA1 ~]# cd /data/
[root@Centos-HA1 ~]# ls
aa.txt  bb.txt  cc.txt  d.txt  f.txt  h.txt  k.txt
a.txt   b.txt   c.txt   e.txt  g.txt  j.txt  lost+found
 
(注:文件没有同步过来)

解决办法
 
(先恢复系统到开始前的状态,就是 eth0 down 之后)
节点1
[root@Centos-HA1 ~]# umount /data
[root@Centos-HA1 ~]# drbdadm secondary r0
 
 
 
节点2
[root@Centos-HA2 ~]# drbdsetup /dev/drbd0 primary
 


[root@Centos-HA2 ~]# mount /dev/drbd0 /data
 
[root@Centos-HA2 ~]# cd /data
[root@Centos-HA2 ~]# ls
aaa.txt  a.txt    bb.txt  ccc.txt  c.txt  e.txt  g.txt  j.txt  lost+found
aa.txt   bbb.txt  b.txt   cc.txt   d.txt  f.txt  h.txt  k.txt
 
 
已经恢复到故障开始的状态,下面开始解决
 
节点1
 
 

[root@Centos-HA1 ~]# drbdadm secondary r0

[root@Centos-HA1 ~]# drbdadm disconnect all

[root@Centos-HA1 ~]# drbdadm --discard-my-data connect r0

切换回来
[root@Centos-HA2 ~]# drbdadm secondary r0
[root@Centos-HA1 ~]# drbdsetup /dev/drbd0 primary
[root@Centos-HA1 ~]# mount /dev/drbd0 /data
[root@Centos-HA1 ~]# cd /data
[root@Centos-HA1 ~]# ls
aaa.txt  a.txt    bb.txt  ccc.txt  c.txt  e.txt  g.txt  j.txt  lost+found
aa.txt   bbb.txt  b.txt   cc.txt   d.txt  f.txt  h.txt  k.txt
(注:本次eth0 宕机实验,如果/data 目录不卸载的话直接 ifdown eth0是不能成功的)
 
完成
 
 
 
 
 
 
 
 
posted on 2014-09-26 10:55  Darren_华子  阅读(1355)  评论(0编辑  收藏  举报