Oracle rac asm 扩容
Oracle rac asm 扩容
Oracle rac asm 扩容-使用udev方式绑定multipath多路径设备
环境介绍：
故事发生在Oracle 11.2.0.4 RAC/ Linux 6.8 的数据库集群服务器上。
DATA/ ARCH/ 磁盘组空间不足，进行扩容。
首先，如果要在线使用udev方式扩容asm，建议先备份两个节点所有在用网卡信息，避免bug误删除网卡信息。备份至/tmp目录下。（包括the ifcfg-eth0 (public), ifcfg-eth1 (private) and ifcfg-eth2 (backup)），注意11.2.0.3及以后在linux上的bug，见如下官网解释：
Network interface going down when dynamically adding disks to storage using udev in RHEL 6 (文档 ID 1569028.1)
Applies to:
Oracle Database - Enterprise Edition - Version 11.2.0.3 and later
Oracle Net Services - Version 11.2.0.4 to 11.2.0.4 [Release 11.2]
Linux x86-64
Symptoms
Dynamically added new disks to storage using udev rules(as per the docs note 414897.1, 1528148.1, 371814.1 etc).
During start_udev, udev has deleted the public network interface and this caused the listener to crash, and clusterware moved all services, scan listeners and the VIP on node 1 to node 2.
Changes
Trying to add disk to storage using udev
Cause
udev has deleted the public network interface.
Solution
As per RHEL you are advised to set HOTPLUG="no" for the network configuration scripts.
Add HOTPLUG="no" to the ifcfg-eth0 (public), ifcfg-eth1 (private) and ifcfg-eth2 (backup) network config files in /etc/sysconfig/network-scripts directory.
建议先做数据库备份
1.停止数据库及集群服务
su - grid
srvctl stop listener
srvctl stop database -d jyrac
rac1/rac2
su - root
/u01/11.2.0/grid/bin/crsctl stop crs
####################### rac1/rac2 begin ##############################
2.网卡信息修改
注意：如果是使用网卡绑定，比如绑定后的网卡为bond0，则要在bond0的配置文件里添加hotplug配置信息，在eth0里添加不起作用。（实战中所有在用网卡均添加）
备份所有网卡信息至/tmp 目录
rac1/rac2
cp ifcfg-eth0 /tmp/ifcfg-eth0bak
cp ifcfg-bond0 /tmp/ifcfg-bond0bak
cp ifcfg-eth1 /tmp/ifcfg-eth1bak
...
实际：
[root@jyrac1 network-scripts]# cp * /tmp/wkbak
[root@jyrac1 network-scripts]#
[root@jyrac2 network-scripts]# cp * /tmp/wkbak/
[root@jyrac2 network-scripts]# 修改网卡信息
HOTPLUG="no"重启网络服务
service network restart
[root@jyrac1 network-scripts]# service network restart
Shutting down interface bond0: [ OK ]
Shutting down interface eth0: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface bond0: Determining if ip address 192.168.20.11 is already in use for device bond0...
[ OK ]
Bringing up interface eth0: Determining if ip address 187.187.2.61 is already in use for device eth0...
[ OK ]
[root@jyrac1 network-scripts]#
[root@jyrac2 network-scripts]# service network restart
Shutting down interface bond0: [ OK ]
Shutting down interface eth0: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface bond0: Determining if ip address 192.168.20.12 is already in use for device bond0...
[ OK ]
Bringing up interface eth0: Determining if ip address 187.187.2.62 is already in use for device eth0...
[ OK ]
[root@jyrac2 network-scripts]#3.修改udev权限文件
备份udev文件至/tmp 目录
[root@jyrac1 network-scripts]# cp /etc/udev/rules.d/12-dm-permissions.rules /tmp/wkbak/vi /etc/udev/rules.d/12-dm-permissions.rules
增加：
ENV{DM_NAME}=="arch4", OWNER:="grid", GROUP:="asmadmin", MODE:="660"
ENV{DM_NAME}=="data3", OWNER:="grid", GROUP:="asmadmin", MODE:="660"4.加载配置文件
[root@rac-db1 /]# /sbin/udevadm control reload-rules5.添加udev磁盘
（生产库千万别重启udev，否则vip会漂移，且网卡信息可能会被删除，甚至宕库，重启监听可解决）
udevadm trigger --subsystem-match=block --action=add[root@jyrac1 network-scripts]# udevadm control reload-rules
unrecognized command
udevadm[27284]: unrecognized command
[root@jyrac1 network-scripts]#
[root@jyrac1 network-scripts]#
[root@jyrac1 network-scripts]#
[root@jyrac1 network-scripts]# /sbin/udevadm control reload-rules
unrecognized command
udevadm[27682]: unrecognized command
[root@jyrac1 network-scripts]# find / -name udevadm
/sbin/udevadm
^C
[root@jyrac1 network-scripts]#
[root@jyrac1 network-scripts]#
[root@jyrac1 network-scripts]#
[root@jyrac1 network-scripts]# cd
[root@jyrac1 ~]# /sbin/udevadm control reload-rules
unrecognized command
udevadm[27987]: unrecognized command
[root@jyrac1 ~]# ls -la /sbin/udevadm
-rwxr-xr-x. 1 root root 137256 Oct 17 2013 /sbin/udevadm
[root@jyrac1 ~]# /sbin/udevadm control reload-rules
unrecognized command
udevadm[28151]: unrecognized command
[root@jyrac1 ~]# /sbin/start_udev
Starting udev: [ OK ]
[root@jyrac1 ~]# 6.检查磁盘属性
ll -l /dev/dm*如果有问题，可以考虑重启multipath 或者udev试试。
/sbin/start_udev
or
service multipathd restart再次检查属性，确认没问题之后考虑启动集群服务。
[root@jyrac1 ~]# ll -l /dev/dm*
brw-rw----. 1 root disk 253, 0 Apr 11 18:23 /dev/dm-0
brw-rw----. 1 root disk 253, 1 Apr 11 18:23 /dev/dm-1
brw-rw----. 1 grid asmadmin 253, 10 Apr 11 18:23 /dev/dm-10
brw-rw----. 1 root disk 253, 11 Apr 11 18:23 /dev/dm-11
brw-rw----. 1 root disk 253, 12 Apr 11 18:23 /dev/dm-12
brw-rw----. 1 grid asmadmin 253, 13 Apr 11 18:23 /dev/dm-13
brw-rw----. 1 grid asmadmin 253, 14 Apr 11 18:23 /dev/dm-14
brw-rw----. 1 root disk 253, 2 Apr 11 18:23 /dev/dm-2
brw-rw----. 1 grid asmadmin 253, 3 Apr 11 18:23 /dev/dm-3
brw-rw----. 1 grid asmadmin 253, 4 Apr 11 18:23 /dev/dm-4
brw-rw----. 1 grid asmadmin 253, 5 Apr 11 18:23 /dev/dm-5
brw-rw----. 1 grid asmadmin 253, 6 Apr 11 18:23 /dev/dm-6
brw-rw----. 1 grid asmadmin 253, 7 Apr 11 18:23 /dev/dm-7
brw-rw----. 1 grid asmadmin 253, 8 Apr 11 18:23 /dev/dm-8
brw-rw----. 1 grid asmadmin 253, 9 Apr 11 18:23 /dev/dm-9
[root@jyrac1 ~]#
[root@jyrac2 ~]# ll -l /dev/dm*
brw-rw---- 1 root disk 253, 0 Apr 11 18:25 /dev/dm-0
brw-rw---- 1 root disk 253, 1 Apr 11 18:25 /dev/dm-1
brw-rw---- 1 grid asmadmin 253, 10 Apr 11 18:25 /dev/dm-10
brw-rw---- 1 root disk 253, 11 Apr 11 18:25 /dev/dm-11
brw-rw---- 1 root disk 253, 12 Apr 11 18:25 /dev/dm-12
brw-rw---- 1 grid asmadmin 253, 13 Apr 11 18:25 /dev/dm-13
brw-rw---- 1 grid asmadmin 253, 14 Apr 11 18:25 /dev/dm-14
brw-rw---- 1 root disk 253, 2 Apr 11 18:25 /dev/dm-2
brw-rw---- 1 grid asmadmin 253, 3 Apr 11 18:25 /dev/dm-3
brw-rw---- 1 grid asmadmin 253, 4 Apr 11 18:25 /dev/dm-4
brw-rw---- 1 grid asmadmin 253, 5 Apr 11 18:25 /dev/dm-5
brw-rw---- 1 grid asmadmin 253, 6 Apr 11 18:25 /dev/dm-6
brw-rw---- 1 grid asmadmin 253, 7 Apr 11 18:25 /dev/dm-7
brw-rw---- 1 grid asmadmin 253, 8 Apr 11 18:25 /dev/dm-8
brw-rw---- 1 grid asmadmin 253, 9 Apr 11 18:25 /dev/dm-9
[root@jyrac2 ~]# 7.启动crs集群服务
rac1/rac2
su - root
/u01/11.2.0/grid/bin/crsctl start crs######################## rac1/rac2 end############################
8.检查crs服务
su - grid
crs_stat -t
crsctl stat res -t9.启动数据库
srvctl start database -d jyrac10.检查数据库状态
crsctl stat res -t11.检查asm是否识别到未添加的磁盘
set linesize 300
col name for a20
select group_number,name,state,type,total_mb,free_mb,usable_file_mb,allocation_unit_size/1024/1024 unit_mb from v$asm_diskgroup order by 1;
GROUP_NUMBER NAME STATE TYPE TOTAL_MB FREE_MB USABLE_FILE_MB UNIT_MB
------------ -------------------- ---------------------- ------------ ---------- ---------- -------------- ----------
1 ARCH MOUNTED EXTERN 614400 120548 120548 1
2 DATA MOUNTED EXTERN 409600 83773 83773 1
3 OCR MOUNTED NORMAL 15360 14434 4657 1set linesize 300
col path for a50
select group_number,name,path,mount_status,state,redundancy,total_mb,free_mb,failgroup,create_date from v$asm_disk order by 1,2;
GROUP_NUMBER NAME PATH MOUNT_STATUS STATE REDUNDANCY TOTAL_MB FREE_MB FAILGROUP CREATE_DATE
------------ -------------------- -------------------------------------------------- -------------- ---------------- -------------- ---------- ---------- ------------------------------------------------------------ -----------------------
0 /dev/mapper/arch4 CLOSED NORMAL UNKNOWN 0 0
0 /dev/mapper/data3 CLOSED NORMAL UNKNOWN 0 0
1 ARCH_0000 /dev/mapper/arch1 CACHED NORMAL UNKNOWN 204800 40180 ARCH_0000 21-MAY-2015 11:00:32
1 ARCH_0001 /dev/mapper/arch2 CACHED NORMAL UNKNOWN 204800 40183 ARCH_0001 21-MAY-2015 11:00:32
1 ARCH_0002 /dev/mapper/arch3 CACHED NORMAL UNKNOWN 204800 40185 ARCH_0002 21-MAY-2015 11:00:32
2 DATA_0000 /dev/mapper/data1 CACHED NORMAL UNKNOWN 204800 41879 DATA_0000 21-MAY-2015 10:59:33
2 DATA_0001 /dev/mapper/data2 CACHED NORMAL UNKNOWN 204800 41894 DATA_0001 21-MAY-2015 10:59:33
3 OCR_0000 /dev/mapper/crs1 CACHED NORMAL UNKNOWN 5120 4812 OCR_0000 21-MAY-2015 10:16:24
3 OCR_0001 /dev/mapper/crs2 CACHED NORMAL UNKNOWN 5120 4810 OCR_0001 21-MAY-2015 10:16:24
3 OCR_0002 /dev/mapper/crs3 CACHED NORMAL UNKNOWN 5120 4812 OCR_0002 21-MAY-2015 10:16:24
10 rows selected.12.ASM扩容
alter diskgroup DATA add disk '/dev/mapper/data3' rebalance power 8;
alter diskgroup ARCH add disk '/dev/mapper/arch4' rebalance power 8;13.检查进度
SQL> select * from v$asm_operation;
GROUP_NUMBER OPERATION STATE POWER ACTUAL SOFAR EST_WORK EST_RATE EST_MINUTES ERROR_CODE
------------ ---------- -------- ---------- ---------- ---------- ---------- ---------- ----------- ----------------------------------------------------------------------------------------
1 REBAL WAIT 8
2 REBAL RUN 8 8 33914 108615 5606 13
SQL> set linesize 300
col path for a50
select group_number,name,path,mount_status,state,redundancy,total_mb,free_mb,failgroup,create_date from v$asm_disk order by 1,2;转自
邮箱：286397146@qq.com
昵称：A这货不是DBA
