使用udev固化存储盘符安装11g grid asm存储管理
在oracle 10g里面，我们可以使用ASMLib库创建asm实例，oracle在RHEL 6后是不单独提供下载，放在了Enterprise Linux Server镜像中。所以我们为了管理方便使用udev管理使用的设备映射。在oracle 11g中，推荐使用clusterware来管理资源情况，如果我们安装grid单实例，配置过程是一样的。
软件环境：
I、第一部分
操作系统安装注意一下几点：
1、为了管理方便我们只创建/boot,swap,/ 三个分区，根据自己的实际情况创建相应的分区信息。
UUID=310e5a20-a456-46a1-8427-fd010eb3815a / ext4 defaults 1 1 UUID=35e50cfb-1206-4c83-879e-af9ccaa526ce /boot ext4 defaults 1 2 UUID=bc56e23e-79b1-451c-bf3d-ba35bc4ae738 swap swap defaults 0 0
2、安装的操作系统只是用来跑数据库的，在安装rpm时，遵循一个原则，尽可能少安装。主要是为性能及安全考虑。
<1>选择最小化安装
<2>选择安装Development tools，Additional Development，System Management，System administration tools
根据系统的相关要求，选择来安装，这儿不过多阐述。
3、系统安装完，我们简单做一些配置。
--修改ip地址
# vi /etc/sysconfig/network-scripts/ifcfg-eth0 # /etc/init.d/network restart
验证：
# ip addr show --关闭不必要的服务 # chkconfig iptables offf # chkconfig iptables off # vi /etc/sysconfig/selinux # vi /etc/hosts
--配置yum repository
/*RHEL 5*/ mkdir /etc/yum.repos.d/bak cd /etc/yum.repos.d && mv *.repo bak cat >> /etc/yum.repos.d/yum.repo <<EOF #redhat release 5.4 [Server] name=Server baseurl=file:///mnt/Server enabled=1 gpgcheck=0 [Cluster] name=Cluster baseurl=file:///mnt/Cluster enabled=1 gpgcheck=0 [ClusterStorage] name=ClusterStorage baseurl=file:///mnt/ClusterStorage enabled=1 gpgcheck=0 [VT] name=VT baseurl=file:///mnt/VT enabled=1 gpgcheck=0 EOF --------------------------------------------------------- /*RHEL 6*/ mkdir /etc/yum.repos.d/bak cd /etc/yum.repos.d && mv *.repo bak cat >> /etc/yum.repos.d/yum.repo <<EOF [RHEL6] name=RHEL6 baseurl=file:///mnt enabled=1 gpgcheck=0 EOF
--在RHEL 6里面需要安装的rpm，我们前面大多数已经安装过。
yum install -y binutils.x86_64 yum install -y compat-libstdc++-33.x86_64 yum install -y elfutils-libelf.x86_64 yum install -y elfutils-libelf-devel.x86_64 yum install -y gcc.x86_64 yum install -y gcc-c++.x86_64 yum install -y glibc.x86_64 yum install -y glibc-common.x86_64 yum install -y glibc-devel.x86_64 yum install -y glibc-headers.x86_64 yum install -y kernel-headers.x86_64 yum install -y ksh.x86_64 yum install -y libaio.x86_64 yum install -y libaio-devel.x86_64 yum install -y libgcc.x86_64 yum install -y libstdc++.x86_64 yum install -y libstdc++-devel.x86_64 yum install -y make.x86_64 yum install -y numactl.x86_64 yum install -y sysstat.x86_64 yum install -y unixODBC.x86_64 yum install -y unixODBC-devel.x86_64 yum install -y libXp.x86_64 yum install -y libXp.i386 yum install -y compat-libstdc++-33.x86_64 yum install -y compat-libcap1.i686 yum install -y compat-libcap1.x86_64 yum install -y libaio-devel.x86_64
II、第二部分
1、检查基本信息
# lsb_release -id Distributor ID: RedHatEnterpriseServer Description: Red Hat Enterprise Linux Server release 6.5 (Santiago) # cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 10.24.34.37 rh-asm-37
2、添加用户和组配置环境变量
# groupadd -g 1000 oinstall # groupadd -g 1300 dba # groupadd -g 1301 oper # groupadd -g 1200 asmadmin # groupadd -g 1201 asmdba # groupadd -g 1202 asmoper # useradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper -d /home/grid -s /bin/bash -c "Grid Infrastructure Owner" grid # useradd -m -u 1101 -g oinstall -G dba,oper,asmdba -d /home/oracle -s /bin/bash -c "Oracle Software Owner" oracle
3、验证用户
# id oracle uid=1101(oracle) gid=1000(oinstall) groups=1000(oinstall),1300(dba),1301(oper),1201(asmdba) # id grid uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1200(asmadmin),1201(asmdba),1202(asmoper)
4、修改用户密码
# echo 'oracle' |passwd --stdin oracle # echo 'grid' |passwd --stdin grid
5、创建目录及修改权限
# mkdir -p /u01/grid # mkdir -p /u01/11.2.0/grid # chown -R grid:oinstall /u01 # mkdir -p /u01/oracle # mkdir -p /u01/oracle/product/11.2.0/db_1 # chown -R oracle:oinstall /u01/oracle # chmod -R 775 /u01
6、配置oracle及grid环境变量
--grid用户 # --------------------------------------------------- # .bash_profile # --------------------------------------------------- # OS User: grid # Application: Oracle Grid Infrastructure # Version: Oracle 11g Release 2 # --------------------------------------------------- export TEMP=/tmp export TMPDIR=/tmp export ORACLE_SID=+ASM export GRID_BASE=/u01/grid export ORACLE_BASE=$GRID_BASE export GRID_HOME=/u01/11.2.0/grid export ORACLE_HOME=$GRID_HOME export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/local/lib export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib export ORACLE_PATH=/u01/oracle/dba_scripts/sql:$ORACLE_HOME/rdbms/admin export SQLPATH=/u01/oracle/dba_scripts/sql:$ORACLE_HOME/rdbms/admin export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/jdk/bin:/usr/sbin:$PATH:$HOME/bin export ORACLE_TERM=xterm stty erase ^H --oracle用户 # --------------------------------------------------- # .bash_profile # --------------------------------------------------- # OS User: oracle # Application: Oracle Database Software Owner # Version: Oracle 11g Release 2 # --------------------------------------------------- export TEMP=/tmp export TMPDIR=/tmp export ORACLE_UNQNAME=rhnode export ORACLE_SID=rhnode export ORACLE_BASE=/u01/oracle export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1 export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/local/lib export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib export ORACLE_PATH=/u01/oracle/dba_scripts/sql:$ORACLE_HOME/rdbms/admin export SQLPATH=/u01/oracle/dba_tools/dba_scripts/sql export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/jdk/bin:/usr/sbin:$PATH:$HOME/bin export ORACLE_TERM=xterm stty erase ^H
7、修改相关参数文件
/etc/sysctl.conf
kernel.shmmax = 68719476736 kernel.shmall = 4294967296 fs.file-max = 6815744 fs.aio-max-nr = 4194304 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default = 4194034 net.core.rmem_max = 4194304 net.core.wmem_default = 262144 net.core.wmem_max = 1048576 net.ipv4.tcp_wmem = 262144 262144 262144 net.ipv4.tcp_rmem = 4194304 4194304 4194304
验证：
sysctl -p
/etc/security/limits.conf
grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 1024 grid hard nofile 65536 oracle soft nproc 2047 oracle hard nproc 16384 oracle soft nofile 1024 oracle hard nofile 65536
/etc/pam.d/login
session required pam_limits.so
/etc/profile
if [ \$USER = "oracle" ] || [ \$USER = "grid" ]; then
if [ \$SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
umask 022
fi
-------可选-------
/etc/csh.login
if ( \$USER == "oracle" || \$USER == "grid" ) then
limit maxproc 16384
limit descriptors 65536
endif
III、第三部分
存储相关配置
这里我们使用openfiler开源软件模拟大部分的存储的使用。具体如何配置，我们这儿不多阐述。google一下，你懂得！
发现一下映射的设备情况。（我们需要做的是发现设备->登陆测试->服务器重启后设备可以自动映射）
# iscsiadm --mode discoverydb --type sendtargets --portal 192.168.1.1 --discover
--登陆
# iscsiadm --mode node --targetname iqn.2015-08.org.asm_01 --portal 192.168.1.1:3260 --login # iscsiadm --mode node --targetname iqn.2015-08.org.asm_02 --portal 192.168.1.1:3260 --login # iscsiadm --mode node --targetname iqn.2015-08.org.asm_03 --portal 192.168.1.1:3260 --login # iscsiadm --mode node --targetname iqn.2015-08.org.asm_04 --portal 192.168.1.1:3260 --login # iscsiadm --mode node --targetname iqn.2015-08.org.asm_05 --portal 192.168.1.1:3260 --login
--验证配置中node.startup = automatic是否使用自动模式
# iscsiadm --mode node --targetname iqn.2015-08.org.asm_01 --portal 192.168.1.1:3260 # iscsiadm --mode node --targetname iqn.2015-08.org.asm_02 --portal 192.168.1.1:3260 # iscsiadm --mode node --targetname iqn.2015-08.org.asm_03 --portal 192.168.1.1:3260 # iscsiadm --mode node --targetname iqn.2015-08.org.asm_04 --portal 192.168.1.1:3260 # iscsiadm --mode node --targetname iqn.2015-08.org.asm_05 --portal 192.168.1.1:3260
--列出iscsi设置情况
# iscsiadm --mode node 192.168.1.1:3260,1 iqn.2015-08.org.asm_03 192.168.1.1:3260,1 iqn.2015-08.org.asm_01 192.168.1.1:3260,1 iqn.2015-08.org.asm_05 192.168.1.1:3260,1 iqn.2015-08.org.asm_04 192.168.1.1:3260,1 iqn.2015-08.org.asm_02
验证映射情况
# ls -ltr /dev/disk/by-path/|grep iscsi lrwxrwxrwx 1 root root 9 Aug 28 01:23 ip-192.168.1.1:3260-iscsi-iqn.2015-08.org.asm_03-lun-0 -> ../../sdb lrwxrwxrwx 1 root root 9 Aug 28 01:23 ip-192.168.1.1:3260-iscsi-iqn.2015-08.org.asm_05-lun-0 -> ../../sde lrwxrwxrwx 1 root root 9 Aug 28 01:23 ip-192.168.1.1:3260-iscsi-iqn.2015-08.org.asm_04-lun-0 -> ../../sdd lrwxrwxrwx 1 root root 9 Aug 28 01:23 ip-192.168.1.1:3260-iscsi-iqn.2015-08.org.asm_01-lun-0 -> ../../sdc lrwxrwxrwx 1 root root 9 Aug 28 01:23 ip-192.168.1.1:3260-iscsi-iqn.2015-08.org.asm_02-lun-0 -> ../../sdf
获取映射后的设备uuid
# for i in `cat /proc/partitions | awk {'print $4'} | grep sd`; do echo "### $i: `scsi_id --whitelist /dev/$i`"; done
### sdb: 14f504e46494c450055325a6665352d525542722d4d324e6a
### sdc: 14f504e46494c45004c56646564382d777765652d664a6561
### sdd: 14f504e46494c45003635686851572d316b4e462d55543172
### sde: 14f504e46494c45006744427654582d41634d542d56724931
### sdf: 14f504e46494c45004f736a786a6e2d5a796c762d6378446a
配置udev固化盘符,写入相关规则文件
# cat /etc/udev/rules.d/99-oracleasm.rules KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c45004c56646564382d777765652d664a6561", NAME="asmdisk1", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c45004f736a786a6e2d5a796c762d6378446a", NAME="asmdisk2", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c450055325a6665352d525542722d4d324e6a", NAME="asmdisk3", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c45003635686851572d316b4e462d55543172", NAME="asmdisk4", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd*", SUBSYSTEM=="block", PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name", RESULT=="14f504e46494c45006744427654582d41634d542d56724931", NAME="asmdisk5", OWNER="grid", GROUP="asmadmin", MODE="0660"
#start_udev
IIII、第四部分
安装grid
创建asmca
创建netca
创建数据库软件
创建库dbca
此部分详细的安装步骤省略。
IIIII、验证
验证grid、oracle安装情况。
grid资源查看：
[grid@rh-asm-37 ~]$ crsctl status res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.AHZHIXIN.dg
ONLINE ONLINE rh-asm-37
ora.DATA.dg
ONLINE ONLINE rh-asm-37
ora.LISTENER.lsnr
ONLINE ONLINE rh-asm-37
ora.asm
ONLINE ONLINE rh-asm-37 Started
ora.ons
OFFLINE OFFLINE rh-asm-37
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
1 ONLINE ONLINE rh-asm-37
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE ONLINE rh-asm-37
ora.rhnode.db
1 ONLINE ONLINE rh-asm-37 Open
asm磁盘使用情况：
[grid@rh-asm-37 ~]$ sqlplus / as sysdasm
SQL> set line 200 pagesize 9999
SQL> col path for a50
SQL> select GROUP_NUMBER,DISK_NUMBER,NAME,FAILGROUP,TOTAL_MB,PATH,state from v$asm_disk;
GROUP_NUMBER DISK_NUMBER NAME FAILGROUP TOTAL_MB PATH STATE
------------ ----------- ------------------------------ ------------------------------ ---------- -------------------------------------------------- --------
1 1 DATA_0001 DATA_0001 5152 /dev/asmdisk2 NORMAL
1 3 DATA_0003 FAIL01 5120 /dev/asmdisk5 NORMAL
1 0 DATA_0000 DATA_0000 5120 /dev/asmdisk1 NORMAL
1 2 DATA_0002 DATA_0002 5120 /dev/asmdisk4 NORMAL
2 0 FRA_0000 FRA_0000 5120 /dev/asmdisk3 NORMAL
db状态：
[oracle@rh-asm-37 ~]$ sqlplus / as sysdba SQL*Plus: Release 11.2.0.4.0 Production on Fri Aug 28 02:00:18 2015 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SQL> select INSTANCE_NAME,INSTANCE_NUMBER,VERSION,STATUS,ACTIVE_STATE from v$instance; INSTANCE_NAME INSTANCE_NUMBER VERSION STATUS ACTIVE_ST ---------------- --------------- ----------------- ------------ --------- rhnode 1 11.2.0.4.0 OPEN NORMAL
---The end---
