RAC（四）-配置linux环境
以下脚本操作较多，不再频繁截图，仅贴出操作脚本供参考。
另，所有操作如非特别注明，均为root用户(注意看操作符前缀)，建议通过ssh远程操作。
此步骤最麻烦是配置RAC的关键
一、配置第一台节点，
避免两台一起配置，我们可以先配置一台部分，然后复制成另外一台虚拟机，然后修改部分参数就行。
1、修改虚拟机配置
在配置linux系统之前，首先到你的虚拟机工作目录，打开.vmx文件，添加下列内容：
disk.locking = "false"
diskLib.dataCacheMaxSize = "0"
diskLib.dataCacheMaxReadAheadSize = "0"
diskLib.DataCacheMinReadAheadSize = "0"
diskLib.dataCachePageSize = "4096"
diskLib.maxUnsyncedWrites = "0"
scsi1:1.deviceType = "disk"
scsi1:2.deviceType = "disk"
scsi1:3.deviceType = "disk"
scsi1:4.deviceType = "disk"
提示：如果你创建有磁盘有多个，注意要按照你的实际情况修改。
2、编辑网络配置文件eth0,rth1，将物理网卡地址注销，避免同一个硬件地址的虚拟机同事存在
[root@rac1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=static
BROADCAST=10.20.141.255
#HWADDR=00:0C:29:62:19:AC
IPADDR=10.20.141.111
NETMASK=255.255.255.0
NETWORK=10.20.141.0
ONBOOT=yes
3、编辑/etc/hosts文件
[root@rac1 ~]# vi /etc/hosts
增加如下内容：
127.0.0.1 localhost.localdomain localhost
10.20.141.111 rac1.hoperun.com rac1
192.168.0.111 rac1-priv.hoperun.com rac1-priv
10.20.141.116 rac1-vip.hoperun.com rac1-vip
10.20.141.112 rac2.hoperun.com rac2
192.168.0.112 rac2-priv.hoperun.com rac2-priv
10.20.141.117 rac2-vip.hoperun.com rac2-vip
提示：192.168.0.116,117为指定的虚IP，供CRS使用，不需要我们配置。
4、创建组oinstall,oinstall,用户oracle
[root@rac1 ~]# groupadd oinstall
[root@rac1 ~]# groupadd oinstall
[root@rac1 ~]# useradd -d /home/oracle -g oinstall -G oinstall oracle
设置oracle用户密码
[root@rac1 ~]# passwd oracle
Changing password for user oracle.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.
提示：
[root@rac1 ~]# id oracle
uid=500(oracle) gid=500(oinstall) groups=500(oinstall),501(oinstall)
如果是非虚机安装，要注意记录下用户和组ID，在其它机器上创建同名组时，务必确认组ID和用户ID与此相同。
5、修改oracle用户的初始化参数文件
[root@rac1 ~]# vi /home/oracle/.bash_profile
增加如下内容：
export TMP=/tmp
export TMPDIR=$TMP
export ORACLE_BASE=/opt/ora10g
export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1
export ORACLE_SID=RACDB1
export ORACLE_TERM=xterm
export PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/bin:/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin/
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
ulimit -u 16384 -n 65536
umask 022
此处注意oracle_sid的名称要与后续创建数据库时指定的sid相同（注意大小写），不然你每次想通过本机登录的方式进入oracle都得重新export ORACLE_SID了。
6、创建/opt/ora10g目录，要注意该目录所有者或oracle用户的权限
[root@rac1 ~]# mkdir /opt/ora10g
由于该目录主要由oracle用户操作，因此我们将其所有者改为oracle
[root@rac1 ~]# chown oracle.oinstall /opt/ora10g
7、配置内核参数
[root@rac1 ~]# vi /etc/sysctl.conf
增加或修改下列内容
kernel.shmall = 2097152
kernel.shmmax = 536870912
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
fs.file-max = 65536
net.ipv4.ip_local_port_range = 1024 65000
net.core.rmem_default = 1048576
net.core.rmem_max = 1048576
net.core.wmem_default = 262144
net.core.wmem_max = 262144
提示，上述参数中通常只有一项需要我们更改，即kernel.shmmax，该参数推荐设定为物理内存的一半，由于安装crs的时候要求内存至少512M，因此我们此处也按照512*1024*1024来设置。
然后：
[root@rac1 Server]# sysctl -p
让设置生效
8、提高 Oracle 用户的 shell 限制
设置oracle使用的文件数权限
[root@rac1 ~]# vi /etc/security/limits.conf
增加下列内容
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
修改安全限制
[root@rac1 ~]# vi /etc/pam.d/login
增加：
session required /lib/security/pam_limits.so
配置Hangcheck计时器
[root@rac1 ~]# vi /etc/rc.local
增加：
modprobe hangcheck-timer hangcheck-tick=30 hangcheck_margin=180
9、安装 oracleasmlib 程序包
所需的程序包可以到这里下载：
http://www.oracle.com/technology/software/tech/linux/asmlib/rhel5.html
http://oss.oracle.com/projects/compat-oracle/files/Enterprise_Linux/
注意一定要与操作系统版本相符。
包还真不少，本着宁可无用，不能无有的原则，能下到的统统装上
俺装的包有下列：
[root@rac1 rhel5]# ls -l
total 225376
-rwxr--r-- 1 oracle oinstall 410476 May 29 15:16 compat-binutils215-2.15.92.0.2-24.i386.rpm
-rwxr--r-- 1 oracle oinstall 4256 May 29 15:16 compat-libcwait-2.1-1.i386.rpm
-rwxr--r-- 1 oracle oinstall 88787 May 29 15:17 compat-libstdc++-egcs-1.1.2-1.i386.rpm
-rwxr--r-- 1 oracle oinstall 3840 May 29 15:17 compat-oracle-el5-1.0-5.i386.rpm
-rwxr--r-- 1 oracle oinstall 1079629 May 29 15:17 openmotif21-2.1.30-11.EL5.i386.rpm
-rwxr--r-- 1 oracle oinstall 122314 May 29 15:17 openmotif21-debuginfo-2.1.30-11.EL5.i386.rpm
-rwxr--r-- 1 oracle oinstall 125313 May 30 15:02 oracleasm-2.6.18-53.el5-2.0.4-1.el5.i686.rpm
-rwxr--r-- 1 oracle oinstall 126085 May 30 15:02 oracleasm-2.6.18-53.el5debug-2.0.4-1.el5.i686.rpm
-rwxr--r-- 1 oracle oinstall 125927 May 30 15:02 oracleasm-2.6.18-53.el5PAE-2.0.4-1.el5.i686.rpm
-rwxr--r-- 1 oracle oinstall 123346 May 30 15:02 oracleasm-2.6.18-53.el5xen-2.0.4-1.el5.i686.rpm
-rwxr--r-- 1 oracle oinstall 13658 May 30 15:13 oracleasmlib-2.0.3-1.el5.i386.rpm
-rwxr--r-- 1 oracle oinstall 22936 May 30 15:13 oracleasm-support-2.0.4-1.el5.i386.rpm
-rwxr--r-- 1 oracle oinstall 10662 May 29 15:17 xorg-x11-libs-compat-6.8.2-1.EL.33.0.1.i386.rpm
安装示例：
[root@rac1 rhel5]# rpm -ivh compat-binutils215-2.15.92.0.2-24.i386.rpm
Preparing... ########################################### [100%]
1:compat-binutils215 ########################################### [100%]
提示，如果在安装过程中提示你缺少其它包，你可以先到操作系统安装光盘中寻找并安装所需系统包，然后再来安装下载到的包。
二、复制RAC1虚拟机到RAC2、添加共享存储、修改RAC2配置：
10、RAC1复制和添加共享存储需要太多截图，详情请见：http://www.cnblogs.com/rhino1030/archive/2011/11/26/2264031.html
11、修改RAC2的主机名：
vim /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=rac2.hoperun.com
GATEWAY=10.20.141.252
12、修改RAC2的eth0 eth1的网卡IP地址：
vim /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=static
BROADCAST=10.20.141.255
#HWADDR=00:0C:29:62:19:AC
IPADDR=10.20.141.112
NETMASK=255.255.255.0
NETWORK=10.20.141.0
ONBOOT=yes
同理将eth1网卡IP修改为：
IPADDR=192.168.0.111
三、配置虚拟机RAC1、RAC2
13、在RAC1上面 磁盘分区
例如:
[root@rac1 ~]# fdisk /dev/sdb
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-102, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-102, default 102):
Using default value 102
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
需要你输入的依次是"n/p/1/回车/回车/w"。
就本例而言，需要我们进行分区的有4个：sdb,sdc,sde,sdf
全分完后fdisk -l看一下，应该是这种形式：
[root@rac1 ~]# fdisk -l
Disk /dev/sda: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 166 1228972+ 82 Linux swap / Solaris
/dev/sda3 167 1044 7052535 83 Linux
Disk /dev/sdb: 107 MB, 107374080 bytes
64 heads, 32 sectors/track, 102 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 102 104432 83 Linux
Disk /dev/sdc: 322 MB, 322122240 bytes
64 heads, 32 sectors/track, 307 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 1 307 314352 83 Linux
Disk /dev/sdd: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 1 261 2096451 83 Linux
Disk /dev/sde: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sde1 1 261 2096451 83 Linux
14、在RAC1、RAC2上 配置裸设备
[root@rac1 ~]# vi /etc/sysconfig/rawdevices
添加
/dev/raw/raw1 /dev/sdb1
/dev/raw/raw2 /dev/sdc1
/dev/raw/raw3 /dev/sdd1
/dev/raw/raw4 /dev/sde1
15、在RAC1、RAC2上面添加RAW 映射配置
由于RHEL5中取消了rawdevices，如果不想将映射命令放到rc.local中的话，我们就需要将其配置到/etc/udev/rules.d/文件夹中
修改文件
[root@rac1 ~]# vi /etc/udev/rules.d/60-raw.rules
增加如下内容：
ACTION=="add", KERNEL=="/dev/sdb1",RUN+="/bin/raw /dev/raw/raw1 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="17",RUN+="/bin/raw /dev/raw/raw1 %M %m"
ACTION=="add", KERNEL=="/dev/sdc1",RUN+="/bin/raw /dev/raw/raw2 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="33",RUN+="/bin/raw /dev/raw/raw2 %M %m"
ACTION=="add", KERNEL=="/dev/sdd1",RUN+="/bin/raw /dev/raw/raw3 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="49",RUN+="/bin/raw /dev/raw/raw3 %M %m"
ACTION=="add", KERNEL=="/dev/sde1",RUN+="/bin/raw /dev/raw/raw4 %N"
ACTION=="add", ENV{MAJOR}=="8",ENV{MINOR}=="65",RUN+="/bin/raw /dev/raw/raw4 %M %m"
KERNEL=="raw[1-4]", OWNER="oracle", GROUP="oinstall", MODE="640"
提示：要根据你的实际情况来配置
另外关于此处的配置有一点非常奇怪，我看到red hat官方文档中关于raw.rule的示例中说KERNEL==..或ENV{MAJOR}...只需要任意配置一个就可以，但我在具体配置过程中发现百试不爽，我尝试了各种组合发现都不生效，必须二者同时配置才可以，难道是俺用的linux内核版本太低了？百思不得其解，如有达人已深究其内幕，望助我解惑~~~~
16、检查RAC1、RAC2裸设备是否配置成功：
重启下服务：
[root@rac1 ~]# start_udev
Starting udev: [ OK ]
验证一下
[root@rac1 ~]# ls /dev/raw/ -l
total 0
crw-r----- 1 oracle oinstall 162, 1 Jun 6 17:57 raw1
crw-r----- 1 oracle oinstall 162, 2 Jun 6 17:57 raw2
crw-r----- 1 oracle oinstall 162, 3 Jun 6 17:57 raw3
crw-r----- 1 oracle oinstall 162, 4 Jun 6 17:57 raw4
确认RAC1、RAC2裸设备配置成功
[root@RAC1 ~]# service rawdevices restart
Assigning devices:
/dev/raw/raw1 --> /dev/sdb1
/dev/raw/raw1: bound to major 8, minor 17
/dev/raw/raw2 --> /dev/sdc1
/dev/raw/raw2: bound to major 8, minor 33
/dev/raw/raw3 --> /dev/sdd1
/dev/raw/raw3: bound to major 8, minor 49
/dev/raw/raw4 --> /dev/sde1
/dev/raw/raw4: bound to major 8, minor 65
done
可能RAC2会不能显示，重启一下RAC2就行
17、给RAC1、RAC2配置裸设备权限
[root@RAC1 disk]# chown oracle:oinstall /dev/raw/raw*
[root@RAC1 ~]# ll /dev/raw/*
crw------- 1 oracle oinstall 162, 1 Mar 17 23:12 /dev/raw/raw1
crw------- 1 oracle oinstall 162, 2 Mar 17 23:12 /dev/raw/raw2
crw------- 1 oracle oinstall 162, 3 Mar 17 23:12 /dev/raw/raw3
crw------- 1 oracle oinstall 162, 4 Mar 17 23:12 /dev/raw/raw4
不知道为什么，每次重启之后络设备的权限都会修改成root组的root用户权限，为了rac正常使用，我们就将chown oracle:oinstall /dev/raw/raw*添加到开机脚本中。
[root@RAC1 disk]# vim /etc/rc.d/rc.local
在里面添加
chown oracle:oinstall /dev/raw/raw*
18、配置RAC1、RAC2时间同步
#此步骤需要分别配置
此处将RAC-1作为时间服务器
[root@RAC1 ~]# vi /etc/ntp.conf
添加
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
driftfile /var/lib/ntp/drift
broadcastdelay 0.008
RAC-2指向RAC-1
[root@RAC2 ~]# vi /etc/ntp.conf
添加
server 192.168.1.11 # local clock
#fudge 127.127.1.0 stratum 10
driftfile /var/lib/ntp/drift
broadcastdelay 0.008
启动ntp
[root@RAC1 /]# service ntpd start
Starting ntpd: [ OK ]
#可以ntsysv配置ntpd开机自动启动
[root@RAC1 /] chkconfig ntpd on
19、RAC1创建ASM
#此步骤需要分别执行.
[root@RAC1 /]# /etc/init.d/oracleasm createdisk ASM1 /dev/sdb1
Marking disk "ASM1" as an ASM disk: [ OK ]
[root@RAC1 /]# /etc/init.d/oracleasm createdisk ASM2 /dev/sdc1
Marking disk "ASM2" as an ASM disk: [ OK ]
[root@RAC1 ~]# /etc/init.d/oracleasm createdisk ASM3 /dev/sdd1
Marking disk "ASM3" as an ASM disk: [ OK ]
[root@RAC1 ~]# /etc/init.d/oracleasm createdisk ASM4 /dev/sde1
Marking disk "ASM4" as an ASM disk: [ OK ]
20、RAC1、RAC2扫描
[root@rac2 ~]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks: [ OK ]
[root@rac2 ~]# /etc/init.d/oracleasm listdisks
ASM1
ASM2
ASM3
ASM4
21、设置RAC1和RAC2下oracle用户的互信机制。
在clusterware (CRS) 和 Rac Database安装过程中，Oracle Universal Installer (OUI) 必须能够以 oracle 的身份自动将软件复制到所有 RAC 节点。这里我们通过配置ssh 让oracle用户拥有不输入密码即可访问各节点的能力。
首先两个节点间互ping，要确保都是通的
[root@rac1 ~]# ping 192.168.0.111
[root@rac1 ~]# ping 10.20.141.111
要建立用户等效性，需要在两个节点上以 oracle 用户身份生成用户的公钥和私钥，首先在RAC1 执行：
[root@rac1 opt]# su - oracle
[oracle@rac1 ~]$ mkdir ~/.ssh
[oracle@rac1 ~]$ chmod 700 ~/.ssh
[oracle@rac1 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
d2:69:eb:ac:86:62:27:50:99:ff:e8:1e:a2:e6:5d:7f oracle@rac1
[oracle@rac1 ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
0a:9a:20:46:a2:28:ec:72:23:82:f2:9d:f8:62:9b:d1 oracle@rac1
然后换RAC2执行
[root@rac2 ~]# ping 192.168.100.102
[root@rac2 ~]# ping 10.10.17.222
[root@rac2 opt]# su - oracle
[oracle@rac2 ~]$ mkdir ~/.ssh
[oracle@rac2 ~]$ chmod 700 ~/.ssh
[oracle@rac2 ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
dd:be:7a:37:e4:b5:f0:b2:24:95:50:61:ea:a1:61:07 oracle@rac2
[oracle@rac2 ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
bc:b5:cb:43:c7:19:53:d6:f7:16:69:85:12:7f:aa:be oracle@rac2
切换回rac1，接着执行：
[oracle@rac1 ~]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[oracle@rac1 ~]$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
提示：下列命令会提示你输入rac2的oracle密码，按照提示输入即可，如果失败可重新尝试执行命令。
[oracle@rac1 ~]$ ssh rac2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
The authenticity of host 'rac2 (192.168.100.102)' can't be established.
RSA key fingerprint is 92:d1:ce:5b:c8:a1:52:d5:ac:00:5f:48:5d:12:06:e4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'rac2,192.168.100.102' (RSA) to the list of known hosts.
oracle@rac2's password:
[oracle@rac1 ~]$ ssh rac2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
oracle@rac2's password:
[oracle@rac1 ~]$ scp ~/.ssh/authorized_keys rac2:~/.ssh/authorized_keys
oracle@rac2's password:
authorized_keys 100% 1992 2.0KB/s 00:00
两机相互执行，看看是否还需要输入密码
[oracle@rac1 ~]$ ssh rac1 date
[oracle@rac1 ~]$ ssh rac2 date
[oracle@rac1 ~]$ ssh rac1-priv date
[oracle@rac1 ~]$ ssh rac2-priv date
[oracle@rac1 ~]$ ssh rac1.hoperun.com date
[oracle@rac1 ~]$ ssh rac2.hoperun.com date
[oracle@rac1 ~]$ ssh rac1-priv.hoperun.com date
[oracle@rac1 ~]$ ssh rac2-priv.hoperun.com date
切换至rac2执行
[oracle@rac2 ~]$ ssh rac1 date
[oracle@rac2 ~]$ ssh rac2 date
[oracle@rac2 ~]$ ssh rac1-priv date
[oracle@rac2 ~]$ ssh rac2-priv date
[oracle@rac2 ~]$ ssh rac1.hoperun.com date
[oracle@rac2 ~]$ ssh rac2.hoperun.com date
[oracle@rac2 ~]$ ssh rac1-priv.hoperun.com date
[oracle@rac2 ~]$ ssh rac2-priv.hoperun.com date
好了RAC1和RAC2的环境设置好了，可以安装RAC的集群组建了
