oracle 12c 之后修改rac 私网ip（FLEXASM）

客户网络调整需要更改私有网络，数据库版本为19c ，集群使用flex asm

查看是否使用flex asm

[+ASM2]@ht02[/home/grid]$asmcmd showclustermode
ASM cluster : Flex mode enabled - Direct Storage Access

更改前私网配置

?

1 2 3 4 5 6 7 8 9

[+ASM2]@ht02[/home/grid]$oifcfg getif enp0s8  192.168.20.0  global  public enp0s9  192.168.0.0  global  cluster_interconnect,asm [+ASM2]@ht02[/home/grid]$oifcfg iflist enp0s3  10.0.2.0 enp0s8  192.168.20.0 enp0s9  192.168.0.0 enp0s10  192.168.1.0 enp0s10  169.254.0.0

　

更改私有网络

?

1 2	/u01/app/grid/bin/oifcfg setif -global enp0s10/192.168.1.0:cluster_interconnect,asm /u01/app/grid/bin/oifcfg delif -global enp0s9

查看私网配置

?

1 2 3

[+ASM2]@ht02[/home/grid]$oifcfg getif enp0s8  192.168.20.0  global  public enp0s10  192.168.1.0  global  cluster_interconnect,asm

　　

重启主机或者重启集群后发现只有一个节点能启动，另一个节点一直卡在启动ora.storage

 

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

[+ASM1]@ht01[/home/grid]$crsctl stat res -t -init -------------------------------------------------------------------------------- Name           Target  State        Server                   State details       -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.asm       1        ONLINE  ONLINE       ht01                     STABLE ora.cluster_interconnect.haip       1        ONLINE  ONLINE       ht01                     STABLE ora.crf       1        ONLINE  ONLINE       ht01                     STABLE ora.crsd       1        ONLINE  OFFLINE                               STABLE ora.cssd       1        ONLINE  ONLINE       ht01                     STABLE ora.cssdmonitor       1        ONLINE  ONLINE       ht01                     STABLE ora.ctssd       1        ONLINE  ONLINE       ht01                     OBSERVER,STABLE ora.diskmon       1        OFFLINE OFFLINE                               STABLE ora.evmd       1        ONLINE  ONLINE       ht01                     STABLE ora.gipcd       1        ONLINE  ONLINE       ht01                     STABLE ora.gpnpd       1        ONLINE  ONLINE       ht01                     STABLE ora.mdnsd       1        ONLINE  ONLINE       ht01                     STABLE ora.storage       1        ONLINE  OFFLINE                               STABLE --------------------------------------------------------------------------------

　　

查看日志如下

+ View Code?

1 2 3

grid alert日志 2022-07-29 09:46:08.994+CRS-5006: Unable to automatically select a network interface which has subnet mask  and subnet number 192.168.0.0 . For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0/grid/diag/crs/ht01/crs/trace/crsd_orarootagent_root.trc".

crsd_orarootagent_root.trc
2022-07-29 09:45:58.166 :CLSDYNAM:723580672: [ora.asmnet1.asmnetwork]{1:4936:2} [check] clsnUtils::error Exception type=2 string=
CRS-5017: The resource action "ora.asmnet1.asmnetwork check" encountered the following error:
CRS-5006: Unable to automatically select a network interface which has subnet mask and subnet number 192.168.0.0
. For details refer to "(:CLSN00109:)" in "/u01/app/11.2.0/grid/diag/crs/ht01/crs/trace/crsd_orarootagent_root.trc"

查看asmnetwork配置

?

1 2 3 4 5 6 7

[+ASM2]@ht02[/home/grid]$srvctl config asmnetwork ASM network 1 exists Subnet IPv4: 192.168.0.0// Subnet IPv6: Network is enabled Network is individually enabled on nodes: Network is individually disabled on nodes:

 修改asmnetwork

?

1 2 3 4 5 6 7 8 9

[root@ht01 ~]# /u01/app/grid/bin/srvctl remove asmnetwork -netnum 1 -force [root@ht01 ~]# /u01/app/grid/bin/srvctl add  asmnetwork -subnet 192.168.1.0 [root@ht01 ~]# /u01/app/grid/bin/srvctl config asmnetwork ASM network 1 exists Subnet IPv4: 192.168.1.0// Subnet IPv6: Network is enabled Network is individually enabled on nodes: Network is individually disabled on nodes:

　在重启集群发现节点1ora.storage还是启动不起来，查看资源属性

?

1 2 3 4 5 6 7

[+ASM2]@ht02[/home/grid]$crsctl stat res -p|grep SUBNET SUBNET=192.168.0.0 REGISTRATION_INVITED_SUBNETS= REGISTRATION_INVITED_SUBNETS= REGISTRATION_INVITED_SUBNETS= USR_ORA_SUBNET=192.168.1.0 USR_ORA_SUBNET=192.168.20.0

　　集群里还是有192.168.0.0的网段，需要修改成192.168.1.0，此时alert日志报错如下

?

1 2 3 4 5 6 7 8 9 10

2019-08-13 19:48:27.006 : USRTHRD:893126400: {0:5:3} 8154 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS   2019-08-13 19:48:27.015 : USRTHRD:893126400: {0:5:3} 8154 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS   2019-08-13 19:48:27.084 : CLSCRED:893126400: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.303a14fb7cd04f7affd9f5ed7ebdcd83.root not found 2019-08-13 19:48:27.084 : USRTHRD:893126400: {0:5:3} 7872 Error 4 opening dom root in 0x7fd00820c3d0   2019-08-13 19:48:30.691 : USRTHRD:893126400: {0:5:3} ORA-12547: TNS:lost contact ORA-12547: TNS:lost contact ORA-15077: could not locate ASM instance serving a required diskgroup

　　需要修改asm监听，查看当前asm监听

?

1 2 3 4 5 6 7 8 9 10

[+ASM1]@ht01[/home/grid]$srvctl config listener -asmlistener Name: ASMNET1LSNR_ASM Type: ASM Listener Owner: grid Subnet: 192.168.0.0 Home: <CRS home> End points: TCP:1525 Listener is enabled. Listener is individually enabled on nodes: Listener is individually disabled on nodes:

　　修改asm监听及查看

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14

[+ASM2]@ht02[/home/grid]$srvctl update listener -listener ASMNET1LSNR_ASM -asm -remove -force [+ASM2]@ht02[/home/grid]$srvctl config listener -asmlistener PRCN-2044 : No listener exists [+ASM1]@ht01[/home/grid]$srvctl add listener -listener ASMNET1LSNR_ASM -oraclehome $ORACLE_HOME -asmlistener -netnum 1 [+ASM1]@ht01[/home/grid]$srvctl config listener -asmlistener Name: ASMNET1LSNR_ASM Type: ASM Listener Owner: grid Subnet: 192.168.1.0 Home: <CRS home> End points: TCP:1525 Listener is enabled. Listener is individually enabled on nodes: Listener is individually disabled on nodes:

　　查看集群属性

?

1 2 3 4 5 6 7

[+ASM2]@ht02[/home/grid]$crsctl stat res -p|grep SUBNET SUBNET=192.168.1.0 REGISTRATION_INVITED_SUBNETS= REGISTRATION_INVITED_SUBNETS= REGISTRATION_INVITED_SUBNETS= USR_ORA_SUBNET=192.168.1.0 USR_ORA_SUBNET=192.168.20.0

　　重启grid，正常启动

查看asm监听

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

[+ASM2]@ht02[/home/grid]$lsnrctl status ASMNET1LSNR_ASM   LSNRCTL for Linux: Version 19.0.0.0.0 - Production on 29-JUL-2022 15:13:14   Copyright (c) 1991, 2019, Oracle.  All rights reserved.   Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=ASMNET1LSNR_ASM))) STATUS of the LISTENER ------------------------ Alias                     ASMNET1LSNR_ASM Version                   TNSLSNR for Linux: Version 19.0.0.0.0 - Production Start Date                29-JUL-2022 11:33:03 Uptime                    0 days 3 hr. 40 min. 11 sec Trace Level               off Security                  ON: Local OS Authentication SNMP                      OFF Listener Parameter File   /u01/app/grid/network/admin/listener.ora Listener Log File         /u01/app/11.2.0/grid/diag/tnslsnr/ht02/asmnet1lsnr_asm/alert/log.xml Listening Endpoints Summary...   (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=ASMNET1LSNR_ASM)))   (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.11)(PORT=1525))) Services Summary... Service "+ASM" has 2 instance(s).   Instance "+ASM1", status READY, has 1 handler(s) for this service...   Instance "+ASM2", status READY, has 2 handler(s) for this service... Service "+ASM_ARCHDG" has 2 instance(s).   Instance "+ASM1", status READY, has 1 handler(s) for this service...   Instance "+ASM2", status READY, has 2 handler(s) for this service... Service "+ASM_DATADG" has 2 instance(s).   Instance "+ASM1", status READY, has 1 handler(s) for this service...   Instance "+ASM2", status READY, has 2 handler(s) for this service... Service "+ASM_OCR" has 2 instance(s).   Instance "+ASM1", status READY, has 1 handler(s) for this service...   Instance "+ASM2", status READY, has 2 handler(s) for this service...

　　总结：12c以后修改私网ip，还须修改asmnetwork、ASMNET1LSNR_ASM，参考mos How to Modify ASM Network And ASM Listener Information in Grid Infrastructure (Doc ID 2856306.1)