asm使用不正确的密码文件导致无法自动启动asm

环境:

OS:Centos 7

DB:18.3.0.0

 

在做2节点rac到2节点rac的dataguard的时候,第一次配置完成后,同步是正常的
但是机器重启后,发现一个实例无法自动启动asm(手工是可以的),查看报错的信息:

复制代码
Trace file /u01/grid/grid_base/diag/crs/rac01/crs/trace/ohasd_orarootagent_root.trc
Oracle Database 18c Clusterware Release 18.0.0.0.0 - Production
Version 18.3.0.0.0 Copyright 1996, 2018 Oracle. All rights reserved.

*** TRACE CONTINUED FROM FILE /u01/grid/grid_base/diag/crs/rac01/crs/trace/ohasd_orarootagent_root_2.trc ***

2021-07-22 05:44:36.243 : default:3567097600: Inited LSF context: 0x7fbbb02abd00 
2021-07-22 05:44:36.248 : CLSCRED:3567097600: clsCredCommonInit: Inited singleton credctx.
2021-07-22 05:44:36.248 : CLSCRED:3567097600: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2021-07-22 05:44:36.302 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2021-07-22 05:44:36.308 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2021-07-22 05:44:36.378 : CLSCRED:3567097600: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.31cb43e9af1ccfa9ffef58465be2c675.root not found
2021-07-22 05:44:36.379 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8626 Error 4 opening dom root in 0x7fbbb0125830

2021-07-22 05:44:36.999 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:37.799 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:37.990 : default:3567097600: clsCredDomClose: Credctx deleted 0x7fbbb0205c30
2021-07-22 05:44:38.019 :   CLSNS:3567097600: clsns_SetTraceLevel:trace level set to 1.
2021-07-22 05:44:38.033 : default:3567097600: Inited LSF context: 0x7fbbb02acd80 
2021-07-22 05:44:38.038 : CLSCRED:3567097600: clsCredCommonInit: Inited singleton credctx.
2021-07-22 05:44:38.038 : CLSCRED:3567097600: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2021-07-22 05:44:38.088 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2021-07-22 05:44:38.096 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2021-07-22 05:44:38.179 : CLSCRED:3567097600: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.31cb43e9af1ccfa9ffef58465be2c675.root not found
2021-07-22 05:44:38.179 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8626 Error 4 opening dom root in 0x7fbbb016fb70

2021-07-22 05:44:38.601 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:39.401 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:39.705 : default:3567097600: clsCredDomClose: Credctx deleted 0x7fbbb046b980
2021-07-22 05:44:39.736 :   CLSNS:3567097600: clsns_SetTraceLevel:trace level set to 1.
2021-07-22 05:44:39.750 : default:3567097600: Inited LSF context: 0x7fbbb008f4d0 
2021-07-22 05:44:39.754 : CLSCRED:3567097600: clsCredCommonInit: Inited singleton credctx.
2021-07-22 05:44:39.754 : CLSCRED:3567097600: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2021-07-22 05:44:39.809 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2021-07-22 05:44:39.820 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2021-07-22 05:44:39.904 : CLSCRED:3567097600: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.31cb43e9af1ccfa9ffef58465be2c675.root not found
2021-07-22 05:44:39.904 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8626 Error 4 opening dom root in 0x7fbbb03d9110

2021-07-22 05:44:40.202 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:40.286 :CLSDYNAM:3577603840: [ ora.crf]{0:5:3} [check] Check return = 0, state detail = NULL
2021-07-22 05:44:40.957 : USRTHRD:3082802944:  HAIP: event GIPCD_METRIC_UPDATE_HAIP
2021-07-22 05:44:40.957 : USRTHRD:3084904192: {0:5:3} Thread:[NetHAMain] this event enp0s8~rac02~99;enp0s8~rac01~99 internal debug only
2021-07-22 05:44:40.957 : USRTHRD:3084904192: {0:5:3} Thread:[NetHAMain] HAIP: inf enp0s8, nodeName: rac02, rank 99 (val 99)
2021-07-22 05:44:40.957 : USRTHRD:3084904192: {0:5:3} Thread:[NetHAMain] HAIP: inf enp0s8, nodeName: rac01, rank 99 (val 99)
2021-07-22 05:44:41.003 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:41.553 : default:3567097600: clsCredDomClose: Credctx deleted 0x7fbbb046b980
2021-07-22 05:44:41.553 : USRTHRD:3567097600: {0:4:146} [ora.storage] -- trace dump on error exit --

2021-07-22 05:44:41.553 : USRTHRD:3567097600: {0:4:146} [ora.storage] Error [kgfoAl06] in [kgfokge] at kgfo.c:3112

2021-07-22 05:44:41.553 : USRTHRD:3567097600: {0:4:146} [ora.storage] ORA-01017: invalid username/password; logon denied


2021-07-22 05:44:41.553 : USRTHRD:3567097600: {0:4:146} [ora.storage] Category: 7

2021-07-22 05:44:41.553 : USRTHRD:3567097600: {0:4:146} [ora.storage] DepInfo: 1017

2021-07-22 05:44:41.554 : USRTHRD:3567097600: {0:4:146} [ora.storage] -- trace dump end --

2021-07-22 05:44:41.554 : USRTHRD:3567097600: {0:4:146} Thread:kgfoCheckMountExt isRunning is reset to false here
2021-07-22 05:44:41.554 : USRTHRD:3567097600: {0:4:146} Thread:kgfoCheckMountExt isFinished set to true
2021-07-22 05:44:41.555 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:41.555 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask(kgfoCheckMountExt) executed in  5 seconds
2021-07-22 05:44:41.555 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] StorageAgent::parsekgforetcodes retcode = 7, kgfoCheckMount(OCR), flag 2
2021-07-22 05:44:41.555 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] (null) category: 7, operation: kgfoAl06, loc: kgfokge, OS error: 1017, other: ORA-01017: invalid username/password; logon denied

2021-07-22 05:44:41.555 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] StorageAgent::check kgfo returncode 1
2021-07-22 05:44:41.555 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] (:CLSN00140:)StorageAgent::parsekgforretcodes OCR dgName OCR state 1
2021-07-22 05:44:41.555 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] waiting for check to not return PARTIALor UNPLANNED_OFFLINE 1
2021-07-22 05:44:42.555 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] StorageAgent::check NODEROLE_HUB getOCRdetails
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt thread constructor exit this:c427d050 m_pThnd:0 m_thndMX:c427d070, m_tintMX:c427d0c0 &m_postMX:0x7fbbc427d0a0
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} TaskletObjManager::manageTasklets START
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt thread destructor entry { this:0x7fbbc47934e0 m_pThnd:0x7fbbc435cf10 m_thndMX:0x7fbbc4793500, m_tintMX:0x7fbbc4793550 &m_postMX:0x7fbbc4793530
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt thread destructor stop
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt stop { m_pThnd:0x7fbbc435cf10 tid:3567097600
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread stop:thread acquire m_pThnd:0x7fbbc435cf10 m_thndMX:0x7fbbc4793500
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread stop:thread sltstjn
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt stop release m_thndMX:0x7fbbc4793500 m_tid:3567097600}
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt thread destructor m_pThnd:(nil) exit }
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} TaskletObjManager::manageTasklets END
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread::start { acquire thndMX:c427d070
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread::start spawn pThnd:0x7fbbc46f7230 thndType:1
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread::start thread spawned tid:3567097600
2021-07-22 05:44:42.558 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread::start spawned release thndMX:c427d070 }
2021-07-22 05:44:42.587 :   CLSNS:3567097600: clsns_SetTraceLevel:trace level set to 1.
2021-07-22 05:44:42.602 : default:3567097600: Inited LSF context: 0x7fbbbc05d920 
2021-07-22 05:44:42.607 : CLSCRED:3567097600: clsCredCommonInit: Inited singleton credctx.
2021-07-22 05:44:42.607 : CLSCRED:3567097600: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2021-07-22 05:44:42.661 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2021-07-22 05:44:42.669 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2021-07-22 05:44:42.733 : CLSCRED:3567097600: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.31cb43e9af1ccfa9ffef58465be2c675.root not found
2021-07-22 05:44:42.733 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8626 Error 4 opening dom root in 0x7fbbbc034940

2021-07-22 05:44:43.359 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:44.159 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:44.361 : default:3567097600: clsCredDomClose: Credctx deleted 0x7fbbbc15e3b0
2021-07-22 05:44:44.396 :   CLSNS:3567097600: clsns_SetTraceLevel:trace level set to 1.
2021-07-22 05:44:44.407 : default:3567097600: Inited LSF context: 0x7fbbbc250220 
2021-07-22 05:44:44.417 : CLSCRED:3567097600: clsCredCommonInit: Inited singleton credctx.
2021-07-22 05:44:44.417 : CLSCRED:3567097600: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2021-07-22 05:44:44.447 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2021-07-22 05:44:44.454 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2021-07-22 05:44:44.507 : CLSCRED:3567097600: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.31cb43e9af1ccfa9ffef58465be2c675.root not found
2021-07-22 05:44:44.508 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8626 Error 4 opening dom root in 0x7fbbbc225180

2021-07-22 05:44:44.959 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:45.760 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:45.961 : USRTHRD:3082802944:  HAIP: event GIPCD_METRIC_UPDATE_HAIP
2021-07-22 05:44:45.961 : USRTHRD:3084904192: {0:5:3} Thread:[NetHAMain] this event enp0s8~rac01~99;enp0s8~rac02~99 internal debug only
2021-07-22 05:44:45.961 : USRTHRD:3084904192: {0:5:3} Thread:[NetHAMain] HAIP: inf enp0s8, nodeName: rac01, rank 99 (val 99)
2021-07-22 05:44:45.961 : USRTHRD:3084904192: {0:5:3} Thread:[NetHAMain] HAIP: inf enp0s8, nodeName: rac02, rank 99 (val 99)
2021-07-22 05:44:46.183 : default:3567097600: clsCredDomClose: Credctx deleted 0x7fbbbc3068a0
2021-07-22 05:44:46.216 :   CLSNS:3567097600: clsns_SetTraceLevel:trace level set to 1.
2021-07-22 05:44:46.227 : default:3567097600: Inited LSF context: 0x7fbbbc25dc30 
2021-07-22 05:44:46.233 : CLSCRED:3567097600: clsCredCommonInit: Inited singleton credctx.
2021-07-22 05:44:46.233 : CLSCRED:3567097600: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2021-07-22 05:44:46.288 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2021-07-22 05:44:46.294 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2021-07-22 05:44:46.371 : CLSCRED:3567097600: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.31cb43e9af1ccfa9ffef58465be2c675.root not found
2021-07-22 05:44:46.371 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8626 Error 4 opening dom root in 0x7fbbbc224f10

2021-07-22 05:44:46.561 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:47.361 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:47.999 : default:3567097600: clsCredDomClose: Credctx deleted 0x7fbbbc3068a0
2021-07-22 05:44:47.999 : USRTHRD:3567097600: {0:4:146} [ora.storage] -- trace dump on error exit --

2021-07-22 05:44:47.999 : USRTHRD:3567097600: {0:4:146} [ora.storage] Error [kgfoAl06] in [kgfokge] at kgfo.c:3112

2021-07-22 05:44:47.999 : USRTHRD:3567097600: {0:4:146} [ora.storage] ORA-01017: invalid username/password; logon denied


2021-07-22 05:44:47.999 : USRTHRD:3567097600: {0:4:146} [ora.storage] Category: 7

2021-07-22 05:44:47.999 : USRTHRD:3567097600: {0:4:146} [ora.storage] DepInfo: 1017

2021-07-22 05:44:48.000 : USRTHRD:3567097600: {0:4:146} [ora.storage] -- trace dump end --

2021-07-22 05:44:48.000 : USRTHRD:3567097600: {0:4:146} Thread:kgfoCheckMountExt isRunning is reset to false here
2021-07-22 05:44:48.000 : USRTHRD:3567097600: {0:4:146} Thread:kgfoCheckMountExt isFinished set to true
2021-07-22 05:44:48.000 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:48.000 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask(kgfoCheckMountExt) executed in  6 seconds
2021-07-22 05:44:48.000 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] StorageAgent::parsekgforetcodes retcode = 7, kgfoCheckMount(OCR), flag 2
2021-07-22 05:44:48.000 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] (null) category: 7, operation: kgfoAl06, loc: kgfokge, OS error: 1017, other: ORA-01017: invalid username/password; logon denied

2021-07-22 05:44:48.000 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] StorageAgent::check kgfo returncode 1
2021-07-22 05:44:48.000 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] (:CLSN00140:)StorageAgent::parsekgforretcodes OCR dgName OCR state 1
2021-07-22 05:44:48.001 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] waiting for check to not return PARTIALor UNPLANNED_OFFLINE 1
2021-07-22 05:44:49.001 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] StorageAgent::check NODEROLE_HUB getOCRdetails
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt thread constructor exit this:c47934e0 m_pThnd:0 m_thndMX:c4793500, m_tintMX:c4793550 &m_postMX:0x7fbbc4793530
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} TaskletObjManager::manageTasklets START
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt thread destructor entry { this:0x7fbbc427d050 m_pThnd:0x7fbbc46f7230 m_thndMX:0x7fbbc427d070, m_tintMX:0x7fbbc427d0c0 &m_postMX:0x7fbbc427d0a0
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt thread destructor stop
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt stop { m_pThnd:0x7fbbc46f7230 tid:3567097600
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread stop:thread acquire m_pThnd:0x7fbbc46f7230 m_thndMX:0x7fbbc427d070
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread stop:thread sltstjn
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt stop release m_thndMX:0x7fbbc427d070 m_tid:3567097600}
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt thread destructor m_pThnd:(nil) exit }
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} TaskletObjManager::manageTasklets END
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread::start { acquire thndMX:c4793500
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread::start spawn pThnd:0x7fbbc46895f0 thndType:1
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread::start thread spawned tid:3567097600
2021-07-22 05:44:49.010 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Thread::start spawned release thndMX:c4793500 }
2021-07-22 05:44:49.041 :   CLSNS:3567097600: clsns_SetTraceLevel:trace level set to 1.
2021-07-22 05:44:49.057 : default:3567097600: Inited LSF context: 0x7fbbb81d7500 
2021-07-22 05:44:49.062 : CLSCRED:3567097600: clsCredCommonInit: Inited singleton credctx.
2021-07-22 05:44:49.062 : CLSCRED:3567097600: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2021-07-22 05:44:49.114 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2021-07-22 05:44:49.123 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2021-07-22 05:44:49.214 : CLSCRED:3567097600: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.31cb43e9af1ccfa9ffef58465be2c675.root not found
2021-07-22 05:44:49.214 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8626 Error 4 opening dom root in 0x7fbbb84366f0

2021-07-22 05:44:49.810 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:50.611 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:50.744 :    AGFW:3575502592: {0:0:2} Agent received the message: AGENT_HB[Engine] ID 12293:7194
2021-07-22 05:44:50.872 : default:3567097600: clsCredDomClose: Credctx deleted 0x7fbbb8378aa0
2021-07-22 05:44:50.901 :   CLSNS:3567097600: clsns_SetTraceLevel:trace level set to 1.
2021-07-22 05:44:50.916 : default:3567097600: Inited LSF context: 0x7fbbb8535d90 
2021-07-22 05:44:50.921 : CLSCRED:3567097600: clsCredCommonInit: Inited singleton credctx.
2021-07-22 05:44:50.921 : CLSCRED:3567097600: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2021-07-22 05:44:50.966 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2021-07-22 05:44:50.973 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2021-07-22 05:44:51.033 : CLSCRED:3567097600: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.31cb43e9af1ccfa9ffef58465be2c675.root not found
2021-07-22 05:44:51.033 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8626 Error 4 opening dom root in 0x7fbbb8488fe0

2021-07-22 05:44:51.412 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:52.212 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:52.682 : default:3567097600: clsCredDomClose: Credctx deleted 0x7fbbb8378aa0
2021-07-22 05:44:52.709 :   CLSNS:3567097600: clsns_SetTraceLevel:trace level set to 1.
2021-07-22 05:44:52.721 : default:3567097600: Inited LSF context: 0x7fbbb8436ee0 
2021-07-22 05:44:52.728 : CLSCRED:3567097600: clsCredCommonInit: Inited singleton credctx.
2021-07-22 05:44:52.728 : CLSCRED:3567097600: (:CLSCRED0101:)clsCredDomInitRootDom: Using user given storage context for repository access.
2021-07-22 05:44:52.782 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_DISCOVERY_ADDRESS

2021-07-22 05:44:52.792 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8908 Error 4 querying length of attr ASM_STATIC_DISCOVERY_ADDRESS

2021-07-22 05:44:52.882 : CLSCRED:3567097600: (:CLSCRED1079:)clsCredOcrKeyExists: Obj dom : SYSTEM.credentials.domains.root.ASM.Self.31cb43e9af1ccfa9ffef58465be2c675.root not found
2021-07-22 05:44:52.883 : USRTHRD:3567097600: {0:4:146} [ora.storage] 8626 Error 4 opening dom root in 0x7fbbb8436310

2021-07-22 05:44:53.013 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:53.757 :CLSDYNAM:3577603840: [ora.ctssd]{0:5:3} [check] translateReturnCodes, return = 0, state detail = OBSERVERCheckcb data [0x7fbbb8567ed0]: mode[0xe6] offset[-181 ms].
2021-07-22 05:44:53.813 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:54.512 : default:3567097600: clsCredDomClose: Credctx deleted 0x7fbbb8366a30
2021-07-22 05:44:54.512 : USRTHRD:3567097600: {0:4:146} [ora.storage] -- trace dump on error exit --

2021-07-22 05:44:54.512 : USRTHRD:3567097600: {0:4:146} [ora.storage] Error [kgfoAl06] in [kgfokge] at kgfo.c:3112

2021-07-22 05:44:54.512 : USRTHRD:3567097600: {0:4:146} [ora.storage] ORA-01017: invalid username/password; logon denied


2021-07-22 05:44:54.513 : USRTHRD:3567097600: {0:4:146} [ora.storage] Category: 7

2021-07-22 05:44:54.513 : USRTHRD:3567097600: {0:4:146} [ora.storage] DepInfo: 1017

2021-07-22 05:44:54.513 : USRTHRD:3567097600: {0:4:146} [ora.storage] -- trace dump end --

2021-07-22 05:44:54.513 : USRTHRD:3567097600: {0:4:146} Thread:kgfoCheckMountExt isRunning is reset to false here
2021-07-22 05:44:54.513 : USRTHRD:3567097600: {0:4:146} Thread:kgfoCheckMountExt isFinished set to true
2021-07-22 05:44:54.513 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask m_cv.timewait returned
2021-07-22 05:44:54.513 : USRTHRD:3583907584: {0:4:146} Thread:kgfoCheckMountExt Tasklet::doTask(kgfoCheckMountExt) executed in  5 seconds
2021-07-22 05:44:54.513 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] StorageAgent::parsekgforetcodes retcode = 7, kgfoCheckMount(OCR), flag 2
2021-07-22 05:44:54.513 :CLSDYNAM:3583907584: [ora.storage]{0:4:146} [start] (null) category: 7, operation: kgfoAl06, loc: kgfokge, OS error: 1017, other: ORA-01017: invalid username/password; logon denied
复制代码

 

 

从日志看到很多ora.storage相关的提示,第一反应感觉是存储的问题,多次检查存储是没有问题的,而且手工是可以启动asm的,就排查存储本身的问题的,接着看下面的日志
ORA-01017: invalid username/password; logon denied
提示用户名密码不对,应该是认证的问题,查看asm实例的配置信息,发现asm实例使用的是做dg同步过来的密码文件,这个应该是不对的,asm实例本身也有自己的密码文件的,
不知道这里为什么使用的是dg的密码文件,尝试修改为asm自身的密码文件,问题解决:

1.查看当前asm实例使用的密码文件
两个实例都查看下
[grid@rac02 ~]$ srvctl config asm
ASM home: <CRS home>
Password file: +DATA/oradg/PASSWORD/pwdoradg
Backup of Password file:
ASM listener: LISTENER
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM


2.修改asm实例本身自己的密码文件
在其中一个节点上操作即可
[grid@rac02 ~]$ srvctl modify asm -pwfile '+OCR/orapwASM'

3.修改完成后检查使用的密码文件
两个实例都检查下
[grid@rac01 ~]$ srvctl config asm
ASM home: <CRS home>
Password file: +OCR/orapwASM
Backup of Password file:
ASM listener: LISTENER
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM


[grid@rac02 ~]$ srvctl config asm
ASM home: <CRS home>
Password file: +OCR/orapwASM
Backup of Password file:
ASM listener: LISTENER
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM

4.重启机器
重启机器后,一切正常.

posted @   slnngk  阅读(647)  评论(0)  编辑  收藏  举报
编辑推荐：
· AI与.NET技术实操系列（二）：开始使用ML.NET
· 记一次.NET内存居高不下排查解决与启示
· 探究高空视频全景AR技术的实现原理
· 理解Rust引用及其生命周期标识（上）
· 浏览器原生「磁吸」效果！Anchor Positioning 锚点定位神器解析
阅读排行：
· DeepSeek 开源周回顾「GitHub 热点速览」
· 物流快递公司核心技术能力-地址解析分单基础技术分享
· .NET 10首个预览版发布：重大改进与新特性概览！
· AI与.NET技术实操系列（二）：开始使用ML.NET
· 单线程的Redis速度为什么快？
点击右上角即可分享
微信分享提示
AI FOR CODE 大赛