19c rac adg其中一节点重启后crsd服务无法启动

系统环境：centos7.6

oracle版本：19c rac adg

adg其中一个节点重启后，使用 crsctl check cluster检查，服务异常。

使用crsctl status res -t -init检查发现crsd服务处于offine状态。

查看启动日志$ORACLE_BASE/diag/crs/rac01/crs/trace/ohasd_orarootagent_root.trc

有报错

?

1	Cluster Ready Service aborted due to Oracle Cluster Registry error [PROC-26: Error while accessing the physical storage Storage layer error [Insufficient quorum to open OCR devices] [0]]. Details at (:CRSD00111:) in /**/grid_base/diag/crs/wwdbdg01/crs/trace/crsd.trc.

　　

?

1	查看crsd.trc日志文件，

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

2022-03-17 17:30:38.819*:kgfn.c@1784: kgfnRecordErrPriv: 1017 error=ORA-01017: invalid username/password; logon denied   2022-03-17 17:30:38.821 : default:1682702656: clsCredDomClose: Credctx deleted 0x56348f3224c0 2022-03-17 17:30:38.821 :  OCRRAW:1682702656: kgfnConnect2: failed to connect   2022-03-17 17:30:38.821*:kgfn.c@5461: kgfnConnect2: failed to connect 2022-03-17 17:30:38.821 :  OCRRAW:1682702656: kgfnConnect2Retry: failed to connect connect after 1 attempts, 136s elapsed   2022-03-17 17:30:38.821 :  OCRRAW:1682702656: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied     2022-03-17 17:30:38.821*:kgfo.c@1003: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied   2022-03-17 17:30:38.821 :  OCRRAW:1682702656: -- trace dump on error exit --   2022-03-17 17:30:38.821 :  OCRRAW:1682702656: Error [kgfoAl06] in [kgfokge] at kgfo.c:2165   2022-03-17 17:30:38.821 :  OCRRAW:1682702656: ORA-01017: invalid username/password; logon denied     2022-03-17 17:30:38.821 :  OCRRAW:1682702656: Category: 7   2022-03-17 17:30:38.821 :  OCRRAW:1682702656: DepInfo: 1017   2022-03-17 17:30:38.822 :  OCRRAW:1682702656: -- trace dump end --   2022-03-17 17:30:38.822 :  OCRASM:1682702656: proprasmo: Failed to open the file in DG [SYSTEMDG] 2022-03-17 17:30:38.822 :  OCRASM:1682702656: proprasmo: Error in open/create file in dg [SYSTEMDG]   OCRASM:1682702656: SLOS : SLOS: cat=7, opn=kgfoAl06, dep=1017, loc=kgfokge   2022-03-17 17:30:38.822 :  OCRASM:1682702656: ASM Error Stack : ORA-01017: invalid username/password; logon denied    default:1682702656: u_set_gbl_comp_error: comptype '108' : error '7' 2022-03-17 17:30:38.849 :  OCRRAW:1682702656: kgfnFindLocalNode03: kgfn_find_node_sid found no members

　　权限验证失败。通过手动启动asm实例，sqlplus / as sysasm      

然后再手动启动crsd   crsctl start crsd -init

 

参考文档&感谢：http://www.askmac.cn/archives/flex-asm-gi-failed-to-start.html

https://www.cnblogs.com/hxlasky/p/15049513.html