使用ansible-playbook自动化安装Oracle RAC数据库19c

【用剧本安装Oracle rac数据库】

前提

首先要分配好共享存储跟分配好IP,可以使用oracleasm或者udev挂载好共享存储,因为脚本中会重启一下udev跟使用oracleasm命令扫描磁盘组

这里OCR盘为三个盘ASM_OCRB/C/D,数据盘为一个盘ASM_DATAE

oracleasm listdisks
ASM_DATAE
ASM_OCRB
ASM_OCRC
ASM_OCRD

 

如果是UDEV设置,注意后面变量的路径变化及名称变化(后面看看能否名称统一)

vim oracleasm_data.sh
for i in e;
do
echo "KERNEL==\"sd$i\", ENV{DEVTYPE}==\"disk\", SUBSYSTEM==\"block\", PROGRAM==\"/usr/lib/udev/scsi_id -g -u -d \$devnode\", RESULT==\"`/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`\", RUN+=\"/bin/sh -c 'mknod /dev/asm_data$i b \$major \$minor; chown grid:asmadmin /dev/asm_data$i; chmod 0660 /dev/asm_data$i'\"">> /etc/udev/rules.d/99-oracle-asmdevices.rules
done

vim oracleasm_ocr.sh
for i in b c d;
do
echo "KERNEL==\"sd$i\", ENV{DEVTYPE}==\"disk\", SUBSYSTEM==\"block\", PROGRAM==\"/usr/lib/udev/scsi_id -g -u -d \$devnode\", RESULT==\"`/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`\", RUN+=\"/bin/sh -c 'mknod /dev/asm_ocr$i b \$major \$minor; chown grid:asmadmin /dev/asm_ocr$i; chmod 0660 /dev/asm_ocr$i'\"">> /etc/udev/rules.d/99-oracle-asmdevices.rules
done

systemctl restart systemd-udev-trigger.service

udevadm control --reload-rules

udevadm trigger

 

设置剧本的环境变量

cat main.yml
---

# OS 安装包及用户密码及目录及节点主机名称前缀名设置
soft_dir: /home/soft
oracle_passwd: oracle
oracle_base_dir: /u01/app
rac_node_name: rac

# RAC IP 数据库的VIP privip publicIP 网卡的名称设置,注意公网私网设置
rac1vip: 10.x.x.xx0
rac2vip: 10.x.x.xx1
rac1privip: 192.xxx.xxx.xx2
rac2privip: 192.xxx.xxx.xx3
racscanip: 10.x.x.xx9
racpublicfcname: ens192
racprivfcname: ens224

# DB 数据库实例名 字符集 
oracle_sid: orcl
characterset: ZHS16GBK

# Diskgroup ocr data 磁盘组属性设置 ocr盘组的路口名称 ocr盘信息跟failure设置注意中间多了逗号 data数据盘设置(多个用逗号隔开)网卡列表信息设置(注意公网私网IP信息对应,注意最后位是0)
ocrp_redundancy: NORMAL
data_redundancy: EXTERNAL
basedisk_path: /dev/oracleasm/disks/ASM*
ocrp_failure_basedisk: /dev/oracleasm/disks/ASM_OCRB,,/dev/oracleasm/disks/ASM_OCRC,,/dev/oracleasm/disks/ASM_OCRD
ocrp_basedisk: /dev/oracleasm/disks/ASM_OCRB,/dev/oracleasm/disks/ASM_OCRC,/dev/oracleasm/disks/ASM_OCRD
data_basedisk: /dev/oracleasm/disks/ASM_DATAE
network_interface_list: ens192:10.x.x.0:1,ens224:192.168.xxx.0:5

# Soft grid db 数据库软件安装包信息
oracle_grid_soft: LINUX.X64_193000_grid_home.zip
oracle_grid_soft_md5: b7c4c66f801f92d14faa0d791ccda721
oracle_oracle_soft: LINUX.X64_193000_db_home.zip
oracle_oracle_soft_md5: 1858bd0d281c60f4ddabd87b1c214a4f
download_target: /home/soft

 

 

所有剧本信息

-rw-r--r-- 1 root root  881 Feb 17 13:53 01_change_os_setting_19cR3.yml
-rw-r--r-- 1 root root 3254 Feb 10 15:45 02_change_oracle_params_19cR3.yml
-rw-r--r-- 1 root root 5190 Feb 27 14:46 03_add_oracle_group_user_19cR3.yml
-rw-r--r-- 1 root root 1380 Feb 20 09:36 04_oracle_grid_ssh_19cR3.yml
-rw-r--r-- 1 root root 1345 Feb 15 17:20 05_install_grid_infrastructure_19cR3.yml
-rw-r--r-- 1 root root  890 Feb 10 15:45 06_run_grid_rootsh_19cR3.yml
-rw-r--r-- 1 root root 1451 Feb 14 17:07 07_configtools_grid_oracle_profile_19cR3.yml
-rw-r--r-- 1 root root  279 Feb 10 18:02 08_asmca_data_diskgroup_create_19cR3.yml
-rw-r--r-- 1 root root 1449 Feb 13 15:16 09_install_oracle_database_soft_19cR3.yml
-rw-r--r-- 1 root root  146 Feb 10 15:45 10_run_oracle_rootsh_node1_19cR3.yml
-rw-r--r-- 1 root root  146 Feb 10 15:45 11_run_oracle_rootsh_node2_19cR3.yml
-rw-r--r-- 1 root root  593 Feb 10 15:45 12_install_oracle_netca_19cR3.yml
-rw-r--r-- 1 root root  867 Feb 16 09:57 13_create_oracle_database_19cR3.yml
-rw-r--r-- 1 root root  904 Feb 27 13:53 14_add_oracle_sqlnet_19cR3.yml
-rw-r--r-- 1 root root  445 Mar 22 10:22 15_add_oracle_initsql_19cR3.yml
-rw-r--r-- 1 root root  432 Feb 20 10:56 copy_grid_oracle_soft.yml
-rw-r--r-- 1 root root  265 Feb 10 15:45 init_cluster.yml
-rw-r--r-- 1 root root 1872 Apr  3 11:27 main.yml
-rw-r--r-- 1 root root 1056 Feb 10 15:45 prepare.yml

 

任务剧本说明

cat main.yml
---

- name: prepare on all node   根据变量设置两个节点的主机名称及expect,tcl,pdksh,compat,oracle-database-preinstall-19c安装包
include: prepare.yml
tags: [prepare]

- name: oracle_rac cluster nodes   根据ansible hosts文件取到对应的物理主机的IP信息
include: init_cluster.yml
tags: [oracle_grid_ssh]

- name: change_os_setting on all node    操作系统参数设置
include: 01_change_os_setting_19cR3.yml
tags: [change_os_setting]

- name: change_oracle_params on all node  根据RAC需求设置hosts文件及用户参数设置
include: 02_change_oracle_params_19cR3.yml
tags: [change_oracle_params]

- name: add_oracle_group_user on all node  创建Oracle跟grid用户及属组
include: 03_add_oracle_group_user_19cR3.yml
tags: [add_oracle_group_user]

- name: oracle_grid_ssh on node1   设置Oracle跟grid用户的节点互相配置
include: 04_oracle_grid_ssh_19cR3.yml
tags: [oracle_grid_ssh]

- name: install_grid_infrastructure   安装GI软件
include: 05_install_grid_infrastructure_19cR3.yml
tags: [install_grid_infrastructure]

- name: run_grid_rootsh   执行GI脚本
include: 06_run_grid_rootsh_19cR3.yml
tags: [run_grid_rootsh]

- name: configtools_grid_oracle_profile 安装configtools
include: 07_configtools_grid_oracle_profile_19cR3.yml
tags: [configtools_grid_oracle_profile]

- name: asmca_data_diskgroup_create  创建磁盘组
include: 08_asmca_data_diskgroup_create_19cR3.yml
tags: [asmca_data_diskgroup_create]

- name: install_oracle_database_soft 安装Oracle软件
include: 09_install_oracle_database_soft_19cR3.yml
tags: [install_oracle_database_soft]

- name: run_oracle_rootsh_node1 执行Oracle的脚本
include: 10_run_oracle_rootsh_node1_19cR3.yml
tags: [run_oracle_rootsh_node1]

- name: run_oracle_rootsh_node2 执行Oracle的脚本
include: 11_run_oracle_rootsh_node2_19cR3.yml
tags: [run_oracle_rootsh_node2]

- name: install_oracle_netca 创建监听
include: 12_install_oracle_netca_19cR3.yml
tags: [install_oracle_netca]

- name: install_oracle_database 创建数据库实例
include: 13_create_oracle_database_19cR3.yml
tags: [install_oracle_database]

- name: add_oracle_sqlnet 设置sqlnet参数
include: 14_add_oracle_sqlnet_19cR3.yml
tags: [add_oracle_sqlnet]

- name: add_oracle_initsql 可以执行一些初始化的SQL脚本
include: 15_add_oracle_initsql_19cR3.yml
tags: [add_oracle_initsql]

 

设置ansible hosts

[oracle_rac]
10.x.x.xx2 ansible_user=root ansible_ssh_pass=kd11111 cluster_role=node1
10.x.x.xx3 ansible_user=root ansible_ssh_pass=kd11111 cluster_role=node2

 

最后安装日志信息,以下屏蔽掉了IP信息

ansible-playbook /root/auto/ansible/oracle_rac.yaml

PLAY [oracle_rac] *******************************************************************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : set hostname on node1] *******************************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : set hostname on node2] *******************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : create soft directory on all node] *******************************************************************************************************************************************************
ok: [10.x.x.xx3]
ok: [10.x.x.xx2]

TASK [oracle_rac : copy expect tcl  rpm package on all node] ************************************************************************************************************************************************
ok: [10.x.x.xx3] => (item=expect-5.45-14.el7_1.x86_64.rpm)
ok: [10.x.x.xx2] => (item=expect-5.45-14.el7_1.x86_64.rpm)
ok: [10.x.x.xx3] => (item=tcl-8.5.13-8.el7.x86_64.rpm)
ok: [10.x.x.xx2] => (item=tcl-8.5.13-8.el7.x86_64.rpm)
ok: [10.x.x.xx3] => (item=pdksh-5.2.14-37.el5.x86_64.rpm)
ok: [10.x.x.xx2] => (item=pdksh-5.2.14-37.el5.x86_64.rpm)
ok: [10.x.x.xx3] => (item=compat-libstdc++-33-3.2.3-72.el7.x86_64.rpm)
ok: [10.x.x.xx2] => (item=compat-libstdc++-33-3.2.3-72.el7.x86_64.rpm)
ok: [10.x.x.xx3] => (item=oracle-database-preinstall-19c-1.0-1.el7.x86_64.rpm)
ok: [10.x.x.xx2] => (item=oracle-database-preinstall-19c-1.0-1.el7.x86_64.rpm)

TASK [oracle_rac : install expect tcl  rpm on all node] *****************************************************************************************************************************************************
[DEPRECATION WARNING]: Invoking "yum" only once while using a loop via squash_actions is deprecated. Instead of using a loop to supply multiple items and specifying `name: "{{soft_dir}}/{{ item }}"`, 
please use `name: ['tcl-8.5.13-8.el7.x86_64.rpm', 'expect-5.45-14.el7_1.x86_64.rpm']` and remove the loop. This feature will be removed in version 2.11. Deprecation warnings can be disabled by setting 
deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: Invoking "yum" only once while using a loop via squash_actions is deprecated. Instead of using a loop to supply multiple items and specifying `name: "{{soft_dir}}/{{ item }}"`, 
please use `name: ['tcl-8.5.13-8.el7.x86_64.rpm', 'expect-5.45-14.el7_1.x86_64.rpm']` and remove the loop. This feature will be removed in version 2.11. Deprecation warnings can be disabled by setting 
deprecation_warnings=False in ansible.cfg.
ok: [10.x.x.xx2] => (item=['/home/soft/tcl-8.5.13-8.el7.x86_64.rpm', '/home/soft/expect-5.45-14.el7_1.x86_64.rpm'])
ok: [10.x.x.xx3] => (item=['/home/soft/tcl-8.5.13-8.el7.x86_64.rpm', '/home/soft/expect-5.45-14.el7_1.x86_64.rpm'])

TASK [oracle_rac : oracle_rac cluster] **********************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : declare host group] **********************************************************************************************************************************************************************
ok: [10.x.x.xx2] => (item=10.x.x.xx2)
ok: [10.x.x.xx2] => (item=10.x.x.xx3)
ok: [10.x.x.xx3] => (item=10.x.x.xx2)
ok: [10.x.x.xx3] => (item=10.x.x.xx3)

TASK [oracle_rac : change SELinux is disable] ***************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : stop firewall] ***************************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : stop auto start a firewall] **************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : install oracle-database-preinstall   rpm] ************************************************************************************************************************************************
ok: [10.x.x.xx3] => (item=oracle-database-preinstall-19c-1.0-1.el7.x86_64.rpm)
ok: [10.x.x.xx2] => (item=oracle-database-preinstall-19c-1.0-1.el7.x86_64.rpm)

TASK [oracle_rac : stop auto start a avahi-daemon] **********************************************************************************************************************************************************
changed: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : stop avahi-daemon] ***********************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : stop auto start a chronyd] ***************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : stop chronyd] ****************************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : backup for /etc/hosts] *******************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : create file for oracle.hosts] ************************************************************************************************************************************************************
ok: [10.x.x.xx2] => (item=###Public IP)
ok: [10.x.x.xx3] => (item=###Public IP)
ok: [10.x.x.xx2] => (item=10.x.x.xx2     rac01)
ok: [10.x.x.xx3] => (item=10.x.x.xx2     rac01)
ok: [10.x.x.xx2] => (item=10.x.x.xx3     rac02)
ok: [10.x.x.xx3] => (item=10.x.x.xx3     rac02)
ok: [10.x.x.xx2] => (item= )
ok: [10.x.x.xx3] => (item= )
ok: [10.x.x.xx2] => (item=###Private IP)
ok: [10.x.x.xx3] => (item=###Private IP)
ok: [10.x.x.xx2] => (item=192.168.100.172          rac01-priv)
ok: [10.x.x.xx3] => (item=192.168.100.172          rac01-priv)
ok: [10.x.x.xx2] => (item=192.168.100.173          rac02-priv)
ok: [10.x.x.xx3] => (item=192.168.100.173          rac02-priv)
ok: [10.x.x.xx2] => (item= )
ok: [10.x.x.xx3] => (item= )
ok: [10.x.x.xx2] => (item=###Virtual IP)
ok: [10.x.x.xx3] => (item=###Virtual IP)
ok: [10.x.x.xx2] => (item=10.0.0.190     rac01-vip)
ok: [10.x.x.xx3] => (item=10.0.0.190     rac01-vip)
ok: [10.x.x.xx2] => (item=10.0.0.191     rac02-vip)
ok: [10.x.x.xx3] => (item=10.0.0.191     rac02-vip)
ok: [10.x.x.xx2] => (item= )
ok: [10.x.x.xx3] => (item= )
ok: [10.x.x.xx2] => (item=###Scan IP)
ok: [10.x.x.xx3] => (item=###Scan IP)
ok: [10.x.x.xx2] => (item=10.0.0.189     rac-scan)
ok: [10.x.x.xx3] => (item=10.0.0.189     rac-scan)
ok: [10.x.x.xx2] => (item= )
ok: [10.x.x.xx3] => (item= )

TASK [oracle_rac : backup for grub.cfg] *********************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : change for grub.cfg] *********************************************************************************************************************************************************************
[WARNING]: Consider using the replace, lineinfile or template module rather than running 'sed'.  If you need to use command because replace, lineinfile or template is insufficient you can add 'warn:
false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message.

changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : make grub2] ******************************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : backup for /etc/sysctl.conf] *************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : sysctl] **********************************************************************************************************************************************************************************
[WARNING]: The value 2 (type int) in a string field was converted to u'2' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change.

ok: [10.x.x.xx3]
ok: [10.x.x.xx2]

TASK [oracle_rac : sysctl] **********************************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : sysctl] **********************************************************************************************************************************************************************************
[WARNING]: The value 6815744 (type int) in a string field was converted to u'6815744' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change.

ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : stat /etc/ntp.conf] **********************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : move file /etc/ntp.conf] *****************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : stat /var/run/ ntpid.pid] ****************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : remove file /var/run/ntpd.pid] ***********************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
[WARNING]: The value 2047 (type int) in a string field was converted to u'2047' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change.

ok: [10.x.x.xx3]
ok: [10.x.x.xx2]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
[WARNING]: The value 16384 (type int) in a string field was converted to u'16384' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change.

ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
[WARNING]: The value 1024 (type int) in a string field was converted to u'1024' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change.

ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
[WARNING]: The value 65536 (type int) in a string field was converted to u'65536' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change.

ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
[WARNING]: The value 10240 (type int) in a string field was converted to u'10240' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change.

ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : pam_limits] ******************************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : check group for oinstall] ****************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : add group for oinstall] ******************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : check group for dba] *********************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : add group for dba] ***********************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : check group for oper] ********************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : add group for oper] **********************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : check group for backupdba] ***************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : add group for backupdba] *****************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : check group for dgdba] *******************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : add group for dgdba] *********************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : check group for kmdba] *******************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : add group for kmdba] *********************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : check group for asmadmin] ****************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : add group for asmadmin] ******************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : check group for asmdba] ******************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : add group for asmdba] ********************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : check group for asmoper] *****************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : add group for asmoper] *******************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : check group for racdba] ******************************************************************************************************************************************************************
ok: [10.x.x.xx2]
ok: [10.x.x.xx3]

TASK [oracle_rac : add group for racdba] ********************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
skipping: [10.x.x.xx3]

TASK [oracle_rac : check user for grid] *********************************************************************************************************************************************************************
fatal: [10.x.x.xx2]: FAILED! => {"changed": false, "cmd": "id grid", "delta": "0:00:00.059661", "end": "2023-03-29 03:01:31.345133", "msg": "non-zero return code", "rc": 1, "start": "2023-03-29 03:01:31.285472", "stderr": "id: grid: no such user", "stderr_lines": ["id: grid: no such user"], "stdout": "", "stdout_lines": []}
...ignoring
fatal: [10.x.x.xx3]: FAILED! => {"changed": false, "cmd": "id grid", "delta": "0:00:00.009653", "end": "2023-03-29 03:01:30.996735", "msg": "non-zero return code", "rc": 1, "start": "2023-03-29 03:01:30.987082", "stderr": "id: grid: no such user", "stderr_lines": ["id: grid: no such user"], "stdout": "", "stdout_lines": []}
...ignoring

TASK [oracle_rac : add user for grid] ***********************************************************************************************************************************************************************
ok: [10.x.x.xx3]
ok: [10.x.x.xx2]

TASK [oracle_rac : check user for oracle] *******************************************************************************************************************************************************************
fatal: [10.x.x.xx2]: FAILED! => {"changed": false, "cmd": "id oracle", "delta": "0:00:00.010554", "end": "2023-03-29 03:01:32.714873", "msg": "non-zero return code", "rc": 1, "start": "2023-03-29 03:01:32.704319", "stderr": "id: oracle: no such user", "stderr_lines": ["id: oracle: no such user"], "stdout": "", "stdout_lines": []}
...ignoring
fatal: [10.x.x.xx3]: FAILED! => {"changed": false, "cmd": "id oracle", "delta": "0:00:00.009216", "end": "2023-03-29 03:01:32.486137", "msg": "non-zero return code", "rc": 1, "start": "2023-03-29 03:01:32.476921", "stderr": "id: oracle: no such user", "stderr_lines": ["id: oracle: no such user"], "stdout": "", "stdout_lines": []}
...ignoring

TASK [oracle_rac : add user for oracle] *********************************************************************************************************************************************************************
ok: [10.x.x.xx3]
ok: [10.x.x.xx2]

TASK [oracle_rac : change password for grid] ****************************************************************************************************************************************************************
changed: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : change password for oracle] **************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : create directory for grid base] **********************************************************************************************************************************************************
[WARNING]: Consider using the file module with state=directory rather than running 'mkdir'.  If you need to use command because file is insufficient you can add 'warn: false' to this command task or set
'command_warnings=False' in ansible.cfg to get rid of this message.

changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : create directory for oracle base] ********************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : create directory for grid home] **********************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : change owner for grid] *******************************************************************************************************************************************************************
[WARNING]: Consider using the file module with owner rather than running 'chown'.  If you need to use command because file is insufficient you can add 'warn: false' to this command task or set
'command_warnings=False' in ansible.cfg to get rid of this message.

changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : change owner for oracle] *****************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : change] **********************************************************************************************************************************************************************************
[WARNING]: Consider using the file module with mode rather than running 'chmod'.  If you need to use command because file is insufficient you can add 'warn: false' to this command task or set
'command_warnings=False' in ansible.cfg to get rid of this message.

changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : add bash_profile for grid] ***************************************************************************************************************************************************************
changed: [10.x.x.xx2] => (item={'line': ' '})
changed: [10.x.x.xx3] => (item={'line': ' '})
changed: [10.x.x.xx2] => (item={'line': '### for grid '})
changed: [10.x.x.xx3] => (item={'line': '### for grid '})
changed: [10.x.x.xx2] => (item={'line': 'export LANG=en_US.UTF-8'})
changed: [10.x.x.xx3] => (item={'line': 'export LANG=en_US.UTF-8'})
changed: [10.x.x.xx2] => (item={'line': 'export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK'})
changed: [10.x.x.xx3] => (item={'line': 'export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK'})
changed: [10.x.x.xx2] => (item={'line': 'export ORACLE_BASE=/u01/app/grid'})
changed: [10.x.x.xx3] => (item={'line': 'export ORACLE_BASE=/u01/app/grid'})
changed: [10.x.x.xx2] => (item={'line': 'export ORACLE_HOME=/u01/app/19.3.0/grid'})
changed: [10.x.x.xx3] => (item={'line': 'export ORACLE_HOME=/u01/app/19.3.0/grid'})

TASK [oracle_rac : add bash_profile for oracle] *************************************************************************************************************************************************************
changed: [10.x.x.xx2] => (item={'line': ' '})
changed: [10.x.x.xx3] => (item={'line': ' '})
changed: [10.x.x.xx3] => (item={'line': '### for oracle '})
changed: [10.x.x.xx2] => (item={'line': '### for oracle '})
changed: [10.x.x.xx3] => (item={'line': 'export LANG=en_US.UTF-8'})
changed: [10.x.x.xx2] => (item={'line': 'export LANG=en_US.UTF-8'})
changed: [10.x.x.xx3] => (item={'line': 'export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK'})
changed: [10.x.x.xx2] => (item={'line': 'export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK'})
changed: [10.x.x.xx3] => (item={'line': 'export ORACLE_BASE=/u01/app/oracle'})
changed: [10.x.x.xx2] => (item={'line': 'export ORACLE_BASE=/u01/app/oracle'})
changed: [10.x.x.xx3] => (item={'line': 'export ORACLE_HOME=/u01/app/oracle/product/19.3.0/db'})
changed: [10.x.x.xx2] => (item={'line': 'export ORACLE_HOME=/u01/app/oracle/product/19.3.0/db'})

TASK [oracle_rac : copy sshUserSetup on node1] **************************************************************************************************************************************************************
skipping: [10.x.x.xx3] => (item=sshUserSetup.sh) 
skipping: [10.x.x.xx3] => (item=auto_grid_ssh.sh) 
skipping: [10.x.x.xx3] => (item=auto_oracle_ssh.sh) 
changed: [10.x.x.xx2] => (item=sshUserSetup.sh)
ok: [10.x.x.xx2] => (item=auto_grid_ssh.sh)
ok: [10.x.x.xx2] => (item=auto_oracle_ssh.sh)

TASK [oracle_rac : clear grid user ssh] *********************************************************************************************************************************************************************
[WARNING]: Consider using the file module with state=absent rather than running 'rm'.  If you need to use command because file is insufficient you can add 'warn: false' to this command task or set
'command_warnings=False' in ansible.cfg to get rid of this message.

changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : clear oracle user ssh] *******************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : config grid user ssh on node1] ***********************************************************************************************************************************************************
skipping: [10.x.x.xx3]
[WARNING]: Module remote_tmp /home/grid/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with
the correct permissions manually

changed: [10.x.x.xx2]

TASK [oracle_rac : config oracle user ssh on node1] *********************************************************************************************************************************************************
skipping: [10.x.x.xx3]
[WARNING]: Module remote_tmp /home/oracle/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with
the correct permissions manually

changed: [10.x.x.xx2]

TASK [oracle_rac : copy sshUserSetup on node2] **************************************************************************************************************************************************************
skipping: [10.x.x.xx2] => (item=auto_grid_ssh02.sh) 
skipping: [10.x.x.xx2] => (item=auto_oracle_ssh02.sh) 
ok: [10.x.x.xx3] => (item=auto_grid_ssh02.sh)
ok: [10.x.x.xx3] => (item=auto_oracle_ssh02.sh)

TASK [oracle_rac : config grid user ssh on node2] ***********************************************************************************************************************************************************
skipping: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : config oracle user ssh on node2] *********************************************************************************************************************************************************
skipping: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : copy grid.rsp] ***************************************************************************************************************************************************************************
skipping: [10.x.x.xx3] => (item=grid.rsp) 
ok: [10.x.x.xx2] => (item=grid.rsp)

TASK [oracle_rac : chenge owner for response file] **********************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : restart udev service] ********************************************************************************************************************************************************************
changed: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : check oracleasm] *************************************************************************************************************************************************************************
[WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'.  If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or
set 'command_warnings=False' in ansible.cfg to get rid of this message.

changed: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : oracleasm scandisks] *********************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : copy and unzip file] *********************************************************************************************************************************************************************
skipping: [10.x.x.xx3]
[WARNING]: Consider using the unarchive module rather than running 'unzip'.  If you need to use command because unarchive is insufficient you can add 'warn: false' to this command task or set
'command_warnings=False' in ansible.cfg to get rid of this message.

changed: [10.x.x.xx2]

TASK [oracle_rac : change dir grid home files permission] ***************************************************************************************************************************************************
changed: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : grid installation] ***********************************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : check orainstRoot.sh] ********************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : run orainstRoot.sh] **********************************************************************************************************************************************************************
changed: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : check root.sh on node1] ******************************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : run root.sh on node1] ********************************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : check root.sh on node2] ******************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : run root.sh on node2] ********************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : grid configtools installation] ***********************************************************************************************************************************************************
skipping: [10.x.x.xx3]
fatal: [10.x.x.xx2]: FAILED! => {"changed": true, "cmd": "/u01/app/19.3.0/grid/gridSetup.sh -executeConfigTools -responseFile /tmp/grid.rsp -silent", "delta": "0:03:09.388664", "end": "2023-03-29 03:27:15.362999", "msg": "non-zero return code", "rc": 255, "start": "2023-03-29 03:24:05.974335", "stderr": "", "stderr_lines": [], "stdout": "Launching Oracle Grid Infrastructure Setup Wizard...\n\nYou can find the logs of this session at:\n/u01/app/oraInventory/logs/GridSetupActions2023-03-29_03-24-06AM\n\nYou can find the log of this install session at:\n /u01/app/oraInventory/logs/UpdateNodeList2023-03-29_03-24-06AM.log\nConfiguration failed.\n[WARNING] [INS-43080] Some of the configuration assistants failed, were cancelled or skipped.\n   ACTION: Refer to the logs or contact Oracle Support Services.", "stdout_lines": ["Launching Oracle Grid Infrastructure Setup Wizard...", "", "You can find the logs of this session at:", "/u01/app/oraInventory/logs/GridSetupActions2023-03-29_03-24-06AM", "", "You can find the log of this install session at:", " /u01/app/oraInventory/logs/UpdateNodeList2023-03-29_03-24-06AM.log", "Configuration failed.", "[WARNING] [INS-43080] Some of the configuration assistants failed, were cancelled or skipped.", "   ACTION: Refer to the logs or contact Oracle Support Services."]}
...ignoring

TASK [oracle_rac : add bash_profile for grid on node1] ******************************************************************************************************************************************************
skipping: [10.x.x.xx3] => (item={'line': 'export ORACLE_SID=+ASM1'}) 
skipping: [10.x.x.xx3] => (item={'line': 'export PATH=$ORACLE_HOME/bin:$PATH'}) 
changed: [10.x.x.xx2] => (item={'line': 'export ORACLE_SID=+ASM1'})
changed: [10.x.x.xx2] => (item={'line': 'export PATH=$ORACLE_HOME/bin:$PATH'})

TASK [oracle_rac : add bash_profile for grid on node2] ******************************************************************************************************************************************************
skipping: [10.x.x.xx2] => (item={'line': 'export ORACLE_SID=+ASM2'}) 
skipping: [10.x.x.xx2] => (item={'line': 'export PATH=$ORACLE_HOME/bin:$PATH'}) 
changed: [10.x.x.xx3] => (item={'line': 'export ORACLE_SID=+ASM2'})
changed: [10.x.x.xx3] => (item={'line': 'export PATH=$ORACLE_HOME/bin:$PATH'})

TASK [oracle_rac : add bash_profile for oracle on node1] ****************************************************************************************************************************************************
skipping: [10.x.x.xx3] => (item={'line': ' '}) 
skipping: [10.x.x.xx3] => (item={'line': '### for oracle '}) 
skipping: [10.x.x.xx3] => (item={'line': 'export ORACLE_SID=orcl1'}) 
skipping: [10.x.x.xx3] => (item={'line': 'export PATH=$ORACLE_HOME/bin:$PATH'}) 
ok: [10.x.x.xx2] => (item={'line': ' '})
ok: [10.x.x.xx2] => (item={'line': '### for oracle '})
changed: [10.x.x.xx2] => (item={'line': 'export ORACLE_SID=orcl1'})
changed: [10.x.x.xx2] => (item={'line': 'export PATH=$ORACLE_HOME/bin:$PATH'})

TASK [oracle_rac : add bash_profile for oracle on node2] ****************************************************************************************************************************************************
skipping: [10.x.x.xx2] => (item={'line': ' '}) 
skipping: [10.x.x.xx2] => (item={'line': '### for oracle '}) 
skipping: [10.x.x.xx2] => (item={'line': 'export ORACLE_SID=orcl2'}) 
skipping: [10.x.x.xx2] => (item={'line': 'export PATH=$ORACLE_HOME/bin:$PATH'}) 
ok: [10.x.x.xx3] => (item={'line': ' '})
ok: [10.x.x.xx3] => (item={'line': '### for oracle '})
changed: [10.x.x.xx3] => (item={'line': 'export ORACLE_SID=orcl2'})
changed: [10.x.x.xx3] => (item={'line': 'export PATH=$ORACLE_HOME/bin:$PATH'})

TASK [oracle_rac : create data diskgroup] *******************************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : create directory for oracle home] ********************************************************************************************************************************************************
changed: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : add bash_profile for oracle] *************************************************************************************************************************************************************
ok: [10.x.x.xx2] => (item={'line': 'export ORACLE_BASE=/u01/app/oracle'})
ok: [10.x.x.xx2] => (item={'line': 'export ORACLE_HOME=/u01/app/oracle/product/19.3.0/db'})
ok: [10.x.x.xx2] => (item={'line': 'export PATH=$ORACLE_HOME/bin:$PATH'})
ok: [10.x.x.xx3] => (item={'line': 'export ORACLE_BASE=/u01/app/oracle'})
ok: [10.x.x.xx3] => (item={'line': 'export ORACLE_HOME=/u01/app/oracle/product/19.3.0/db'})
ok: [10.x.x.xx3] => (item={'line': 'export PATH=$ORACLE_HOME/bin:$PATH'})

TASK [oracle_rac : copy db.rsp] *****************************************************************************************************************************************************************************
skipping: [10.x.x.xx3] => (item=db.rsp) 
changed: [10.x.x.xx2] => (item=db.rsp)

TASK [oracle_rac : chenge owner for response file] **********************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : copy and unzip db file] ******************************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : change dir oracle home files permission] *************************************************************************************************************************************************
changed: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : execute database  soft installation] *****************************************************************************************************************************************************
skipping: [10.x.x.xx3]
fatal: [10.x.x.xx2]: FAILED! => {"changed": true, "cmd": "/u01/app/oracle/product/19.3.0/db/runInstaller -silent -force -responseFile /tmp/db.rsp -ignorePrereq -ignoreInternalDriverError -waitforcompletion", "delta": "0:05:53.333003", "end": "2023-03-29 03:36:03.135703", "msg": "non-zero return code", "rc": 6, "start": "2023-03-29 03:30:09.802700", "stderr": "", "stderr_lines": [], "stdout": "Launching Oracle Database Setup Wizard...\n\n[WARNING] [INS-13013] Target environment does not meet some mandatory requirements.\n   CAUSE: Some of the mandatory prerequisites are not met. See logs for details. /u01/app/oraInventory/logs/InstallActions2023-03-29_03-30-09AM/installActions2023-03-29_03-30-09AM.log\n   ACTION: Identify the list of failed prerequisite checks from the log: /u01/app/oraInventory/logs/InstallActions2023-03-29_03-30-09AM/installActions2023-03-29_03-30-09AM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually.\nThe response file for this session can be found at:\n /u01/app/oracle/product/19.3.0/db/install/response/db_2023-03-29_03-30-09AM.rsp\n\nYou can find the log of this install session at:\n /u01/app/oraInventory/logs/InstallActions2023-03-29_03-30-09AM/installActions2023-03-29_03-30-09AM.log\n\nAs a root user, execute the following script(s):\n\t1. /u01/app/oracle/product/19.3.0/db/root.sh\n\nExecute /u01/app/oracle/product/19.3.0/db/root.sh on the following nodes: \n[rac01, rac02]\n\n\nSuccessfully Setup Software with warning(s).", "stdout_lines": ["Launching Oracle Database Setup Wizard...", "", "[WARNING] [INS-13013] Target environment does not meet some mandatory requirements.", "   CAUSE: Some of the mandatory prerequisites are not met. See logs for details. /u01/app/oraInventory/logs/InstallActions2023-03-29_03-30-09AM/installActions2023-03-29_03-30-09AM.log", "   ACTION: Identify the list of failed prerequisite checks from the log: /u01/app/oraInventory/logs/InstallActions2023-03-29_03-30-09AM/installActions2023-03-29_03-30-09AM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually.", "The response file for this session can be found at:", " /u01/app/oracle/product/19.3.0/db/install/response/db_2023-03-29_03-30-09AM.rsp", "", "You can find the log of this install session at:", " /u01/app/oraInventory/logs/InstallActions2023-03-29_03-30-09AM/installActions2023-03-29_03-30-09AM.log", "", "As a root user, execute the following script(s):", "\t1. /u01/app/oracle/product/19.3.0/db/root.sh", "", "Execute /u01/app/oracle/product/19.3.0/db/root.sh on the following nodes: ", "[rac01, rac02]", "", "", "Successfully Setup Software with warning(s)."]}
...ignoring

TASK [oracle_rac : root.sh script for node1] ****************************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : root.sh script for node2] ****************************************************************************************************************************************************************
skipping: [10.x.x.xx2]
changed: [10.x.x.xx3]

TASK [oracle_rac : copy netca.rsp] **************************************************************************************************************************************************************************
skipping: [10.x.x.xx3] => (item=netca.rsp) 
changed: [10.x.x.xx2] => (item=netca.rsp)

TASK [oracle_rac : chenge owner for response file] **********************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : execute netca installation] **************************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : execute database installation] ***********************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : set sqlnet.ora for rac node] *************************************************************************************************************************************************************
changed: [10.x.x.xx2] => (item=##OracleBegin)
changed: [10.x.x.xx3] => (item=##OracleBegin)
changed: [10.x.x.xx3] => (item=SQLNET.ALLOWED_LOGON_VERSION_CLIENT=8)
changed: [10.x.x.xx2] => (item=SQLNET.ALLOWED_LOGON_VERSION_CLIENT=8)
changed: [10.x.x.xx2] => (item=SQLNET.ALLOWED_LOGON_VERSION_SERVER=8)
changed: [10.x.x.xx2] => (item=SQLNET.ALLOWED_LOGON_VERSION=8)
changed: [10.x.x.xx2] => (item=##OracleEnd)
changed: [10.x.x.xx3] => (item=SQLNET.ALLOWED_LOGON_VERSION_SERVER=8)
changed: [10.x.x.xx3] => (item=SQLNET.ALLOWED_LOGON_VERSION=8)
changed: [10.x.x.xx3] => (item=##OracleEnd)

TASK [oracle_rac : stop lsnrctl for rac node1] **************************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : start lsnrctl for rac node1] *************************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

TASK [oracle_rac : copy init sql to node1] ******************************************************************************************************************************************************************
skipping: [10.x.x.xx3] => (item=oracle_init.sql) 
ok: [10.x.x.xx2] => (item=oracle_init.sql)

TASK [oracle_rac : create read only user on node1] **********************************************************************************************************************************************************
skipping: [10.x.x.xx3]
changed: [10.x.x.xx2]

PLAY RECAP **************************************************************************************************************************************************************************************************
10.x.x.xx2                 : ok=100  changed=57   unreachable=0    failed=0    skipped=21   rescued=0    ignored=4   
10.x.x.xx3                 : ok=82   changed=40   unreachable=0    failed=0    skipped=39   rescued=0    ignored=2   

 

说明

由于脚本较多,可以留言发送脚本。

 

posted @ 2023-05-19 16:41  zetan·chen  阅读(370)  评论(10编辑  收藏  举报