KingbaseES V8R3集群运维案例之---ksql连接数据库system密码认证失败

案例说明：
在测试主机上部署了KingbaseES V8R3和V8R6的集群环境，在V8R3下执行ksql连接数据库时，出现动态库加载故障，并且无法正常连接数据库。

适用版本：
KingbaseES V8R3/R6

一、问题现象

现象一：ksql动态库加载失败，并无法连接数据库：

[kingbase@node201 bin]$ ./ksql -U SYSTEM -W 123456 test
ksql (V008R003C002B0370)
Type "help" for help.

./ksql: symbol lookup error: ./ksql: undefined symbol: createPQExpBuffer

现象二：ksql连接数据库，system用户认证失败

[kingbase@node201 bin]$ ./ksql -U system -W 123456 test
ksql: FATAL:  password authentication failed for user "system"

二、问题分析

1、通过ksql远程连接
如下所示，在另外的节点，通过ksql远程连接此节点，连接正常：

[kingbase@node202 bin]$ ./ksql -U system -W 123456 test -h 192.168.1.201
ksql (V008R003C002B0370)
Type "help" for help.

test=# select sys_is_in_recovery();
 sys_is_in_recovery
--------------------
 t
(1 row)

---通过以上判断，此节点的ksql存在问题，应该不是认证问题。

2、查看ksql的动态库

1）当前节点

---ksql版本：
[kingbase@node201 bin]$ ./ksql -V
ksql (Kingbase) V008R003C002B0370

---动态库加载：
[kingbase@node201 bin]$ ldd ksql
        linux-vdso.so.1 =>  (0x00007ffc33da4000)
        libkci.so.5 => /opt/Kingbase/ES/R6_C8/Server/lib/libkci.so.5 (0x00007f7cdc815000)
        libxml2.so.2 => /opt/Kingbase/ES/R6_C8/Server/lib/libxml2.so.2 (0x00007f7cdc4a4000)
        libssl.so.10 => /lib64/libssl.so.10 (0x00007f7cdc232000)
        libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f7cdbdcf000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f7cdbbb8000)
        libreadline.so.5 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libreadline.so.5 (0x00007f7cdb97c000)
        libtermcap.so.2 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libtermcap.so.2 (0x00007f7cdb777000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f7cdb56e000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f7cdb337000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f7cdb133000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f7cdae30000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f7cdaa6f000)
        libssl.so.1.1 => /opt/Kingbase/ES/R6_C8/Server/lib/libssl.so.1.1 (0x00007f7cda7d7000)
        libcrypto.so.1.1 => /opt/Kingbase/ES/R6_C8/Server/lib/libcrypto.so.1.1 (0x00007f7cda2d5000)
        libldap_r-2.4.so.2 => /opt/Kingbase/ES/R6_C8/Server/lib/libldap_r-2.4.so.2 (0x00007f7cda08a000)
        libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f7cd9d82000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f7cd9b6b000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f7cd994f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f7cdcc06000)
        liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f7cd972a000)
        libgssapi_krb5.so.2 => /opt/Kingbase/ES/R6_C8/Server/lib/libgssapi_krb5.so.2 (0x00007f7cd94d9000)
        libkrb5.so.3 => /opt/Kingbase/ES/R6_C8/Server/lib/libkrb5.so.3 (0x00007f7cd91eb000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f7cd8fe7000)
        libk5crypto.so.3 => /opt/Kingbase/ES/R6_C8/Server/lib/libk5crypto.so.3 (0x00007f7cd8db2000)
        libfreebl3.so => /lib64/libfreebl3.so (0x00007f7cd8baf000)
        liblber-2.4.so.2 => /opt/Kingbase/ES/R6_C8/Server/lib/liblber-2.4.so.2 (0x00007f7cd899e000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f7cd8784000)
        libsasl2.so.3 => /opt/Kingbase/ES/R6_C8/Server/lib/libsasl2.so.3 (0x00007f7cd8565000)
        libkrb5support.so.0 => /opt/Kingbase/ES/R6_C8/Server/lib/libkrb5support.so.0 (0x00007f7cd8352000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f7cd814e000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f7cd7f26000)
        libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f7cd7cc5000)

2）其他节点ksql动态库（正常节点）

---ksql版本：
[kingbase@node202 bin]$ ./ksql -V
ksql (Kingbase) V008R003C002B0370

---动态库加载：
[kingbase@node202 bin]$ ldd ksql
        linux-vdso.so.1 =>  (0x00007ffd3f3f3000)
        libkci.so.5 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libkci.so.5 (0x00007f5298575000)
        libxml2.so.2 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libxml2.so.2 (0x0000003c36600000)
        libssl.so.10 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libssl.so.10 (0x00007f5298304000)
        libcrypto.so.10 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libcrypto.so.10 (0x00007f5297edb000)
        libz.so.1 => /usr/lib64/libz.so.1 (0x00007f5297ca2000)
        libreadline.so.5 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libreadline.so.5 (0x00007f5297a66000)
        libtermcap.so.2 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libtermcap.so.2 (0x00007f5297861000)
        librt.so.1 => /usr/lib64/librt.so.1 (0x00007f5297658000)
        libcrypt.so.1 => /usr/lib64/libcrypt.so.1 (0x00007f5297421000)
        libdl.so.2 => /usr/lib64/libdl.so.2 (0x00007f529721d000)
        libm.so.6 => /usr/lib64/libm.so.6 (0x00007f5296f1a000)
        libc.so.6 => /usr/lib64/libc.so.6 (0x00007f5296b4c000)
        libpthread.so.0 => /usr/lib64/libpthread.so.0 (0x00007f5296930000)
        libfreebl3.so => /usr/lib64/libfreebl3.so (0x00007f529672c000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f52987b8000)

3）动态库加载对比
通过正常节点和当前异常节点ksql的动态库加载对比看，异常节点的ksql动态库和正常节点有许多不同点，加载库文件的目录不同。
异常节点：

正常节点：

4）查看异常节点LD_LIBRARY_PATH
如下所示，在异常节点的LD_LIBRARY_PATH变量指向了V8R6的lib目录，ksql的动态库加载了V8R6的库文件：

[kingbase@node201 ~]$ cat .bashrc
export LD_LIBRARY_PATH=/opt/Kingbase/ES/R6_C8/Server/lib:/lib:/usr/lib:/lib64

三、解决方案
在异常节点，注释LD_LIBRARY_PATH变量配置

查看ksql动态库加载：

[kingbase@node201 bin]$ ldd ksql
        linux-vdso.so.1 =>  (0x00007ffffbbc2000)
        libkci.so.5 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libkci.so.5 (0x00007f2301a98000)
        libxml2.so.2 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libxml2.so.2 (0x0000003c36600000)
        libssl.so.10 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libssl.so.10 (0x00007f2301827000)
        libcrypto.so.10 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libcrypto.so.10 (0x00007f23013fe000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f23011cd000)
        libreadline.so.5 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libreadline.so.5 (0x00007f2300f91000)
        libtermcap.so.2 => /home/kingbase/cluster/R3HA/db/bin/./../lib/libtermcap.so.2 (0x00007f2300d8c000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f2300b83000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f230094c000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f2300748000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f2300445000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f2300084000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f22ffe68000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f2301cdb000)
        libfreebl3.so => /lib64/libfreebl3.so (0x00007f22ffc64000)

数据库登录：（数据库登录正常）

[kingbase@node201 bin]$ ./ksql -h 192.168.1.201 -U system -W 123456 test
ksql (V008R003C002B0370)
Type "help" for help.

四、总结
在主机上存在多个版本数据库时，注意系统变量LD_LIBRARY_PATH的配置。