Doris forentend: Environment invalid because of previous exception: (JE 18.3.12)

错误日志

2023-07-18 19:45:58,493 INFO (main|1) [PaloFe.start():124] Palo FE starting...
2023-07-18 19:45:58,503 INFO (main|1) [FrontendOptions.analyzePriorityCidrs():107] configured prior_cidrs value: 10.XXX.XXX.2/24
2023-07-18 19:45:58,525 INFO (main|1) [FrontendOptions.init():87] local address: /10.XXX.XXX.2.
*******
2023-07-18 19:45:59,945 INFO (main|1) [Env.getFeNodeTypeAndNameFromHelpers():1158] get fe node type FOLLOWER, name 10.XXX.XXX.2_9010_1689680924169 from 10.XXX.XXX.1:8130
2023-07-18 19:46:00,101 WARN (main|1) [Env.getNewImage():1591] get an image with a lower version, localImageVersion: 0, got version: 0
2023-07-18 19:46:00,101 INFO (main|1) [Env.getClusterIdAndRole():1103] finished to get cluster id: 1832780415, role: FOLLOWER and node name: 10.XXX.XXX.2_9010_1689680924169
2023-07-18 19:46:00,123 INFO (main|1) [Env.loadImage():1668] image does not exist: /data/doris/doris-meta/image/image.0
2023-07-18 19:46:00,826 INFO (UNKNOWN 10.XXX.XXX.2_9010_1689680924169(-1)|1) [BDBEnvironment.setup():162] add helper[10.XXX.XXX.1:9010] as ReplicationGroupAdmin
2023-07-18 19:46:00,826 INFO (UNKNOWN 10.XXX.XXX_9010_1689680924169(-1)|1) [BDBEnvironment.setup():168] add self[10.XXX.XXX.2:9010] as ReplicationGroupAdmin
2023-07-18 19:46:00,835 WARN (UNKNOWN 10.XXX.XXX.2_9010_1689680924169(-1)|1) [Env.notifyNewFETypeTransfer():2377] notify new FE type transfer: UNKNOWN
2023-07-18 19:46:00,920 WARN (RepNode 10.XXX.XXX.2_9010_1689680924169(-1)|67) [BDBStateChangeListener.stateChange():57] this node is DETACHED
2023-07-18 19:46:10,877 ERROR (UNKNOWN 10.XXX.XXX.2_9010_1689680924169(-1)|1) [BDBEnvironment.setup():201] error to open replicated environment. will exit.
com.sleepycat.je.EnvironmentFailureException: (JE 18.3.12) Environment must be closed, caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 18.3.12)
 10.XXX.XXX.2_9010_1689680924169(-1):/data/doris/doris-meta/bdb Clock delta: 166523 ms. between Feeder: 10.XXX.XXX.1_9010_1689680864166 and this Replica exceeds max permissible delta: 5000 ms.
 HANDSHAKE_ERROR: Error during the handshake between two nodes. Some validity or compatibility check failed, preventing further communication between the nodes. Environment is invalid and must be closed. 
Originally thrown by HA thread: RepNode 10.XXX.XXX.2_9010_1689680924169(-1) Originally thrown by HA thread: RepNode 10.XXX.XXX.2_9010_1689680924169(-1)
        at com.sleepycat.je.EnvironmentFailureException.wrapSelf(EnvironmentFailureException.java:230) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.dbi.EnvironmentImpl.checkIfInvalid(EnvironmentImpl.java:1835) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:151) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Environment.makeEnvironmentImpl(Environment.java:278) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.Environment.<init>(Environment.java:258) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.ReplicatedEnvironment.<init>(ReplicatedEnvironment.java:605) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.ReplicatedEnvironment.<init>(ReplicatedEnvironment.java:464) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.ReplicatedEnvironment.<init>(ReplicatedEnvironment.java:538) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at org.apache.doris.journal.bdbje.BDBEnvironment.setup(BDBEnvironment.java:154) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.journal.bdbje.BDBJEJournal.open(BDBJEJournal.java:303) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.persist.EditLog.open(EditLog.java:1038) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.initialize(Env.java:863) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.PaloFe.start(PaloFe.java:138) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.PaloFe.main(PaloFe.java:73) ~[doris-fe.jar:1.2-SNAPSHOT]
Caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 18.3.12) 10.XXX.XXX.2_9010_1689680924169(-1):/data/doris/doris-meta/bdb Clock delta: 166523 ms.
 between Feeder: 10.XXX.XXX.1_9010_1689680864166 and this Replica exceeds max permissible delta: 5000 ms. HANDSHAKE_ERROR: Error during the handshake between two nodes. 
Some validity or compatibility check failed, preventing further communication between the nodes. Environment is invalid and must be closed. 
Originally thrown by HA thread: RepNode 10.XXX.XXX.2_9010_1689680924169(-1) Originally thrown by HA thread: RepNode 10.XXX.XXX.2_9010_1689680924169(-1)
        at com.sleepycat.je.rep.stream.ReplicaFeederHandshake.checkClockSkew(ReplicaFeederHandshake.java:432) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.stream.ReplicaFeederHandshake.execute(ReplicaFeederHandshake.java:269) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.Replica.initReplicaLoop(Replica.java:709) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoopInternal(Replica.java:485) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.Replica.runReplicaLoop(Replica.java:412) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]
        at com.sleepycat.je.rep.impl.node.RepNode.run(RepNode.java:1869) ~[je-18.3.14-doris-SNAPSHOT.jar:18.3.14-doris-SNAPSHOT]

异常原因

服务器时间同步问题 延迟大于5000ms