Ceph 时钟偏移问题 clock skew detected 解决方案--- 部署内网NTP服务
告警:HEALTH_WARN clock skew detected on mon.ip-10-25-195-6; 8 requests are blocked > 32 sec; Monitor clock skew detected
原因:
MON可能被MON节点之间的重要的时钟偏移激烈的影响。这经常会转变为没有明显原因的诡异的行为。为了避免这种问题,你应该在MON节点上运行一个时间同步的工具。
默认最大容忍的时钟偏移为0.05s,不建议修改这个官方开发推荐值。私自未经测试修改虽然无数据丢失风险,可能会对MON集群和总体集群健康导致意外的作用。
如果你遇到这个告警,同步时钟,在MON上运行NTP的客户端会有帮助。如果经常遇到这个问题,可能是因为使用了远端的NTP服务器,请考虑在内网部署NTP服务器。
在MON/OSD/RGW节点部署NTP服务的操作步骤:
1. apt-get install ntp
2. 修改ntp配置 /etc/ntp.conf
3. 重启ntp服务
/etc/ntp.conf
# /etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help #sync time interval tinker step 0.5 driftfile /var/lib/ntp/ntp.drift # Enable this if you want statistics to be logged. #statsdir /var/log/ntpstats/ statistics loopstats peerstats clockstats filegen loopstats file loopstats type day enable filegen peerstats file peerstats type day enable filegen clockstats file clockstats type day enable # Specify one or more NTP servers. # Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board # on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for # YY NTP servers server 121.14.36.27 iburst server 221.228.209.14 iburst server 183.57.81.221 iburst server 113.142.33.209 iburst server 106.38.198.9 iburst # Use Ubuntu's ntp server as a fallback. #pool ntp.ubuntu.com # Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for # details. The web page <http://support.ntp.org/bin/view/Support/AccessRestrictions> # might also be helpful. # # Note that "restrict" applies to both servers and clients, so a configuration # that might be intended to block requests from certain clients could also end # up blocking replies from your own upst