Ceph: HEALTH_WARN clock skew detected

17,052

I don't know why but it have happened because of command in ntpd-gq. This command updates your datatime from servers which you have written in ntp.conf and then stops.
I can't figure out why ceph fails after this command but when I changed it to:

ntpdate 0.ua.pool.ntp.org

Ceph became work.

Share:
17,052

Related videos on Youtube

Oleksandr
Author by

Oleksandr

Updated on September 18, 2022

Comments

  • Oleksandr
    Oleksandr over 1 year

    I have configured ntp on Ceph nodes and time is synchronized! But ceph status always shows clock skew.

    ceph health detail shows:
    mon.node2 aadr 192.168.56.102:6789/0 clock skew 7192.45s > max 0.05s (latency 0.0129368s)
    mon.node3 aadr 192.168.56.103:6789/0 clock skew 7192.46s > max 0.05s (latency 0.00915873s)
    

    How to force Ceph to update status? All time on nodes are equal! If I execute date command on all nodes then it shows correct and same time. But Ceph thinks that time is skewed on 7192.45s!

    It is my configuration on nodes /etc/ntp.conf:

    driftfile /vat/lib/ntp/drift
    
    restrict 0.0.0.0 mask 0.0.0.0
    
    server 0.ua.pool.ntp.org iburst
    server 1.ua.pool.ntp.org iburst
    server 2.ua.pool.ntp.org iburst
    server 3.ua.pool.ntp.org iburst
    
    includefile /etc/ntp/crypto/pw
    keys /etc/ntp/keys
    disable monitor
    

    It is my /etc/rc.local on nodes:

    touch /var/lock/subsys/local
    /sbin/iptables-restore < /etc/sysconfig/iptables
    /sbin/ntpd -gq
    /sbin/hwclock --systohc
    systemctl enable ntpd.service
    systemctl start ntpd.service
    

    systemctl status ntpd.service shows that service is active and running.

    In addition some strange things happens with ceph if I put peers in /etc/ntp.conf. Ceph becomes unresponsible and ceph -s shows nothing (it freezes).

    Please, help me.