I remember having had a problem with two servers when the time 
discrepency reached 5mn (It was for kerberos authentication). We 
discovered that one of the two was not synchronized by NTP at all. The 
problem showed up after 4 years with many reboots and no clock 
adjustment on the non-synchronized host.

     NTP does not adjust the RTC brutally; it seems to adjust slowly the 
frequency so that synchronization happens without the process being 
noticeable to other apps - it can take hours. On shutdown it saves the 
RTC settings in /var/lib/ntp/ntp.drift, and (AFAIU) 
/var/lib/hwclock/adjtime. After some days of running NTP, your RTC is 
well trimmed and does not drift much. It does not stop running when you 
power-off your computer. Therefore, at startup the discrepancy is very 
small - unless the battery of the RTC is dead or you stopped the 
computer for a year.

     Everything above is non-authoritative; it is the result of my own 
observations.

     Didier