Re: [Xen-devel] timekeepeing in XEN - request fo comments

[Jeremy Fitzhardinge <jeremy@xxxxxxxx>]

On 11/09/09 04:21, Bartosz Lis wrote:
> I have some issues with time synchronization among dom0 and domUs. In my case 
> helped reconfiguring kernel with HZ=1000 for dom0 and HZ=100 for domUs and 
> running ntpd also on domUs. I think, running ntpd on domU is not a good idea, 
>   

Why not?

> but it works acceptably good as a workaround for my configuration: XEN 3.4.1 
> + 
> kernel 3.6.31.5 with pv_ops (taken from git last week). When I don't run ntpd 
> on domU clock slowly drifts. I have choosen xen clocksource in both dom0 and 
> domU.
>   

Drifts with respect to what?

If you run all domains without ntp then they should remain in lockstep,
because they are all being run off the same timebase.  But the xen
clocksource does have a drift correction factor which can be updated via
ntpd/adjtimex, so if each kernel updates its factor differently then
they will appear to drift with respect to each other.

> I'm not the only one to have time synchronization issues. Please write some 
> document specifying how time synchronization between dom0 and domU should be 
> achieved. I have a number of special questions, but probably there are more 
> questions that should be aswered:
>
> 1. What is the mechanics behind the clocksource named xen? How does it depend 
> on phisical time sources available on different platforms?
>   

You can think of it as an augmented tsc; the tsc is used to convey
fine-grained time information, and a set of additional parameters allow
the guest to correct for things like tsc skew between cpus.  (And,
ideally, differing tsc rates, but it is hard to get things perfectly
monotonic in that case.)

> 2. How gets domU's clock initialized when domU boots? (I observe that just 
> after domU has started its clock differs from dom0. The difference is about 
> 10s.)
>   

It reads the Xen wallclock at boot to initialize the time of day.  Xen's
wallclock time may not be correct/match dom0 if dom0 has failed to
update it (which is currently the case in pvops dom0 kernels).

> 3. What are the consequences of not selecting xen clocksource (by selecting 
> one of hpet, acpi_pm, etc.) for dom0 and for domU?
>   
Xen itself uses and claims ownership of hardware resources like hpet and
acpi_pm, so they aren't available to guest kernels.  DomU kernels simply
don't have any hardware access; dom0 is specifically not allowed to
touch hpet (and I think acpi_pm).

You could force a kernel to use tsc a clocksource, but that will only
work correctly if the system has inherently synchronized tscs, and there
would be discontinuities over suspend/resume/migrate.

> 4. What are the best practices to achieve clock synchronization?
>   

ntp is it at present.

> 5. What has changed recently in timekeeping? What is going to change? Are 
> there any improvements/bugfixes added to redhat or suse kernels? Are these 
> bugfixes planned to be added upstream?
>   

As far as I know the pvops kernel is state of the art.  It currently
fails to set the Xen wallclock time when asked (not implemented), but
aside from that it should be OK.

The basic principle is that Xen doesn't know all that much about time of
day, and doesn't really care.  It offers a token get/set time of day
interface to access the corresponding hardware, but very little beyond
that.  Time is really considered to be a local matter for domains to
work out.  If they want to synchronize, then ntp is a perfectly
reasonable mechanism for doing that.

Would you like to see any particular changes?  What are your requirements?

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel