Re: [Xen-devel] BUG: NOW() seems to (sometimes) go backwards!

[Joao Martins <joao.m.martins@xxxxxxxxxx>]

>>>> Yet when the scaling values get set only once at boot, monotonic
>>>> (cross-CPU) TSC means monotonic (cross-CPU) returns from NOW().
>>>>
>>> Yep. And at this point, this is what needs to be verified, I guess...
>> I think get_s_time_fixed doesn't guarantee monotonicity across CPUs being it
>> different socket or (SMT) siblings. local_tsc_stamp is seeded differently on 
>> each CPU
>> i.e. rdtsc() right after doing the platform time read on the calibration 
>> rendezvous.
>> Plus stime_local_stamp is seeded with values taken from platform timer 
>> (HPET, ACPI,
>> PIT) on local_time_calibration which means that get_s_time isn't solely 
>> based on TSC
>> and that there will always be a gap between stime_local_stamp and 
>> local_tsc_stamp.
>> TSC is indeed monotonic on a TSC invariant box, but the delta that is 
>> computed
>> differs from cpu to cpu according to when time calibration happens on each 
>> CPU - thus
>> not guaranteeing the desired monotonicity property. Having stime_local_stamp 
>> be based
>> on the same timestamp that of the local_tsc_stamp plus having a single
>> local_tsc_stamp as reference would address this behaviour - see also below.
> 
> The quality of get_s_time_fixed() output indeed heavily depends on
> t->local_tsc_stamp and t->stime_local_stamp being a properly
> matched pair. Yet in local_time_calibration()'s constant-TSC case,
> they're a direct copy from the respective cpu_calibration fields.
> The
> main issue I could see here is that on SMT siblings the hardware
> switching between the two may introduce arbitrary delays between
> them. And with CPU frequency changes, the latency between the
> rdtsc() and the execution of get_s_time() finishing could also be
> pretty variable. I wonder whether c->local_tsc_stamp wouldn't
> better be written with the TSC value used by get_s_time() (or,
> which should amount to the same effect, whether we shouldn't
> simply call get_s_time_fixed() here with the just sampled TSC value).
Indeed, but notice that in this copy for the constant-TSC case:
t->stime_local_stamp is written with c->stime_master_stamp - ending
up the former being discarded. So even changing that pair to correctly
match it wouldn't change the result. At point of which I wonder if copying
rendezvous c->stime_master_stamp to t->stime_local_stamp on
local_time_calibration is correct?

Joao

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel