Re: [Xen-devel] BUG: NOW() seems to (sometimes) go backwards!

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 09.06.16 at 12:24, <joao.m.martins@xxxxxxxxxx> wrote:
>>>>> Yet when the scaling values get set only once at boot, monotonic
>>>>> (cross-CPU) TSC means monotonic (cross-CPU) returns from NOW().
>>>>>
>>>> Yep. And at this point, this is what needs to be verified, I guess...
>>> I think get_s_time_fixed doesn't guarantee monotonicity across CPUs being it
>>> different socket or (SMT) siblings. local_tsc_stamp is seeded differently 
>>> on 
> 
>>> each CPU
>>> i.e. rdtsc() right after doing the platform time read on the calibration 
>>> rendezvous.
>>> Plus stime_local_stamp is seeded with values taken from platform timer 
>>> (HPET, ACPI,
>>> PIT) on local_time_calibration which means that get_s_time isn't solely 
>>> based on TSC
>>> and that there will always be a gap between stime_local_stamp and 
>>> local_tsc_stamp.
>>> TSC is indeed monotonic on a TSC invariant box, but the delta that is 
>>> computed
>>> differs from cpu to cpu according to when time calibration happens on each 
>>> CPU - thus
>>> not guaranteeing the desired monotonicity property. Having 
>>> stime_local_stamp 
> 
>>> be based
>>> on the same timestamp that of the local_tsc_stamp plus having a single
>>> local_tsc_stamp as reference would address this behaviour - see also below.
>> 
>> The quality of get_s_time_fixed() output indeed heavily depends on
>> t->local_tsc_stamp and t->stime_local_stamp being a properly
>> matched pair. Yet in local_time_calibration()'s constant-TSC case,
>> they're a direct copy from the respective cpu_calibration fields.
>> The
>> main issue I could see here is that on SMT siblings the hardware
>> switching between the two may introduce arbitrary delays between
>> them. And with CPU frequency changes, the latency between the
>> rdtsc() and the execution of get_s_time() finishing could also be
>> pretty variable. I wonder whether c->local_tsc_stamp wouldn't
>> better be written with the TSC value used by get_s_time() (or,
>> which should amount to the same effect, whether we shouldn't
>> simply call get_s_time_fixed() here with the just sampled TSC value).
> Indeed, but notice that in this copy for the constant-TSC case:
> t->stime_local_stamp is written with c->stime_master_stamp - ending
> up the former being discarded. So even changing that pair to correctly
> match it wouldn't change the result. At point of which I wonder if copying
> rendezvous c->stime_master_stamp to t->stime_local_stamp on
> local_time_calibration is correct?

Yeah, I stumbled across that too, and I am in the process of
trying whether this in fact was just a copy-and-paste mistake
many years ago.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel