Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

[Dario Faggioli <dario.faggioli@xxxxxxxxxx>]

On Mon, 2017-02-20 at 19:38 +0000, Julien Grall wrote:
> On 20/02/17 19:20, Dario Faggioli wrote:
> > E.g., if vCPU x of domain A wants to go idle with a WFI/WFE, but
> > the
> > host is overbooked and currently really busy, Xen wants to run some
> > other vCPU (of either the same of another domain).
> > 
> > That's actually the whole point of virtualization, and the reason
> > why
> > overbooking an host with more vCPUs (from multiple guests) than it
> > has
> > pCPUs works at all. If we start letting guests put the host's pCPUs
> > to
> > sleep, not only the scheduler, but many things would break, IMO!
> 
> I am not speaking about general case but when you get 1 vCPU pinned
> to 1 
> pCPU (I think this is Stefano use case). No other vCPU will run on
> this 
> pCPU. So it would be fine to let the guest do the WFI.
> 
Mmm... ok, yes, in that case, it may make sense and work, from a, let's
say, purely functional perspective. But still I struggle to place this
in a bigger picture.

For instance, as you say, executing a WFI from a guest directly on
hardware, only makes sense if we have 1:1 static pinning. Which means
it can't just be done by default, or with a boot parameter, because we
need to check and enforce that there's only 1:1 pinning around.

Is it possible to decide whether to trap and emulate WFI, or just
execute it, online, and change such decision dynamically? And even if
yes, how would the whole thing work? When the direct execution is
enabled for a domain we automatically enforce 1:1 pinning for that
domain, and kick all the other domain out of its pcpus? What if they
have their own pinning, what if they also have 'direct WFI' behavior
enabled?

If it is not possible to change all this online and on a per-domain
basis, what do we do? When dooted with the 'direct WFI' flag, we only
accept 1:1 pinning? Who should enforce that, the setvcpuaffinity
hypercall?

These are just examples, my point being that in theory, if we consider
a very specific usecase or set of usecase, there's a lot we can do. But
when you say "why don't you let the guest directly execute WFI", in
response to a patch and a discussion like this, people may think that
you are actually proposing doing it as a solution, which is not
possible without figuring out all the open questions above (actually,
probably, more) and without introducing a lot of cross-subsystem
policing inside Xen, which is often something we don't want.

But, if you let me say this again, it looks to me we are trying to
solve too many problem all at once in this thread, should we try
slowing down/refocusing? :-)

> If you run multiple vCPU in the same pCPU you would have a bigger 
> interrupt latency. And blocked the vCPU or yield will likely have
> the 
> same number unless you know the interrupt will come right now. 
>
Maybe. At least on x86, that would depend on the actual load. If all
your pCPUs are more than 100% loaded, yes. If the load is less than
that, you may still see improvements.

> But in 
> that case, using WFI in the guest may not have been the right things
> to do.
> 
But if the guest is, let's say, Linux, does it use WFI or not? And is
it the right thing or not?

Again, the fact you're saying this probably means there's something I
am either missing or ignoring about ARM.

> I have heard use case where people wants to disable the scheduler
> (e.g a 
> nop scheduler) because they know only 1 vCPU will ever run on the
> pCPU. 
> This is exactly the use case I am thinking about.
> 
Sure! Except that, in Xen, we don't know whether we have, and always
will, 1 vCPU ever run on each pCPU. Nor we have a way to enforce that,
neither in toolstack nor in the hypervisor. :-P

> > So, I'm not sure what we're talking about, but what I'm quite sure
> > is
> > that we don't want a guest to be able to decide when and until what
> > time/event, a pCPU goes idle.
> 
> Well, if the guest is not using the WFI/WFE at all you would need an 
> interrupt from the scheduler to get it running. 
>
If the guest is not using WFI, it's busy looping, isn't it?

> So here it is similar, 
> the scheduler would have setup a timer and the processor will awake
> when 
> receiving the timer interrupt to enter in the hypervisor.
> 
> So, yes in fine the guest will waste its slot. 
>
Did I say it already that this concept of "slots" does not apply here?
:-D

> Cheers,
> 
Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel