Re: [Xen-devel] Linux spin lock enhancement on xen

[Jeremy Fitzhardinge <jeremy@xxxxxxxx>]

 On 08/17/2010 06:58 PM, Mukesh Rathor wrote:
>> How does this compare with Jeremy's existing paravirtualised spinlocks
>> in pv_ops? They required no hypervsior changes. Cc'ing Jeremy.
>> -- Keir
> Yeah, I looked at it today. What pv-ops is doing is forcing a yield
> via a fake irq/event channel poll, after storing the lock pointer in
> a per cpu area. The unlock'er then IPIs the vcpus waiting. The lock
> holder may not be running tho, and there is no hint to hypervisor
> to run it. So you may have many waitor's come and leave for no
> reason.

(They don't leave for no reason; they leave when they're told they can
take the lock next.)

I don't see why the guest should micromanage Xen's scheduler decisions. 
If a VCPU is waiting for another VCPU and can put itself to sleep in the
meantime, then its up to Xen to take advantage of that newly freed PCPU
to schedule something.  It may decide to run something in your domain
that's runnable, or it may decide to run something else.  There's no
reason why the spinlock holder is the best VCPU to run overall, or even
the best VCPU in your domain.

My view is you should just put any VCPU which has nothing to do to
sleep, and let Xen sort out the scheduling of the remainder.

> To me this is more of an overhead than needed in a guest. In my
> approach, the hypervisor is hinted exactly which vcpu is the 
> lock holder.

The slow path should be rare.  In general locks should be taken
uncontended, or with brief contention.  Locks should be held for a short
period of time, so risk of being preempted while holding the lock should
be low.  The effects of the preemption a pretty disastrous, so we need
to handle it, but the slow path will be rare, so the time spent handling
it is not a critical factor (and can be compensated for by tuning the
timeout before dropping into the slow path).

>  Often many VCPUs are pinned to a set of physical cpus
> due to licensing and other reasons. So this really helps a vcpu
> that is holding a spin lock, wanting to do some possibly real
> time work, get scheduled and move on.

I'm not sure I understand this point.  If you're pinning vcpus to pcpus,
then presumably you're not going to share a pcpu among many, or any
vcpus, so the lock holder will be able to run any time it wants.  And a
directed yield will only help if the lock waiter is sharing the same
pcpu as the lock holder, so it can hand over its timeslice (since making
the directed yield preempt something already running in order to run
your target vcpu seems rude and ripe for abuse).

>  Moreover, number of vcpus is
> going up pretty fast.

Presumably the number of pcpus are also going up, so the amount of
per-pcpu overcommit is about the same.

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel