WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] credit scheduler and HYPERVISOR_yield()

To: John Levon <levon@xxxxxxxxxxxxxxxxx>
Subject: Re: [Xen-devel] credit scheduler and HYPERVISOR_yield()
From: Emmanuel Ackaouy <ackaouy@xxxxxxxxx>
Date: Sun, 14 Oct 2007 21:20:50 +0200
Cc: George Dunlap <gdunlap@xxxxxxxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Sun, 14 Oct 2007 12:21:19 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:in-reply-to:references:mime-version:content-type:message-id:content-transfer-encoding:cc:from:subject:date:to:x-mailer; bh=0bLfMH1Ob8VX0RJ1f2FqD6JVbrTtIxuv7ZL8njCbr+4=; b=AXvDOqD4tbc+/a2Wd3ONET9We6Qxcw2vLVonaRHbgT5M6A8GurNijHHnTEAkiMSX/VjLFts7R8B/hb+LG8j8wkEcHWcI1Qd19IvnGKMroqW/b5/aoaD508fyqFtQDgehnXV53XrvmjCyllwynFwJLP3fSRoD4cS2Vq2SxEhqEYI=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:in-reply-to:references:mime-version:content-type:message-id:content-transfer-encoding:cc:from:subject:date:to:x-mailer; b=AEYmozRYCuyE0epsDCDJJgYBvUOqHHA8+Yl4vbugjqyqHe1xzyji8vW/slRI2LqHobkB09HeMwZWuOTI7qpQmrWNCtXxt0K4sYSv4sAHGQGcRF4MAXSWFIFZif01SgFzH85YSUPF3/IjhTi/1oxYcRLi94W4haatucKv7lrS+bc=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <20071014184528.GB16827@xxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <20071008234118.GA1396@xxxxxxxxxxxxxxxxxxxxxxx> <7e903d73fbea1bb8ca97396fef058b2b@xxxxxxxxx> <20071009121533.GA30258@xxxxxxxxxxxxxxxxxxxxxxx> <de76405a0710090622x63a1c34exc7a14c391782211b@xxxxxxxxxxxxxx> <20071014184528.GB16827@xxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
On Oct 14, 2007, at 20:45, John Levon wrote:
Emmanuel Ackaouy wrote:
It may be worthwhile to consider if yield() can be replaced with
more intelligent mechanisms for VCPU synchronization of SMP
guests. In the case of ACKed IPIs for example, if all target VCPUs
are not running at the time of the IPI initiation, it might be a good
idea to put the source to sleep until all targets have ACKed.
If all target VCPUs are running though, I suspect things will work
best if the IPI initiator does not yield at all.

This seems like a bad idea since we may be IPIing to several CPUs and we
don't want to sleep whilst we can usefully move on and IPI the other
CPUs (even if they can't quite respond yet).

Why can't you initiate the IPI to all the destination CPUs first
and then wait for them to ACK, going to sleep if it looks like at
least one of them won't be able to ACK in a reasonable
timeframe (for example if it is asleep or on the run queue of
the running VCPU's physical CPU)?

I'm probably not understanding what you're trying to do?


On Tue, Oct 09, 2007 at 02:22:13PM +0100, George Dunlap wrote:
A simple option would be, for the credit scheduler, to temporarily
reduce the priority from TS_UNDER to TS_OVER.  This will cause it to

We prototyped this change and it made quite a difference (though didn't
solve our problems entirely). Would it be possible to get a proper fix
available?

Doing the change that George proposed may help in your case
but I suspect that, as I described in my previous post, it will cause
problems for other workloads.

I think it is reasonable for a yield() operation to yield to runnable
VCPUs of equal or higher priority than the running VCPU. That
is the behavior of the scheduler today. Maybe your problem can
be addressed without changing the behavior of yield?

With that said, it's unlikely that I'll be making a change to the
scheduler myself: I haven't worked at XenSource for some time
now and don't have the resources (not to mention time) to test
any such change. I'm happy to learn about your problem and
suggest potential fixes but I'm probably not the person you need
to convince if you want to make a significant scheduler change
these days. Arguably, a number of things need to be done in
the Xen scheduler and synchronization primitives to improve
the performance of SMP guests. It may be worthwhile to have
a generic discussion about that on top of the specific problem
you're encountering.

Cheers,
Emmanuel.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel