RE: [Xen-devel] Re: Xen scheduler

["Petersson, Mats" <Mats.Petersson@xxxxxxx>]

> -----Original Message-----
> From: Emmanuel Ackaouy [mailto:ackaouy@xxxxxxxxx] 
> Sent: 24 April 2007 15:47
> To: Petersson, Mats
> Cc: pak333@xxxxxxxxxxx; xen-devel@xxxxxxxxxxxxxxxxxxx; 
> ncmike@xxxxxxxxxx
> Subject: Re: [Xen-devel] Re: Xen scheduler
> 
> Mats,
> 
> As I've just explained in my previous post to the mailing list, what
> you say is not totally correct.

Yes, I realized from your previous post that what I said was not
entirely correctly. 

> 
> On partially idle systems, the credit scheduler does schedule across
> sockets and cores before co-scheduling them. At least it used to but
> I've not looked at the code in a while so perhaps someone changed
> this?

Not that I'm aware of. I apologize for the misinformation. 

> 
> On a system with no idle PCPU, when time slicing becomes
> necessary, what you say is true: The scheduler doesn't care
> which PCPU hosts which VCPU. On NUMA systems, this
> means a VCPU may not always run closest to the memory
> it is using at the time.
> 
> On Apr 24, 2007, at 13:07, Petersson, Mats wrote:
> 
> >
> >
> >> -----Original Message-----
> >> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> >> [mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of
> >> pak333@xxxxxxxxxxx
> >> Sent: 23 April 2007 20:34
> >> To: ncmike@xxxxxxxxxx
> >> Cc: Mike D. Day; xen-devel@xxxxxxxxxxxxxxxxxxx
> >> Subject: Re: [Xen-devel] Re: Xen scheduler
> >>
> >> Thanks. A little more clarification.
> >>
> >> Here is an example.
> >>
> >> I have multiple VMs, each with 2 vcpus. There is no user
> >> affinity. So i will let the vcpus run whereever the Xen
> >> scheduler chooses. My system has 2 dual core sockets.
> >>
> >>      If all 4 pcpus are idle, then will the scheduler assign
> >> the vcpus of a VM to the same socket pcpus.
> >>      If while running, 2 pcpus from different sockets become
> >> available, the scheduler will assign 2 vcpus to those two
> >> pcpus. Does the scheduler do any optimization as to moving
> >> the vcpus of a vm to the same socket or just assign the vcpus
> >> as they become ready.
> >
> > Xen's current schedulers doesn't have any clue about CPU cores and 
> > their
> > relationship to sockets, memory locations or any other such 
> things. So
> > if you get two VCPUS from the same domain or two different 
> domains on
> > one of your physical dual core CPU is entirely random, and 
> will remain
> > so at any time a VCPU is rescheduled. Whichever PCPU happens to be 
> > ready
> > when the VCPU is schedule will be used unless you 
> specifically restrict
> > the VCPU to a (set of) PCPU(s).
> >>
> >> or if 3 pcpus are idle will the scheduler assign vcpus from a
> >> VM to the same socket.
> >
> > It will assign "any VCPU to any PCPU that is allowed for 
> that VCPU", 
> > and
> > it doesn't really care which VM or which socket any particular 
> > VCPU/PCPU
> > combination belongs to.
> >>
> >> Basically all my questions boil down to this: Does the
> >> Scheduler know about the pcpu layout(same socket) and does it
> >> do any scheduling based on that.
> >
> > Not at present. There's been some discussions on this, and 
> whilst it's
> > easy to solve some of the obvious cases, there are also some harder 
> > nuts
> > to crack. What do you do when the system is really busy and 
> there's not
> > a "good" PCPU to schedule a particular VCPU on - do you wait for the
> > PCPU that is ideal to become available, or do you schedule 
> it on a less
> > ideal PCPU? How long do you allow the wait for that ideal PCPU?
> >
> > Whilst it's easy to say "Just do it right", solving the rather hairy
> > problems of when there's congestion and making the right 
> "judgement" of
> > the situation is much harder.
> >
> > --
> > Mats
> >>
> >> Thanks
> >> Prabha
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>    -------------- Original message --------------
> >>    From: "Mike D. Day" <ncmike@xxxxxxxxxx>
> >>    
> >>    > On 21/04/07 06:03 +0000, pak333@xxxxxxxxxxx wrote:
> >>    > >
> >>    > > Hi,
> >>    > >
> >>    > >
> >>    > >
> >>    > > On running on a dual/quad core does the Xen
> >> scheduler take into
> >>    > > account the physical layout of the cores.
> >>    > >
> >>    > > For example if a VM has two vcpus, and there are 4
> >> physical cpus
> >>    > > free, will it take care to assign the 2vcpus (from
> >> a VM) to 2 pcpus
> >>    > > on the same socket.
> >>    >
> >>    >
> >>    > The scheduler only knows the affinity of vcpus for physical
> >>    > cpus. The affinity is determined by a userspace
> >> application and can
> >>    > be modified using a domain control hypercall. Look in
> >>    > xen/common/domctl.c around line 568 for the following:
> >>    >
> >>    > case XEN_DOMCTL_setvc puaffinity:
> >>    > case XEN_DOMCTL_getvcpuaffinity:
> >>    >
> >>    >
> >>    >
> >>    > When the credit scheduler migrates a vcpu to a pcpu,
> >> it only considers
> >>    > pcpus for which the affinity bit is set. If the
> >> userspace application
> >>    > sets affinity such that only the bits set for pcpus
> >> on the same
> >>    > socket, then the vcpu will only run on pcpu's sharing
> >> the same
> >>    > socket.
> >>    >
> >>    >
> >>    > Mike
> >>    >
> >>    > --
> >>    > Mike D. Day
> >>    > IBM LTC
> >>    > Cell: 919 412-3900
> >>    > Sametime: ncmike@xxxxxxxxxx AIM: ncmikeday Yahoo:
> >> ultra.runner
> >>    > PGP key: http://www.ncultra.org/ncmike/pubkey.asc
> >>    >
> >>    > _______________________________________________
> >>    > Xen-devel mailing list
> >>    > Xen-devel@xxxxxxxxxxxxxxxxxxx
> >>    > http://lists.xensource.com/xen-devel
> >>
> >>
> >
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
> >
> 
> 
> 
> 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel