[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Credit scheduler


  • To: pak333@xxxxxxxxxxx,Atsushi SAKAI <sakaia@xxxxxxxxxxxxxx>
  • From: Mats Petersson <mats@xxxxxxxxxxxxxxxxx>
  • Date: Thu, 12 Jul 2007 22:25:23 +0100
  • Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
  • Delivery-date: Tue, 17 Jul 2007 05:09:02 -0700
  • Domainkey-signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=beta; h=received:x-mailer:date:to:from:subject:cc:in-reply-to:references:mime-version:content-type:sender:message-id; b=MCSxejrgRKQhfxl+NjVCZA0ThfBHKtlJ30f6WWRBpVpn6h5sM1uUSJvUHOXitQSDXHnxzfltsE//HAmerCCpwXTYQq4MMc5q5RJxl5kH8uBV4+pTCqoYcNk2NioLV55moJorH+MxZqiG7XfNMitYF9d9fqDGmC/dthsdkmk/nxg=
  • List-id: Xen developer discussion <xen-devel.lists.xensource.com>

At 21:46 12/07/2007, pak333@xxxxxxxxxxx wrote:
Hi Atsushi

I am still confused, so let me explain what i think should happen and understand from you why it may not happen.

I have 2 cpu intensive VMs and 1 IO intensive VM. My system has 4 physical cpus and 8 virtual cpus.

Now using default parameters for the credit scheduler, the cpu intersive vcpu should run for 30 millsec. It does not, it runs for less (few microsecs).

You said this could be because of softirq which are raised after a system call. Can i disable the softirqs. What happens if i do? If i cannot disable is there a way to see what VM is raising the softirq. If the cpu-VM raises the softirq it gets preempted or does it continue to run. How can I montior the softirqs raised by this VM. Can I tell if the premption is from a softirq or from something else.

I don't believe you can "disable" a softirq - you could of course avoid doing the call to "do_softirq", but that is most likely just going to completely remove any scheduling capability at all in the system, making the system "single-tasking". Not a recommended way, in my opinion.


Also, if the IO VM requests an IO, it will block and dom0 wakes up and gets scheduled to run to service the IO. Will it preempt the cpu intensive VM. If so why? Shouldn't the cpuintensiveV M get its quantum of time. Or does the IOVM get higher priority to preempt the cpu intensive VM. How does the scheduler pick which cpu to run dom0 on ( if all the vcpus running are cpu intensive)? If there is a mix of cpu-vcpus and Io-vcpus, which will be the victim


The whole idea of having separate queues for IO and CPU intensive VMs (or processes in a normal kernel scenario) is to allow the IO-intensive tasks to avoid waiting for the next time-slot when a CPU-hogging VM/process is running [1]. The normal behaviour for the scheduler is to determine dynamically if the current VM/process is IO or CPU intensive.

Unless Dom0 is used for doing some silly CPU-intensive task (calculating PI with a million decimal points or compiling Xen + Linux kernel, for example), it will most likely look to the scheduler like a IO-intensive VM. So it will have the same priority as any other IO-intensive VM (assuming Dom0 has equal scheduler parameters as other guests, which I'm pretty sure is the default behaviour).

If, like in your example, there are a number of processors busy with CPU-intensive tasks, and others with IO-intensive tasks, the most likely scenario is that any new IO-request will go be run on a "previously CPU-intensive" CPU - but on the other hand, if you have a lot of IO-intensive processing, there should be some processor(s) that are "asleep" - unless all CPU's are busy running CPU-intensive tasks...

Note that scheduling is a complex subject, and there have been dozens of PhD dissertations written on the subject.

[1] The idea being that if we process the IO-request that just completed, we can set up another IO-request, and this will get better IO-throughput than waiting a long time (relatively speaking 30ms is an eternity to the processor).

--
Mats

As you see i have a lot of questions, so please bear with me and help me understand the scheduling behavior of Xen

Thanks
Prabha




-------------- Original message --------------
From: Atsushi SAKAI <sakaia@xxxxxxxxxxxxxx>

> Hi Prabha
>
> Please check do_softirq in assembler.
> for x86
> call do_softirq
> is in.
>
> And usually do_softirq is executed at the end of system call.
>
> Thanks
> Atsushi SAKAI
>
>
>
> pak333@xxxxxxxxxxx wrote:
>
> > Thanks for the explanation, i am not sure i understood it correctly.
> >
> >
> > when does softirq get set. So when softirq is set a processor enters the
> scheduler every 3 micro secs? where in the source code is this handled. please
> could you send me some pointers
> >
> > thanks
> > -Prabha
> >
> >
> >
> >
> > -------------- Or iginal message --------------
> > From: Atsushi SAKAI
> >
> > > Hi,$B!!(BPrabha
> > >
> > > $B!!(B30msec is the maximum time slice.
> > > And I guess your 3 microsecond is the response after
> > > SCHEDULE_SOFTIRQ is set.
> > > As you know,
> > > If SCHEDULE_SOFTIRQ is set,
> > > it waits softirq calls schedule(). (this time interval is 3microsec)
> > >
> > > And I/O intensive has higher priority than CPU intensive.
> > > So I/O intensive job is first dispatched domain in runq.
> > > This is because latency improvement for I/O intensive guest.
> > >
> > > So your behavior is not strange.
> > >
> > > Thanks
> > > Atsushi SAKAI
> > >
> > >
> > > pak333@xxxxxxxxxxx wrote:
> &g
t; >
> > > > Hi
> > > >
> > > > I have noticed that VMs are typically spending 3 microsecs or less before
> they
> > > are being prempted. I thought that the credit schduler time slice was 30 ms.
> I
> > > have 4 VMs running and they are all cpu intensive except for 1 (which is IO > > > intensive) but having a VM spend max 3 micro secs before being kicked out
> seems
> > > strange.
> > > >
> > > > Is there something else going on that i am not aware of. Is the time slice > > > really 30 millisecs? I am using default parameters of the credit scheduler.
> > > >
> > > > Thanks
> > > > -Prabha
> > >
> > >
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxx source.com
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.