Re: [Xen-users] DomU slower with smp?

["Janusz Ulanowski" <janul666@xxxxxxxxx>]

unfortunatelly I had no chance to do tests yet.
I have intel cpus
/proc/cpuinfo shows me: 
cpu cores       : 1
I checked on generic kernel 2.6.24 and it shows: cpu cores       : 4
it's strange isn't it?
what are tools to check which cpus (intel) are in exactly the same core?



processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Xeon(R) CPU           E7340  @ 2.40GHz
stepping        : 11
cpu MHz         : 2393.888
cache size      : 4096 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu de tsc msr pae cx8 apic sep mtrr cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc pni cx16 lahf_lm
bogomips        : 4789.82
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Xeon(R) CPU           E7340  @ 2.40GHz
stepping        : 11
cpu MHz         : 2393.888
cache size      : 4096 KB
physical id     : 2
siblings        : 1
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu de tsc msr pae cx8 apic sep mtrr cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc pni cx16 lahf_lm
bogomips        : 4789.82
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:


....

processor       : 15
vendor_id       : GenuineIntel
cpu family      : 6
model           : 15
model name      : Intel(R) Xeon(R) CPU           E7340  @ 2.40GHz
stepping        : 11
cpu MHz         : 2393.888
cache size      : 4096 KB
physical id     : 6
siblings        : 1
core id         : 3
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu de tsc msr pae cx8 apic sep mtrr cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc pni cx16 lahf_lm
bogomips        : 4789.82
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

--

2008/5/16 Apparao, Padmashree K <padmashree.k.apparao@xxxxxxxxx>:

/proc/cpuinfo only shows what the OS sees the cpu as (aka logical cpu).
As to what physical cores the logical cpus are mapped can be obtained by
using special tools( I don't know if you have any for AMD).

What you show below sows dom0 having 4 vcpus, but what your guest?
What is the cpu utilization of the dom0 cores? Can you get that
information from Xentop. This very much seems to be a scheduling/context
switching problem and it would be interesting to drill into this.

Thanks
-Padma



-----Original Message-----
From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx

[mailto:xen-users-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Janusz
Ulanowski
Sent: Friday, May 16, 2008 1:29 AM
To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-users] DomU slower with smp?

Hi,
I have 4 QuadCore cpus.
I've done as you said and no difference. I missed my file with all
results (I do again).
Also when I was increasing number of cpus for DomU I got worse result.
"xm vcpu-list 0" always shown me the same running vcpu: something like
that:
xm vcpu-list 0
Name                                ID  VCPU   CPU State   Time(s) CPU
Affinity
Domain-0                             0     0     0   -b-     366.6 0
Domain-0                             0     1     1   r--    1068.2 1
Domain-0                             0     2     2   -b-    1629.7 2
Domain-0                             0     3     3   -b-    1410.6 3
...
...
I'm a bit confused about numbering cpus. doesn't it matter what `cat
/proc/cpuinfo |grep ^core` in Dom0 shows ?

Thanks
Janusz


Apparao, Padmashree K wrote:
>
> Here's one more suggestion
>
> *        Try interrupt binding. Find out the irq of the nic/IO device
>
> Now affinitize the interrupts to a particular cpu (usually 0)
>
> Turn off interrupt balancing in Xen
>
> Service Irq balance stop (in linux). There is a config parameter in
Xen.
>
>
>
> *        Xm vcpu-set 0 2
>
> *        Xm vcpu-pin 0 0 0
>
> *        Xm vcpu-pin 0 1 2
>
>
>
> *        Now start your smp (with 4 vcpus) domU (say domid 2)
>
>
>
> *        Xm vcpu-pin 2 0  1
>
> *        Xm vcpu-pin 2 1 3
>
> *        Xm vcpu-pin 2 2 5
>
> *        Xm vcpu-pin 2 3 7
>
>
>
> *        Typically the cpus are numbered as cores 0,2, 4 and 6 on 1
> die (1 quad core) and the cores 1,3,5,7 on another die. You want to
> separate the domU from dom0 altogether. Also by putting the domU on a
> different die you will eliminate any cache corruption.
>
>
>
> *
>
> *        I am not familiar with how the AMD machines enumerate the
> cores but this is how the Intel machines do.
>
> *        Can you please run this config and let us know what the
> thruput is. Similarly you can run the domU with 1 vcpu and bind it to
> one of the cores.
>
> *        Also observe what the cpu% is for dom0 and domU.
>
>
>
> I hope this helps. I am interested in seeing the data on this
>
>
>
> Thanks
>
> -Padma
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> -----Original Message-----
> From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-users-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Nick
Anderson
> Sent: Thursday, May 15, 2008 5:46 PM
> To: Luke S Crawford
> Cc: Janusz Ulanowski; xen-users@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Xen-users] DomU slower with smp?
>
>
>
> Luke S Crawford wrote:
>
> > "Janusz Ulanowski" <janul666@xxxxxxxxx> writes:
>
> >
>
> >> I just made simple peformance test with iozone tool on xen 3.2.1.
>
> > ...
>
> >> It seems DomU  is much slower if using more than one cpu.
>
> Its a dual socket quad core amd. So I have 8 cores. cpus for dom0 is
set
>
> to 0 and according to xend-config.sxp that will give dom0 access to
all
>
> cpus. I havent adjusted any weighting so im not sure what would be
>
> stepping on it. The only thing running during the test was that one
>
> domU. and the domU was idle when running the test on dom0.
>
>
>
> --
>
> Nick Anderson <nick@xxxxxxxxxxxx>
>
> http://www.cmdln.org
>
> http://www.anders0n.net
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users