[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] NUMA problem with AMD G34 system



Brian Marcotte wrote:
I'm having a problem with NUMA on an AMD G34 system. Xen seems to think
that out of the 16 total cores, there are 12 on node0 and 4 on node3.

The box is:

    SuperMicro H8DGU (dual socket G34)
    Pair of AMD 6128 CPUs (8 core each)
    32 GB of RAM.
    BIOS has been upgraded three times - no help.

    xen-unstable from Feb 12 (same problem on 3.4.x).

I'm including the output from the "u" command below to show Xen's idea
of the CPU->node mapping. Complete boot log is here:

    http://www.panix.com/~marcotte/g34/boot.txt

Any help would be appreciated.

Have you checked that you populated the DIMM slots properly?
The SRAT CPU table seems sane, but the memory table seems to say that there is no memory on node 1 and 3, thus Xen will simply put those CPUs to node 0 as it cannot cope with memory-less nodes. If I understand the manual right, you first have to populate all the blue slots, then the black ones.

Hope that helps,
Andre.



Thanks.

- Brian

(XEN) Xen version 4.1.0-rc5-pre (root@xxxxxxxxxxxxxx) (gcc version 4.3.2 
(Debian 4.3.2-1.1) ) Sat Feb 12 18:53:49 EST 2011
(XEN) Command line: dom0_mem=600M com1=57600,8n1 console=com1 no-mce 
loglvl=debug

(XEN) 'u' pressed -> dumping numa info (now-0x2:5A61B881)
(XEN) idx0 -> NODE0 start->0 size->4325376
(XEN) phys_to_nid(0000000000001000) -> 0 should be 0
(XEN) idx3 -> NODE3 start->4325376 size->4194304
(XEN) phys_to_nid(0000000420001000) -> 3 should be 3
(XEN) CPU0 -> NODE0
(XEN) CPU1 -> NODE0
(XEN) CPU2 -> NODE0
(XEN) CPU3 -> NODE0
(XEN) CPU4 -> NODE0
(XEN) CPU5 -> NODE0
(XEN) CPU6 -> NODE0
(XEN) CPU7 -> NODE0
(XEN) CPU8 -> NODE3
(XEN) CPU9 -> NODE3
(XEN) CPU10 -> NODE3
(XEN) CPU11 -> NODE3
(XEN) CPU12 -> NODE0
(XEN) CPU13 -> NODE0
(XEN) CPU14 -> NODE0
(XEN) CPU15 -> NODE0
(XEN) Memory location of each domain:
(XEN) Domain 0 (total: 153600):
(XEN)     Node 0: 132232
(XEN)     Node 3: 21368

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel


--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.