WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] Problem with VT-d on Mainboard Intel DQ45CB

To: "Franz Regensburger" <Franz.Regensburger@xxxxxxxx>
Subject: Re: [Xen-users] Problem with VT-d on Mainboard Intel DQ45CB
From: "Andrew Lyon" <andrew.lyon@xxxxxxxxx>
Date: Mon, 27 Oct 2008 19:43:09 +0000
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Mon, 27 Oct 2008 12:44:35 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=XmehafujnlU44DrZxlXRhZkP2Saen1dNLbJVFCTKBow=; b=H9hTV52S/wxGAhIC7flLj/NxrsaLG9Np/p0lzh+IR1mcAgAIR5htda8FmdHyurHymz iuc51BMOEzxQNHUNbvEQkSliQriLTxmCbrWnFxsAqbj9v4nguDH1BLhKiEFl0Z3eyyqb puOLbi4hwkzQuMMWmwbwDGF8kLz6xcGrCtJ2s=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=jvkty2FJdbfIYAZpxXA+uoQES11S+YRZKzLwcuGvi6/WvzMnzQBNVW7GSEy4Uq1ruq A92V+y2obLsKtqF42KJQZBxphY9HPy84zVJk1xZg1Vs9j44g0tiEwdK9VZYfBcczPcXZ Gc6iKaSpo7H5m0+XwODAnovNxvTDmPCifXseY=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <200810271204.40859.Franz.Regensburger@xxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <200810230925.04813.Franz.Regensburger@xxxxxxxx> <1e16a9ed0810251710h29d3d45g54e2772f1d32a855@xxxxxxxxxxxxxx> <200810271204.40859.Franz.Regensburger@xxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
On Mon, Oct 27, 2008 at 11:04 AM, Franz Regensburger
<Franz.Regensburger@xxxxxxxx> wrote:
> Hello Todd,
>
> thank you for your comments.
> I did not have time yet to follow your suggestions (I certainly will).
>> To get to the point where you can be helpful, you will need to get the
>> latest xen-unstable.
>>
>> hg clone http://xenbits.xensource.com/xen-unstable.hg
>> hg clone http://xenbits.xensource.com/linux-2.6.18-xen.hg
>> cd xen-unstable
>> make world
>
> However, yesterday I was able to make some progress with respect to the Intel 
> board DQ45CB that
> solves most of my problems.
>
> It seems that most problems were caused by the board and not Xen!
> I think it is important to share my new experience as early as possible in 
> order to prevent others for
> suffering in vain or doing unneccessary error analysis.
>
> Some days ago Intel put a new firmware for the board onto their WEB-site
>        CBQ4510H.86A.0061.BI.ZIP
>
> I upgraded my board from CBQ4510H.86A.0059 with that new BIOS and the 
> following problems seem to have vanished:
>
> Summary:
> ========
> Update to BIOS CBQ4510H.86A.0061.BI.ZIP solved the following issues:
>
> - Whenever I boot my Xen 2.6.18.8 (Xen 3.3) with kernel parameter iommu=1
> the boot process freezes.
> SOLVED (details below)
>
> and related to that
>
> - VT-d broken. No PCI backend hiding with HVM guest possible
> SOLVED (details below)
>
> - When I insert a PCIe x16 Graphics card (ATI Radeon HD 2400 Pro) the BIOS 
> menu display
> somehow becomes slow. The experience is like if you use a 300 Baut serial 
> line connection.
> SOLVED (details below)
>
> - PCIe x16 graphics with Xen kernel broken
> SOLVED (details below)
>
>
> I will also post the above short note on xen-devel
>
> Regards,
>
> Franz
>
> ##############################
>
> Details:
>
> My current setup:
> Xen Source: http://bits.xensource.com/oss-xen/release/3.3.0/xen-3.3.0.tar.gz
> Xen Kernel: hg clone http://xenbits.xensource.com/linux-2.6.18-xen.hg
>
> Details for the solved issues:
> ======================
>
> - Whenever I boot my Xen 2.6.18.8 (Xen 3.3) with kernel parameter iommu=1
> the boot process freezes. (SOLVED)
>
> After the BIOS update the Xen kernel boots without problems even if I use the 
> iommu=1
>
> #----------------------------
>
> - VT-d broken. No PCI backend hiding with HVM guest possible (SOLVED)
>
> PCI backend hideing and VT-d seem to work now!
>
> The functionality 'pciback' is compiled fixed into my kernel (not a module).
>
> I use the following grub config
>
> title           Xen 3.3.0 / kernel 2.6.18.8-xen with IOMMU=1 and backhide 1G 
> NIC
> root            (hd0,0)
> kernel          /xen-3.3.0.gz console=vga noreboot max_loop=128 
> dom0_mem=1024M iommu=1
> module          /vmlinuz-2.6.18.8-xen root=/dev/mapper/RootVG-root ro 
> pciback.hide=(03:00.0)
> module          /initrd-2.6.18.8-xen.img
>
> The messages/dmesg show
>
> kernel: Bootdata ok (command line is root=/dev/mapper/RootVG-root ro 
> pciback.hide=(03:00.0))
> ...
> kernel: pciback 0000:03:00.0: seizing device
>
> The xm dmesg shows:
> (XEN) Xen version 3.3.0 (root@xxxxxxxxx) (gcc version 4.1.2 20061115 
> (prerelease) (Debian 4.1.1-21)) Tue Oct 21 16:22:47 CEST 2008
> (XEN) Latest ChangeSet: unavailable
> (XEN) Command line: console=vga noreboot max_loop=128 dom0_mem=1024M iommu=1
> (XEN) Video information:
> (XEN)  VGA is text mode 80x25, font 8x16
> (XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
> (XEN)  EDID info not retrieved because of reasons unknown
> (XEN) Disc information:
> (XEN)  Found 2 MBR signatures
> (XEN)  Found 2 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN)  0000000000000000 - 0000000000096400 (usable)
> (XEN)  0000000000096400 - 00000000000a0000 (reserved)
> (XEN)  00000000000e0000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 00000000bfade000 (usable)
> (XEN)  00000000bfade000 - 00000000bfb21000 (ACPI NVS)
> (XEN)  00000000bfb21000 - 00000000bfc3f000 (reserved)
> (XEN)  00000000bfc3f000 - 00000000bfc42000 (ACPI NVS)
> (XEN)  00000000bfc42000 - 00000000bfd42000 (reserved)
> (XEN)  00000000bfd42000 - 00000000bfd43000 (ACPI NVS)
> (XEN)  00000000bfd43000 - 00000000bfd48000 (reserved)
> (XEN)  00000000bfd48000 - 00000000bfd50000 (ACPI data)
> (XEN)  00000000bfd50000 - 00000000bfd6b000 (ACPI NVS)
> (XEN)  00000000bfd6b000 - 00000000bfd8a000 (reserved)
> (XEN)  00000000bfd8a000 - 00000000bfd90000 (ACPI NVS)
> (XEN)  00000000bfd90000 - 00000000bff00000 (usable)
> (XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
> (XEN)  00000000ff000000 - 0000000100000000 (reserved)
> (XEN)  0000000100000000 - 000000023c000000 (usable)
> (XEN) System RAM: 8123MB (8318864kB)
> (XEN) ACPI: RSDP 000F03C0, 0024 (r2  INTEL)
> (XEN) ACPI: XSDT BFD4EE18, 005C (r1 INTEL  DQ45CB         3D MSFT    10013)
> (XEN) ACPI: FACP BFD4DD98, 00F4 (r4  INTEL    A M I  6222004 MSFT    10013)
> (XEN) ACPI: DSDT BFD48018, 4E9A (r1 INTEL  DQ45CB         3D INTL 20051117)
> (XEN) ACPI: FACS BFD56F40, 0040
> (XEN) ACPI: APIC BFD4DF18, 006C (r2 INTEL  DQ45CB         3D MSFT    10013)
> (XEN) ACPI: MCFG BFD58E18, 003C (r1 INTEL  DQ45CB         3D MSFT       97)
> (XEN) ACPI: ASF! BFD57D18, 00A0 (r32 INTEL  DQ45CB         3D TFSM    F4240)
> (XEN) ACPI: SPCR BFD58D18, 0050 (r1 INTEL  DQ45CB         3D AMI.        3)
> (XEN) ACPI: TCPA BFD58C98, 0032 (r2 INTEL  DQ45CB         3D MSFT  1000013)
> (XEN) ACPI: DMAR BFD42F18, 00D8 (r1 INTEL  DQ45CB         3D INTL        1)
> (XEN) Xen heap: 14MB (14496kB)
> (XEN) Domain heap initialised
> (XEN) Processor #0 7:7 APIC version 20
> (XEN) Processor #1 7:7 APIC version 20
> (XEN) Processor #2 7:7 APIC version 20
> (XEN) Processor #3 7:7 APIC version 20
> (XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> (XEN) Intel VT-d has been enabled
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Detected 2833.076 MHz processor.
> (XEN) HVM: VMX enabled
> (XEN) CPU0: Intel(R) Core(TM)2 Quad  CPU   Q9550  @ 2.83GHz stepping 07
> (XEN) Booting processor 1/1 eip 8c000
> (XEN) CPU1: Intel(R) Core(TM)2 Quad  CPU   Q9550  @ 2.83GHz stepping 07
> (XEN) Booting processor 2/2 eip 8c000
> (XEN) CPU2: Intel(R) Core(TM)2 Quad  CPU   Q9550  @ 2.83GHz stepping 07
> (XEN) Booting processor 3/3 eip 8c000
> (XEN) CPU3: Intel(R) Core(TM)2 Quad  CPU   Q9550  @ 2.83GHz stepping 07
> (XEN) Total of 4 processors activated.
> (XEN) ENABLING IO-APIC IRQs
> (XEN)  -> Using new ACK method
> (XEN) checking TSC synchronization across 4 CPUs: passed.
> (XEN) Platform timer is 3.579MHz ACPI PM Timer
> (XEN) Brought up 4 CPUs
> (XEN) I/O virtualisation enabled
> (XEN) I/O virtualisation for PV guests disabled
> (XEN) mtrr: your CPUs had inconsistent fixed MTRR settings
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN)  Xen  kernel: 64-bit, lsb, compat32
> (XEN)  Dom0 kernel: 64-bit, lsb, paddr 0xffffffff80200000 -> 
> 0xffffffff805c050c
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN)  Dom0 alloc.:   0000000230000000->0000000232000000 (253952 pages to be 
> allocated)
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN)  Loaded kernel: ffffffff80200000->ffffffff805c050c
> (XEN)  Init. ramdisk: ffffffff805c1000->ffffffff81506800
> (XEN)  Phys-Mach map: ffffffff81507000->ffffffff81707000
> (XEN)  Start info:    ffffffff81707000->ffffffff817074a4
> (XEN)  Page tables:   ffffffff81708000->ffffffff81717000
> (XEN)  Boot stack:    ffffffff81717000->ffffffff81718000
> (XEN)  TOTAL:         ffffffff80000000->ffffffff81800000
> (XEN)  ENTRY ADDRESS: ffffffff80200000
> (XEN) Dom0 has maximum 4 VCPUs
> (XEN) Scrubbing Free RAM: 
> .....................................................................done.
> (XEN) Xen trace buffers: disabled
> (XEN) Std. Loglevel: Errors and warnings
> (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to 
> Xen)
> (XEN) Freed 108kB init memory.
> (XEN) mtrr: type mismatch for c0000000,10000000 old: write-back new: 
> write-combining
>
> In the config of the HVM WinXP guest I use:
>
> pci=[ '03:00.0' ]
>
> In the running guest I see a new (native) 1Gbit NIC.
> I did not do any performance test yet.
> However the VT-d/PCI backend hide mechanism seems to work :-)
>
> #----------------------------
>
> - When I insert a PCIe x16 Graphics card (ATI Radeon HD 2400 Pro) the BIOS 
> menu display
> somehow becomes slow.  The experience is like if you use a 300 Baut serial 
> line connection.
> (SOLVED)
> Details see next paragraph
>
> - PCIe x16 graphics with Xen kernel broken
> SOLVED
>
> After the BIOS update I tested two PCIe x16 graphics adapters:
>  - ATI Radeon HD 2400 Pro
>  - MSI NX 8400GS (Nvidia 8400GS)
> Both cards run now without problems on a non-Xen standard Debian etch 
> 2.6.18.6 kernel.
> Especially, the BIOS menus are no longer slowed down.
> 3D runs perfectly with the propriatary drivers
>        ATI: ati-driver-installer-8-5-x86.x86_64.run
>        NVIDIA: NVIDIA-Linux-x86_64-177.80-pkg2.run
>
> Caveat: after testing the Nvida Card/Driver you need to remove some links 
> that point to
> NVIDIA drivers before you test again the ATI card/driver.
> Search /usr/lib for files like with '*177.80*' in them.
>
> With the xen-kernel 2.6.18.8 booted (details about version above) I was able 
> to configure the ATI card for X11
> by using the ATI driver ati-driver-installer-8-5-x86.x86_64.run.
> However, 3D s broken due to a problem with symbols
>
> Oct 27 10:09:09 susi kernel: fglrx: Unknown symbol xen_invlpg_mask
>
> With the xen-kernel 2.6.18.8 booted (details about version above) I was not 
> able to compile the NVIDIA driver
>  NVIDIA-Linux-x86_64-177.80-pkg2.run
> The build script from NVIDIA expclicitly states that it won't not run on a 
> XEN-kernel

The build script can be told to ignore Xen, I am running 177.80 on
2.6.27.3 (kernel.org + modified suse xen patches) with a single 9500GT
driving two 1920x1200 TFT's (not xinerama), I have a non xen kernel of
the same version installed so when I need to install nvidia drivers I
do the following:

rm /usr/src/linux
ln -s /usr/src/linux-2.6.27 /usr/src/linux
emerge nvidia-drivers #substitute the appropriate command for your distro
rm /usr/src/linux
ln -s /usr/src/linux-2.6.27-xen /usr/src/linux
cd /usr/src
sh /home/andy/Desktop/NVIDIA-Linux-x86_64-177.80-pkg2.run -x
cd NVIDIA-Linux-x86_64-177.80-pkg2/usr/src/nv/
IGNORE_XEN_PRESENCE=y SYSSRC=/usr/src/linux make module
IGNORE_XEN_PRESENCE=y SYSSRC=/usr/src/linux make install
modprobe nvidia

The drivers seem to work quite well under Xen, I can't really say they
are stable as my system occasionally locks up, but that happens
without the nvidia module/drivers loaded or X running at all and it
doesn't happen more frequently with them loaded so I think they are
probably ok.

Andy






>
> #-----------------------
>
> Details about BIOS update
>
> I download the archive
>        CBQ4510H.86A.0061.BI.ZIP
> from the Intel page
>
> I had quite some hard time to install the update via the DOS-Utility 
> IFLASH.EXE on a host without FD-drive
> and only linux installed on the HDs. After frantically searching for some old 
> Win boot media I came up with
> a good old Win98SE2 boot CD. It has the nice feature to recognize USB sticks 
> as FD disk drive B:
>
> Then I proceeded as follows.
> (Shame on me that I do not know how to do it better with purely Linux and 
> freeDos basis
>  Is anybody out there who can educate me?)
>
> Unpack the zip archive.
> This yields
>
> CB0061.BIO
> CB0061.ITK
> IFLASH.EXE
>
> Copy these files to a USB stick with FAT FS
>
> Reboot the host and enter BIOS
> Set Advanced/Drives Disk mode from RAID to IDE
> This allows for a proper Win98SE2 boot from CD
>
> Boot a Win98SE2 CD.
> It will recognize the USB stick as drive b:
>
> b:
> iflash /PF CB0061.BIO
>
> When prompted
> - remove CD
> - remove USB stick
>
> type return an wait until BIOS flash has finished
>
> It takes about 3-5 minutes
>
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users