Re: [Xen-devel] Nvidia, Xen, and Vt-d

[Pasi Kärkkäinen <pasik@xxxxxx>]

On Wed, Sep 09, 2009 at 04:47:39AM -0400, Michael J Coss wrote:
> If this is the right list I apologies in advance, but I am a kernel 
> developer, and I'm not afraid of getting my hands dirty so...
> 
> Like a number of others, I want to play with Vt-d and graphics pass
> through.  But first I need to just get Xen up and running. with a dom0
> and an operational X server.  I've been trying various combinations
> looking for some way to have a working dom0.  The hardware is a ASUS
> p6t7 w/ 2 Nvidia GTX 295 graphics cards.  Each card occupies 2 pci 
> slots, and have 2 GPUs plus quite a bit of memory.
> 
> I keep getting the follow X server crash when I boot Xen and a dom0 kerenl.
> 
> >[54149.822514] X: Corrupted page table at address b61a1000
> >[54149.822519] *pdpt = 0000000027690001
> >[54149.822525] Bad pagetable: 000f [#1] SMP
> >[54149.822531] last sysfs file: 
> >/sys/devices/pci0000:00/0000:00:1c.5/0000:10:00.0/resource
> >[54149.822535] Modules linked in: nvidia(P) [last unloaded: nvidia]
> >[54149.822544]
> >[54149.822548] Pid: 7148, comm: X Tainted: P           (2.6.30-rc3-tip 
> >#2) System Product Name
> >[54149.822552] EIP: 0073:[<b622da45>] EFLAGS: 00210202 CPU: 0
> >[54149.822555] EIP is at 0xb622da45
> >[54149.822558] EAX: b61a1000 EBX: b6580950 ECX: 0979a410 EDX: 00000001
> >[54149.822561] ESI: 00000020 EDI: 097961e0 EBP: 0979a410 ESP: bfd3cec8
> >[54149.822565]  DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
> >[54149.822568] Process X (pid: 7148, ti=e0c38000 task=e1615d60 
> >task.ti=e0c38000)
> >[54149.822571]
> >[54149.822574] EIP: [<b622da45>] 0xb622da45 SS:ESP 007b:bfd3cec8
> >[54149.822583] ---[ end trace 410a0a71e695876c ]---
> 
> Running strace on X, I see that it's crashing doing a series of mmap2.
> The other odd thing is that if I cat the card files
> (/proc/driver/nvidia/cards/0), it reports the video bios as ??:??:??:??. 
>  It looks like either the Nvidia driver is poking around the system
> using non-standard interfaces or somehow the presented pci interface to
> the cards is broken somehow.  Turning on or off the Vt-d support in the
> bios makes no difference, and if I boot the system without the Xen
> hypervisor, the video bios is reported correctly, and the driver works
> just fine.
> 
> I've compared the lspci output with and without the Xen hypervisor, and 
> there are a number of discrepancies.  Notably in the capabilities listed 
> for the graphics cards.  Capability 78 is misreported as Express (v1) 
> instead of Express (v2).  I'm not sure what to make of this.
> 
> Unfortunately, I don't have any other older Nvidia cards to see if they
> behave differently.
> 
> I've tried 3.4.1, and the lastest xen-unstable.  I've tried the pv-ops
> git tree as I really need a 2.6.31 dom0 kernel for other reasons, but
> for the moment I'd just like to get to the point where I have Xen, and
> dom0 and Nvidia playing nicely with one another, so I can move on to
> working on Vt-d and graphic pass through.
> 
> Any suggestions?
> 

So I take it you're using the binary/proprietary Nvidia driver?

It is known to have problems with Xen dom0 kernels.. I'm not sure of the
latest status. I think some people have gotten it to work..

-- Pasi


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel