Ok, I’ve finally had some more
time to work on this.
The hardware I’m using is a
DQ45CB, a C2D E6600, and HIS ATI Radeon 4770.
The kernel I’m currently using
is a pv_ops dom0 from Jeremy’s xen-tip/master branch, version 2.6.30-rc3. I
will attach the config.
I built xen from xen-unstable
c/s 20180. This includes Weidong’s gfx passthrough v2 patch. I
applied the qemuv2 patch.
The interesting thing about this
configuration is I’m having problems passing through devices that are sharing
IRQs with devices that dom0 still has loaded. IR is not enabled, so that
may be expected, but I’m certain I’ve managed to do this with other kernels
(maybe it’s a difference between using pciback and pci-stub?)
Other than that this
configuration was working just fine and perfectly stable for everything not gfx
passthrough.
I passed my videocard through to
the guest, without enabling gfx_passthru. I installed the driver for the
device, and rebooted. The device was recognized properly, but device
manager said “Could not start”, which I expected. I shut down and enabled
gfx_passthru(=2) and started the guest again. Nothing happened on the
screen and xm list was showing that the domu wasn’t using and cputime. Xm
dmesg output stopped at HVM: Loading ROMBIOS, and there was no BIOS output as
there usually is.
Next I applied a patch I made
(well, in two pieces, also attached) adapted from Weidong’s v1 vBAR:pBAR patch
and rebuilt tools. The guest started successfully, and I was able to read
the bios messages and see the normal startup screens. The domU was
perfectly stable; I logged on and looked through the event log and device
manager using the keyboard and mouse attached to the usb controllers I passed
through. Windows reverted to using the VgaSave (I guess roughly
equivalent to X’s vesa driver?) driver, saying the 4770 could not get enough free
resources.
I tried out a few basic tasks
and everything was very stable. Strangely, even though the card had the
same MAC address, Windows recognized the emulated network adapter as a different
card and defaulted to using DHCP instead of using the static IP I’d assigned to
that card. Perhaps windows remembers something else about the cards other
than MAC (pci slot, io/mem region?) that changes when the graphics card is
passed through, as it went back to recognizing the original adapter when I boot
with gfx_passthru=0.
In device manager, if I select
view->Resources by connection, and look at memory resources, I see my
graphics device listed by itself, whereas I see all of the other PCI devices listed
under a couple of PCI Bus entries.
Next I applied a patch to map
the proper memory regions for my device (attached). I booted the guest,
and watched the normal output during startup. When windows was about to
start (switching from that Starting Windows screen that clearly uses BIOS vga
capability, to the graphical shell with the login prompt), the screen shut off,
and the dom0 froze completely. I rebooted and tried a couple more times, with
the same result. The qemu log is attached.
I believe the memory regions are
mapped properly (using that word loosely) and it tried to start the device, as
it seems to have enabled MSI on the device, which it had never done before,
according to the lines in the qemu log:
pt_msgctrl_reg_write: guest
enabling MSI, disable MSI-INTx translation
pt_msi_update: Update msi with
pirq 37 gvec b0 gflags 0
That IRQ corresponds with the
one assigned to the graphics device earlier in the log. It’s almost
immediately after this that the machine totally locks up.
I’d like to figure out why the
BIOS never seems to execute without the vBAR=pBAR patch. Is it possible
that the gpu’s BIOS has some values hardcoded that don’t let it operate with
different i/o or memory regions? Or is it possible that xen is not
properly assigning resources through it’s normal code path (which is skipped
using the vBAR=pBAR patch)? When I get another minute, I will try again
with an unpatched xen to see what resources are assigned to the device when I pass
it through with gfx_passthru disabled.
Let me know if there’s anything I
can do to help resolve this and test this functionality further.
Doug Magee
From:
xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of djmagee@xxxxxxxxxxxx
Sent: Thursday, September 03, 2009 11:08 AM
To: enming.teo@xxxxxxxxxxxxxxx; Han, Weidong;
xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] RE: xen-unstable pci passthrough
I had a little bit of time to
work on this the other night. As the revised passthrough patch had
already been applied to xen-unstable, I only had to apply the qemu branch.
When I tried to passthrough my
video device with gfx_passthru=2, the guest BIOS never seemed to execute.
Xm dmesg, went as far as “Loading ROMBIOS”, but no further. When guests
successfully start, I can see the bios messages in xm dmesg, and that was not
the case. I tried leaving the vnc enabled to see if there was any output,
and all I had was some qemu monitor and it was not clear what I was supposed to
do there. I was attempting to passthrough my addon card, which is the only
graphics device enabled in my system.
I then tried to passthrough the
graphics device without the gfx_passthru option just to make sure the guest
would boot properly. I was immediately hit with a dom0 kernel BUG and at
that point I had no more time to experiment.
This test was while I was
running a xenified 2.6.29.6 kernel, which has been perfectly stable for me in
all my other uses. I want to try with a 2.6.18.8-xen kernel as
well. Hopefully I’ll get a chance this afternoon. I built and
tested a 2.6.31-rc6 dom0 kernel the other day but it was difficult to use as
there was really high latency over my ssh connection for some reason and the
machine was generally unresponsive.
When I have time to fully test
all of these combinations I will report back with the results.
From: Teo En Ming (Zhang
Enming) [mailto:enming.teo@xxxxxxxxxxxxxxx]
Sent: Thursday, September 03, 2009 12:12 AM
To: djmagee@xxxxxxxxxxxx; 'Han, Weidong'; xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] RE: xen-unstable pci passthrough
Dear Magee,
Any luck with the Intel vga passthrough patches to xen 3.5-unstable
on Intel DQ45CB with extra PCI-e x16 graphics card? Are you using pvops dom 0
kernels 2.6.30-rc3 and 2.6.31-rc6?
Regards,
Mr. Teo En Ming (Zhang Enming)
Dip(Mechatronics Engineering) BEng(Hons)(Mechanical Engineering)
Technical Support Engineer
Information Technology Department
Asiasoft Online Pte Ltd
Tampines Central 1 #04-01 Tampines Plaza
Singapore 529541
Republic of Singapore
Mobile: +65-9648-9798
MSN: teoenming@xxxxxxxxxxx
From:
xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of djmagee@xxxxxxxxxxxx
Sent: Wednesday, September 02, 2009 6:59 PM
To: Han, Weidong; xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] RE: xen-unstable pci passthrough
That was the problem, thank
you. Now I’ll work on testing the gfx-passthrough patches.
From:
xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Han, Weidong
Sent: Tuesday, September 01, 2009 6:55 PM
To: djmagee@xxxxxxxxxxxx; 'xen-devel@xxxxxxxxxxxxxxxxxxx'
Subject: [Xen-devel] RE: xen-unstable pci passthrough
I suspect you are using old hvm config file. The device_model is
changes in config file.
in old config file:
# New stuff
device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
in new config file:
# Device Model to be used
device_model = 'qemu-dm'
Pls check it, and use the latest config file to create guest.
Regards,
Weidong
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of djmagee@xxxxxxxxxxxx
Sent: 2009年9月2日 6:40
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] xen-unstable pci passthrough
I have not been able to passthrough any PCI devices using the
latest xen-unstable. I have a DQ45CB, and have successfully passed
devices to guests using 3.4.1.
The latest c/s in my copy of xen-unstable is 20145. I
just started playing around with unstable yesterday, so I can’t tell you if
earlier revisions worked. I’ve tried with various dom0 kernels, the
current 2.6.18.8-xen branch, a xenified 2.6.29.6, and a pvops 2.6.31-rc6, and
in every case I get the same error. I’ve tried both putting pci= in the
config file, and hot-adding the device using xm pci-attach. In every
case, the xm command (either create or pci-attach) fails with the message
“Error: Timed out waiting for device model action”. The guests in every
case are HVM guests, some flavors of Windows, as well as the Knoppix 5.3.1 DVD.
The relevant xm dmesg output is:
(XEN) PCI add device 00:1b.0
(XEN) [VT-D]iommu.c:1292:d0 domain_context_unmap:PCIe: bdf =
0:1b.0
(XEN) [VT-D]iommu.c:1178:d0 domain_context_mapping:PCIe: bdf
= 0:1b.0
(XEN) [VT-D]io.c:284:d0 VT-d irq bind: m_irq = 37 device = 3
intx = 0
(XEN) [VT-D]iommu.c:1292:d0 domain_context_unmap:PCIe: bdf =
0:1b.0
(XEN) [VT-D]iommu.c:1178:d0 domain_context_mapping:PCIe: bdf
= 0:1b.0
And the messages from qemu-log:
dm-command: hot insert pass-through pci dev
hot add pci slot -2 exceed.
Please let me know what else I need to supply to help
resolve this problem. If I need to enable debugging messages, let me know
the best way to do this.
Doug Magee
djmagee@xxxxxxxxxxxx
No virus
found in this incoming message.
Checked by AVG - www.avg.com
Version: 8.5.409 / Virus Database: 270.13.75/2340 - Release Date: 09/01/09
20:03:00
Home
