Richie,
Thanks for the response - unfortunately it sounds like you're running a
PV domU - I need to forward these PCI devices to an HVM domU, so the
iommu=pv option on dom0 and the others for domU won't really work for
me. I'll check on the restrictive option to see if that will help me -
I think I had already played with that option a little bit, but it's
worth another shot!
Thanks,
Nick
On Thu, 2010-10-21 at 09:43 -0400, Richie wrote:
> I'm not sure if this will be of help to you. I run a Xenified 2.6.34
> kernel and use iommu=pv on dom0 and iommu=soft swiotlb=force on domU
> and haven't seen this. You may want to forward lspci -t output, dom0
> grub.cfg and domU .cfg to the list and also check the xend-config.sxp
> file. There is a "restrictive" (don't recall the exact name) setting
> that is default for assigning pci devices that I will typically set to no.
>
> On 10/21/2010 9:28 AM, Nick Couchman wrote:
> > I've asked a couple of these questions, already, but haven't really had
> > any response, so I thought I'd try, again, and try to provide a little
> > more detail. I need to pass through a PCIe 4-port Serial card. I've
> > done this successfully multiple times with different serial cards, but
> > the current one, a PCIe Perle quad-port RJ45, is causing some problems.
> > I believe the problem stems from the fact that the card is a PCI card
> > and that Perle just built a PCIe-PCI bridge so that they could reuse an
> > existing PCI design. The following three PCI devices are all on the
> > same card:
> >
> > # lspci
> > ...
> > 01:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
> > Express-to-PCI Bridge (rev aa)
> > 02:00.0 Serial controller: Perle Systems Ltd Device 0331
> > 02:00.1 Bridge: Perle Systems Ltd Device 0331
> >
> > I've tried various combinations of hiding 01:00.0, 02:00.0, and/or
> > 02:00.1 from dom0 and passing those through. First, if I try all three,
> > I put the following in my dom0 kernel arguments:
> >
> > pciback.hide=(01:00.0)(02:00.0)(02:00.1)
> >
> > Before someone suggests that maybe I have my kernel arguments wrong,
> > this is a SuSE kernel, so it is not pvops, and that is the correct
> > syntax - I can add other PCI(e) devices on the same exact host (like a
> > graphics card) and pass those through without any problem at all. So
> > syntax is correct. After I boot with these arguments, I check the dmesg
> > output for pciback messages:
> >
> > # dmesg|grep pciback
> > [ 0.000000] Command line: root=/dev/local/dom0 resume=/dev/local/swap
> > pciback.hide=(01:00.0)(02:00.0)(02:00.1) splash=silent quiet showopts
> > [ 0.000000] Kernel command line: root=/dev/local/dom0
> > resume=/dev/local/swap pciback.hide=(01:00.0)(02:00.0)(02:00.1)
> > splash=silent quiet showopts
> > [ 3.646529] pciback 0000:01:00.0: seizing device
> > [ 3.646536] pciback 0000:02:00.0: seizing device
> > [ 3.646540] pciback 0000:02:00.1: seizing device
> > [ 3.646594] pciback 0000:02:00.1: PCI INT A -> GSI 16 (level, low) ->
> > IRQ 16
> > [ 3.646599] pciback 0000:02:00.1: PCI INT A disabled
> > [ 3.646637] pciback 0000:02:00.0: PCI INT A -> GSI 16 (level, low) ->
> > IRQ 16
> > [ 3.646642] pciback 0000:02:00.0: PCI INT A disabled
> >
> > So, everything points to the devices having been removed from dom0's
> > visibility and placed under the control of pciback. A check with "xm"
> > confirms this:
> >
> > # xm pci-list-assignable-devices
> > 0000:02:00.0
> > 0000:02:00.1
> > 0000:01:00.0
> >
> > Next, I set up the pci pass through in my HVM domU configuration. By
> > the way, VTd is enabled and working fine - xm dmesg shows it as enabled,
> > and, as already mentioned, I can forward through other PCI(e) devices
> > without problem. I set the following in my domU configuration:
> >
> > pci=['01:00.0','02:00.0','02:00.1']
> >
> > and then I try to start the domU:
> >
> > # xm start XP
> > Error: pci: PCI Backend and pci-stub don't own device 0000:01:00.0
> > Usage: xm start<DomainName>
> >
> > ??? Ummm, actually, the device is owned by pciback, so why is Xen
> > telling me it's not?? I can confirm via lspci -v:
> >
> > # lspci -v
> > ...
> > 01:00.0 PCI bridge: PLX Technology, Inc. PEX8112 x1 Lane PCI
> > Express-to-PCI Bridge (rev aa) (prog-if 00 [Normal decode])
> > Flags: bus master, fast devsel, latency 0
> > Memory at d0000000 (64-bit, prefetchable) [size=64K]
> > Bus: primary=01, secondary=02, subordinate=02, sec-latency=64
> > I/O behind bridge: 0000d000-0000dfff
> > Memory behind bridge: fe500000-fe5fffff
> > Capabilities: [40] Power Management version 2
> > Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
> > Capabilities: [60] Express PCI/PCI-X Bridge, MSI 00
> > Capabilities: [100] Power Budgeting<?>
> > Kernel driver in use: pciback
> >
> > Any hints on this particular part of the problem would be appreciated.
> > My next step was to leave this one out and just try forwarding through
> > 02:00.0 and 02:00.1. However, when I modify the domU configuration and
> > remove the 01:00.0 device and try to start the domU:
> >
> > # xm start XP
> > Error: Failed to assign device to IOMMU
> > (0000:02:00.0@100,msitranslate=1,power_mgmt=0)
> > Usage: xm start<DomainName>
> >
> > I've been able to find very, very little useful information on this
> > particular error - most of the issues are with trying to forward through
> > one or two of the devices on a card to one domU and the others to
> > another or not at all - splitting the devices on a card between domains.
> > I'm not really trying to do this, but I suspect that there's some sort
> > of tie between the PCIe-PCI bridge device (01:00.0) and the actual
> > serial card (02:00.0) that is causing Xen to need to have both those
> > devices present in the same domain. Of course, because of the problems
> > I'm having forwarding the bridge through, that's not really working for
> > me.
> >
> > Environment:
> > Xen: 4.0.0 (4.0.0_21091_06-0.1)
> > Kernel: 2.6.34.7 (2.6.34.7-0.4-xen, not pvops)
> > Hardware: Dell Optiplex 780
> >
> > Any help, hints, information, would be greatly appreciated!
> >
> > -Nick
> >
> >
> >
> >
> > --------
> >
> > This e-mail may contain confidential and privileged material for the sole
> > use of the intended recipient. If this email is not intended for you, or
> > you are not responsible for the delivery of this message to the intended
> > recipient, please note that this message may contain SEAKR Engineering
> > (SEAKR) Privileged/Proprietary Information. In such a case, you are
> > strictly prohibited from downloading, photocopying, distributing or
> > otherwise using this message, its contents or attachments in any way. If
> > you have received this message in error, please notify us immediately by
> > replying to this e-mail and delete the message from your mailbox.
> > Information contained in this message that does not relate to the business
> > of SEAKR is neither endorsed by nor attributable to SEAKR.
> >
> > _______________________________________________
> > Xen-users mailing list
> > Xen-users@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-users
>
--------
This e-mail may contain confidential and privileged material for the sole use
of the intended recipient. If this email is not intended for you, or you are
not responsible for the delivery of this message to the intended recipient,
please note that this message may contain SEAKR Engineering (SEAKR)
Privileged/Proprietary Information. In such a case, you are strictly
prohibited from downloading, photocopying, distributing or otherwise using this
message, its contents or attachments in any way. If you have received this
message in error, please notify us immediately by replying to this e-mail and
delete the message from your mailbox. Information contained in this message
that does not relate to the business of SEAKR is neither endorsed by nor
attributable to SEAKR.
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home