[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] pt_msix_init: Error: Can't map physical MSI-X table: Invalid argument

  • To: "'xen-devel@xxxxxxxxxxxxxxxxxxx'" <xen-devel@xxxxxxxxxxxxxxxxxxx>
  • From: "Cinco, Dante" <Dante.Cinco@xxxxxxx>
  • Date: Tue, 13 Oct 2009 16:51:15 -0600
  • Accept-language: en-US
  • Acceptlanguage: en-US
  • Delivery-date: Tue, 13 Oct 2009 15:51:43 -0700
  • List-id: Xen developer discussion <xen-devel.lists.xensource.com>
  • Thread-index: AcpMV6t4VNorGkOyS1SqOgVpDsm4xQ==
  • Thread-topic: pt_msix_init: Error: Can't map physical MSI-X table: Invalid argument

I get the above error message when I boot the domU (Linux I'm using Xen 3.5-unstable (changeset 20303) with pv_ops kernel I use pci-stub to hide the device so that I can assign/pass it through to domU. The error is coming from ioemu-xxx/hw/pt-msi.c. I think it is complaining about the table_off that is part of the mmap() call:
    dev->msix->phys_iomem_base = mmap(0, total_entries * 16,
                          PROT_WRITE | PROT_READ, MAP_SHARED | MAP_LOCKED,
                          fd, dev->msix->table_base + table_off);
When I printed table_off, its value is 0x4100 (not page-aligned) which is consistent with the BAR4 offset of the PCI device (see below). I think mmap() requires the last argument to be a multiple of the page size and results is a map error if it is not. Here's the partial output of "lspci -vv -s 0:07:0.1" from dom0:
07:00.0 Fibre Channel: PMC-Sierra Inc. Device 8032 (rev 05)
        Subsystem: Atto Technology Device 003c
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 11
        Capabilities: [60] Message Signalled Interrupts: Mask- 64bit+ Queue=0/1 Enable+
                Address: 00000000fee11000  Data: 405e
        Capabilities: [70] Express (v2) Endpoint, MSI 01
                DevCap: MaxPayload 4096 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
                        ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
        Capabilities: [b0] MSI-X: Enable- Mask- TabSize=9
                Vector table: BAR=4 offset=00004100
                PBA: BAR=4 offset=00004000
I'm able to workaround this problem by modifying pt-msi.c: force offset to zero in mmap() and on the next line, adjust the pointer by table_off:
    table_off_adjust = table_off & 0x0fff;
    dev->msix->phys_iomem_base = mmap(0, total_entries * 16 + table_off_adjust,
                          PROT_WRITE | PROT_READ, MAP_SHARED | MAP_LOCKED,
                          fd, dev->msix->table_base + table_off - table_off_adjust);
    dev->msix->phys_iomem_base = (void *)((char *)dev->msix->phys_iomem_base + table_off_adjust);
Ideally, I should only be applying this to my device but since I'm not passing any other device that use MSI, I don't see any negative effect. In addition, my device only uses MSI and not MSI-X.
Obviously, this is a custom hack but I was wondering if there is a cleaner solution for this. Is a non-page aligned BAR offset unusual? When I boot Linux on bare metal (no Xen), I don't see this error.
Dante Cinco
Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.