Xen project Mailing List

[Xen-devel] [patch] fix iommu interrupt setup: setup_vector() may fail

From: Simon Horman <horms@xxxxxxxxxxxx>

Date: Mon, 16 Feb 2009 18:07:33 +1100

Cc: "Kay, Allen M" <allen.m.kay@xxxxxxxxx>, "Jiang, Yunhong" <yunhong.jiang@xxxxxxxxx>, Espen Skoglund <espen.skoglund@xxxxxxxxxxxxx>, Alexia Benington <alexbenington@xxxxxxxxx>, Isaku Yamahata <yamahata@xxxxxxxxxxxxx>, Keir Fraser <keir.fraser@xxxxxxxxxxxxx>, Kouya Shimura <kouya@xxxxxxxxxxxxxx>

Delivery-date: Sun, 15 Feb 2009 23:08:03 -0800

List-id: Xen developer discussion <xen-devel.lists.xensource.com>

Since changeset 19175 (Cleanup IOMMU interrupt setup) I am experiencing a boot failure on my HP dc7800 running in x86_64 mode. The most recent version that I have tested is 19210 (x86: Clean up vpt-align patch.), which is after 19207 (iommu: Fix interrupt setup.) which Shimura-san created to fix some of the problems caused by 19175. After investigating this problem I found that the cause is that for some vectors setup_vector() fails (returning -EBUSY). setup_vector() is called by request_irq_vector(). And request_irq_vector() is called by iommu_set_interrupt(). Which is in turn called by init_vtd_hw(). And 19175 essentailly adds error checking code which now checks for errors returned by iommu_set_interrupt() and thus indirectly the error returned by setup_vector(). Unfortunately as things stands, the way that init_vtd_hw() handles such errors causes my dc7800 not to boot. I am not entirely sure of the correctness of my proposed change, but by calling iommu_set_interrupt() before iommu_set_root_entry() in init_vtd_hw() my system boots correctly. In particularl I'm not sure: a) if it is always valid to call iommu_set_interrupt() before iommu_set_root_entry() b) if this would adversely affect a situation where iommu_set_root_entry() fails. - This seems to be ok, I tested replacing the body of iommu_set_root_entry() with return -EINVAL; Signed-off-by: Simon Horman <horms@xxxxxxxxxxxx> --- (XEN) Xen version 3.4-unstable (horms@xxxxxxxxxxxxxxxxxxxxxxxx) (gcc version 4.0.2) Mon Feb 16 14:00:20 EST 2009 (XEN) Latest ChangeSet: Fri Feb 13 09:48:56 2009 +0000 19210:09a6fa059b37 (XEN) Command line: com1=115200,8n1,0x1248,0 console=com1 loglvl=all guest_loglvl=all vtd=1 iommu=1 (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds (XEN) EDID info not retrieved because no DDC retrieval method detected (XEN) Disc information: (XEN) Found 1 MBR signatures (XEN) Found 1 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009fc00 (usable) (XEN) 000000000009fc00 - 00000000000a0000 (reserved) (XEN) 00000000000e8000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000defafe00 (usable) (XEN) 00000000defafe00 - 00000000defb1ea0 (ACPI NVS) (XEN) 00000000defb1ea0 - 00000000e0000000 (reserved) (XEN) 00000000f4000000 - 00000000f8000000 (reserved) (XEN) 00000000fec00000 - 00000000fed40000 (reserved) (XEN) 00000000fed45000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 000000011c000000 (usable) (XEN) System RAM: 4015MB (4111672kB) (XEN) ACPI: RSDP 000E5C10, 0014 (r0 COMPAQ) (XEN) ACPI: RSDT DEFC1E40, 0044 (r1 HPQOEM SLIC-BPC 20080318 0) (XEN) ACPI: FACP DEFC1EE8, 0074 (r1 COMPAQ BEARLAKE 1 0) (XEN) ACPI: DSDT DEFC2427, A370 (r1 COMPAQ DSDT_PRJ 1 MSFT 100000E) (XEN) ACPI: FACS DEFC1E00, 0040 (XEN) ACPI: APIC DEFC1F5C, 0084 (r1 COMPAQ BEARLAKE 1 0) (XEN) ACPI: ASF! DEFC1FE0, 0063 (r32 COMPAQ BEARLAKE 1 0) (XEN) ACPI: MCFG DEFC2043, 003C (r1 COMPAQ BEARLAKE 1 0) (XEN) ACPI: TCPA DEFC207F, 0032 (r1 COMPAQ BEARLAKE 1 0) (XEN) ACPI: SLIC DEFC20B1, 0176 (r1 HPQOEM SLIC-BPC 1 0) (XEN) ACPI: HPET DEFC2227, 0038 (r1 COMPAQ BEARLAKE 1 0) (XEN) ACPI: DMAR DEFC225F, 0198 (r1 COMPAQ BEARLAKE 1 0) (XEN) NUMA turned off (XEN) Faking a node at 0000000000000000-000000011c000000 (XEN) Domain heap initialised (XEN) found SMP MP-table at 000f9bf0 (XEN) DMI 2.5 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0xf808 (XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[f804,460], pm1x_evt[f800,0] (XEN) ACPI: wakeup_vec[defc1e0c], vec_size[20] (XEN) ACPI: Local APIC address 0xfee00000 (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) (XEN) Processor #0 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled) (XEN) Processor #1 7:7 APIC version 20 (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x00] disabled) (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x00] disabled) (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1]) (XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1]) (XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0]) (XEN) IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23 (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) (XEN) ACPI: IRQ0 used by override. (XEN) ACPI: IRQ2 used by override. (XEN) ACPI: IRQ9 used by override. (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000 (XEN) [VT-D]dmar.c:460: Host address width 36 (XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:337: dmaru->address = fed90000 (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1b.0 (XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:337: dmaru->address = fed91000 (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:337: dmaru->address = fed92000 (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:3.0 (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:3.2 (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:3.3 (XEN) [VT-D]dmar.c:469: found ACPI_DMAR_DRHD (XEN) [VT-D]dmar.c:337: dmaru->address = fed93000 (XEN) [VT-D]dmar.c:346: found INCLUDE_ALL (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:2.0 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.7 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1a.7 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.0 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.1 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1d.2 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1a.0 (XEN) [VT-D]dmar.c:473: found ACPI_DMAR_RMRR (XEN) [VT-D]dmar.c:288: found endpoint: bdf = 0:1a.1 (XEN) Intel VT-d has been enabled (XEN) Using ACPI (MADT) for SMP configuration information (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Initializing CPU#0 (XEN) Detected 2826.324 MHz processor. (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 0 (XEN) VMX: Supported advanced features: (XEN) - APIC MMIO access virtualisation (XEN) - APIC TPR shadow (XEN) - Virtual NMI (XEN) - MSR direct-access bitmap (XEN) HVM: VMX enabled (XEN) MCE: Intel newly family MC Init (XEN) Intel machine check reporting enabled on CPU#0. (XEN) CPU0: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU0 has no CMCI support (XEN) CPU0: Intel(R) Core(TM)2 Duo CPU E8300 @ 2.83GHz stepping 06 (XEN) Booting processor 1/1 eip 8c000 (XEN) Initializing CPU#1 (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K (XEN) CPU: L2 cache: 6144K (XEN) CPU: Physical Processor ID: 0 (XEN) CPU: Processor Core ID: 1 (XEN) MCE: Intel newly family MC Init (XEN) Intel machine check reporting enabled on CPU#1. (XEN) CPU1: Thermal monitoring enabled (TM2) (XEN) CMCI: CPU1 has no CMCI support (XEN) CPU1: Intel(R) Core(TM)2 Duo CPU E8300 @ 2.83GHz stepping 06 (XEN) Total of 2 processors activated. (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1 (XEN) checking TSC synchronization across 2 CPUs: passed. (XEN) Platform timer is 14.318MHz HPET (XEN) Brought up 2 CPUs (XEN) microcode.c:73:d32767 microcode: CPU1 resumed (XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found (XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found (XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found (XEN) [VT-D]iommu.c:1707: Queued Invalidation hardware not found (XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found (XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found (XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found (XEN) [VT-D]iommu.c:1715: Interrupt Remapping hardware not found (XEN) Intel VT-d snoop control disabled (XEN) I/O virtualisation enabled (XEN) I/O virtualisation for PV guests disabled (XEN) ACPI sleep modes: S3 (XEN) mcheck_poll: Init_mcheck_timer (XEN) mcheck_poll: Machine check polling timer started. (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:0.0 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:3.0 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:3.2 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:3.3 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:19.0 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1246:d32767 domain_context_mapping:PCIe: bdf = 0:1b.0 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1f.0 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1f.2 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1f.5 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.1 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.0 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.2 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.1 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.0 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1a.7 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:1d.7 (XEN) [VT-D]iommu.c:1253:d32767 domain_context_mapping:PCI: bdf = 0:2.0 (XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg = ffff828bfff57000 (XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg = ffff828bfff56000 (XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg = ffff828bfff55000 (XEN) [VT-D]iommu.c:618: iommu_enable_translation: iommu->reg = ffff828bfff54000 (XEN) *** LOADING DOMAIN 0 *** (XEN) elf_init: not an ELF binary (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) Could not set up DOM0 guest OS (XEN) **************************************** (XEN) (XEN) Reboot in five seconds... (XEN) [VT-D]iommu.c:721: iommu_page_fault: iommu->reg = ffff828bfff54000 (XEN) [VT-D]iommu.c:690: iommu_fault_status: Fault Overflow (XEN) [VT-D]iommu.c:693: iommu_fault_status: Primary Pending Fault (XEN) [VT-D]iommu.c:675: iommu_fault:DMA Read: 0:1d.2 addr defce000 REASON 6 iommu->reg = ffff828bfff54000 (XEN) print_vtd_entries: iommu = ffff83011bfd4310 bdf = 0:1d:2 gmfn = defce (XEN) root_entry = ffff83011bfcb000 (XEN) root_entry[0] = 118453001 (XEN) context = ffff830118453000 (XEN) context[ea] = 101_11bfb1001 (XEN) l3 = ffff83011bfb1000 (XEN) l3_index = 3 (XEN) l3[3] = 11862d003 (XEN) l2 = ffff83011862d000 (XEN) l2_index = f7 (XEN) l2[f7] = 118535003 (XEN) l1 = ffff830118535000 (XEN) l1_index = 1ce (XEN) l1[1ce] = 0 (XEN) l1[1ce] not present Index: xen-unstable.hg/xen/drivers/passthrough/vtd/iommu.c =================================================================== --- xen-unstable.hg.orig/xen/drivers/passthrough/vtd/iommu.c 2009-02-16 17:14:35.000000000 +1100 +++ xen-unstable.hg/xen/drivers/passthrough/vtd/iommu.c 2009-02-16 18:06:09.000000000 +1100 @@ -1674,12 +1674,6 @@ static int init_vtd_hw(void) for_each_drhd_unit ( drhd ) { iommu = drhd->iommu; - ret = iommu_set_root_entry(iommu); - if ( ret ) - { - gdprintk(XENLOG_ERR VTDPREFIX, "IOMMU: set root entry failed\n"); - return -EIO; - } vector = iommu_set_interrupt(iommu); if ( vector < 0 ) @@ -1687,6 +1681,12 @@ static int init_vtd_hw(void) gdprintk(XENLOG_ERR VTDPREFIX, "IOMMU: interrupt setup failed\n"); return vector; } + ret = iommu_set_root_entry(iommu); + if ( ret ) + { + gdprintk(XENLOG_ERR VTDPREFIX, "IOMMU: set root entry failed\n"); + return -EIO; + } dma_msi_data_init(iommu, vector); dma_msi_addr_init(iommu, cpu_physical_id(first_cpu(cpu_online_map))); iommu->vector = vector; _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxxxxxxxx http://lists.xensource.com/xen-devel

©2013 Xen Project, A Linux Foundation Collaborative Project. All Rights Reserved.
Linux Foundation is a registered trademark of The Linux Foundation.
Xen Project is a trademark of The Linux Foundation.