Xen project Mailing List

Hi Konrad and all,

Patch was applied against kernel 4.4.12-9.pvops.qubes.x86_64 and tested against Xen 4.6.1-17.fc23.

Quick result:
user@user-ThinkPad-X200:~$ grep "DMA ops" XEN\{iommu\=*
XEN{iommu=0} KERNEL{rhgb drm.debug=255}:intel_gtt_init: Using DMA ops
XEN{iommu=1} KERNEL{drm.debug=255}:intel_gtt_init: Using DMA ops
XEN{iommu=1} KERNEL{rhgb drm.debug=255 intel_iommu=igfx_off}:intel_gtt_init: Using DMA ops
XEN{iommu=no-igfx} KERNEL{rhgb drm.debug=255}:intel_gtt_init: Using DMA ops
XEN{iommu=no-igfx} KERNEL{rhgb drm.debug=255 intel_iommu=igfx_off}:intel_gtt_init: Using DMA ops

With iommu=1:

Unfortunately, the behavior haven't changed: it's impossible to boot with default xen cmd line (iommu=1). System hangs at "clearing unused GTT space".

The following message seems problematic:
"XEN{iommu=1} KERNEL{drm.debug=255}:(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr 200000000, iommu reg = ffff82c000203000" seems to be problematic.

With iommu=no-igfx, it hangs after vt-d remappings:
XEN{iommu=no-igfx} KERNEL{rhgb drm.debug=255 intel_iommu=igfx_off}:(XEN) [VT-D]d0:PCI: unmap 0000:00:19.0
XEN{iommu=no-igfx} KERNEL{rhgb drm.debug=255 intel_iommu=igfx_off}:(XEN) [VT-D]d1:PCI: map 0000:00:19.0
XEN{iommu=no-igfx} KERNEL{rhgb drm.debug=255 intel_iommu=igfx_off}:(XEN) [VT-D]d0:PCIe: unmap 0000:03:00.0
XEN{iommu=no-igfx} KERNEL{rhgb drm.debug=255 intel_iommu=igfx_off}:(XEN) [VT-D]d1:PCIe: map 0000:03:00.0

Those devices are network devices, which netvm attempts to use exclusively.

With iommu=0, it boots.

See attached logs collected with AMT SOL and previous post for lspci -v output. Thanks for your time.

Regards,

Thierry

Le mar. 5 juil. 2016 à 22:07, Thierry Laurion <thierry.laurion@xxxxxxxxx> a écrit :

I Konrad, first, thanks for your input and your time, it is much appreciated.

I understand that those changes are torward the linux kernel, which is used by xen compilation. I applied the changes and i'm rebuilding Qubes with xen 4.6.1 based on a kernel-4.1.24. Will test the build in the next days and post back the results.

output of sudo lspci -v from dom0:
00:00.0 Host bridge: Intel Corporation Mobile 4 Series Chipset Memory Controller Hub (rev 07)
   Subsystem: Lenovo Device 20e0
   Flags: bus master, fast devsel, latency 0
   Capabilities: [e0] Vendor Specific Information: Len=0a <?>
   Kernel driver in use: agpgart-intel

00:02.0 VGA compatible controller: Intel Corporation Mobile 4 Series Chipset Integrated Graphics Controller (rev 07) (prog-if 00 [VGA controller])
   Subsystem: Lenovo Device 20e4
   Flags: bus master, fast devsel, latency 0, IRQ 47
   Memory at e1000000 (64-bit, non-prefetchable) [size=4M]
   Memory at d0000000 (64-bit, prefetchable) [size=256M]
   I/O ports at 3400 [size=8]
   Expansion ROM at <unassigned> [disabled]
   Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
   Capabilities: [d0] Power Management version 3
   Kernel driver in use: i915
   Kernel modules: i915

00:02.1 Display controller: Intel Corporation Mobile 4 Series Chipset Integrated Graphics Controller (rev 07)
   Subsystem: Lenovo Device 20e4
   Flags: fast devsel
   Memory at e1400000 (64-bit, non-prefetchable) [size=1M]
   Capabilities: [d0] Power Management version 3

00:19.0 Ethernet controller: Intel Corporation 82567LF Gigabit Network Connection (rev 03)
   Subsystem: Lenovo Device 20ee
   Flags: bus master, fast devsel, latency 0, IRQ 60
   Memory at e1600000 (32-bit, non-prefetchable) [size=128K]
   Memory at e1624000 (32-bit, non-prefetchable) [size=4K]
   I/O ports at 3000 [size=32]
   Capabilities: [c8] Power Management version 2
   Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [e0] PCI Advanced Features
   Kernel driver in use: pciback
   Kernel modules: e1000e

00:1a.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 03) (prog-if 00 [UHCI])
   Subsystem: Lenovo Device 20f0
   Flags: bus master, medium devsel, latency 0, IRQ 16
   I/O ports at 3020 [size=32]
   Capabilities: [50] PCI Advanced Features
   Kernel driver in use: pciback
   Kernel modules: uhci_hcd

00:1a.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 03) (prog-if 00 [UHCI])
   Subsystem: Lenovo Device 20f0
   Flags: bus master, medium devsel, latency 0, IRQ 17
   I/O ports at 3040 [size=32]
   Capabilities: [50] PCI Advanced Features
   Kernel driver in use: pciback
   Kernel modules: uhci_hcd

00:1a.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 (rev 03) (prog-if 00 [UHCI])
   Subsystem: Lenovo Device 20f0
   Flags: bus master, medium devsel, latency 0, IRQ 18
   I/O ports at 3060 [size=32]
   Capabilities: [50] PCI Advanced Features
   Kernel driver in use: pciback
   Kernel modules: uhci_hcd

00:1a.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 03) (prog-if 20 [EHCI])
   Subsystem: Lenovo Device 20f1
   Flags: bus master, medium devsel, latency 0, IRQ 18
   Memory at e1626000 (32-bit, non-prefetchable) [size=1K]
   Capabilities: [50] Power Management version 2
   Capabilities: [58] Debug port: BAR=1 offset=00a0
   Capabilities: [98] PCI Advanced Features
   Kernel driver in use: pciback
   Kernel modules: ehci_pci

00:1b.0 Audio device: Intel Corporation 82801I (ICH9 Family) HD Audio Controller (rev 03)
   Subsystem: Lenovo Device 20f2
   Flags: bus master, fast devsel, latency 0, IRQ 48
   Memory at e1620000 (64-bit, non-prefetchable) [size=16K]
   Capabilities: [50] Power Management version 2
   Capabilities: [60] MSI: Enable+ Count=1/1 Maskable- 64bit+
   Capabilities: [70] Express Root Complex Integrated Endpoint, MSI 00
   Capabilities: [100] Virtual Channel
   Capabilities: [130] Root Complex Link
   Kernel driver in use: snd_hda_intel
   Kernel modules: snd_hda_intel

00:1c.0 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 1 (rev 03) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 40
   Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
   Capabilities: [40] Express Root Port (Slot-), MSI 00
   Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
   Capabilities: [90] Subsystem: Lenovo Device 20f3
   Capabilities: [a0] Power Management version 2
   Capabilities: [100] Virtual Channel
   Capabilities: [180] Root Complex Link
   Kernel driver in use: pcieport
   Kernel modules: shpchp

00:1c.1 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 2 (rev 03) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 41
   Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
   Memory behind bridge: e1500000-e15fffff
   Capabilities: [40] Express Root Port (Slot-), MSI 00
   Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
   Capabilities: [90] Subsystem: Lenovo Device 20f3
   Capabilities: [a0] Power Management version 2
   Capabilities: [100] Virtual Channel
   Capabilities: [180] Root Complex Link
   Kernel driver in use: pcieport
   Kernel modules: shpchp

00:1c.2 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 3 (rev 03) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 42
   Bus: primary=00, secondary=03, subordinate=03, sec-latency=0
   Capabilities: [40] Express Root Port (Slot-), MSI 00
   Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
   Capabilities: [90] Subsystem: Lenovo Device 20f3
   Capabilities: [a0] Power Management version 2
   Capabilities: [100] Virtual Channel
   Capabilities: [180] Root Complex Link
   Kernel driver in use: pcieport
   Kernel modules: shpchp

00:1c.3 PCI bridge: Intel Corporation 82801I (ICH9 Family) PCI Express Port 4 (rev 03) (prog-if 00 [Normal decode])
   Flags: bus master, fast devsel, latency 0, IRQ 43
   Bus: primary=00, secondary=04, subordinate=04, sec-latency=0
   I/O behind bridge: 00002000-00002fff
   Memory behind bridge: e0800000-e0ffffff
   Prefetchable memory behind bridge: 00000000e0000000-00000000e07fffff
   Capabilities: [40] Express Root Port (Slot-), MSI 00
   Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit-
   Capabilities: [90] Subsystem: Lenovo Device 20f3
   Capabilities: [a0] Power Management version 2
   Capabilities: [100] Virtual Channel
   Capabilities: [180] Root Complex Link
   Kernel driver in use: pcieport
   Kernel modules: shpchp

00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 03) (prog-if 00 [UHCI])
   Subsystem: Lenovo Device 20f0
   Flags: bus master, medium devsel, latency 0, IRQ 16
   I/O ports at 3080 [size=32]
   Capabilities: [50] PCI Advanced Features
   Kernel driver in use: pciback
   Kernel modules: uhci_hcd

00:1d.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 03) (prog-if 00 [UHCI])
   Subsystem: Lenovo Device 20f0
   Flags: bus master, medium devsel, latency 0, IRQ 17
   I/O ports at 30a0 [size=32]
   Capabilities: [50] PCI Advanced Features
   Kernel driver in use: pciback
   Kernel modules: uhci_hcd

00:1d.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 03) (prog-if 00 [UHCI])
   Subsystem: Lenovo Device 20f0
   Flags: bus master, medium devsel, latency 0, IRQ 18
   I/O ports at 30c0 [size=32]
   Capabilities: [50] PCI Advanced Features
   Kernel driver in use: pciback
   Kernel modules: uhci_hcd

00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 03) (prog-if 20 [EHCI])
   Subsystem: Lenovo Device 20f1
   Flags: bus master, medium devsel, latency 0, IRQ 16
   Memory at e1627000 (32-bit, non-prefetchable) [size=1K]
   Capabilities: [50] Power Management version 2
   Capabilities: [58] Debug port: BAR=1 offset=00a0
   Capabilities: [98] PCI Advanced Features
   Kernel driver in use: pciback
   Kernel modules: ehci_pci

00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev 93) (prog-if 01 [Subtractive decode])
   Flags: fast devsel
   Bus: primary=00, secondary=05, subordinate=05, sec-latency=32
   Capabilities: [50] Subsystem: Lenovo Device 20f4

00:1f.0 ISA bridge: Intel Corporation ICH9M LPC Interface Controller (rev 03)
   Subsystem: Lenovo Device 20f5
   Flags: bus master, medium devsel, latency 0
   Capabilities: [e0] Vendor Specific Information: Len=0c <?>
   Kernel driver in use: lpc_ich
   Kernel modules: lpc_ich

00:1f.2 SATA controller: Intel Corporation 82801IBM/IEM (ICH9M/ICH9M-E) 4 port SATA Controller [AHCI mode] (rev 03) (prog-if 01 [AHCI 1.0])
   Subsystem: Lenovo Device 20f8
   Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 46
   I/O ports at 3408 [size=8]
   I/O ports at 3418 [size=4]
   I/O ports at 3410 [size=8]
   I/O ports at 341c [size=4]
   I/O ports at 30e0 [size=32]
   Memory at e1625000 (32-bit, non-prefetchable) [size=2K]
   Capabilities: [80] MSI: Enable+ Count=1/16 Maskable- 64bit-
   Capabilities: [70] Power Management version 3
   Capabilities: [a8] SATA HBA v1.0
   Capabilities: [b0] PCI Advanced Features
   Kernel driver in use: ahci

00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 03)
   Subsystem: Lenovo Device 20f9
   Flags: medium devsel, IRQ 18
   Memory at e1628000 (64-bit, non-prefetchable) [size=256]
   I/O ports at 0400 [size=32]
   Kernel modules: i2c_i801

02:00.0 Network controller: Qualcomm Atheros AR9285 Wireless Network Adapter (PCI-Express) (rev 01)
   Subsystem: Foxconn International, Inc. T77H126.00 802.11bgn Wireless Half-size Mini PCIe Card
   Flags: bus master, fast devsel, latency 0, IRQ 17
   Memory at e1500000 (64-bit, non-prefetchable) [size=64K]
   Capabilities: [40] Power Management version 3
   Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit-
   Capabilities: [60] Express Legacy Endpoint, MSI 00
   Capabilities: [100] Advanced Error Reporting
   Capabilities: [140] Virtual Channel
   Capabilities: [160] Device Serial Number 00-15-17-ff-ff-24-14-12
   Capabilities: [170] Power Budgeting <?>
   Kernel driver in use: pciback
   Kernel modules: ath9k

Le jeu. 30 juin 2016 à 09:37, Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx> a écrit :
On Sun, Jun 26, 2016 at 11:48:44PM +0000, Thierry Laurion wrote:
> Sorry for the precedent post that was written a bit too fast. Libreboot was
> flashed when I wrote it, which is the equivalent of a having vt-d
> deactivated (iommu=0). Thanks to a user that read this post and wrote to me
> personally so I could do my mea culpa. Sorry for the precedent misleading
> post.
>
> Xen on a GM45 chipset and with IGD i915 driver is still getting the system
> hanged when vt-d is activated. I'm willing to borrow a machine to the Xen
> developer that could fix the iommu=no-igfx code for gm45 chipset to
> actually work.

This sounds like http://wiki.xenproject.org/wiki/Paravirtualized_DRM
issues.

Can you try and also attach lspci -v ?

diff --git a/drivers/char/agp/intel-gtt.c b/drivers/char/agp/intel-gtt.c
index aef87fd..cf31aad 100644
--- a/drivers/char/agp/intel-gtt.c
+++ b/drivers/char/agp/intel-gtt.c
@@ -35,7 +35,7 @@
#ifdef CONFIG_INTEL_IOMMU
#define USE_PCI_DMA_API 1
#else
-#define USE_PCI_DMA_API 0
+#define USE_PCI_DMA_API 1
#endif

struct intel_gtt_driver {
@@ -654,6 +654,7 @@ static int intel_gtt_init(void)

intel_private.needs_dmar = USE_PCI_DMA_API && INTEL_GTT_GEN > 2;

+ printk("%s: %s DMA ops\n", __func__,intel_private.needs_dmar ? "Using" : "Not using");
ret = intel_gtt_setup_scratch_page();
if (ret != 0) {
intel_gtt_cleanup();
>
> A ticket is opened here with current states of thing:
> https://github.com/QubesOS/qubes-issues/issues/1594#issuecomment-209213917
>
> Sorry about that (and repost since I wrote the same misleading post to two
> places)
> Thierry
>
> Le dim. 28 févr. 2016 à 14:03, Thierry Laurion <thierry.laurion@xxxxxxxxx>
> a écrit :
>
> > The problem wasn't with xen iommu support but kms/drm and i915 driver.
> >
> > Passing to the kernel i915.preliminary_hw_support=1 fixes it all :)
> >
> > Thanks
> >
> > Le mer. 6 janv. 2016 à 22:11, Thierry Laurion <thierry.laurion@xxxxxxxxx>
> > a écrit :
> >
> >> Nope. That commit is present in 4.6 and results in x200 being able to
> >> boot xen.
> >>
> >> Not having that option makes xen hang at boot.
> >>
> >> If present, it works until other vm access pass-through devices, which
> >> I'm not able to troubleshoot even through amt SOL.
> >>
> >> See here for debug logs:
> >> https://groups.google.com/forum/m/#!topic/qubes-users/bHQHjXqinaU
> >>
> >> Le mer. 6 janv. 2016 09:35, Jan Beulich <JBeulich@xxxxxxxx> a écrit :
> >>
> >>> >>> On 22.12.15 at 19:04, <thierry.laurion@xxxxxxxxx> wrote:
> >>> > iommu=no-igfx is a gamechanger for Qubes support through 3.1 RC1
> >>> release,
> >>> > thanks to Xen 4.6 :)
> >>> >
> >>> > The Lenovo X200 supports vt-x, vt-d and TPM as reported and required by
> >>> > Qubes in the HCL attached to this e-mail. The problem is that when
> >>> Qubes
> >>> > launches it's netvm which uses IOMMU to talk to it's network card, it
> >>> > freezes the whole system up. Even when specifying sync_console, I
> >>> don't get
> >>> > much more verbosity. I ordered a PCMCIA to serial adapter which will be
> >>> > shipped to my door late January... Meanwhile, booting with iommu=0
> >>> makes
> >>> > things work, but a potential hardware component being compromised has
> >>> > chances to compromise the whole system since compartmentalization is
> >>> not
> >>> > guaranteed without IOMMU (vt-d).
> >>> >
> >>> > A little more love is needed from xen to make that laptop line
> >>> supported by
> >>> > Qubes and a nice alternative to the costy Librem currently promoted by
> >>> > Qubes-Purism
> >>> > partnership
> >>>
> >>> Is all of the above and below a quite complicated way of expressing
> >>> that you'd like to see commit 146341187a backported to 4.6.x?
> >>>
> >>> Jan
> >>>
> >>> > <
> >>> http://arstechnica.com/gadgets/2015/12/qubes-os-will-ship-pre-installed-on-p
> >>> > urisms-security-focused-librem-13-laptop/>which
> >>> > suggest that the laptop will be Respect Your Freedom compliant in the
> >>> > future with Intel participation in removing ME and AMT
> >>> > <http://libreboot.org/faq/#intelme>, which is not guaranteed at all.
> >>> > <
> >>> http://www.phoronix.com/scan.php?page=news_item&px=Purism-Librem-Still-Blobbe
> >>> > d>
> >>> > If Xen 4.6 can cooperate with Penryn GM45 chipset, it's all MiniFree
> >>> laptops
> >>> > <http://minifree.org/product-category/laptops/> (and Libreboot
> >>> support of
> >>> > those <http://libreboot.org/docs/hcl/x200.html>) that will be
> >>> potential
> >>> > candidates!
> >>> > Please share the love so that the community has a cheap alternative.
> >>> >
> >>> > Requirements to replicate bug:
> >>> > Model: X200 745434U with p8700 CPU running 1067a microcode(important),
> >>> > upgrable to 8go
> >>> > BIOS: Lenovo 3.22/1.07 (latest from 2013
> >>> > <http://support.lenovo.com/ca/en/downloads/ds015007>)
> >>> > Network card supports FLReset+ as requested here
> >>> > <http://wiki.xen.org/wiki/VTd_HowTo>.
> >>> > Bios settings: vt-d and vt-x needs to be enforced.
> >>> > Xen command line option required
> >>> > <http://www.gossamer-threads.com/lists/xen/devel/393647> to boot:
> >>> > iommu=no-igfx
> >>> >
> >>> > Here is the current debug trace/status on Qubes side of things
> >>> > <https://groups.google.com/forum/#!topic/qubes-users/bHQHjXqinaU>.
> >>> > If you have any hint, please contribute :)
> >>> >
> >>> > Help me say happy new years to all security conscious people out there
> >>> :)
> >>> >
> >>> > Merry Christmas all,
> >>> > Thierry Laurion
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Thierry Laurion
> >>>
> >>>
> >>>
> >>>

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel

Re: [Xen-devel] Lenovo X200 IOMMU support through Xen 4.6 iommu=no-igfx switch