Re: SecureBoot and PCI passthrough with kernel lockdown in place (on Xen)

["marmarek@xxxxxxxxxxxxxxxxxxxxxx" <marmarek@xxxxxxxxxxxxxxxxxxxxxx>]

On Mon, Feb 14, 2022 at 03:25:31PM +0000, Andrew Cooper wrote:
> On 14/02/2022 15:02, Dario Faggioli wrote:
> > Hello,
> >
> > We have run into an issue when trying to use PCI passthrough for a Xen
> > VM running on an host where dom0 kernel is 5.14.21 (but we think it
> > could be any kernel > 5.4) and SecureBoot is enabled.
> 
> Back up a bit...
> 
> Xen doesn't support SecureBoot and there's a massive pile of work to
> make it function, let alone work in a way that MSFT aren't liable to
> revoke your cert on 0 notice.
> 
> >
> > The error we get, when (for instance) trying to attach a device to an
> > (HVM) VM, on such system is:
> >
> > # xl pci-attach 2-fv-sles15sp4beta2 0000:58:03.0 
> > libxl: error: libxl_qmp.c:1838:qmp_ev_parse_error_messages: Domain 
> > 12:Failed to initialize 12/15, type = 0x1, rc: -1
> > libxl: error: libxl_pci.c:1777:device_pci_add_done: Domain 
> > 12:libxl__device_pci_add failed for PCI device 0:58:3.0 (rc -28)
> > libxl: error: libxl_device.c:1420:device_addrm_aocomplete: unable to add 
> > device
> >
> > QEMU, is telling us the following:
> >
> > [00:04.0] xen_pt_msix_init: Error: Can't open /dev/mem: Operation not 
> > permitted
> > [00:04.0] xen_pt_msix_size_init: Error: Internal error: Invalid 
> > xen_pt_msix_init.
> >
> > And the kernel reports this:
> >
> > Jan 27 16:20:53 narvi-sr860v2-bps-sles15sp4b2 kernel: Lockdown: 
> > qemu-system-i38: /dev/mem,kmem,port is restricted; see man kernel_lockdown.7
> >
> > So, it's related to lockdown. Which AFAIUI it's consistent with the
> > fact that the problem only shows up when SecureBoot is enabled, as
> > that's implies lockdown. It's also consistent with the fact that we
> > don't seem to have any problems doing the same with a 5.3.x dom0
> > kernel... As there's no lockdown there!
> >
> > Some digging revealed that QEMU tries to open /dev/mem in
> > xen_pt_msix_init():
> >
> >     fd = open("/dev/mem", O_RDWR);
> >     ...
> >     msix->phys_iomem_base =
> >             mmap(NULL,
> >                  total_entries * PCI_MSIX_ENTRY_SIZE + 
> > msix->table_offset_adjust,
> >                  PROT_READ,
> >                  MAP_SHARED | MAP_LOCKED,
> >                  fd,
> >                  msix->table_base + table_off - msix->table_offset_adjust);
> >     close(fd);
> 
> Yes.  Use of /dev/mem is not permitted in lockdown mode.  This wants
> reworking into something which is lockdown compatible.

FWIW, Qubes has PCI passthrough working with qemu in stubdomain, which
works without access to /dev/mem in dom0. We do this, by disabling
MSI-X, including the above piece of code...

https://github.com/QubesOS/qubes-vmm-xen-stubdom-linux/blob/master/qemu/patches/0005-Disable-MSI-X-caps.patch

> The real elephant in the room is that privcmd is not remotely safe to
> use in a SecureBoot environment, because it lets any root userspace
> trivially escalate privilege into the dom0 kernel, bypassing the
> specific protection that SecureBoot is trying to achieve.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab

Attachment: signature.asc
Description: PGP signature