Re: [PATCH v6] xen/pt: reserve PCI slot 2 for Intel igd-passthru

[Chuck Zmudzinski <brchuckz@xxxxxxx>]

On 1/4/23 3:47 PM, Chuck Zmudzinski wrote:
> On 1/3/23 10:14 AM, Alex Williamson wrote:
> 
>> 
>> It's necessary to configure the assigned IGD at slot 2 to make it
>> functional, yes, but I don't really understand this notion of
>> "reserving" slot 2.  If something occupies address 00:02.0 in the
>> config, it's the user's or management tool's responsibility to move it
>> to make this configuration functional.  Why does QEMU need to play a
>> part in reserving this bus address.  IGD devices are not generally
>> hot-pluggable either, so it doesn't seem we need to reserve an address
>> in case an IGD device is added dynamically later.
> 
> The capability to reserve a bus address for a quirky device need not
> be limited to the case of hotplugged or dynamically added devices. The
> igd is a quirky device, and its presence in an emulated system like
> qemu requires special handling. The slot_reserved_mask member of PCIBus
> is also well-suited to the case of quirky device like Intel the igd that
> needs to be at slot 2. Just because it is not dynamically added later
> does not change the fact that it needs special handling at its initial
> configuration when the guest is being created.
> 
>>  
> 
> Here's the problem that answers Michael's question why this patch is
> specific to xen:
> 
> ---snip---
> #ifdef CONFIG_XEN
> 
> ...
> 
> static void pc_xen_hvm_init(MachineState *machine)
> {
>     PCMachineState *pcms = PC_MACHINE(machine);
> 
>     if (!xen_enabled()) {
>         error_report("xenfv machine requires the xen accelerator");
>         exit(1);
>     }
> 
>     pc_xen_hvm_init_pci(machine);
>     pci_create_simple(pcms->bus, -1, "xen-platform");
> }
> #endif
> ---snip---
> 
> This code is from hw/i386/pc_piix.c. Note the call to
> pci_create_simple to create the xen platform pci device,
> which has -1 as the second argument. That -1 tells
> pci_create_simple to autoconfigure the pci bdf address.
> 
> It is *hard-coded* that way. That means no toolstack or
> management tool can change it. And what is hard-coded here
> is that the xen platform device will occupy slot 2, preventing
> the Intel igd or any other device from occupying slot 2.
> 

Actually, today I discovered it is possible to workaround
this issue with libxl that appears to hard-code the xen
platform device to slot 2.

The code referenced here that effectively hard codes the xen
platform device to slot 2 is only executed with the
'-machine xenfv' qemu option, which is the default that
libxl uses for hvm guests, but this behavior can be changed
by setting xen_platform_pci = '0' in the xl guest configuration
file, in which case libxl configures qemu with the
'-machine pc,accel=xen' option instead, and then one can add
the xen platform device at a different slot by adding, for
example,

device_model_args = ['-device','xen-platform,addr=03']

to the xl guest configuration file which adds the device at
slot 3 instead of slot 2.

This approach is an ugly workaround which has other side effects,
such as by having -machine as pc,accel=xen instead of xenfv, the
machine type is not exactly the same because the xenfv machine
type is based on a much older version of qemu's i440fx emulation
(qemu version 3.1 IIRC), so with this workaround there could be
some unintended side effects, although I just tested this
workaround and it does seem to work, but I have not been using
this approach to the problem for very long.

Also, this approach requires setting type=vif in the vif network
setting of the xl guest configuration to suppress the creation of
the qemu emulated network device that would grab slot 2 if it is
created by the ordinary libxl network configuration settings and,
for guests that do not have the xen pv network drivers installed,
this also would require manually building the qemu command line
using device_model_args to allow adding an emulated qemu network
device at a different slot and probably also manually configuring
the correct networking scripts outside of the normal xen networking
scripts, because the ordinary xl guest configuration options do not
have a setting for assigning an emulated qemu network device to a
specific slot, and the device would otherwise grab slot 2.

So,it is possible to configure the guest to have the Intel igd at slot
2 without patching either libxl or qemu, but it is a real PITA with
some possible unintended side effects, and that is why I think patching
qemu to reserve slot 2 is a much better solution to the problem.

Kind regards,

Chuck