Hi,
Keir,
These patches are rebased
version of Yunhong’s original patches, which were sent
out before XEN 3.2 was released. These patches enable MSI support and limited
MSI-X support in XEN. Here is the original description of the patches from Yunhong’s mail.
The basic idea including:
1)
Keep vector global resource owned by xen, while split
pirq into per-domain information.
2) Domain0 kernel
will operate msi resource for domain0/domU, while QEMU will
operate MSI resource for HVM
domain.
3) Xen will do EOI for MSI
interrupt.
Signed-off-by: Yunhong Jiang <yunhong.jiang@xxxxxxxxx>
There are no much changes
made compared with the original patches. But
there do have some issues that we need your kind
comments.
1> ACK-NEW method is necessary to
avoid IRQ storm. But it causes the deadlock.
During my tests, I do find
there can be deadlock with patches
applied. When assigned a NIC device to HVM domain, the
scenario is: Dom0 is waiting to IDE interrupt (vector 0x21); HVM domain is
waiting for qemu’s IDE emulation and thus blocked; NIC
interrupt (MSI vector 0x31) is waiting for injection to HVM domain since it is
blocked now; IDE interrupt is waiting for NIC interrupt since NIC interrupt is
of high priority but not ACKed by XEN now. When IDE interrupt and NIC interrupt are delivered to
the same CPU, and when guest OS is Vista, the phenomenon is easy to be
observed.
2> Without ACK-NEW, some
naughty NIC devices as we observed will bring IRQ storms. For this phenomenon, I
think Yunhong can comment more. Basically, writing EOI without mask the source
of MSI will bring IRQ storm. Although the reason is under investigation, XEN
should anyhow handle such bogous device, right?
3> Using ACK-OLD and masking the
MSI when writing EOI can be solution. However, XEN does not own PCI
configuration spaces.
We also tried some work
arounds.
One work around might be
using a timer to force a EOI within some time interval.
This method is already implemented in VT-D’s code. However, with this approach, if the timer is fired and
EOI is written, this is essentially the same apporach as option
2.
Another approach is to never
deliver these two IRQs to the same CPU. But this is really ugly and can not be
applied to UP.
We have also considered using VT-D 2
interrupt remapping feature. According to the spec, there is no bit in the
remapping table to mask the interrupt. Therefore, this can not be combined with
option 2 to solve the issue. Masking the interrupt still needs accessing PCI
configuration spaces.
We think the most clean method may
be to move ownership from dom0 to VMM. However, this is a great
change. This should be well discussed in community and need your
comments.
These patch series sent out can be
served as a discussion materials. What is your comments on the patches and the
issues, Keir?
Thanks!
Haitao Shan
Home