RE: [Xen-ia64-devel] consoles, iosapics, and device interrupts

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

(Sorry for a late mail on this thread and so aggregate some small pieces 
together ;-)

>From: Alex Williamson
>Sent: 2005年11月18日 1:16
>      * PCDP info is fairly unreliable for getting proper
>        trigger/polarity data.  ACPI namespace would be preferred for
>        non-PCI UARTs.  We don't currently start ACPI namespace, nor am
>        I convinced we want to.

Alex, sorry I'm not familiar with PCDP table used here. What's its major 
purpose there? Why doesn't it provide proper trigger/polarity data while ACPI 
already has?

>   I also need to go back and further digest the threads Kevin
>referenced, but I'm thinking we'll likely need to virtualize the IOSAPIC
>RTEs to the domains.

Depends on what "virtualization" actually means, it can route to different 
direction. If you let dom0 still aware existence of IOSAPIC without change, you 
have to trap MMIO access to IOSAPIC from dom0 to ensure those mapping entries 
from guest not inserted into machine TLB. So first step is to modify current 
vTLB (1 entry dtc/itc) implementation to cache more guest entries like this 
one. Then mmio instruction decoder is required. Reference code is 
arch/xen/vmx/{vtlb.c,vmmu.c,mmio.c} for VTI domain since that's a must feature 
for the latter.

Another way is to modify dom0 kernel to let it aware of existence of a virtual 
IOSAPIC or similar para-controller. That's current x86 approach with virtual 
interrupt controller working upon hypercall.

>From: Tristan Gingold
>Sent: 2005年11月18日 19:53
>Dom0 linux (or dom-driver linuxes) see and parses MADT, but does not
>directly modify IOSAPIC: it does an hypercall, which may fails if the
>interrupt is already allocated by the hypervisor (or another domain).  I
>think this approach is simple and follows the transparent para-virtualization
>spirit.
>
>Should I go ?

Agree on this approach, though I'm not sure whether it can be done easily 
without modifying core interrupt code of xenlinux. But that's preferred way to 
go, IMO. ;-)

>From: Alex Williamson
>Sent: 2005年11月18日 23:53
>   Yes, please, sounds like a good start.  After contemplating the MADT
>issue some more, what will dom0 do w/ the IOSAPICs?  We're going to need
>extra xen specific kludges in xenlinux to see IOSAPICS in the MADT and
>ignore them.  There are no flags in the IOSAPIC MADT entry to mark it
>disabled like there is for the local APIC.  I think maybe we should just
>remove the IOSAPICs from the MADT to avoid xenlinux registering support
>for them.

Maybe a fierce hack is just to modify some fields in MADT entry and let dom0 
think it broken. ;-) However the most important is, after MADT is removed, how 
do we provide an alternative hw_interrupt_type controller to guest? Once the 
IOSAPIC and related interrupt vector resources are controlled by hypervisor, 
there gonna add hypercall between dom0 and hypervisor to setup relationship 
between virtual interrupts and actual interrupt lines. To better accommodate 
current interrupt sub-system of Linux, a natural way is to provide a virtual 
interrupt controller as current x86. Because what we virtualized is the 
interrupt vector, the ack/enable/disable/eoi actions are still needed to be 
issued from domain.

>From: Alex Williamson
>Sent: 2005年11月19日 0:53
>The security risk of the debugging console doesn't seem so big to me.
>Don't you have to have physical access to the console device (VGA or
>serial)?  Adding a password to access the xen console would probably be
>pragmatic, but I don't think we need to go overboard when access is
>already so limited.

Agree. That's the issue we need to track down, instead of avoid. I don't expect 
to see that debug feature missing or changed on XEN/IA64, especially when some 
type of boxes have legacy interrupts for serial ports.

>   I think we're currently just trying to get up to parity w/ x86.  x86
>has a different architecture for interrupts, so doesn't really have the
>same IOSAPIC issue.  AFAICT though, interrupts are virualized on x86.  I
>need to get a better feel for the code to see what they've done around
>ACPI though.  Power likely doesn't care about ACPI at all.  Thanks,

Currently on x86, IOAPIC code is still there for dom0 to setup RTEs for PCI 
devices however with some modification. Long time ago (xen2.0), all IOAPIC 
related initialization including PCI devices are owned by Xen and then dom0 
doesn't see IOAPIC. Then several months ago, Arun kept legacy interrupts 
initialization in Xen and then moved all PCI related stuff (together with ACPI 
namespace) into dom0. Then it's dom0's responsibility to:
        - parse ACPI namespace
        - Get PCI routing table information
        - For each existing PCI devices:
                * allocate a dynamic interrupt vector as native linux, which 
however is only a virtual vector meaningful to dom0 itself
                * hypercall into Xen and then Xen will allocate a real global 
interrupt vector and bound to that virtual vector
                * try to update RTE for this device, by hypercall into Xen 
which will further look at the virtual vector and replace it with real vector.
        - Later when one PCI decice is found and tries to setup irq handler, an 
event channel port will be created and bound to that virtual interrupt vector.

After all these preparations are done, you can easily see how interrupt is 
handled currently on x86:
        - One PCI device triggers interrupt and trap into interrupt handler of 
xen
        - Xen checks the descriptor whether owned by guest. If yes, just notify 
guest by bound event channel port.
        - event dispatcher (or similar component) in guest further searches 
local pirq_to_vector mapping table, and finally deliver to handler of virtual 
interrupt vector.

By far this flow seems natural to me, and it would be great if people can come 
up better and clearer way to do it as it does modify a lot. ;-)

Thanks,
Kevin

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel