WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] [Experimental PATCH] PCI and IO device emulation

To: "Neugebauer, Rolf" <rolf.neugebauer@xxxxxxxxx>
Subject: RE: [Xen-devel] [Experimental PATCH] PCI and IO device emulation
From: Stefan Berger <stefanb@xxxxxxxxxx>
Date: Fri, 30 Sep 2005 12:14:49 -0500
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Delivery-date: Fri, 30 Sep 2005 17:12:32 +0000
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <39CC97884CA19A4D8D6296FE94357BCB0341103B@swsmsx404>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx

"Neugebauer, Rolf" <rolf.neugebauer@xxxxxxxxx> wrote on 09/30/2005 05:53:36 AM:

>
>
> > -----Original Message-----
> > From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-devel-
> > bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Stefan Berger
> > Sent: 30 September 2005 02:16
> > To: Keir Fraser
> > Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> > Subject: Re: [Xen-devel] [Experimental PATCH] PCI and IO device
> emulation
> >
> > Keir Fraser <Keir.Fraser@xxxxxxxxxxxx> wrote on 09/28/2005 08:41:19
> AM:
> >
> > >
> > > On 28 Sep 2005, at 03:50, Stefan Berger wrote:
> > >
>
> >
> > >
> > > The only mechanism I think we should have in Xen is a protected
> > > interface for allowing domU's to access PCI config space. I would
> make
> > > it an explicit hypercall interface rather than bothering with
> emulating
> > > I/O port accesses -- we have to make modifications to the PCI stack
> > > anyway (otherwise we get into having to do crap like providing fake
> > > BIOS tables to provide dummy bus and irq info), and adding a new
> type
> > > of pci read/write access method is trivial in Linux. I expect the
> same
> > > is true of most other OSes.
> >
> > I think the dummy bus could have a potential (double) benefit for
> running
> > unmodified legacy OSes as well and would not just serve 'driver
> domains'.
>
> At the moment the PCI emulation for unmodified guests is dealt with in
> the device model in dom0. Are you suggesting to pull this into Xen
> instead?


If we can come up with architecture that can serve both purposes, enable driver domains and provide access to an emulated PCI bus (or even real devices) for unmodified guests, then I would do it.

>
>
> > >
> > > This interface will reject all config accesses by default, but
> domain0
> > > can change access privileges on a per-device basis. All that Xen has
> to
> >
> >
> > > do is then mask off some of the registers for write access (e.g.,
> don;t
> > > allow domU to arbitrarily rewrite resource base addresses) and
> possibly
> > > fake out reads for certain registers (e.g., perhaps the IRQ number
> > > register).
> >
> > At least that part you could also solve by providing a PCI emulation
> > layer. Some devices can be accessible, others remain hidden.
>
> Sure, but then you have to PCI emulation layer in Xen. Just doing what
> keir suggests requires one file...
>
> > You can also
> > intercept requests to the IRQ number register and maybe return the
> > system's real IRQ number without having it translated in the user
> domain.
>
> That's what Keir suggested. The problem is that at least Linux seems to
> largely ignore the IRQ number in the PCI config space.


I am saying that this can also be done in the PCI emulation layer.

>
> > >
> > > All other smarts belong in domain0 imo. The only reason for not
> doing
> > > the whole lot in domain0 is that a pcifront/pciback split driver
> would
> > > be a lot more pain to write and to debug.
> >
> > :-) I would push those smarts downwards to avoid changes in OS(es).
>
> With Keir' proposal there are three types of changes:
>
> - changes required to non-dom0 OSes. I think these are minimal in keir's
> proposal and only require changes to arch-dependent code. Linux already
> supports three PCI config probing methods on ia32. adding a 4th one
> which uses a hypercall interface is trivial. The other change would be

> to the IRQ setup. You'd have to convince the OS to believe the IRQ
> number in the config space (or get it by some other means). I think that


I also think it would be important that the OS use the IRQ number in the config space. The Hypervisor can always do translation of IRQ numbers from machine physical to VM in the hypervisor before propagating the IRQ to the domain.

> is straight forward as well and both has been done in Xen 2.X. You can
> even delete/remove an lot of code related to quirks in the real HW.

>
> - changes to dom0 kernels. These are trickier in Xen 3.x. dom0 has to do
> the entire setup for the device as it normally would but then not start
> a device driver for devices which are supposed to be hidden from it.
> This is likely to require a small change to arch-independent code in
> Linux. It would also be nice if dom0 could still see all devices in the
> system via things like lspci to make it easier to do device assignment
>
> Another requirement is that both these changes need to coexist in the
> same kernel so that no separate kernels are required for dom0 and domUs

>
> - addition to the xen hypercall interface to allow domUs to probe the
> PCI config space. This is not strictly required as one could just
> emulate the io reads/writes of a guests, but it seems cleaner to do it
> via a hypercall interface. There is also only a small addition to the
> hypervisor to deal with the hypercalls. But in xen 2x this is about


Access to PCI config space is protected by spinlocks. First one writes bus/device/function and address into the config register then one reads or writes from/to the data register affecting the previously given address. To properly protect access to the config space it really should be done in a common place and accessed in the same way by domain 0 and any other domain.

>
> Your proposal doesn't require changes to guests but adds significantly
> more code to Xen. I think the changes required to device driver domains
> are small.
>
> Could you elaborate a bit more on how your proposal interacts with Dom0
> and the IRQ routing/IO APIC set up it is doing at the moment. If you
> hide the device completely, as I understand you are doing, how will that
> be done?


I don't think you would like to move ownership of the IO-APIC to Xen and let Xen do that part of the setup.  Therefore, I would build a request list for domain 0 to do the necessary setup for those devices it does not see and provide the HV with the necessary information (like machine physical interrupt) it needs to properly route the IRQs to a domain that is accessing a device. This would require an additional module in the domain 0 kernel and some more hypercalls.

   Stefan
 
>
> Thanks
> Rolf
>
>
>
> >
> >   Stefan
> > >
> > >   -- Keir
> > >
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel