WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

[Xen-devel] Re: [Xen-users] Passthrough support ?

To: Mark Williamson <mark.williamson@/cl.cam.ac.uk>
Subject: [Xen-devel] Re: [Xen-users] Passthrough support ?
From: sanket hase <shase@xxxxxxxxxxxxxx>
Date: Tue, 27 Nov 2007 20:27:52 -0500
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx, xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Tue, 27 Nov 2007 17:28:34 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <200711272055.39363.mark.williamson@xxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <474B5E98.1060302@xxxxxxxxxxxxxx> <200711272055.39363.mark.williamson@xxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
User-agent: Thunderbird 2.0.0.6 (Windows/20070728)
Good brainer. ... I  am geting back to few basics.
I will leave HVM and emulation aside :)

1. When PCI Passthrough support 'is not enabled' , how does domU access PCI device ? I suppose they continue to communicate using pcifront/back split drivers and now dom0's
   drivers are used ( right ? )

2. PCI-passthrough enables domU to own the PCI device. Now dom0 can no longer use that device . If domU has mapped address space of the required PCI device , why need it talk to dom0 any further ? What kind of hand shake is involved ( setup and tear down as you have mentioned ) between domU and
   domU  " in passthrough case" ?

3. DomU discovers all PCI devices through  Xenstore.

Mark Williamson wrote:
I  have been trying to understand PCI -passthrough support.  Please
correct me if I am wrong in my following inferences.

1. Device emulation and pass through are both implemented using split
drivers.

I'm going to be pedantic now ;-)

Device "emulation" is really what we do for fully virtualised (HVM) guests: the device models provided by QEmu emulate real world devices in terms of their responses to particular port IOs, mmapped IO operations, etc. This isn't done using a front / back model because the guest is just using it's normal drivers for the "real" devices.

The virtual devices used by the PV drivers are implemented using split drivers, though, as you say.

The PCI passthrough support for PV guest is also implemented using a split driver that implements the functions of the PCI bus in order to give the guest the information it
Which is this information we are referring to here ?
requires to talk to its assigned device(s).
How are devices "assigned" to domU ? I am specifically talking about late binding.
Having obtained this information, communication with the device is possible directly using IO ports, memory IO regions, and DMA.

2. But, in case of emulation drivers of dom0 are used  where as in case
of passthrough ( as the name suggests ) native drivers in domU are used .

For true emulation (qemu device model), a userspace process in dom0 handles modelling a "real" device and then issues IO using normal userspace APIs. These get serviced by the dom0 kernel using the normal device driver.

For PV drivers, the frontend driver in the domU kernel issues requests which are picked up by the backend driver in dom0's kernel, which then issues requests into the IO stack. Again this uses the normal device driver in dom0 to talk to the actual device, it's just that the request is made using a kernel-internal API rather than a userspace API (which results in slightly different actions being taken).

3. dom0 provides a virtual PCI device { an interface for device-OPs and
status  of this virtual device} to domU and  through associated event
channel domU makes
   "synchronous" use of this device.

domU uses this for control plane operations, but for most work it can talk to its PCI device directly without going through dom0.

===
Queries:

1. What i am really not so sure about is ... passthrough case
    Will there be requirement to map the address space of this PCI
device in domU ?  Will the page which was being shared  so-far
{xen_pci_sharedinfo}
     for emulation , be "flipped"  ( transferred ) into domU ?

xen_pci_sharedinfo - is that the page used to talk to the PCI backend from pcifront? If so, then no, that's just used for dom0-domU communications.
Well As I read in the xen0linux code , xen_pci_sharedinfo contains xen_pci_op where front end specifies which operation is to be done on which pci device. And then it keeps checking the status of this device through another flag in xen_pci_sharedinfo. I infer here that dom0's drivers are being used. Can you point me towards a code path where domU actually
uses its own drivers having owned the PCI device ?

In order to map the address space of the PCI device directly, the domU is given permissions to map the IO memory regions of that device into it's page tables. I think this is now possible to do using a grant table operation...

It is also given permission to access certain IO port ranges so that it can use the device's port IO interfaces.

2. Well , Having read the code for linux (dom0,domU) I see that there are split
device  drivers for PCI. (pci front and PCIback). Which are normally
   communicating over xenbus.  which looks almost like other split
drivers.  How exactly then passthrough enables use of domU's drivers ?

The key thing to understand is that the pcifront / pciback is basically just used for setup and teardown, not for the actual IO. The real IO is done directly by the domU without going through dom0. For the block and net drivers, *all* IO goes through dom0.

3. And if passthrough support isnt provided how will communication
between pcifront-pciback  be different ? ( I guess netbsd , freebsd do
not have passthrough support yet )

I'm not entirely clear what you're asking here, but I'll take a stab at it:

if pcifront (in domU) and pciback (in dom0) aren't available then passthrough won't work. The dom0 has to support the backend functions of PCI passthrough and the domU has to know how to talk to it. It's also implicit that they're using the same interface version to talk to each other - I'm not sure whether that's frozen stable or not.

So *if* NetBSD lacks pciback support, it can't pass PCI devices to guests that do. Similarly, *if* it lacks pcifront support, it can't have devices passed to it.

4. What  restricts other domUs from accessing PCI device given to other
domU via passrthrough support.

There are some restrictions on what can be done in PCI config space to prevent a guest fouling things up. These need to be relaxed for some awkward devices, though.

For the device IO itself, domUs are only allowed to map mmio regions and access io ports that are relevant to their device. It's possible for these to overlap with those for other devices, in which case you're trusting the domU to be well behaved. More crucially, though, giving a domain a device with DMA capabilities is equivalent to giving it the ability to subvert the entire machine. DMA can't be sandboxed on most current hardware, so if you give DMA rights to a VM it's automatically just as trusted as dom0 with respect to not fooling about with other domains, hardware, etc.

Cheers,
Mark

Thanks,
Sanket

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel