WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

RE: [Xen-devel] [PATCH 0/5] pcie io space multiplexing for bootable pass

To: Isaku Yamahata <yamahata@xxxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: RE: [Xen-devel] [PATCH 0/5] pcie io space multiplexing for bootable pass through HVM domain
From: "Tian, Kevin" <kevin.tian@xxxxxxxxx>
Date: Thu, 28 May 2009 18:27:43 +0800
Accept-language: en-US
Acceptlanguage: en-US
Cc: "shimada-yxb@xxxxxxxxxxxxxxx" <shimada-yxb@xxxxxxxxxxxxxxx>, "Ian.Jackson@xxxxxxxxxxxxx" <Ian.Jackson@xxxxxxxxxxxxx>
Delivery-date: Thu, 28 May 2009 03:28:28 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <20090528034648.GA9392@xxxxxxxxxxxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
References: <20090528034648.GA9392@xxxxxxxxxxxxxxxxxxxxxxxxxx>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcnfR0ARDb1nWNGETsS9JkanWMF9xQANpscQ
Thread-topic: [Xen-devel] [PATCH 0/5] pcie io space multiplexing for bootable pass through HVM domain
Curious question. :-)

Is this very virtualization specific usage? How about same box used
in native environment, where how admin can judge which PCI-e slot
is allocated with I/O ports for bootable purpose?

Thanks,
Kevin

>From: Isaku Yamahata
>Sent: 2009年5月28日 11:47
>
>This patch series is for PCIe IO space multiplexing patch for commit.
>
>It is not uncommon that a big iron for server consolidation has
>many (e.g. > 16) PCIe slots. It will hold many domains, and
>the administrator wants them to boot from pass through devices.
>But currently up to 16 hvm domains can boot from pass through device.
>This patch series address it by multiplexing PCI IO space access.
>
>Usage:
>Add the following options to dom0 kernel command line
>
>  guestdev=<device path>+iomul
>  (append "+iomul")
>or
>  guestiomuldev=[<segment>:]<bus>:<dev>[,[<segment:><bus>:dev]][,...]
>  (In this case, don't forget to add related options. pciback.hide,
>   reassign_resources or guestdev.)
>
>Then dom0 Linux will allocate IO ports which are shared by 
>specified devices.
>And ioemu will automatically recognize IO-port-shared devices.
>The unspecified devices will be treated same as before.
>Note: Specifying unit to share IO port is PCI slot (device), 
>on the other
>      hand the unit of guestdev is function.
>      If you specify a function to guestdev with "+iomul", all the
>      functions of the given slot will share IO port even if 
>you specify
>      some of functions.
>
>
>This patch series addresses the issue by multiplexing IO space access.
>The patches are composed of
>       Linux part: backport: preliminary patch
>       Linux part: IO space ressignment code and multiplexing driver
>       Linux part: guestdev kernel parameter support.
>       Linux part: add kernel command line to reserve io/memory space
>       xen part:   udev script for the driver
>       ioemu part: make use of the PCIe io space multiplexing driver
>
>
>Details:
>PCI expansion ROM BIOS often uses IO port access to boot from 
>its device
>and Linux as dom0 exclusively assigns IO space to downstream 
>PCI bridges
>and the assignment unit of PCI bridge IO space is 4K. So the only up to
>16 PCIe device can be accessed via IO space within 64K IO ports.
>So on virtualized environment, it means only up to 16 guest domains
>can boot from such pass-through devices.
>
>The solution is to assign the same IO port region to pci devices
>under same PCIe switch and disable IO bit in command register.
>When accessing to one of IO port shared devices, the IO bit
>of the device is enabled, and then issues IOIO.
>
>
>Limitation:
>- PCI devices or root complex integrated endpoints aren't supported.
>- IO port of IO shared devices can't be accessed from dom0 Linux device
>  driver.
>  But those wouldn't be big issues because PCIe specification 
>discourages
>  the use of IO space and recommends that IO space should be used only
>  for bootable device with ROM code. OS device driver should 
>work without
>  IO space access.
>
>Test:
>I don't have a machine with complicated PCIe topology nor many 
>PCIe cards.
>- PCI hotplug was tested with Linux fakephp.
>- Only pci device (not bridge) hot plug/remove was tested.
>- I have tested with only single multifunction PCIe
>
>Changes from take2:
>- rebased
>- add kernel command line to reserve memory/io space for 
>unused pci slot
>
>change take 2:
>- support PCI hotplug
>- guestdev kernel paremeter.
>
>thanks 
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@xxxxxxxxxxxxxxxxxxx
>http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel