[Xen-devel] Poor performance of xend on servers with more than 10000 FC device paths

["Dube, Lutz" <lutz.dube@xxxxxxxxxxxxxx>]

On server startup xend start or a later xend restart needs approx. 30 min to

start/restart. Without attached FC devices xend start/restart needs only some

seconds.

 

server type: Fujitsu Primergy RX600-S5

 

The time gets lost in xen/xend/XendNode.py line 329 while calling

vscsi_util.get_all_scsi_device().

 

329         for pscsi_record in vscsi_util.get_all_scsi_devices():

330             scsi_id = pscsi_record['scsi_id']

331             if scsi_id:

332                 saved_HBA_uuid = None

 

I think, in most cases we don't need all the PSCSI devices registered in

xend, but only a few of it.

So a good solution for this perforamce issue is to scan only the SCSI device

paths we need, controlled by a new option in xend-config.sxp.

 

I have made a patch to allow specification of scsi devices we need in xend

in the config file xend-config.sxp.

The new options pscsi-device-mask expects a list of device ids oder partial

device ids like the option of lsscsi, e.g.

(pscsi-device-mask ('<partial-dev-id1' 'partial-dev-id2' ...))

 

Without this option set in xend-config.sxp or if lsscsi is not support, all

device paths are process like today.

 

Signed-off-by: Lutz Dube Lutz.Dube@xxxxxxxxxxxxxx

 

diff -r fac9d58d3c8a -r 02d55e7d0e94 tools/examples/xend-config.sxp

--- a/tools/examples/xend-config.sxp  Wed Jul 21 13:15:33 2010 +0200

+++ b/tools/examples/xend-config.sxp              Fri Jul 23 08:23:19 2010 +0200

@@ -294,3 +294,11 @@

 # we have to realize this may incur security issue and we can't make sure the

 # device assignment could really work properly even after we do this.

 #(pci-passthrough-strict-check yes)

+

+# If we have a very big scsi device configuration, start of xend is slow,

+# because xend scans all the device paths to build its internal PSCSI device

+# list.  If we need only a few devices for assigning to a guest, we can reduce

+# the scan to this device. Set list list of device paths in same syntax like in

+# command lsscsi, e.g. ('16:0:0:0' '15:0')

+# (pscsi-device-mask ('*'))

+

diff -r fac9d58d3c8a -r 02d55e7d0e94 tools/python/xen/util/vscsi_util.py

--- a/tools/python/xen/util/vscsi_util.py            Wed Jul 21 13:15:33 2010 +0200

+++ b/tools/python/xen/util/vscsi_util.py         Fri Jul 23 08:23:19 2010 +0200

@@ -148,11 +148,12 @@ def _vscsi_get_scsidevices_by_sysfs():

     return devices

 

 

-def vscsi_get_scsidevices():

+def vscsi_get_scsidevices(mask=""):

     """ get all scsi devices information """

 

-    devices = _vscsi_get_scsidevices_by_lsscsi("")

-    if devices:

+    devices = _vscsi_get_scsidevices_by_lsscsi("[%s]" % mask)

+    if devices or (len(mask) and mask[0] != "*"):

+        # devices found or partial device scan

         return devices

     return _vscsi_get_scsidevices_by_sysfs()

 

@@ -274,9 +275,9 @@ def get_scsi_device(pHCTL):

             return _make_scsi_record(scsi_info)

     return None

 

-def get_all_scsi_devices():

+def get_all_scsi_devices(mask=""):

     scsi_records = []

-    for scsi_info in vscsi_get_scsidevices():

+    for scsi_info in vscsi_get_scsidevices(mask):

         scsi_record = _make_scsi_record(scsi_info)

         scsi_records.append(scsi_record)

     return scsi_records

diff -r fac9d58d3c8a -r 02d55e7d0e94 tools/python/xen/xend/XendNode.py

--- a/tools/python/xen/xend/XendNode.py     Wed Jul 21 13:15:33 2010 +0200

+++ b/tools/python/xen/xend/XendNode.py Fri Jul 23 08:23:19 2010 +0200

@@ -326,7 +326,12 @@ class XendNode:

         pscsi_table = {}

         pscsi_HBA_table = {}

 

-        for pscsi_record in vscsi_util.get_all_scsi_devices():

+        pscsi_records = []

+        for pscsi_mask in xendoptions().get_pscsi_device_mask():

+            pscsi_records += vscsi_util.get_all_scsi_devices(pscsi_mask)

+        log.debug("pscsi record count: %s" % len(pscsi_records))

+

+        for pscsi_record in pscsi_records:

             scsi_id = pscsi_record['scsi_id']

             if scsi_id:

                 saved_HBA_uuid = None

diff -r fac9d58d3c8a -r 02d55e7d0e94 tools/python/xen/xend/XendOptions.py

--- a/tools/python/xen/xend/XendOptions.py                Wed Jul 21 13:15:33 2010 +0200

+++ b/tools/python/xen/xend/XendOptions.py            Fri Jul 23 08:23:19 2010 +0200

@@ -164,6 +164,9 @@ class XendOptions:

         """

         print >>sys.stderr, "xend [ERROR]", fmt % args

 

+    """Default mask for pscsi device scan."""

+    xend_pscsi_device_mask = ['*']

+

 

     def configure(self):

         self.set_config()

@@ -429,6 +432,10 @@ class XendOptions:

     def get_pci_dev_assign_strict_check(self):

         return self.get_config_bool("pci-passthrough-strict-check",

                                     self.pci_dev_assign_strict_check_default)

+

+    def get_pscsi_device_mask(self):

+        return self.get_config_value("pscsi-device-mask",

+                                      self.xend_pscsi_device_mask)

 

 class XendOptionsFile(XendOptions):

 

 

xen-devl-patch-pscsi
Description: xen-devl-patch-pscsi

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel