Re: [Xen-devel] Poor performance of xend on servers with more than 10000 FC device paths

[Masaki Kanno <kanno.masaki@xxxxxxxxxxxxxx>]

Hi Lutz,

Well done!!

Best regards,
 Kan

Fri, 23 Jul 2010 08:31:14 +0200, "Dube, Lutz" wrote:

>On server startup xend start or a later xend restart needs approx. 30 min to
>start/restart. Without attached FC devices xend start/restart needs only some
>seconds.
>
>server type: Fujitsu Primergy RX600-S5
>
>The time gets lost in xen/xend/XendNode.py line 329 while calling
>vscsi_util.get_all_scsi_device().
>
>329         for pscsi_record in vscsi_util.get_all_scsi_devices():
>330             scsi_id = pscsi_record['scsi_id']
>331             if scsi_id:
>332                 saved_HBA_uuid = None
>
>I think, in most cases we don't need all the PSCSI devices registered in
>xend, but only a few of it.
>So a good solution for this perforamce issue is to scan only the SCSI device
>paths we need, controlled by a new option in xend-config.sxp.
>
>I have made a patch to allow specification of scsi devices we need in xend
>in the config file xend-config.sxp.
>The new options pscsi-device-mask expects a list of device ids oder partial
>device ids like the option of lsscsi, e.g.
>(pscsi-device-mask ('<partial-dev-id1' 'partial-dev-id2' ...))
>
>Without this option set in xend-config.sxp or if lsscsi is not support, all
>device paths are process like today.
>
>Signed-off-by: Lutz Dube Lutz.Dube@xxxxxxxxxxxxxx
>
>diff -r fac9d58d3c8a -r 02d55e7d0e94 tools/examples/xend-config.sxp
>--- a/tools/examples/xend-config.sxp  Wed Jul 21 13:15:33 2010 +0200
>+++ b/tools/examples/xend-config.sxp              Fri Jul 23 08:23:19 2010 
>+0200
>@@ -294,3 +294,11 @@
> # we have to realize this may incur security issue and we can't make sure 
>the
> # device assignment could really work properly even after we do this.
> #(pci-passthrough-strict-check yes)
>+
>+# If we have a very big scsi device configuration, start of xend is slow,
>+# because xend scans all the device paths to build its internal PSCSI device
>+# list.  If we need only a few devices for assigning to a guest, we can 
>reduce
>+# the scan to this device. Set list list of device paths in same syntax 
>like in
>+# command lsscsi, e.g. ('16:0:0:0' '15:0')
>+# (pscsi-device-mask ('*'))
>+
>diff -r fac9d58d3c8a -r 02d55e7d0e94 tools/python/xen/util/vscsi_util.py
>--- a/tools/python/xen/util/vscsi_util.py            Wed Jul 21 13:15:33 
>2010 +0200
>+++ b/tools/python/xen/util/vscsi_util.py         Fri Jul 23 08:23:19 2010 
>+0200
>@@ -148,11 +148,12 @@ def _vscsi_get_scsidevices_by_sysfs():
>     return devices
>
>
>-def vscsi_get_scsidevices():
>+def vscsi_get_scsidevices(mask=""):
>     """ get all scsi devices information """
>
>-    devices = _vscsi_get_scsidevices_by_lsscsi("")
>-    if devices:
>+    devices = _vscsi_get_scsidevices_by_lsscsi("[%s]" % mask)
>+    if devices or (len(mask) and mask[0] != "*"):
>+        # devices found or partial device scan
>         return devices
>     return _vscsi_get_scsidevices_by_sysfs()
>
>@@ -274,9 +275,9 @@ def get_scsi_device(pHCTL):
>             return _make_scsi_record(scsi_info)
>     return None
>
>-def get_all_scsi_devices():
>+def get_all_scsi_devices(mask=""):
>     scsi_records = []
>-    for scsi_info in vscsi_get_scsidevices():
>+    for scsi_info in vscsi_get_scsidevices(mask):
>         scsi_record = _make_scsi_record(scsi_info)
>         scsi_records.append(scsi_record)
>     return scsi_records
>diff -r fac9d58d3c8a -r 02d55e7d0e94 tools/python/xen/xend/XendNode.py
>--- a/tools/python/xen/xend/XendNode.py     Wed Jul 21 13:15:33 2010 +0200
>+++ b/tools/python/xen/xend/XendNode.py Fri Jul 23 08:23:19 2010 +0200
>@@ -326,7 +326,12 @@ class XendNode:
>         pscsi_table = {}
>         pscsi_HBA_table = {}
>
>-        for pscsi_record in vscsi_util.get_all_scsi_devices():
>+        pscsi_records = []
>+        for pscsi_mask in xendoptions().get_pscsi_device_mask():
>+            pscsi_records += vscsi_util.get_all_scsi_devices(pscsi_mask)
>+        log.debug("pscsi record count: %s" % len(pscsi_records))
>+
>+        for pscsi_record in pscsi_records:
>             scsi_id = pscsi_record['scsi_id']
>             if scsi_id:
>                 saved_HBA_uuid = None
>diff -r fac9d58d3c8a -r 02d55e7d0e94 tools/python/xen/xend/XendOptions.py
>--- a/tools/python/xen/xend/XendOptions.py                Wed Jul 21 13:15:
>33 2010 +0200
>+++ b/tools/python/xen/xend/XendOptions.py            Fri Jul 23 08:23:19 
>2010 +0200
>@@ -164,6 +164,9 @@ class XendOptions:
>         """
>         print >>sys.stderr, "xend [ERROR]", fmt % args
>
>+    """Default mask for pscsi device scan."""
>+    xend_pscsi_device_mask = ['*']
>+
>
>     def configure(self):
>         self.set_config()
>@@ -429,6 +432,10 @@ class XendOptions:
>     def get_pci_dev_assign_strict_check(self):
>         return self.get_config_bool("pci-passthrough-strict-check",
>                                     self.pci_dev_assign_strict_check_defau
>lt)
>+
>+    def get_pscsi_device_mask(self):
>+        return self.get_config_value("pscsi-device-mask",
>+                                      self.xend_pscsi_device_mask)
>
> class XendOptionsFile(XendOptions):
>
>
>
>-------------------------------text/plain-------------------------------
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@xxxxxxxxxxxxxxxxxxx
>http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel