[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Poor HVM performance with 8 vcpus

  • To: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
  • From: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
  • Date: Wed, 14 Oct 2009 10:16:25 +0200
  • Cc: Gianluca Guida <gianluca.guida@xxxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Wed, 14 Oct 2009 01:17:00 -0700
  • Domainkey-signature: s=s1536a; d=ts.fujitsu.com; c=nofws; q=dns; h=X-SBRSScore:X-IronPort-AV:Received:X-IronPort-AV: Received:Received:Message-ID:Date:From:Organization: User-Agent:MIME-Version:To:CC:Subject:References: In-Reply-To:X-Enigmail-Version:Content-Type: Content-Transfer-Encoding; b=RNz0uPDN5DcKK3Z8ICOpFCVYVj4DV6dtd4jvXMt/Q4Q5H9TG2fkeD0iH LiZNhCG+8jW5szl/TnDmZmXYb8BoJBKKV0g2E2n+FoWTYnv6qYUHC4v2F E4rc3E2PBJZxcnZJLJZdthWo2S/qfLSrtOZ8d83bvVQFVHOEVXXaIZHVx 9Fsk+CKMhfwyvDnQZG3pCHthsfYV6+yyudMfa+Zu+nckLsC2JfGJm/Bc0 y12cogOU8+t9EzPBGM/kR38NnZbaw;
  • List-id: Xen developer discussion <xen-devel.lists.xensource.com>


as the performance of BS2000 seems to be hit by OOS optimization, I'm
thinking of making a patch to disable this feature by a domain parameter.

Is there a way to do this without having to change all places where the
#if statements are placed?
I think there should be some central routines where adding an "if" could
be enough (setting oos_active to 0 seems not to be enough, I fear).

Do you have any hint?


Juergen Gross wrote:
> Hi,
> Gianluca Guida wrote:
>> Hi,
>> On Wed, Oct 7, 2009 at 8:55 AM, Juergen Gross
>> <juergen.gross@xxxxxxxxxxxxxx> wrote:
>>> we've got massive performance problems running a 8 vcpu HVM-guest (BS2000)
>>> under XEN (xen 3.3.1).
>>> With a specific benchmark producing a rather high load on memory management
>>> operations (lots of process creation/deletion and memory allocation) the 8
>>> vcpu performance was worse than the 4 vcpu performance. On other platforms
>>> (/390, MIPS, SPARC) this benchmark scaled rather well with the number of 
>>> cpus.
>>> The result of the usage of the software performance counters of XEN seemed
>>> to point to the shadow lock being the reason. I modified the Hypervisor to
>>> gather some lock statistics (patch will be sent soon) and found that the
>>> shadow lock is really the bottleneck. On average 4 vcpus are waiting to get
>>> the lock!
>>> Is this a known issue?
>> Acutally, I think so. The OOS optimization is widely known not to be
>> too scalable at 8vcpus in the current state, since its weak point is
>> the CR3 switching time increasing linearly with the number of cpus. If
>> you have lot of processes switches together with lot of PTE writings
>> (as it seems to be the case for your benchmark) then that's probably
>> the cause.
>> Could you try disabling the OOS optimization from the
> Great!
> First performance data looks okay!
> We will have to run different benchmarks in different configurations, but I
> think you gave an excellent hint. :-)

Juergen Gross                 Principal Developer Operating Systems
TSP ES&S SWE OS6                       Telephone: +49 (0) 89 636 47950
Fujitsu Technolgy Solutions               e-mail: juergen.gross@xxxxxxxxxxxxxx
Otto-Hahn-Ring 6                        Internet: ts.fujitsu.com
D-81739 Muenchen                 Company details: ts.fujitsu.com/imprint.html

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.