[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Issue with PV superpage handling

  • To: Dave McCracken <dcm@xxxxxxxx>
  • From: Juergen Gross <juergen.gross@xxxxxxxxxxxxxx>
  • Date: Mon, 09 Jul 2012 08:02:24 +0200
  • Cc: George Dunlap <george.dunlap@xxxxxxxxxxxxx>, Ian Campbell <Ian.Campbell@xxxxxxxxxx>, Xen Developers <xen-devel@xxxxxxxxxxxxx>
  • Delivery-date: Mon, 09 Jul 2012 06:03:01 +0000
  • Domainkey-signature: s=s1536a; d=ts.fujitsu.com; c=nofws; q=dns; h=X-SBRSScore:X-IronPort-AV:Received:X-IronPort-AV: Received:Received:Message-ID:Date:From:Organization: User-Agent:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=YJytrZSzZLGOP9TJqpN07IAVg6zJaxoPEYX/qtb05fDVXn5/awsC6ppJ Q11iIt7QmKfBWcI+wnUd208t8w0c+PA9os5f1T52oymWn1VUjuOQY7iVf qcrolri2H73uSDo2mtjBNt17VvmBnkxnG8S2t+RMkq9fxvTnj6+skizSL kGJoOPilXMtAcfJbRQB1OGta0MLcjNrySwo4jOurcvVa1awa2WTD2Kb7B Tsu5Gwd8L7XxwNk8/PXpNWqfY10Ov;
  • List-id: Xen developer discussion <xen-devel.lists.xen.org>

Am 25.06.2012 16:38, schrieb Dave McCracken:

Awhile back I added the domain config flag "superpages" to support Linux
hugepages in PV domains.  When the flag is set, the PV domain is populated
entirely with superpages.  If not enough superpage-sized chunks can be found,
the domain creation fails.

At some time after my patch was accepted, the code I added to domain restore
was removed because I broke page allocation batching.  I put it on my TODO
list to reimplement it, then it got lost, for which I apologize.

Now I have gotten back to reimplementing PV superpage support in restore, I
find that recently other code was added to restore that, while triggered by
the superpage flag, only allocates superpages opportunistically and falls back
to small pages if it fails.  This breaks the original semantics of the flag
and could cause any OS that depends on the semantics to fail catastrophically.

I have a patch that implements the original semantics of the superpage flag
while preserving the batch allocation behavior.  I can remove the competing
code and submit mine, but I have a question.  What value is there in
implementing opportunistic allocation of superpages for a PV (or an HVM)
domain in restore?  It clearly can't be based on the superpages flag.
Opportunistic superpage allocation is already the default behavior for HVM
domain creation.  Should it also be a default on HVM restore?  What about for
PV domains?  Is there any real benefit?

There is a real benefit.

We are seeing severe performance penalties after migrating a HVM domain.
Performance is going down by 10% or more! Our OS (BS2000) is trying to use
superpages where possible. Before live migration I can see that the complete
memory for the domain is allocated in at least 2MB chunks, after the
migration not a single superpage is left.

With EPT this not only makes each TLB-miss more expensive, but there are much
more TLB-misses, as no 2MB TLB-entries are possible at all!


Juergen Gross                 Principal Developer Operating Systems
PDG ES&S SWE OS6                       Telephone: +49 (0) 89 3222 2967
Fujitsu Technology Solutions              e-mail: juergen.gross@xxxxxxxxxxxxxx
Domagkstr. 28                           Internet: ts.fujitsu.com
D-80807 Muenchen                 Company details: ts.fujitsu.com/imprint.html

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.