[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Shadow Page Tables in Xen


  • To: priya sehgal <priyagps@xxxxxxxxxxx>
  • From: Gianluca Guida <glguida@xxxxxxxxx>
  • Date: Wed, 22 Apr 2009 15:18:39 +0200
  • Cc: Tim Deegan <Tim.Deegan@xxxxxxxxxxxxx>, "xen-devel@xxxxxxxxxxxxxxxxxxx" <xen-devel@xxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Wed, 22 Apr 2009 06:19:12 -0700
  • Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=Wfq0DbG9ltl2P7b62GyAB+4l2I9A7kSq5z41oxQQ0NJjbhE+PI1OLVR91XTlyXv8EU z4jo8LY33PAneDH3bHB6Ad3ucyeRWnmKQi46Ao9Z7RDvtV6J8RU0/03LsFBRF58J1RSQ aQe1GI/34TOmqZqVAq3ZGHS5PG1FzU752O2h0=
  • List-id: Xen developer discussion <xen-devel.lists.xensource.com>


On Apr 22, 2009, at 3:02 PM, priya sehgal wrote:


Hi Gianluca,

There are various problems I can see with this approach:


- As Tim suggested, this will make the bandwidth required
to do live
migration much bigger (you're talking about increasing the
granularity
of memory to be sent from 1 to 1000 pages). So you should
take into
account that yes, making bigger logdirty chunks will
decrease the
pagefaults, but will increase the required network
bandwidth, which is
a very important parameter for live migration.

I think there is one thing missing in what I explained.
If a page in a group is written for the first time, all its neighbors
are marked as RW. But, their dirty bit might be off. Now, we maintain
a dirty group bitmap, which just stores the page groups that are dirty.
During the end of an epoch, when we are about to ship the pages
to the destination machine, we first check the groups which are dirty. Scan through *each* page's PTE corresponding to that group to determine
if it is DIRTY. Only if the page is DIRTY is it sent over the network.
So, we do not send unwanted pages , only the ones marked DIRTY in the
RW-enabled groups.

Ok that seems, but please note that at the moment shadows always set the DIRTY bit (unless we're tracking a VRAM address), so you might have to set the dirty bit clean when propagating the shadow, which is a bit too intrusive perhaps. But yes, that would work.

Also, you mentioned that the main performance bottleneck is due to blowing up of the shadow page tables. Why is this required? Can't the page table entries that were migrated in the last epoch be cleaned up only and reset. Could you please elaborate on this ?

Yes. We could do that, as Ian suggested. There was no effort in making live migration's shadow code fast, so nobody ever did it. Might be good to do it, if you're interested in this case, and will be much cleaner than your suggested effort, and perhaps way more interesting.

This comment, found in shadow_clean_dirty_bitmap might explain you a lot:

    /* Need to revoke write access to the domain's pages again.
     * In future, we'll have a less heavy-handed approach to this,
     * but for now, we just unshadow everything except Xen. */
    shadow_blow_tables(d);



Thanks,
Gianluca

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.