WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

[Xen-devel] [PATCH] tmem: reclaim minimal memory proactively

To: "Xen-Devel (E-mail)" <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: [Xen-devel] [PATCH] tmem: reclaim minimal memory proactively
From: Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>
Date: Tue, 8 Dec 2009 11:16:38 -0800 (PST)
Delivery-date: Tue, 08 Dec 2009 11:19:03 -0800
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
When a single domain is using most/all of tmem memory
for ephemeral pages belonging to the same object, e.g.
when copying a single huge file larger than ephemeral
memory, long lists are traversed looking for a page to
evict that doesn't belong to this object (as pages in
the object for which a page is currently being inserted
are locked and cannot be evicted).  This is essentially
a livelock.

Avoid this by proactively ensuring there is a margin
of available memory (1MB) before locks are taken on
the object.

Signed-off-by: Dan Magenheimer <dan.magenheimer@xxxxxxxxxx>

diff -r 7f611de6b93c xen/common/tmem.c
--- a/xen/common/tmem.c Tue Dec 08 14:14:27 2009 +0000
+++ b/xen/common/tmem.c Tue Dec 08 12:09:05 2009 -0700
@@ -1091,6 +1091,24 @@ static unsigned long tmem_relinquish_npa
     if ( avail_pages )
         tmh_release_avail_pages_to_host();
     return avail_pages;
+}
+
+/* Under certain conditions (e.g. if each client is putting pages for exactly
+ * one object), once locks are held, freeing up memory may
+ * result in livelocks and very long "put" times, so we try to ensure there
+ * is a minimum amount of memory (1MB) available BEFORE any data structure
+ * locks are held */
+static inline void tmem_ensure_avail_pages(void)
+{
+    int failed_evict = 10;
+
+    while ( !tmh_free_mb() )
+    {
+        if ( tmem_evict() )
+            continue;
+        else if ( failed_evict-- <= 0 )
+            break;
+    }
 }
 
 /************ TMEM CORE OPERATIONS ************************************/
@@ -2315,10 +2333,12 @@ EXPORT long do_tmem_op(tmem_cli_op_t uop
                               op.u.new.uuid[0], op.u.new.uuid[1]);
         break;
     case TMEM_NEW_PAGE:
+        tmem_ensure_avail_pages();
         rc = do_tmem_put(pool, op.u.gen.object,
                          op.u.gen.index, op.u.gen.cmfn, 0, 0, 0, NULL);
         break;
     case TMEM_PUT_PAGE:
+        tmem_ensure_avail_pages();
         rc = do_tmem_put(pool, op.u.gen.object,
                     op.u.gen.index, op.u.gen.cmfn, 0, 0, PAGE_SIZE, NULL);
         if (rc == 1) succ_put = 1;
diff -r 7f611de6b93c xen/include/xen/tmem_xen.h
--- a/xen/include/xen/tmem_xen.h        Tue Dec 08 14:14:27 2009 +0000
+++ b/xen/include/xen/tmem_xen.h        Tue Dec 08 12:09:05 2009 -0700
@@ -252,6 +252,11 @@ static inline unsigned long tmh_freeable
             (20 - PAGE_SHIFT);
 }
 
+static inline unsigned long tmh_free_mb(void)
+{
+    return (tmh_avail_pages() + total_free_pages()) >> (20 - PAGE_SHIFT);
+}
+
 /*
  * Memory allocation for "infrastructure" data
  */

Attachment: tmem-livelock.patch
Description: Binary data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-devel] [PATCH] tmem: reclaim minimal memory proactively, Dan Magenheimer <=