Re: [Xen-devel] VM_BUG_ON_PAGE(!PageOffline(page), page) in alloc_xenballooned_pages

[David Hildenbrand <david@xxxxxxxxxx>]

On 26.09.19 23:34, Marek Marczykowski-Górecki wrote:
> Hi,
> 
> I've hit VM_BUG_ON_PAGE(!PageOffline(page), page) in
> alloc_xenballooned_pages, when trying to use gnttab from userspace
> application. It happens on Xen PV, but not on Xen PVH or HVM with the
> same kernel. This happens at least with 5.1.6, but also 5.2.15
> (as seen below). Based on this, it looks related to 0266def91377
> (xen/balloon: Fix mapping PG_offline pages to user space) and probably
> 77c4adf6a6df (xen/balloon: mark inflated pages PG_offline).
> 
> Any idea? Below is full message.

Now that's weird. Weird because half a year passed since 
0266def91377 (xen/balloon: Fix mapping PG_offline pages to user space).

> 
> 
> page:ffffea0003e7ffc0 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0
> flags: 0xffffe00001000(reserved)

So we have a PageReserved page that is not PageOffline. I assume this
happens when we do a __ClearPageOffline() in alloc_xenballooned_pages().

That means, that we get such a page via balloon_retrieve(true). Which
means that we have such a page sitting in the ballooned_pages list, which
is weird.

Pages enter ballooned_pages via __balloon_append() only.

1. Via xen_online_page(). We have a __SetPageOffline() right in front
   of it.
2. Via balloon_add_region(). I don't see a __SetPageOffline().
3. Via decrease_reservation(). We seem to do a __SetPageOffline() on all
   pages in the previous loop.
4. Via free_xenballooned_pages(). We have a __SetPageOffline() right
   in front of it.


So this smells like #2 (and matches your PV only observation). Also,
it makes sense that the page is PageReserved that way.


Wonder if it is as easy as:

From 0955beef5aa11da4a8398472ce3106a92599cbe6 Mon Sep 17 00:00:00 2001
From: David Hildenbrand <david@xxxxxxxxxx>
Date: Fri, 27 Sep 2019 09:39:31 +0200
Subject: [PATCH v1] xen/balloon: Set pages PageOffline() in
 balloon_add_region()
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

We are missing a __SetPageOffline(), which is why we can get
!PageOffline() pages onto the balloon list, where
alloc_xenballooned_pages() will complain:

page:ffffea0003e7ffc0 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0
flags: 0xffffe00001000(reserved)
raw: 000ffffe00001000 dead000000000100 dead000000000200 0000000000000000
raw: 0000000000000000 0000000000000000 00000001ffffffff 0000000000000000
page dumped because: VM_BUG_ON_PAGE(!PageOffline(page))
------------[ cut here ]------------
kernel BUG at include/linux/page-flags.h:744!
invalid opcode: 0000 [#1] SMP NOPTI

Reported-by: Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>
Fixes: 77c4adf6a6df ("xen/balloon: mark inflated pages PG_offline")
Cc: stable@xxxxxxxxxxxxxxx # v5.1+
Signed-off-by: David Hildenbrand <david@xxxxxxxxxx>
---
 drivers/xen/balloon.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c
index 05b1f7e948ef..d31149068448 100644
--- a/drivers/xen/balloon.c
+++ b/drivers/xen/balloon.c
@@ -687,6 +687,7 @@ static void __init balloon_add_region(unsigned long 
start_pfn,
                /* totalram_pages and totalhigh_pages do not
                   include the boot-time balloon extension, so
                   don't subtract from it. */
+               __SetPageOffline(page)
                __balloon_append(page);
        }
 
-- 
2.21.0


-- 

Thanks,

David / dhildenb

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel