[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] questions of vm save/restore on arm64





On 6/6/2016 7:58 PM, Stefano Stabellini wrote:
On Fri, 3 Jun 2016, Chenxiao Zhao wrote:
On 6/3/2016 4:02 AM, Julien Grall wrote:
Hello,

First thing, the time in the mail headers seems to be wrong. Maybe
because of a wrong timezone?

I got: 04/06/16 02:32 however we are still the 3rd in my timezone.

On 04/06/16 02:32, Chenxiao Zhao wrote:


On 6/3/2016 3:16 AM, Julien Grall wrote:
Hello,

On 03/06/16 18:05, Chenxiao Zhao wrote:
I finally found out that the problem is that the toolstack did not get
corret p2m_size while sending all pages on save(always be zero).
After I
fixed that, the guest could be restored but guest kernel caught
handle_mm_fault().

where do you think I'm going to investigate, guest kernel hibernation
restore or xen?

The hibernation support for ARM64 has only been merged recently in the
kernel. Which kernel are you using?

Hi Julien,

I'm using a linaro ported Linux kernel 4.1 for hikey from this link.

https://github.com/96boards/linux/tree/android-hikey-linaro-4.1

I also applied following patches to make the kernel support hibernation.

This looks the wrong way to do it as this series may requires some
patches which have been upstreamed before hand.

Linux upstream seems support to the hikey board [1]. Any reason to not
using it?

I tried a newer version of kernel 4.4, but got no luck to start dom0 with xen.
so I decide to stay in 4.1 for now.


[1] http://www.spinics.net/lists/arm-kernel/msg477769.html
[2] http://lists.xen.org/archives/html/xen-devel/2015-12/msg01068.html


Also, what are the modifications you have made to support Xen
suspend/resume for ARM64?

I believe I have posted my modifications on xen in the first mail of
this thread.

I mean in Linux. The patch from Ian Campbell does not have any kind of
support for ARM64.

For instance arch/arm/xen/suspend.c needs to be built for ARM64. So I am
wondering if your kernel has support of hibernation...

Oh, yes, I most forgot I added this file in arch/arm64/xen/Makefile to let it
build for arm64.


 From my understanding, a kernel hibernation will cause kernel to save
memories to disk(swap partition). But on guest save progress, the
hibernation for domU does not make the guest save memories to disk. it's
more like suspend all processes in guest, and memors actually depends on
xen toolstack to save the pages to file. Am I correct?

You are using an older tree with a patch series based on a newer tree.

So I would recommend you to move to a newer tree. If it is not possible,
please test that hibernation works on baremetal.

I think the suspend/resume in guest is working, cause I can use pause/unpause
command in toolstack to suspend/resume guest without problem. I can also see
the suspend/resume kernel messages from guest's console. The only problem is
it's can not resume from restore.

But can you still connect to the guest after resume, maybe over the network?
If you cannot, then something is likely wrong.

Hi Stefano,

I can connect to the guest after resume from xen console. It responds by 'return' key, but I can not run any other commands, e.g. ls or ps. I think the guest is not 'fully' restored.



One thing that confused me is that the kernel's hibernation means the guest
kernel will save the memory state to disk and power off VM at last. The guest
will also take care of the memory restore itself. But I do not see the
save/restore on xen works that way. So my question is why it requires
hibernation (aka. suspend to disk) instead of the real suspend (aka. suspend
to RAM and standby)?

Xen suspend/resume has nothing to do with guest suspend to RAM or guest
hibernation.

Xen suspend/resume is a way for the hypervisor to save to file the
entire state of the VM, including RAM and the state of any devices.
Guest suspend to RAM and guest hibernation are two guest driven
technologies to save the state of the operating system to RAM or to
disk. The only link between Xen suspend and guest suspend is that when
Xen issues a domain suspend, it notifies the guest of it so that it can
ease the process.  The code in Linux to support Xen suspend/resume is:

drivers/xen/manage.c:do_suspend

and makes use of some of the Linux internal hooks provided for
hibernations (see CONFIG_HIBERNATE_CALLBACKS). But that's just for
better integration with the rest of Linux: hibernation is NOT what is
happening.

I hope that this clarifies things a bit, I realize that it is confusing.


Thanks for your explanation, It clear enough and just as my understanding from the code. I think the problem might caused by incompatible of arm p2m and xen save/restore mechanism. I'll try a core-dump and compare the memory after save and restore. I suppose this two dumps should be identical but there already pages are different. I'll let you know if I got some progress.

Regards.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.