|
xen-devel
Re: [Xen-devel] [PATCH] balloon: selfballooning and post memory info via
hey,
thanks for the tip, I've already memory hotplug activated. Now it works
fine with 7 domains, but no one uses more than 256MB... I'd like to test
the ballooning with more than 2GB memory but at the moment I'ven't a
live machine which needs so much memory...
but with maxmen and hotplug, this defines the maximum or?
greetings
Torben Viets
Dan Magenheimer wrote:
memory = 256
maxmem = 8192
By the way, I'm not sure if you knew this, but the above two
lines don't work as you might want. The maxmem is ignored.
The domain is launched (in this example) with 256MB of
memory and (at least without hot-plug memory support in the
guest) memory can only be decreased from there, not increased.
So to run a guest which adjusts between 256MB and 8192MB
of memory, you must launch it with 8192MB and balloon it
down to 256MB. If Xen does not have 8192MB free at
launch, launching the domain will fail.
Dan
-----Original Message-----
From: Torben Viets [mailto:viets@xxxxxxx]
Sent: Friday, May 16, 2008 10:51 AM
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Cc: dan.magenheimer@xxxxxxxxxx
Subject: Re: [Xen-devel] [PATCH] balloon: selfballooning and
post memory
info via xenbus,
Dan Magenheimer wrote:
(Your reply came to me but not to the list... not sure why.
So I've attached your full reply below.)
thanks, hope this time it works....
ah ok, that is my failure, I need a bigger swapdisk ;)
Yes, definitely. If you are creating the swapdisk on an ext3
filesystem, you might try using sparse files. They won't
take up much disk space unless/until they get swapped-to.
There might be some performance ramifications though.
(My testing has been with swap disk as a logical volume
so I can't try sparse.)
Ok, our plan is to have a high availbilty xen farm. Now we're
beginning
with 2 Suns X2200, each has 16GB RAM. The idea, why we like to use
selfballooning, because of peak traffic on a server,
normal a server
needs about 256MB, but when it needs more, it shouldn't be a
problem to
give it 4GB. The idea is not to overbook the memory, but have the
ability to get rid of memory failures because of peaks.
Exactly what it is intended for!
I'd be interested in how it works for guests with memory=4096
and higher. All of my testing so far has been on a machine with
only 2GB of physical memory so I can test lots of guests but
no large guests.
I'll test it on monday, now I'm going into my weekend ;) but I think,
that I wasn't able to get more than 2GB RAM allocated, but I will test
it on monday again.
PS: In my first mail I've attached my whole signatur, I remove it
because I get enough spam ;)
Thanks
Torben Viets
Thanks,
Dan
-----Original Message-----
From: viets@xxxxxxx [mailto:viets@xxxxxxx]
Sent: Friday, May 16, 2008 9:49 AM
To: dan.magenheimer@xxxxxxxxxx;
xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] [PATCH] balloon: selfballooning and
post memory
info via xenbus,
Dan Magenheimer wrote:
thanks for the patch, I was waiting for this feature.
Thanks very much for the testing and feedback! Could you
comment on what you plan to use it for? (Keir hasn't accepted
it yet, so I am looking for user support ;-)
Ok, our plan is to have a high availbilty xen farm. Now we're
beginning
with 2 Suns X2200, each has 16GB RAM. The idea, why we like to use
selfballooning, because of peak traffic on a server,
normal a server
needs about 256MB, but when it needs more, it shouldn't be a
problem to
give it 4GB. The idea is not to overbook the memory, but have the
ability to get rid of memory failures because of peaks.
First question: Do you have a swap (virtual) disk configured and,
if so, how big is it? (Use "swapon -s" and the size shows in KB.)
Selfballooning shouldn't be run in a domain with no swap disk.
Also, how big is your "memory=" in your vm.cfg file?
#kernel = "/boot/xen-3.2.0/vmlinuz-2.6.18.8-xenU"
#kernel = "/boot/vmlinuz-2.6.18.8-xenU"
kernel = "/boot/vmlinuz-selfballooning"
memory = 256
maxmem = 8192
vcpu = 4
name = "test.work.de"
vif = [ 'bridge=xenvlan323' ]
disk = [ 'phy:/dev/sda,hda,w', 'file:/var/swap.img,hdb,w' ]
root = "/dev/hda ro"
extra = 'xencons=tty'
swap_size = 256M
I'm not able to reproduce your dd failure at all, even with
bs=2047M (dd doesn't permit larger values for bs).
Your program (I called it "mallocmem") does eventually fail for
me but not until i==88. However, I have a 2GB swap disk
configured.
ah ok, that is my failure, I need a bigger swapdisk ;)
I think both tests are really measuring the total virtual memory
space configured, e.g. the sum of physical memory (minus kernel
overhead) and configured swap space. I think you will find that
both will fail similarly with ballooning off and even on
a physical
system, just at different points in virtual memory usage.
Indeed, by adding additional output to mallocmem, I can see that
it fails exactly when it attempts to malloc memory larger than
the CommitLimit value in /proc/meminfo. I expect the same is
true for the dd test.
Note that CommitLimit DOES go down when memory is ballooned-out
from a guest. So your test does point out to me that I should
include a warning in the documentation not only that a swap disk
should be configured, but also that the swap disk should be
configured larger for a guest if selfballooning will be turned on.
Thanks,
Dan
-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]On Behalf Of
viets@xxxxxxx
Sent: Friday, May 16, 2008 3:36 AM
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] [PATCH] balloon: selfballooning and
post memory
info via xenbus,
Hello,
thanks for the patch, I was waiting for this feature.
I've tried this patch and I've seen that if I malloc a great
size of memory in time, this fails, but if I malloc a small
size first and then resize it slowly, it works.
this highly suffisticated (:p) program I use to test the
ballooning:
#include <stdio.h>
#include <unistd.h>
int main () {
void *v;
int i;
for(i=40; i < 50; ++i) {
v = malloc((i*32*1024*1024));
printf("%i\n", i);
if ( v != NULL) {
system("cat /proc/xen/balloon");
sleep(1);
free(v);
}
}
}
same effect I've got if I change the blocksize in a dd:
works: dd if=zero of=/test.img count=1 bs=32M
doesn't work: dd if=zero of=/test.img count=1 bs=256M
Don't know whether this is the right test for this...
greetings
Torben Viets
Dan Magenheimer wrote
OK, here's the promised patch. The overall objective of the
patch is to enable limited memory load-balancing capabilities
as a step toward allowing limited memory overcommit. With
this and some other minor hackery, I was able to run as
many as 15 lightly loaded 512MB domains on a 2GB system
(yes, veerrrryyy slooowwwlly).
Review/comments appreciated.
With this patch, balloon.c communicates (limited) useful
memory usage information via xenbus. It also implements
"selfballooning" which applies the memory information
locally to immediately adjust the balloon, giving up memory
when it is not needed and asking for it back when it is needed,
implementing a first-come-first-served system-wide ballooning
"policy". When a domain asks for memory but none is available,
it must use its own configured swap disk, resulting in
(potentially severe) performance degradation. Naturally,
it is not recommended to turn on selfballooning in a domain
that has no swap disk or if performance is more important
than increasing the number of VMs runnable on a
physical machine.
A key assumption is that the Linux variable vm_committed_space
is a reasonable first approximation of memory needed by
a domain.
This approximation will probably improve over time, but is
a good start for now. The variable is bound on the lower end
by the recently submitted minimum_target() algorithm patch;
thus O-O-M conditions should not occur.
The code is a bit complicated in a couple of places because of
race conditions involving xenstored startup relative to
turning on selfballooning locally. Because the key variable
(vm_committed_space) is not exported by Linux, I implemented
a horrible hack which still allows the code to work in a
module, however I fully expect that this part of the patch
will not be accepted (which will limit the functionality to
pvm domains only... probably OK for now).
Existing balloon functionality which is unchanged:
- Set target for VM from domain0
- Set target inside VM by writing to /proc/xen/balloon
Existing balloon info on xenbus which is unchanged:
- /local/domain/X/memory/target
To turn on selfballooning:
- Inside a VM: "echo 1 > /proc/xen/balloon"
- From domain0: "xenstore-write
/local/domain/X/memory/selfballoon 1"
To turn off selfballooning:
- Inside a VM: "echo 0 > /proc/xen/balloon"
- From domain0: "xenstore-write
/local/domain/X/memory/selfballoon 0"
New balloon info now on xenbus:
- /local/domain/X/memory/selfballoon [0 or 1]
- /local/domain/X/memory/actual [kB] *
- /local/domain/X/memory/minimum [kB] *
- /local/domain/X/memory/selftarget [kB] * (only valid if
selfballoon==1)
* writeable only by balloon driver in X when either
selfballooning is first enabled, or target is changed
by domain0
Thanks,
Dan
===================================
Thanks... for the memory
I really could use more / My throughput's on the floor
The balloon is flat / My swap disk's fat / I've OOM's in store
Overcommitted so much
(with apologies to the late great Bob Hope)
--
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel
|
|
|
Home