[Xen-users] domU has better I/O performance than dom0?
I've done some I/O benchmarks on an RHEL 5.1-based xen setup. The main
(dom0) server is an x86_64 host with a FC-connected IBM SAN. The guest
servers are paravirtualized.
I used bonnie++ to stress-test and to try to analyze I/O performance.
Bonnie++ was run with this command, current working directory being the
relevant part of the file system:
bonnie++ -n 4 -s 20g -x 5 -u nobody
The chunk size (20g) was much larger than the memory available for the
dom0/domU (both having ½GB RAM available).
Testing ran for several hours. No stability problems were seen.
Bonnie++ was run on the dom0 and one of the domUs, making sure that the
involved disks were otherwise idle, or very close to being idle. I.e., a
quiet (and somewhat slow) part of the SAN was used.
The storage area was made available for the dom0 as a file system on "raw
device" (no use of logical volume management on the operating system).
After testing on the dom0, the same device was unmounted, the file system
was re-created and subsequently allocated to the domU as a "phy"-device;
the phy-device was then mounted to a suitable mountpoint on the domU.
The tests were run on different times of the day. The dom0 test was
partly run during work hours where there may have been contention on the
SAN signalling infrastructure (switch/storage HBAs), although we
generally believe that we don't have a major bottleneck on the SAN
optical pathways. The domU test was run during night-time where I/O
pressure on the SAN infrastructure is probably somewhat lower (although
various backup and batch jobs make sure that the SAN is never sleeping).
The results were averaged, after filtering out result which seemed
Bonnie wasn't able to detect a difference in the file-creation
performance, so these values aren't included.
Host | Sequential Output | Sequential Input | Random seeks |
| (K/sec) | (K/sec) | (/sec) |
| Per Char | Block | Rewrite | Per Char | Block | |
dom0 | 56739 | 96529 | 46524 | 58346 | 119830 | 94 |
domU | 56186 | 112796 | 50178 | 65325 | 202569 | 148 |
domU | | | | | | |
gain% | -1 | 17 | 8 | 12 | 69 | 56 |
In other words: I've found that my domU's I/O performance generally
surpasses that of my dom0(!).
On a side note: Running mke2fs went much, much faster on the dom0 than on
the domU. So for this kind of I/O, the pattern seems to break.
Am I just being an ignorant benchmark-idiot, or could this kind of result
actually be expected and/or explained?
Is bonnie++ a bad storage benchmarking tool? - If so: What else is better?
Troels Arvin <troels@xxxxxxxx>
Xen-users mailing list