This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


Re: [Xen-users] best practices in using shared storage for XEN VirtualMa

To: Jeff Sturm <jeff.sturm@xxxxxxxxxx>
Subject: Re: [Xen-users] best practices in using shared storage for XEN VirtualMachines and auto-failover?
From: Rudi Ahlers <Rudi@xxxxxxxxxxx>
Date: Fri, 15 Oct 2010 11:11:25 +0200
Cc: xen-users <xen-users@xxxxxxxxxxxxxxxxxxx>
Delivery-date: Fri, 15 Oct 2010 02:13:12 -0700
Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=softdux.com; h=MIME-Version:Reply-To:In-Reply-To:References:From:Date:Message-ID:Subject:To:Cc:Content-Type:Content-Transfer-Encoding:X-Assp-Whitelisted:X-Assp-Envelope-From:X-Assp-Intended-For:X-Assp-ID:X-Assp-Version:X-Source:X-Source-Args:X-Source-Dir; b=QaJ9KsSlFIpPpstSPFhDTEv1+kFTlaCLnGvlIX5oZMRc58YXUfoz01/+/YL/YXOK894INEK8Wa2naSWqlXFyayVf1AVR5Zul+8Nz/KK4FZURQNN5zQPwP0MW/WlmbtHb;
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <64D0546C5EBBD147B75DE133D798665F06A12480@xxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <AANLkTinDO4=ruP+bPcsdc6_hhJpmDZSBs_Es8hTqi81e@xxxxxxxxxxxxxx> <64D0546C5EBBD147B75DE133D798665F06A12480@xxxxxxxxxxxxxxxxx>
Reply-to: rudi@xxxxxxxxxxx
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
On Thu, Oct 14, 2010 at 3:42 PM, Jeff Sturm <jeff.sturm@xxxxxxxxxx> wrote:
>> -----Original Message-----
>> From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-users-
>> bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Rudi Ahlers
>> Sent: Thursday, October 14, 2010 7:25 AM
>> To: xen-users
>> Subject: [Xen-users] best practices in using shared storage for XEN
> VirtualMachines
>> and auto-failover?
>> Hi all,
>> Can anyone pleas tell me what would be best practice to use shared
> storage with
>> virtual machines, especially when it involved high availability /
> automated failover
>> between 2 XEN servers?
> With 2 servers, I hear good things about DRBD, if you don't want to go
> the SAN route.  If you have a SAN make sure it is sufficiently
> redundant--i.e. two (or more) power supplies, redundant Ethernet, spare
> controllers, etc.  And of course RAID 10 or similar RAID level to guard
> against single-drive failure.

I am planning on setting up a SAN with a few Gluster / CLVM servers -
just need to decide which one first, but I'm going to attemp high
availability + load balancing + ease-of-upgrade-with-no-downtime. Each
server will run RAID10 (maybe RAID6?)

> Pay close attention to power and networking.  With 4 NICs available per
> host, I'd go for a bonded pair for general network traffic, and a
> multipath pair for I/O.  Use at least two switches.  If you get it right
> you should be able to lose one switch or one power circuit and maintain
> connectivity to your critical hosts.

So would you bond eth0 & eth1, and then eth2 & eth3 together? But then
connect the bonded eth0+1 one one switch, and eth2+3 on another switch
for failover? Or would you have eth0 & eth2 on one switch, and eth1 &
eth3 on the other? Is this actually possible? I presume the 2 switches
should also be connected together (preferably via fiber?) and then
setup Spanning Tree? Or should I seperate the 2 networks,and connect
them indivually to the internet?

> In my experience with high availability, the #1 mistake I see is
> overthinking the esoteric failure modes and missing the simple stuff.
> The #2 mistake is inadequate monitoring to detect single device
> failures.  I've seen a lot of mistakes that are simple to correct:
> - Plugging a bonded Ethernet pair into the same switch.
> - Connecting dual power supplies to the same PDU.
> - Oversubscribing a power circuit.  When a power supply fails, power
> draw on the remaining supply will increase--make sure this increase
> doesn't overload and trip a breaker.
> - Ignoring a drive failure until the 2nd drive fails.
> You can use any of a variety of clustering tools, like heartbeat, to
> automate the domU failover.  Make sure you can't get into split-brain
> mode, where a domU can start on two nodes at once--that would quickly
> corrupt a shared filesystem.  With any shared storage configuration,
> node fencing is generally an essential requirement.
>> What is the best way to connect a NAS / SAN to these 2 servers for
> this kind of setup
>> to work flawlessly? The NAS can export iSCSI, NFS, SMB, etc. I'm sure
> I could even
>> use ATAOE if needed
> For my money I'd go with iSCSI (or AoE), partition my block storage and
> export whole block devices as disk images for the domU guests.  If your
> SAN can't easily partition your storage, consider a clustered logical
> volume manager like CLVM on RHCS.
> -Jeff

I am considering CLVM, or Gluster - just need to play with them and
decide which one I prefer :)

> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users

Kind Regards
Rudi Ahlers

Website: http://www.SoftDux.com
Technical Blog: http://Blog.SoftDux.com
Office: 087 805 9573
Cell: 082 554 7532

Xen-users mailing list