Re: [Xen-users] Scary!!! Lost domU!!!

[Jamon Camisso <jamonation@xxxxxxxxx>]

James Pifer wrote:
> I've been on vacation (and still am) but had to work on a couple
> problems. I have a couple Citrix servers that are domU's on a sles10sp2
> server that has local storage and connects to two ocfs2 volumes. 
Is there more than just the sles server using both volumes? If not, have 
you considered using another filesystem? Personally I've had nothing but 
trouble with ocfs2 in Debian and Centos -- clusters would just randomly 
fall apart. I've also found that unless filesystem throughput is very 
good, ocfs2 would end up loosing writes by getting ahead of itself 
somehow. All depends on the storage backend I suppose.
> I tried to restart one of the Citrix servers and it would not restart,
> giving an error that the disk was already mounted in a loopback, etc. I
> looked at mount and didn't see anything mounted and I had just shut the
> domU down. I assumed it had not shut down completely. This domU runs
> from the local disk. 
>
> So I decided to a restart of the host was in order. I downed the rest of
> the domU's, including an oracle server running off one of the ocfs2
> clusters. This servers has been being used for the last three weeks from
> this location. 
>
> After restarting dom0 I started bringing the domU's back up. All of them
> came back up fine, except for the oracle server. It gave an error that
> the disk files did not exist, and they don't, they aren't there
> anymore. 
>
> I checked and double checked history to see if any rm commands had been
> given and I didn't find any. 
Do you still have the system logs from the reboots? You might see the 
cluster falling apart there depending on how your dom0 shutdown the domUs.
> When I restarted, there was an error on one of the local file systems
> that said "JDB: barrier-based sync failed...".
>
> Luckily I have a copy of this domU from a few weeks ago BEFORE I copied
> it to the ocfs2 volume. What could explain the sudden deletion of a
> directory like this?
Per above, what kind of storage architecture are you using underneath 
your ocfs2 volumes? I recall reading a bug that described the lost 
writes I mentioned, though I can't for the life of me find it now.
> If this happened on some of the other domU's it could be ugly. 
>
> Any advice is appreciated!!!!
I've not used SLES, so maybe that's the determining factor, but have you 
tried gfs2 or lustre (http://wiki.lustre.org/index.php/Main_Page)? I 
particularly want to try the latter.
Jamon


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users