WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

[Xen-users] Problem with time in 64bit domUs

To: xen-users@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-users] Problem with time in 64bit domUs
From: "Paul Reiber" <reiber@xxxxxxxxx>
Date: Thu, 20 Dec 2007 17:17:55 -0800
Delivery-date: Thu, 20 Dec 2007 17:18:50 -0800
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; bh=u9p2sKm9AOsF3eKF8XDEEFmFtu1ZnZuDkAenl+XY994=; b=xCQb8BQTdgGU7TrapJ8E4tCCE8KX7s8K1fonZBxGY2ZjfppHfPccKigs3R9m/LZVF8qrjTSdI36XBfndyfGrR6f4E9CwwIfbxPZA35ibYzgHBYB3fmM3QqoFpVb3kD4dYFY8CKN/OW9oyV+MwTTui6vpXCJG2hPw6Yfb1XUhxAE=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:reply-to:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=CjSduRxAs+Vfe4bYAhsVoHpg5B5QaUAP1K1hlx/ELIuvECAOoBKoe0GT3hTOxWj53GF28iJb0XzrHCsthUJiHuwK3uLGwbXu9hZ0FiozeKF6wYLpLs1Ku7nMn1XOQMMgeIG7Qy55Jr3ZC1VTgCE0Uu8y6BRWYLqRGUtREOqpsNE=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
Reply-to: reiber@xxxxxxxxx
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
Greetings, Xen developers and power-users!

I have a client who's using Xen in their staging environment.  This client
is technically advanced, well beyond your typical sysadmin shop.  For example,
they're comfortable building Xen locally and packaging their own RPMs.

We've been noticing some issues with TIME within 64bit HVM-based domU's -
their time-of-day clock appears to get out-of-whack in short order.

For example, in our early testing, running "sleep 1; date" from the
commandline, repeatedly, exhibited the following behavior:
- the "real time" that passes each time is indeed approx. one second
- the "system clock" time that's reported can jump forward randomly
from between 1 through 6-7 seconds per execution.

Various changes (newest version of Xen, etc) have obfuscated the problem
but not eliminated it.  The above test no longer fails, but the following
test still shows there are problems.

The client used the shmux shell multiplexer to execute 'date' in parallel
across various domU's:

[root@hostname]# SHMUX_SSH_OPTS='-xa -o "StrictHostKeyChecking no"'
shmux -S all `grep -vh '#' /etc/hosts.stage-orion` -c 'date'
orion-xen01.stage2: Thu Dec 20 16:49:02 PST 2007
orion-xen03.stage2: Thu Dec 20 17:14:38 PST 2007
orion-xen02.stage2: Thu Dec 20 16:31:36 PST 2007
orion-xen08.stage2: Thu Dec 20 16:31:37 PST 2007
orion-xen05.stage2: Thu Dec 20 16:37:47 PST 2007
orion-xen04.stage2: Thu Dec 20 16:31:37 PST 2007
orion-xen06.stage2: Thu Dec 20 16:31:39 PST 2007
orion-xen10.stage2: Thu Dec 20 16:37:55 PST 2007
orion-xen07.stage2: Thu Dec 20 16:31:39 PST 2007
orion-xen09.stage2: Thu Dec 20 16:31:38 PST 2007
orion-xen12.stage2: Thu Dec 20 16:59:18 PST 2007
orion-xen11.stage2: Thu Dec 20 16:39:24 PST 2007
orion-xen14.stage2: Thu Dec 20 16:47:35 PST 2007
orion-xen13.stage2: Thu Dec 20 16:40:47 PST 2007

14 targets processed in 2 seconds.
Summary: 14 successes

You can see from the variations in timestamps above that the various domUs are
indeed quite far out of sync.  This isn't being off by a second or two; it's
quite beyond that.

While there are some work-arounds (like running ntpdate periodically,
for example)
it's disconcerting to both me and the client that this is occurring,
and we'd like
to help actually _fix_ the problem rather than just work around it.

I've learned there are three different processor schedulers within Xen.  Should
my client focus on testing whether changes at this level affect this problem?

Are there any other "obvious" things to check/test regarding this?

I believe this client could provide valuable help with problem investigation and
debugging. Please feel free to contact them with any initial questions you
might have regarding the specifics.  Brendan Wood - bwood@xxxxxxxxxxxxx - will
be happy to provide in-depth answers to any questions you might have.

Thanks so much for your time and attention to this issue!

Happy Holidays!
-Paul Reiber,
Linux specialist, Taos Mountain Consulting

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

<Prev in Thread] Current Thread [Next in Thread>
  • [Xen-users] Problem with time in 64bit domUs, Paul Reiber <=