WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-users

Re: [Xen-users] FreeBSD HVM Guest boots very slow.

To: "Mark Williamson" <mark.williamson@xxxxxxxxxxxx>
Subject: Re: [Xen-users] FreeBSD HVM Guest boots very slow.
From: "John Mathews" <mathjm@xxxxxxxxx>
Date: Fri, 22 Aug 2008 20:13:00 -0700
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Delivery-date: Fri, 22 Aug 2008 20:13:40 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type:references; bh=U4q0GbYVsa0aonV3GhD4+SDKBqThdziIMh6sigzIhrQ=; b=KEdDD28kLg4xqq7Jo1XLTrCHPi6baC9KwCbT6nVnKNkfkv+SkZb6y25gQzRZjBzdsg kiFmjNiE4ZChtQW+UBt3Os5JI2vbBs6sFewwVElZMMABqCyfWHKwvQxHipSk1zkoFCUb bsdpUL/kTjr1kbQxI/cIh0sjPubdg97uIAQsI=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:references; b=hHyVNxP4qJeDzlPrKGXhq1JRbhqcZ2EiwsbwjXMVPOsbabk12pGWWc9TWC75WSoq40 RAJyV0wSiS59y15Y7fzzzd7fgonExfTKRfSzsl1ZbJ7LOPfErUNq6a1155gf1dUAi/j5 4mmYwmoW5QvOYKYIV4D4yUQxtqLYVzg6xLem4=
Envelope-to: www-data@xxxxxxxxxxxxxxxxxxx
In-reply-to: <200808212302.09932.mark.williamson@xxxxxxxxxxxx>
List-help: <mailto:xen-users-request@lists.xensource.com?subject=help>
List-id: Xen user discussion <xen-users.lists.xensource.com>
List-post: <mailto:xen-users@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-users>, <mailto:xen-users-request@lists.xensource.com?subject=unsubscribe>
References: <a31dfe7c0808202018x4f54ac89u11b5de765d8e996e@xxxxxxxxxxxxxx> <200808211944.33830.mark.williamson@xxxxxxxxxxxx> <a31dfe7c0808211246s6af9a07cs316bbe8ce0916b5d@xxxxxxxxxxxxxx> <200808212302.09932.mark.williamson@xxxxxxxxxxxx>
Sender: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
Hi,

On Thu, Aug 21, 2008 at 3:02 PM, Mark Williamson <mark.williamson@xxxxxxxxxxxx> wrote:
Hi,

> Sorry, I forgot to mention about the pciback module part.
> I run the pciback as a module in Dom0 and I do run the below commands
> before I start the guest OS.
>
> modprobe pciback
> echo -n 0000:xx:00.0 > /sys/bus/pci/drivers/pciback/new_slot
> echo -n 0000:xx:00.0 > /sys/bus/pci/drivers/pciback/bind
>
> So i guess this ensures that this device is hidden from dom0.

You also need to unbind it from any existing device driver that may have
grabbed it on behalf of dom0.  The disadvantage of having pciback as a module
is that any "real" device driver in dom0 that can handle that device is
likely to grab it and not let go...

> How can I verify that this device is really hidden from dom0 ? If I do an
> lspci from dom0 to dump the pci config space of this deivce after its
> hidden, should that work ?

Yes, that would work regardless, I think.

The way to tell would be to do an:

ls -l /sys/bus/pci/devices/<the id>/

and take a look at where the "driver" symlink is pointing.  If it's pointing
to the pciback driver's directory, you're all set.  If it's pointing to some
other driver's directory you need to go there and unbind the device, then
rebind it to pciback.  If there's no "driver" symlink you should just need to
bind it to pciback.

Also, take a look in dmesg to see if pciback is spitting out any useful debug
output.

> My intention is just to make sure that my PCI device is hidden from DOM0
> and is visible in my FreeBSD guest.
> So If I hide my device in DOM0 and direct xen to enable the PCI passthrough
> with the PCI  line in the config file, the FreeBSD guest boots very slow.
> And if I comment out the PCI line in Config file, it boots superfast.
>
> Do you think the PCI passthrough logic in Xen can by any chance degrade the
> Disk IO performance for HVM guests ? (Just a wild guess. I am not that good
> with the xen code.)

Well, I'd be slightly surprised if the disk IO performance was directly
affected but it's certainly possible that attempting PCI passthrough is
*somehow* slowing down *something* (I can think of a few general ways this
might happen but nothing very directed).  I.e. definitely sounds like the
cause but exactly why is hard for me to suggest at this point...

So yes, I think your guess is justified although the exact causality is hard
to say.

Just another quick check for me: is your hardware VT-d capable?  (that's
Intel's IOMMU technology).  It's needed for successful passthrough to HVM
guests.  Have you tried it with other guest OSes e.g. Linux?  It's possible
it's some kind of FreeBSD-specific bug in the PCI passthrough...

Yes my hardware is VT-d enabled. I have linux HVM guest and I dont see this problem.
Moreover freeBSD boots fine with 1 CPU, even after enabling passthrough. But if I make the vcpu to 2 then FreeBSd gets slow on passthrough.
 

Cheers,
Mark

> > > I was wondering if anyone had any information on this. Please post your
> > > responses if you could
> > > think of any possible cause for this or you have some suggestion to
> > > make
> >
> > my
> >
> > > FreeBSD guest faster.
> >
> > I guess the ideal way to make the guest faster would be to get someone to
> > port
> > the PV drivers to run under FreeBSD.  There was an existing paravirt
> > FreeBSD
> > port which could be drawn upon here but - ironically - one of the things
> > that
> > kept it out of the FreeBSD mainline was the need to modify the drivers
> > support FreeBSD's Newbus architecture.  I guess this would still be a
> > problem
> > now with respect to mainlining - however, the PV drivers definitely
> > *worked*
> > in the PV port at one stage.  We (as a community) still would need to
> > find someone who'd take on this work though :-/
> >
> >
> > Other than that, I'm afraid all I can suggest is that you apply any
> > FreeBSD /
> > Xen / virtualisation tuning tips you can find and see what effect they
> > have.
> > HVM usually hurts most in networking performance.  HVM also has fairly
> > limited GUI performance so you may find (despite the network limitations)
> > a networked GUI like X11-over-SSH or Nomachine X (if FreeBSD can run it)
> > would
> > work best.
> >
> > Hope that helps,
> > Cheers,
> > Mark
> >
> > --
> > Push Me Pull You - Distributed SCM tool (
> > http://www.cl.cam.ac.uk/~maw48/pmpu/<http://www.cl.cam.ac.uk/%7Emaw48/pmp
> >u/> )



--
Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
<Prev in Thread] Current Thread [Next in Thread>