The pvops kernel and Dullor's patch are very new and did not get enough
testing. I am not surprised that using that did not work.
Your best option is using the 2.6.18-xen kernel. This has been used for a
while, although I suspect nobody has actually used active mode profiling or we
would have seen the bug you found earlier. Anyway that is the kernel most
likely to work for you.
Renato
> -----Original Message-----
> From: Dante Cinco [mailto:dantecinco@xxxxxxxxx]
> Sent: Monday, November 16, 2009 3:33 PM
> To: Santos, Jose Renato G
> Cc: Xen-users
> Subject: Re: xenoprof: operation 9 failed for dom0 (status: -1)
>
> Renato,
>
> Is there a list of Linux guests that have support for
> Xenoprofile or do they all need to be patched? What are the
> components that are needed in the Linux guest to support Xenoprofile?
>
> I saw Dulloor's xenoprof.patch for the pvops kernel
> (http://xen.markmail.org/search/?q=performance+tools+on+xen#qu
> ery:performance%20tools%20on%20xen%20list%3Acom.xensource.list
> s.xen-devel+page:1+mid:mouwof65pvnyan4p+state:results). I
> tried applying it to my 2.6.30.1 domU and most of it applied
> cleanly except for mmu.c and xenoprofile.c. I tried
> reconciling the changes to these 2 files and was able to
> compile the kernel but still could not get OProfile to run
> with --active-domains.
>
> BTW, I tried the kernel I'm using for dom0 and it won't event
> boot as a domU.
>
> Dante
>
>
> On Thu, Nov 12, 2009 at 5:54 PM, Santos, Jose Renato G
> <joserenato.santos@xxxxxx> wrote:
>
>
> It looks like your domU does not have the right version
> of oprofile.
> You should have seen a few printks for "dom 1". In
> particular dom1 should have called XENOPROF_enable_virq when
> you run "opcontrol --start" in the guest.
> That is why is failing.
> Either your kernel or your user level tools do not have
> Xen support. Did you apply the patch
> "oprofile-0.9.3-xen-r2.patch" to Oprofile in the guest? f so
> your kernel does not have the Xenoprof support.
>
> Renato
>
>
>
>
> ________________________________
>
>
> From: Dante Cinco [mailto:dantecinco@xxxxxxxxx]
>
> Sent: Thursday, November 12, 2009 5:44 PM
>
> To: Santos, Jose Renato G
> Cc: Xen-users
> Subject: Re: xenoprof: operation 9 failed for
> dom0 (status: -1)
>
>
> I added this line (after the close bracket of
> "switch(op)") to xenoprof.c so that I can see the state
> transitions and the values of the static variables:
>
> printk("xenoprof: operation %d for dom %d
> (xenoprof_state : %d, ret : %d, activated : %d, adomains : %d)\n",
> op, current->domain->domain_id,
> xenoprof_state, ret, activated, adomains);
>
> Here's the Xen console output after running
> "opcontrol --start-daemon --active-domains=0,1"
>
> (XEN) xenoprof: operation 1 for dom 0
> (xenoprof_state : 1, ret : 0, activated : 0, adomains : 0)
> (XEN) xenoprof: operation 3 for dom 0
> (xenoprof_state : 1, ret : 0, activated : 0, adomains : 1)
> (XEN) xenoprof: operation 3 for dom 0
> (xenoprof_state : 1, ret : 0, activated : 0, adomains : 2)
> (XEN) xenoprof: operation 14 for dom 0
> (xenoprof_state : 1, ret : 0, activated : 0, adomains : 2)
> (XEN) xenoprof: operation 5 for dom 0
> (xenoprof_state : 2, ret : 0, activated : 0, adomains : 2)
> (XEN) xenoprof: operation 6 for dom 0
> (xenoprof_state : 2, ret : 0, activated : 0, adomains : 2)
> (XEN) xenoprof: operation 6 for dom 0
> (xenoprof_state : 2, ret : 0, activated : 0, adomains : 2)
> (XEN) xenoprof: operation 7 for dom 0
> (xenoprof_state : 3, ret : 0, activated : 0, adomains : 2)
> (XEN) xenoprof: operation 8 for dom 0
> (xenoprof_state : 3, ret : 0, activated : 1, adomains : 2)
>
> Notice that "activated" is incremented only to
> 1 and it is the mismatch when compared against "adomains"
> that causes XENOPROF_start to fail after running "opcontrol
> --start" in dom0:
>
> (XEN) xenoprof: operation 9 for dom 0
> (xenoprof_state : 3, ret : -1, activated : 1, adomains : 2)
> (XEN) xenoprof: operation 9 failed for dom 0
> (status : -1)
>
> My question is what is supposed to trigger
> "activated" to go to 2? It seems that XENOPROF_enable_virq
> (calls set_active which increments "activated") needs to be
> called twice (once for each active domain). I found only one
> instance in the dom0 kernel that calls
> "HYPERVISOR_xenoprof_op(XENOPROF_enable_virq, NULL)" and it
> is in drivers/xen/xenoprof/xenoprofile.c inside the function
> xenoprof_setup(void).
>
> I did run "opcontrol --start" in domU before
> running it in dom0. Here's the sequence:
>
> domU: opcontrol --shutdown; opcontrol --reset
> dom0: opcontrol --shutdown; opcontrol --reset;
> opcontrol --start-daemon --active-domains=0,1
> domU: opcontrol --start
> dom0: opcontrol --start
>
> Dante
>
>
>
> On Wed, Nov 11, 2009 at 6:06 PM, Santos, Jose
> Renato G <joserenato.santos@xxxxxx> wrote:
>
>
> This is not the right way to use active domains.
> You should specify "--active-domains=1"
> and not "--passive-domains=1".
> By specifying passive domains the
> samples for domain 1 are being delivered to dom0.
> Probably the samples you are seeing in
> the guest are from a previous run.
> This definetely not right.
> Also you do not need to specify dom0 as
> an active domain. The option --active-domain=0 should have no effect
>
> If "--active-domains=1" is giving you
> the same error as before, this is probably because you are
> not following the right sequence of commands between dom0 and
> the guest.
> It is working fine for me after the patch
>
> Renato
>
>
> ________________________________
>
>
> From: Dante Cinco
> [mailto:dantecinco@xxxxxxxxx]
>
> Sent: Wednesday, November 11,
> 2009 5:55 PM
>
> To: Santos, Jose Renato G
> Cc: Xen-users
> Subject: Re: xenoprof:
> operation 9 failed for dom0 (status: -1)
>
>
> Yes, your patch made a big
> difference. I can now get profile data in domU. Thanks.
>
> There's a slight twist though
> in order to make this work. I have to use this command line in dom0:
>
> opcontrol --start-daemon
> --passive-domains=1 --passive-images=/boot/vmlinux-2.6.30.1
> --active-domains=0 --vmlinux=/boot/vmlinux-2.6.30.3
> --xen=/boot/xen-syms-3.4.1
>
> If I try to use
> "--active-domains=1" and not specify --passive-domains and
> --passive-images, I still get the same error message when I
> run "opcontrol --start" in dom0. I thought passive-domains
> are domains that are not running OProfile but I am running
> OProfile in domU (dom ID#1).
>
> When I run "opreport -l" in
> dom0, I see samples from domain1-kernel and when I run
> "opreport -l" in domU, I see samples from vmlinux-2.6.30.1.
> My question is how does Xenoprof know which samples to route
> to domain1-kernel in dom0 and which samples to routed in
> vmlinux-2.6.30.1 in domU?
>
> Dante
>
>
> On Wed, Nov 11, 2009 at 12:16
> AM, Santos, Jose Renato G <joserenato.santos@xxxxxx> wrote:
>
>
> I think I found the bug.
> This patch should fix it.
> Please let me know if it works
>
> Thanks
>
> Renato
>
> diff -r 7422afed66ee
> xen/common/xenoprof.c
> ---
> a/xen/common/xenoprof.c Mon Sep 07 08:53:07 2009 +0100
> +++
> b/xen/common/xenoprof.c Tue Nov 10 23:45:48 2009 -0800
> @@ -681,8 +681,9 @@ int
> do_xenoprof_op(int op, XEN_GUEST_HAN
> {
> case XENOPROF_init:
> ret =
> xenoprof_op_init(arg);
> - if ( !ret )
> -
> xenoprof_state = XENOPROF_INITIALIZED;
> + if ( (ret == 0) &&
> +
> (current->domain == xenoprof_primary_profiler) )
> +
> xenoprof_state = XENOPROF_INITIALIZED;
> break;
>
> case XENOPROF_get_buffer:
>
>
>
> > -----Original Message-----
> > From: Dante Cinco
> [mailto:dantecinco@xxxxxxxxx]
>
> > Sent: Tuesday,
> November 10, 2009 11:21 AM
> > To: Santos, Jose Renato G
> > Cc: Xen-users
> > Subject: Re:
> xenoprof: operation 9 failed for dom0 (status: -1)
> >
> > As you can see from
> the output of "opreport -l" below, most
> > of the cycles are
> coming from domain1-modules so I do need to
> > focus on
> --active-domains=1 since --passive-domains=1 does
> > not provide the
> profiling details for the domU modules.
> >
> > The steps described
> in the xenoprof tutorial for active
> > domains is pretty
> straightforward but I cannot get past the
> > "write error" when I
> run "opcontrol --start" in dom0.
> >
> > After running
> "opcontrol --start" in domU, I see the response
> > below and I use lsmod
> to verify that oprofile module is
> > loaded. Given the
> results from --passive-domains=1, the dom0
> > and Xenoprof
> interface is working. It's the domU and dom0
> > interface that has
> some problem. Is there some other way I
> > can tell from dom0
> that domU is "ready" before running
> > "opcontrol --start" in dom0?
> >
> > FYI: I'm using the
> same oprofile-0.9.3 with xen-r2.patch in
> > dom0 and domU. If I
> boot the Debian 2.6.30.1 kernel (the same
> > one I'm using in
> domU) in bare-metal (no Xen VMM), I'm able
> > to successfully run
> oprofile-0.9.3.
> >
> > opcontrol --start
> (from domU before "opcontrol --start" in dom0):
> > Using 2.6+ OProfile
> kernel interface.
> > Reading module info.
> > Using log file
> /var/lib/oprofile/samples/oprofiled.log
> > Daemon started.
> > Profiler running.
> >
> > opreport -l (from
> dom0 using --passive-domains=1
> >
> --passive-images=/boot/vmlinux-2.6.30.1):
> > CPU: Core 2, speed
> 2533.51 MHz (estimated) Counted
> > CPU_CLK_UNHALTED
> events (Clock cycles when not halted) with a
> > unit mask of 0x00
> (Unhalted core cycles) count 100000
> > samples %
> image name app name
> > symbol name
> > 617835 38.6156
> domain1-modules domain1-modules
> > (no symbols)
> > 448911 28.0576
> domain1-xen-unknown
> > domain1-xen-unknown
> (no symbols)
> > 72460 4.5289
> domain1-kernel domain1-kernel
> > __down
> > 43294 2.7059
> domain1-kernel domain1-kernel
> > __down_killable
> > 34145 2.1341
> domain1-kernel domain1-kernel
> > validate_slab_slab
> >
> > Dante
> >
> >
> > On Tue, Nov 10, 2009
> at 10:22 AM, Santos, Jose Renato G
> >
> <joserenato.santos@xxxxxx> wrote:
> >
> >
> > With passive
> domains you cannot have detailed profiling
> > information on
> modules, only in kernel builtin functions and
> > on Xen. All the
> samples associated with modules will be
> > grouped under the
> same symbol "domain1-modules".
> > If you are
> interested in one particular module you
> > should try to
> recompile the kernel with the associated code
> > builtin (or you can
> use active domains, but follow the steps
> > on the xenoprof
> tutorial to coordinate opcontrol in dom0 and
> > in the guest)
> >
> > Renato
> >
> >
> >
> ________________________________
> >
> >
> > From:
> Dante Cinco [mailto:dantecinco@xxxxxxxxx]
> >
> > Sent:
> Tuesday, November 10, 2009 10:13 AM
> >
> > To:
> Santos, Jose Renato G
> > Cc: Xen-users
> >
> Subject: Re: xenoprof: operation 9 failed for
> > dom0 (status: -1)
> >
> >
> > Renato,
> >
> > I think
> I'm making progress. I followed your
> > suggestion of using
> --passive-domains and --passive-images.
> > When I run opreport,
> it is unable to find /domain1-modules
> > and
> /domain1-xen-unknown. Where or how do I specify the
> > kernel modules I have
> running in domU/domain1? I tried
> > copying the *.ko
> files in /boot in dom0 and used
> > --image-path=/boot in
> opreport and it is still not finding them.
> >
> > Thanks.
> >
> > Dante
> >
> >
> > On Mon,
> Nov 9, 2009 at 6:44 PM, Santos, Jose
> > Renato G
> <joserenato.santos@xxxxxx> wrote:
> >
> >
> >
> Try replacing "--active-domains=1" with
> > "--passive-domains=1
> passive-images=<domU-kernel-image>" (use
> > the uncompressed
> version of your kernel image for the guest,
> > vmlinux-*)
> >
> >
> To use "active-domains" you need to run
> > opcontrol in the
> guest in addition to running it in dom0 and
> > you need to
> coordinate the execution of both instances.. This
> > require the guest
> opcontrol to be ready before running
> > "opcontrol --start"
> in dom0. That is why it is failing. I
> > suspect you have not
> executed opcontrol in the guest
> >
> Using active-domains is very tricky. I
> > suggest that you use
> --passive-domains, unless you really
> > need active domains
> (it is only useful in case you need
> > detailed profiles for
> user processes running in the guest)
> >
> > Renato
> >
> >
> >
> ________________________________
> >
> >
> >
> From: Dante Cinco
> > [mailto:dantecinco@xxxxxxxxx]
> >
> >
> Sent: Monday, November 09, 2009 6:13 PM
> >
> >
> To: Santos, Jose Renato G
> >
> >
> Cc: Xen-users
> >
> Subject: Re: xenoprof:
> > operation 9 failed
> for dom0 (status: -1)
> >
> >
> >
> Renato,
> >
> >
> I've narrowed down the
> > opcontrol command
> sequence that causes the "write error" I'm
> > having. If I just run
> "opcontrol --start" in dom0, it runs
> > with no error and
> after "opcontrol --shutdown" I can run
> > "opreport" and get a
> real report.
> >
> >
> If I run "opcontrol
> > --start-daemon
> --active-domains=1" in dom0, run "opcontrol
> > --start" in domU
> (ID#1) and go back to dom0 and run
> > "opcontrol --start" I
> get the "write error" message. It's as
> > if "--start-daemon"
> is grabbing the file handle for
> > /dev/oprofile/enable
> so when "--start" tries to write "1" to
> > /dev/oprofile/enable,
> it is unable to do so because it is
> > already locked.
> >
> >
> So I can run OProfile in normal
> > (non-Xen) mode but as
> soon I start using "--start-daemon" I
> > have problems. To me
> it seems like a Xenoprofile problem.
> >
> >
> I'm using OProfile 0.9.3 and
> >
> oprofile-0.9.3-xen-r2.patch applied successfully.
> >
> >
> Dante
> >
> >
> >
> On Thu, Nov 5, 2009 at 6:05 PM,
> > Santos, Jose Renato G
> <joserenato.santos@xxxxxx> wrote:
> >
> >
> >
> What version of
> > OProfile are you using?
> >
> Did you apply the Xen
> > patch available in
> http://xenoprof.sourceforge.net ?
> >
> >
> Renato
> >
> >
> ________________________________
> >
> >
> From: Dante
> > Cinco
> [mailto:dantecinco@xxxxxxxxx]
> >
> Sent: Thursday,
> > November 05, 2009 5:16 PM
> >
> To: Santos,
> > Jose Renato G
> >
> Cc: Xen-devel
> >
> Subject:
> > xenoprof: operation 9
> failed for dom0 (status: -1)
> >
> >
> Renato,
> >
> >
> When I tried
> > running "opcontrol
> --start" (after previously running
> > "opcontrol
> --start-daemon") in dom0, I get this error message:
> >
> >
> >
> /usr/local/bin/opcontrol: line 1639: echo: write error:
> > Operation not permitted
> >
> >
> and this
> > message in the Xen console:
> >
> >
> (XEN) xenoprof:
> > operation 9 failed
> for dom 0 (status : -1)
> >
> >
> It looks like
> > opcontrol is trying
> to do this: echo 1 > /dev/oprofile/enable
> >
> >
> and it is
> > failing. "operation
> 9" maps to XENOPROF_start which is
> > consistent with
> running "opcontrol --start." At first, I
> > ignored the error
> because it gave the indication "Profiler
> > running" but after I
> ran "opcontrol --shutdown" followed by
> > "opreport" in dom0, I
> got this error message:
> >
> >
> error: no
> > sample files found:
> profile specification too strict ?
> >
> >
> Do you know why
> > the write error is
> occurring? I followed the steps in
> > xenoprof_tutorial.ppt.
> >
> >
> Dante
> >
> >
> >
> >
> >
> >
>
>
>
>
>
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home