This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
Home Products Support Community News


RE: [Xen-devel] RE: [PATCH] Xenoprof passive domain support fixes

To: "Ray Bryant" <raybry@xxxxxxxxxxxxxxxxx>, <xen-devel@xxxxxxxxxxxxxxxxxxx>
Subject: RE: [Xen-devel] RE: [PATCH] Xenoprof passive domain support fixes
From: "Santos, Jose Renato G" <joserenato.santos@xxxxxx>
Date: Tue, 11 Jul 2006 13:51:37 -0700
Cc: "Yang, Xiaowei" <xiaowei.yang@xxxxxxxxx>
Delivery-date: Tue, 11 Jul 2006 13:52:07 -0700
Envelope-to: www-data@xxxxxxxxxxxxxxxxxx
In-reply-to: <200607111402.13475.raybry@xxxxxxxxxxxxxxxxx>
List-help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-id: Xen developer discussion <xen-devel.lists.xensource.com>
List-post: <mailto:xen-devel@lists.xensource.com>
List-subscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
List-unsubscribe: <http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-devel>, <mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
Sender: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
Thread-index: AcalHJkKLbXph0qEQjeRcTx2kZEwpgACveSw
Thread-topic: [Xen-devel] RE: [PATCH] Xenoprof passive domain support fixes

Please, see my comments below.

>> -----Original Message-----
>> From: Ray Bryant [mailto:raybry@xxxxxxxxxxxxxxxxx] 
>> Sent: Tuesday, July 11, 2006 12:02 PM
>> To: xen-devel@xxxxxxxxxxxxxxxxxxx
>> Cc: Santos, Jose Renato G; Yang, Xiaowei
>> Subject: Re: [Xen-devel] RE: [PATCH] Xenoprof passive domain 
>> support fixes
>> Renato,
>> One thing I am still puzzled by in all of this is the 
>> following:  In the "opreport -x" output, I get entries for 
>> both pxen*-syms (which is a symbolic link to the xen-syms I 
>> supplied on the opreport command line) AND I get entries for 
>> xen-syms itself:
>>   samples|      %|
>> ------------------
>>     76273 23.7270 papps2-syms
>>     73306 22.8040 pxen2-syms
>>     63550 19.7691 vmlinux
>>     43024 13.3839 libc-2.4.so
>>     24406  7.5922 xen-syms
>>     17278  5.3748 jbd
>>     12228  3.8039 ext3
>>      9845  3.0626 oprofiled
>>       395  0.1229 qemu-dm
>> <snip>
>> Why is that?  (I've been told the xen-syms samples are 
>> samples in the hypervisor due to dom0 activity, but I've 
>> been unable to verify this).

Yes, that is correct. Every sample (including the ones in the
hypervisor)are associated with a domain ( the current domain running on
the CPU as indicated by the Xen macro "current"). Therefore opreport
distinguish hypervisor samples based on the domain that was running at
the time the sample was generated.

>> Additionally, I find that "opreport -lx" will report "no symbols" for
>> papps*-syms:
>> samples  %        app name                 symbol name
>> 76273    23.7738  papps2-syms              (no symbols)
>> 19131     5.9630  pxen2-syms               l2e_rw_fault
>> 17278     5.3854  jbd                      (no symbols)
>> 12228     3.8114  ext3                     (no symbols)
>> 11840     3.6905  libc-2.4.so              vfprintf
>> 10256     3.1967  libc-2.4.so              
>> _IO_file_xsputn@@GLIBC_2.2.5
>> 8587      2.6765  xen-syms                 general_protection
>> 7374      2.2984  pxen2-syms               vmx_asm_vmexit_handler
>> 5212      1.6245  pxen2-syms               resync_all
>> 5128      1.5984  xen-syms                 write_cr3
>> <snip>
>> unless I do an "ln -s /boot/vmlinux2-syms 
>> /boot/papps2-syms".  (It appears that opreport should be 
>> creating papps2-syms instead of vmlinux2-syms??)

papps2-syms represent samples collected in user level for domain2 (i.e.
ring 3). Remember that passive domain profiling cannot decode
application level samples since domain0 does not know the current memory
mappings of user level processes in domain 2. Therefore it is expected
that opreport will report "no symbols" for papps2-syms.

What is suspicious to me is that opreport is not reporting any samples
in the kernel for domain2 (they should have appeared under the name
This is probably a bug. Maybe this is triggered if you do not specify
the option --passive-images. Did you specify this option? If not, try
running the command with --passive-images=<linux image file for xenU>
(e.g. --passive-images=/boot/vmlinux-syms-2.6.16-xenU)

>> Finally, I'm not convinced yet that the sample reports for 
>> the HVM guest (papps2-syms or pvmlinux2-syms, in this case) 
>> are correct.  I'm going to run some native and xen profile 
>> sessions using the same benchmark and see if I can correlate 
>> the results at all.

 There is a problem with the current version of xenoprof for passive
domains. Samples are being assigned to wrong samples. I posted a patch
last week, that fix this problem but it seems that it was not pushed
into the main unstable tree yet. 
Try applying that patch and check if they match what you get from
(If you cannot find the patch, please let me know I will forward it to
I would appreciate if you could send me the results of your tests,
either if you find problems or if they are successfull. I think not many
people have used passive domain support yet, and any feedback would be



>> --
>> Ray Bryant
>> AMD Performance Labs                   Austin, Tx
>> 512-602-0038 (o)                 512-507-7807 (c)

Xen-devel mailing list