Re: [win-pv-devel] poor winpv performance with iommu=on + disk driver domain

[James Dingwall <james-xen@xxxxxxxxxxxxxx>]

On Tue, May 22, 2018 at 05:12:31PM +0100, James Dingwall wrote:
> I did some tracing in the disk driver domain for the block back driver:
> 
> cd /sys/kernel/debug/tracing
> echo function_graph > current_tracer
> echo ':mod:xen_blkback' > set_ftrace_filter
> echo pid_of_blback_worker > set_ftrace_pid
> 
> For WinPV there are some very long periods at the end of some 
> operations which are not seen when tracing with a Linux guest, some 
> parts of the captured traces below.  I assume that something extra is 
> happening with WinPV due to the extra {} set after xen_blkbk_map 
> [xen_blkback]().

Based on the traces I had a look at the blkback driver source and it has 
a code path in xen_blkbk_map which is conditional on persistent-grants 
being enabled.  When a frontend device is connected the backend logs the 
following messages:

Linux: iommu=off/on

May 23 10:37:00 zdiskdd0000-0063-00-00 kernel: [  572.830571] 
xen-blkback: ring-pages:1, event-channel 19, protocol 1 (x86_64-abi) 
persistent grants

WinPV: iommu=off/on

May 23 10:40:50 zdiskdd0000-0063-00-00 kernel: [  802.362511] 
xen-blkback: ring-pages:1, event-channel 5, protocol 1 (x86_64-abi) 
May 23 10:40:50 zdiskdd0000-0063-00-00 kernel: [  802.512790] 
xen-blkback: ring-pages:1, event-channel 6, protocol 1 (x86_64-abi) 
May 23 10:40:50 zdiskdd0000-0063-00-00 kernel: [  802.537699] 
xen-blkback: ring-pages:1, event-channel 7, protocol 1 (x86_64-abi)

So it seems that not using persistent grants severely impacts the 
performance when the disk controller is passed through to a driver 
domain.  (The same issue is not seen if the disk controller is in the 
dom0.)  Looking at the xenvbd source code it seems there is some 
awareness of feature-persistent but I don't know if it is fully 
supported, any answers in that regard?

Thanks,
James

_______________________________________________
win-pv-devel mailing list
win-pv-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/win-pv-devel