[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [PATCH v3 0/7] xen/x86: various XPTI speedups



On Wed, 2018-03-21 at 13:51 +0100, Juergen Gross wrote:
> On my machine (Intel i7-4600M) using the PCID feature in the non-XPTI
> case showed a slightly worse performance than using global pages
> instead (using PCID and global pages is a bad idea as invalidating
> global pages in this case would need a complete TLB flush). For this
> reason I've decided to use PCID for XPTI only as the default. That
> can easily be changed by using the command line parameter "pcid=all".
> 
>        XPTI off     Jan, XPTI on        +this series, XPTI on
> real   1m21.169s    1m52.149s (+38%)    1m25.692s (+6%)
> user   2m47.652s    2m50.054s (+1%)     2m46.428s (-1%)
> sys    1m11.949s    2m21.767s (+97%)    1m23.053s (+15%)
> 
> A git branch of that series (+ Jan's patches) is available:
> 
> https://github.com/jgross1/xen.git xpti
> 
As usual, I've run a few benchmarks. Long story short, this series
mitigates the performance hit of XPTI, in pretty much all the cases. In
a couple of cases, even significantly so (~10%).

The improvement is not as good as in the above numbers, even for
similar workloads (compile ones), but I guess that depends on the
hardware. But anyway, things do improve, and I think we should have the
series in 4.11.

I can also confirm that using PCID with XPTI=no, does not bring much of
an improvement, if any at all.

There's the weird case of STREAM, where using PCID makes things go
significantly worse, not only than no-PCID and xpti=no, but also than 
PCID and xpri=true.
Basically, when using PCID, xpti boosts STREAM performance, which is a
little weird. I'll try to manually re-run the benchmark in a couple of
configurations, and let's see what I will find.

Here's the results. Basically:
- if you want to compare xpti=on, with and without this series, look at
   'Jan, XPTI on' (without) and '+this series, XPTI on' (with);
- if you want to compare, with this series, xpti on and off, look at
  'XPTI off' (off) and '+this series, XPTI on' (on);
- if you want to compare xpti=off, with and without PCID, look at
  'XPTI off' (without) and '+this series, XPTI off, PCID all' (with).

https://openbenchmarking.org/result/1803232-DARI-180323589

Also available here:

AIO-Stress 0.21
Test: Random Write
    MB/s > Higher Is Better
    XPTI off .......................... 1783.11 
|===================================================================================================================================================
    +this series, XPTI on ............. 1967.75 
|==================================================================================================================================================================
    +this series, XPTI off, PCID all .. 1827.43 
|======================================================================================================================================================
    Jan, XPTI on ...................... 1866.80 
|==========================================================================================================================================================

Stream 2013-01-17
Type: Copy
    MB/s > Higher Is Better
    XPTI off .......................... 19442.54 
|=================================================================================================================================================================
    +this series, XPTI on ............. 18939.40 
|=============================================================================================================================================================
    +this series, XPTI off, PCID all .. 15478.48 
|================================================================================================================================
    Jan, XPTI on ...................... 15638.96 
|==================================================================================================================================

Stream 2013-01-17
Type: Scale
    MB/s > Higher Is Better
    XPTI off .......................... 12978.74 
|=================================================================================================================================================================
    +this series, XPTI on ............. 12851.56 
|===============================================================================================================================================================
    +this series, XPTI off, PCID all .. 10699.02 
|=====================================================================================================================================
    Jan, XPTI on ...................... 10778.84 
|======================================================================================================================================

Stream 2013-01-17
Type: Triad
    MB/s > Higher Is Better
    XPTI off .......................... 14280.68 
|=================================================================================================================================================================
    +this series, XPTI on ............. 14022.30 
|==============================================================================================================================================================
    +this series, XPTI off, PCID all .. 12149.18 
|=========================================================================================================================================
    Jan, XPTI on ...................... 12218.22 
|==========================================================================================================================================

Stream 2013-01-17
Type: Add
    MB/s > Higher Is Better
    XPTI off .......................... 16093.28 
|=================================================================================================================================================================
    +this series, XPTI on ............. 15815.04 
|==============================================================================================================================================================
    +this series, XPTI off, PCID all .. 12077.40 
|=========================================================================================================================
    Jan, XPTI on ...................... 12269.30 
|===========================================================================================================================

t-test1 2017-01-13
Threads: 1
    Seconds < Lower Is Better
    XPTI off .......................... 76.16  
|===============================================================================================
    +this series, XPTI on ............. 113.93 
|=============================================================================================================================================
    +this series, XPTI off, PCID all .. 74.68  
|=============================================================================================
    Jan, XPTI on ...................... 131.26 
|===================================================================================================================================================================

t-test1 2017-01-13
Threads: 2
    Seconds < Lower Is Better
    XPTI off .......................... 24.28 
|====================================================================================================
    +this series, XPTI on ............. 34.91 
|================================================================================================================================================
    +this series, XPTI off, PCID all .. 23.92 
|===================================================================================================
    Jan, XPTI on ...................... 39.73 
|====================================================================================================================================================================

Timed Linux Kernel Compilation 4.13
Time To Compile
    Seconds < Lower Is Better
    XPTI off .......................... 155.35 
|==============================================================================================================================================
    +this series, XPTI on ............. 170.36 
|===========================================================================================================================================================
    +this series, XPTI off, PCID all .. 155.81 
|==============================================================================================================================================
    Jan, XPTI on ...................... 178.89 
|===================================================================================================================================================================

Timed MPlayer Compilation 1.0-rc3
Time To Compile
    Seconds < Lower Is Better
    XPTI off .......................... 45.94 
|=======================================================================================================================================================
    +this series, XPTI on ............. 48.85 
|================================================================================================================================================================
    +this series, XPTI off, PCID all .. 45.95 
|=======================================================================================================================================================
    Jan, XPTI on ...................... 49.97 
|====================================================================================================================================================================

Timed PHP Compilation 7.1.9
Time To Compile
    Seconds < Lower Is Better
    XPTI off .......................... 119.18 
|================================================================================================================================================
    +this series, XPTI on ............. 129.24 
|============================================================================================================================================================
    +this series, XPTI off, PCID all .. 120.29 
|=================================================================================================================================================
    Jan, XPTI on ...................... 134.76 
|===================================================================================================================================================================

Parallel BZIP2 Compression 1.1.12
256MB File Compression
    Seconds < Lower Is Better
    XPTI off .......................... 8.97 
|===========================================================================================================================================================
    +this series, XPTI on ............. 8.43 
|==================================================================================================================================================
    +this series, XPTI off, PCID all .. 9.00 
|===========================================================================================================================================================
    Jan, XPTI on ...................... 9.55 
|=====================================================================================================================================================================

Hackbench 
Count: 1 - Type: Process
    Seconds < Lower Is Better
    XPTI off .......................... 11.11 
|=====================================================================================================
    +this series, XPTI on ............. 16.62 
|=======================================================================================================================================================
    +this series, XPTI off, PCID all .. 11.05 
|====================================================================================================
    Jan, XPTI on ...................... 18.11 
|====================================================================================================================================================================

Hackbench 
Count: 4 - Type: Process
    Seconds < Lower Is Better
    XPTI off .......................... 37.82 
|===============================================================================================
    +this series, XPTI on ............. 59.66 
|=====================================================================================================================================================
    +this series, XPTI off, PCID all .. 37.58 
|==============================================================================================
    Jan, XPTI on ...................... 65.47 
|====================================================================================================================================================================

Hackbench 
Count: 16 - Type: Process
    Seconds < Lower Is Better
    XPTI off .......................... 142.24 
|==========================================================================================
    +this series, XPTI on ............. 232.55 
|===================================================================================================================================================
    +this series, XPTI off, PCID all .. 141.58 
|==========================================================================================
    Jan, XPTI on ...................... 257.20 
|===================================================================================================================================================================

Redis 4.0.8
Test: LPOP
    Requests Per Second > Higher Is Better
    XPTI off .......................... 402887.81 
|================================================================================================================================================================
    +this series, XPTI on ............. 146349.66 
|==========================================================
    +this series, XPTI off, PCID all .. 341182.95 
|=======================================================================================================================================
    Jan, XPTI on ...................... 128094.76 
|===================================================

Redis 4.0.8
Test: SADD
    Requests Per Second > Higher Is Better
    XPTI off .......................... 378952.25 
|================================================================================================================================================================
    +this series, XPTI on ............. 165097.91 
|======================================================================
    +this series, XPTI off, PCID all .. 371460.27 
|=============================================================================================================================================================
    Jan, XPTI on ...................... 141056.13 
|============================================================

Redis 4.0.8
Test: LPUSH
    Requests Per Second > Higher Is Better
    XPTI off .......................... 327837.31 
|=============================================================================================================================================================
    +this series, XPTI on ............. 146894.11 
|=======================================================================
    +this series, XPTI off, PCID all .. 333126.30 
|================================================================================================================================================================
    Jan, XPTI on ...................... 126966.19 
|=============================================================

Redis 4.0.8
Test: GET
    Requests Per Second > Higher Is Better
    XPTI off .......................... 397632.29 
|================================================================================================================================================================
    +this series, XPTI on ............. 169390.22 
|====================================================================
    +this series, XPTI off, PCID all .. 393664.83 
|==============================================================================================================================================================
    Jan, XPTI on ...................... 143045.55 
|==========================================================

Redis 4.0.8
Test: SET
    Requests Per Second > Higher Is Better
    XPTI off .......................... 352699.10 
|================================================================================================================================================================
    +this series, XPTI on ............. 153986.24 
|======================================================================
    +this series, XPTI off, PCID all .. 350564.06 
|===============================================================================================================================================================
    Jan, XPTI on ...................... 133726.65 
|=============================================================

Stress-NG 0.07.26
Test: Crypto
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 689.86 
|===================================================================================================================================================================
    +this series, XPTI on ............. 688.27 
|===================================================================================================================================================================
    +this series, XPTI off, PCID all .. 689.49 
|===================================================================================================================================================================
    Jan, XPTI on ...................... 688.75 
|===================================================================================================================================================================

Stress-NG 0.07.26
Test: Bsearch
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 3939.99 
|==================================================================================================================================================================
    +this series, XPTI on ............. 3936.73 
|==================================================================================================================================================================
    +this series, XPTI off, PCID all .. 3936.85 
|==================================================================================================================================================================
    Jan, XPTI on ...................... 3930.97 
|==================================================================================================================================================================

Stress-NG 0.07.26
Test: Forking
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 6056.42 
|=================================================================================================================================================================
    +this series, XPTI on ............. 5222.55 
|===========================================================================================================================================
    +this series, XPTI off, PCID all .. 6107.00 
|==================================================================================================================================================================
    Jan, XPTI on ...................... 4515.24 
|========================================================================================================================

Stress-NG 0.07.26
Test: Hsearch
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 26376.03 
|=================================================================================================================================================================
    +this series, XPTI on ............. 26278.84 
|================================================================================================================================================================
    +this series, XPTI off, PCID all .. 26391.88 
|=================================================================================================================================================================
    Jan, XPTI on ...................... 26258.08 
|================================================================================================================================================================

Stress-NG 0.07.26
Test: Lsearch
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 60.27 
|====================================================================================================================================================================
    +this series, XPTI on ............. 60.26 
|====================================================================================================================================================================
    +this series, XPTI off, PCID all .. 60.27 
|====================================================================================================================================================================
    Jan, XPTI on ...................... 60.27 
|====================================================================================================================================================================

Stress-NG 0.07.26
Test: Tsearch
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 124.15 
|==================================================================================================================================================================
    +this series, XPTI on ............. 124.12 
|==================================================================================================================================================================
    +this series, XPTI off, PCID all .. 125.04 
|===================================================================================================================================================================
    Jan, XPTI on ...................... 124.75 
|===================================================================================================================================================================

Stress-NG 0.07.26
Test: CPU Stress
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 1709.34 
|============================================================================================================================================================
    +this series, XPTI on ............. 1751.09 
|================================================================================================================================================================
    +this series, XPTI off, PCID all .. 1746.89 
|================================================================================================================================================================
    Jan, XPTI on ...................... 1772.44 
|==================================================================================================================================================================

Stress-NG 0.07.26
Test: Semaphores
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 2208236.65 
|============================================================================================================================================================
    +this series, XPTI on ............. 2044183.40 
|================================================================================================================================================
    +this series, XPTI off, PCID all .. 2252052.71 
|===============================================================================================================================================================
    Jan, XPTI on ...................... 2092475.64 
|====================================================================================================================================================

Stress-NG 0.07.26
Test: Matrix Math
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 20279.06 
|=================================================================================================================================================================
    +this series, XPTI on ............. 20261.14 
|=================================================================================================================================================================
    +this series, XPTI off, PCID all .. 20287.40 
|=================================================================================================================================================================
    Jan, XPTI on ...................... 20264.35 
|=================================================================================================================================================================

Stress-NG 0.07.26
Test: Vector Math
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 10506.60 
|=================================================================================================================================================================
    +this series, XPTI on ............. 10500.10 
|=================================================================================================================================================================
    +this series, XPTI off, PCID all .. 10509.78 
|=================================================================================================================================================================
    Jan, XPTI on ...................... 10493.20 
|=================================================================================================================================================================

Stress-NG 0.07.26
Test: Memory Copying
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 1051.84 
|==================================================================================================================================================================
    +this series, XPTI on ............. 1012.27 
|============================================================================================================================================================
    +this series, XPTI off, PCID all .. 983.86  
|========================================================================================================================================================
    Jan, XPTI on ...................... 992.60  
|=========================================================================================================================================================

Stress-NG 0.07.26
Test: Socket Activity
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 1587.44 
|=================================================================================================================================================================
    +this series, XPTI on ............. 1087.62 
|==============================================================================================================
    +this series, XPTI off, PCID all .. 1594.80 
|==================================================================================================================================================================
    Jan, XPTI on ...................... 922.89  
|==============================================================================================

Stress-NG 0.07.26
Test: Context Switching
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 696817.28 
|================================================================================================================================================================
    +this series, XPTI on ............. 343513.18 
|===============================================================================
    +this series, XPTI off, PCID all .. 657917.78 
|=======================================================================================================================================================
    Jan, XPTI on ...................... 280122.99 
|================================================================

Stress-NG 0.07.26
Test: Glibc C String Functions
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 268516.17 
|=============================================================================================================================================================
    +this series, XPTI on ............. 269261.27 
|=============================================================================================================================================================
    +this series, XPTI off, PCID all .. 273643.72 
|================================================================================================================================================================
    Jan, XPTI on ...................... 266052.71 
|============================================================================================================================================================

Stress-NG 0.07.26
Test: Glibc Qsort Data Sorting
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 57.80 
|====================================================================================================================================================================
    +this series, XPTI on ............. 56.66 
|=================================================================================================================================================================
    +this series, XPTI off, PCID all .. 57.77 
|====================================================================================================================================================================
    Jan, XPTI on ...................... 56.88 
|=================================================================================================================================================================

Stress-NG 0.07.26
Test: System V Message Passing
    Bogo Ops/s > Higher Is Better
    XPTI off .......................... 3645928.49 
|===============================================================================================================================================================
    +this series, XPTI on ............. 1743511.46 
|============================================================================
    +this series, XPTI off, PCID all .. 3573098.79 
|============================================================================================================================================================
    Jan, XPTI on ...................... 1596148.22 
|======================================================================

PyBench 2018-02-16
Total For Average Test Times
    Milliseconds < Lower Is Better
    XPTI off .......................... 3007 
|=====================================================================================================================================================================
    +this series, XPTI on ............. 3000 
|=====================================================================================================================================================================
    +this series, XPTI off, PCID all .. 2995 
|====================================================================================================================================================================
    Jan, XPTI on ...................... 3000 
|=====================================================================================================================================================================

NGINX Benchmark 1.9.9
Static Web Page Serving
    Requests Per Second > Higher Is Better
    XPTI off .......................... 15930.51 
|=================================================================================================================================================================
    +this series, XPTI on ............. 11537.88 
|=====================================================================================================================
    +this series, XPTI off, PCID all .. 15915.73 
|=================================================================================================================================================================
    Jan, XPTI on ...................... 10454.68 
|==========================================================================================================

PHPBench 0.8.1
PHP Benchmark Suite
    Score > Higher Is Better
    XPTI off .......................... 279331 
|==================================================================================================================================================================
    +this series, XPTI on ............. 253206 
|===================================================================================================================================================
    +this series, XPTI off, PCID all .. 280877 
|===================================================================================================================================================================
    Jan, XPTI on ...................... 247288 
|================================================================================================================================================

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.