[Xen-devel] apic-v reduce network performance in my test case

["Liuqiming (John)" <john.liuqiming@xxxxxxxxxx>]

Hi all,

    Recently I met an odd performance problem: when I turn on APIC 
Virtualization feature (apicv=1), the network performance of a windows 
guest become worse.
    My test case like this: host only have one windows 2008 R2 HVM 
guest running,and this guest has a SR-IOV VF network passthrough to it.
Guest using this network access a NAS device. No fontend or backend of 
network and storage, all data transfered through network.
    The xentrace data shows: the mainly difference between apicv and 
non-apicv, is the way guest write apic registers, and 
EXIT_REASON_MSR_WRITE vmexit cost much more time than 
EXIT_REASON_APIC_WRITE, but when using WRMSR, the PAUSE vmexit is much 
less than using APIC-v.
This is the odd part,any ideas?

APIC-v OFF:
     4099582 VMEXIT   3467051359128 TSC HLT
    10135140 VMEXIT     42484175528 TSC WRMSR
     1651714 VMEXIT      9785961276 TSC I/O instruction
      532702 VMEXIT      3887971388 TSC External interrupt
      290546 VMEXIT      2262312440 TSC PAUSE
      588077 VMEXIT       914905312 TSC Control-register accesses
      383617 VMEXIT       453329940 TSC Exception or non-maskable 
interrupt (NMI)
      132717 VMEXIT       232289792 TSC Interrupt window
       25534 VMEXIT       198718764 TSC EPT violation
       53969 VMEXIT        62886752 TSC TPR below threshold
        7996 VMEXIT        34735376 TSC RDMSR
        1615 VMEXIT        16042768 TSC VMCALL
         147 VMEXIT          272320 TSC CPUID
           7 VMEXIT            6484 TSC WBINVD
           2 VMEXIT            6308 TSC MOV DR  

APIC-v ON:
     3717629 VMEXIT   3459905385332 TSC HLT
     2282403 VMEXIT     23099880196 TSC APIC write
     3900448 VMEXIT     13073253548 TSC PAUSE
     1643729 VMEXIT     11719626776 TSC I/O instruction
     2194667 VMEXIT      5321640708 TSC WRMSR
      214425 VMEXIT      2198994944 TSC External interrupt
      566795 VMEXIT      1940710108 TSC Control-register accesses
      342688 VMEXIT       659665532 TSC Exception or non-maskable 
interrupt (NMI)
      190623 VMEXIT       644411612 TSC VMCALL
      188657 VMEXIT       295956932 TSC Virtualized EOI
       24350 VMEXIT       194817152 TSC EPT violation
        4393 VMEXIT        23282044 TSC RDMSR
         179 VMEXIT         1688676 TSC CPUID
           7 VMEXIT            6884 TSC WBINVD
           1 VMEXIT            4200 TSC MOV DR

In commit 7f2e992b824ec62a2818e64390ac2ccfbd74e6b7
"VMX/Viridian: suppress MSR-based APIC suggestion when having APIC-V", 
msr based apic is disabled when apic-v is on, I wonder can they co-exist 
in some way? seems for windows guest msr-based apic has better performance.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel