Xen network domain performance for 10Gb NIC

[tosher 1 <akm2tosher@xxxxxxxxx>]

 Hi everyone,

Lately, I have been experimenting with 10Gb NIC performance on Xen domains. I 
have found that network performance is very poor for PV networking when a 
driver domain is used as a network backend.

My experimental setup is I have two machines connected by the 10Gb network: a 
server running the Xen hypervisor and a desktop machine working as a client. I 
have Ubuntu 18.04.3 LTS running on the Dom0, Domus, Driver Domain, and client 
desktop, where the Xen version is 4.9. I measured the network bandwidth using 
iPerf3.

The network bandwidth between a DomU using Dom0 as backend and the client 
desktop is like 9.39Gbits/sec. However, when I use a network driver domain, 
which has the 10Gb NIC by PCI pass through, the bandwidth between the DomU and 
the client desktop is like 2.41Gbit/sec is one direction and 4.48Gbits/sec in 
another direction. Here, by direction, I mean the client-server direction for 
iPerf3.

These results indicate a huge performance degradation, which is unexpected. I 
am wondering if I am missing any key points here which I should have taken care 
of or if there is any tweak that I can apply.

Thanks,
Mehrab