[Xen-devel] PROBLEM: xen-netfront: ethtool settings changed in 4.4.134 causing packet loss

[Liam Shepherd <liam@xxxxxxxxx>]

When upgrading from 4.4.132 to 4.4.135 I started observing about 3%
packet loss on PV on HVM domU interfaces. Specifically TX dropped
packets (As reported by ifconfig).

It seems that the default ethtool settings have changed.

Previously (4.4.132):
tx-checksumming: on
    tx-checksum-ipv4: on [fixed]
    tx-checksum-ip-generic: off [fixed]
    tx-checksum-ipv6: on
    tx-checksum-fcoe-crc: off [fixed]
    tx-checksum-sctp: off [fixed]
scatter-gather: on
    tx-scatter-gather: on
    tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
    tx-tcp-segmentation: on
    tx-tcp-ecn-segmentation: off [fixed]
    tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on

Now (4.4.135):
tx-checksumming: on
    tx-checksum-ipv4: on [fixed]
    tx-checksum-ip-generic: off [fixed]
    tx-checksum-ipv6: off
    tx-checksum-fcoe-crc: off [fixed]
    tx-checksum-sctp: off [fixed]
scatter-gather: off
    tx-scatter-gather: off
    tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
    tx-tcp-segmentation: off
    tx-tcp-ecn-segmentation: off [fixed]
    tx-tcp6-segmentation: off
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off [requested on]

Manually running 'ethtool -K INT tso on sg on' on all interfaces stops
the packet loss again. They're then set like this:
tx-checksumming: on
    tx-checksum-ipv4: on [fixed]
    tx-checksum-ip-generic: off [fixed]
    tx-checksum-ipv6: off
    tx-checksum-fcoe-crc: off [fixed]
    tx-checksum-sctp: off [fixed]
scatter-gather: on
    tx-scatter-gather: on
    tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
    tx-tcp-segmentation: on
    tx-tcp-ecn-segmentation: off [fixed]
    tx-tcp6-segmentation: off [requested on]
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on

I've traced it to a commit in 4.4.134:
6be4fe832954db3b359b72107a6c60e34d939b26
xen-netfront: Fix race between device setup and open
[ Upstream commit f599c64fdf7d9c108e8717fb04bc41c680120da4 ]
https://patchwork.kernel.org/patch/10328969/

Once that is reverted on 4.4.135, the ethtool settings are as 4.4.132
and there is no packet loss.

The VMs and hypervisor are all running CentOS 7.5. The hypervisor is
on the centos-virt provided kernel, and the VMs are using kernel-lt
from elrepo.

# xl info
host                   : blah
release                : 4.9.86-30.el7.x86_64
version                : #1 SMP Mon Mar 5 16:58:23 UTC 2018
machine                : x86_64
nr_cpus                : 40
max_cpu_id             : 191
nr_nodes               : 2
cores_per_socket       : 10
threads_per_core       : 2
cpu_mhz                : 2197
hw_caps                :
b7ebfbff:77fef3ff:2c100800:00000121:00000001:001cbfbb:00000000:00000100
virt_caps              : hvm hvm_directio
total_memory           : 81826
free_memory            : 13237
sharing_freed_memory   : 0
sharing_used_memory    : 0
outstanding_claims     : 0
free_cpus              : 0
xen_major              : 4
xen_minor              : 8
xen_extra              : .3-5.el7
xen_version            : 4.8.3-5.el7
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : Thu Apr 26 23:28:44 2018 -0700 git:10d7690-dirty
xen_commandline        : placeholder dom0_mem=4096M,max:4096M cpuinfo
com1=115200,8n1 console=com1,tty loglvl=all guest_loglvl=all
gnttab_max_frames=1024 cpufreq=none max_cstate=0
cc_compiler            : gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
cc_compile_by          : mockbuild
cc_compile_domain      : centos.org
cc_compile_date        : Tue May  8 17:17:55 UTC 2018
build_id               : 484fda280c1d4a21346da47907cd1a8e9aba99df
xend_config_format     : 4

Please let me know anything else I can provide.

Thanks

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel