My dom0 is sending packets to the domU with incorrect TCP checksums
(not in all cases, but in some). I've looked at the various FAQ and
documentation pages but I'm afraid I'm still stumped. (ethtool -k
didn't work.)
Below are tcpdumps of the sessions, and the output of route -vn and
ifconfig and brctl show on both dom0 and domU. I'm not using bridges;
I'm using a slightly modified vif-route, so I also include the output
of iptables -L -v -n and a copy of the vif-route script.
The topology is fairly simple:
real ethernet dom0 as router domU
172.18.45/25 ------ 172.18.45.97 -------- 172.18.45.65
gw 172.18.45.11 eth0 vif*.0 eth0
to real Internet
I have tried a variety of different networking configs on the domU to
try to get it not to check the tcp checksums (since the dom0
apparently insists on not generating them correctly), without any
success. I _am_ able to ssh from another machine on my network to
domU via the routing in dom0, showing that tcp checksums are at least
being generated correctly in one direction.
Ideally I would like to COMPLETELY DISABLE this fragile optimisation.
Is there a way to do that ?
Failing that I need to either (a) persuade dom0 to generate proper
checksums on packets leaving for domU via vif*, or (b) persuade domU
to accept broken checksums but only on some packets (the ones from
dom0 itself rather than routed via dom0).
Versions:
I'm using the Debian Xen packages from Ralph Passgang (3.0.1-0tha3)
locally compiled on sarge but without patches. Both host and guest
are running the same 2.6.12, which is vanilla except for the Xen
patches.
The host is Debian sarge; the guest is Ubuntu dapper (constructed with
pbuilder/debootstrap and some home-grown scripts I'm working on).
Ian.
dom0:
lalonde:~# tcpdump -vvs500 -lnivif31.0
tcpdump: listening on vif31.0, link-type EN10MB (Ethernet), capture size 500
bytes
17:56:39.806453 IP (tos 0x0, ttl 64, id 19458, offset 0, flags [DF], length:
60) 172.18.45.97.37227 > 172.18.45.65.22: S [tcp sum ok]
1738938563:1738938563(0) win 5840 <mss 1460,sackOK,timestamp 31872193
0,nop,wscale 2>
17:56:39.807082 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], length: 60)
172.18.45.65.22 > 172.18.45.97.37227: S [tcp sum ok] 1245122588:1245122588(0)
ack 1738938564 win 5792 <mss 1460,sackOK,timestamp 273577 31872193,nop,wscale 2>
17:56:39.807114 IP (tos 0x0, ttl 64, id 19460, offset 0, flags [DF], length:
52) 172.18.45.97.37227 > 172.18.45.65.22: . [tcp sum ok] 1:1(0) ack 1 win 1460
<nop,nop,timestamp 31872193 273577>
17:56:39.823210 IP (tos 0x0, ttl 64, id 15543, offset 0, flags [DF], length:
90) 172.18.45.65.22 > 172.18.45.97.37227: P [tcp sum ok] 1:39(38) ack 1 win
1448 <nop,nop,timestamp 273578 31872193>
17:56:39.823483 IP (tos 0x0, ttl 64, id 19462, offset 0, flags [DF], length:
52) 172.18.45.97.37227 > 172.18.45.65.22: . [tcp sum ok] 1:1(0) ack 39 win 1460
<nop,nop,timestamp 31872195 273578>
17:56:39.824197 IP (tos 0x0, ttl 64, id 19464, offset 0, flags [DF], length:
93) 172.18.45.97.37227 > 172.18.45.65.22: P [bad tcp cksum b316 (->4d05)!]
1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872195 273578>
17:56:40.028407 IP (tos 0x0, ttl 64, id 19466, offset 0, flags [DF], length:
93) 172.18.45.97.37227 > 172.18.45.65.22: P [bad tcp cksum b316 (->4cf0)!]
1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872216 273578>
17:56:40.448393 IP (tos 0x0, ttl 64, id 19468, offset 0, flags [DF], length:
93) 172.18.45.97.37227 > 172.18.45.65.22: P [bad tcp cksum b316 (->4cc6)!]
1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872258 273578>
17:56:41.288386 IP (tos 0x0, ttl 64, id 19470, offset 0, flags [DF], length:
93) 172.18.45.97.37227 > 172.18.45.65.22: P [bad tcp cksum b316 (->4c72)!]
1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872342 273578>
17:56:42.968366 IP (tos 0x0, ttl 64, id 19472, offset 0, flags [DF], length:
93) 172.18.45.97.37227 > 172.18.45.65.22: P [bad tcp cksum b316 (->4bca)!]
1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872510 273578>
10 packets captured
10 packets received by filter
0 packets dropped by kernel
lalonde:~# ifconfig
eth0 Link encap:Ethernet HWaddr 00:13:20:21:DF:C1
inet addr:172.18.45.97 Bcast:172.18.45.255 Mask:255.255.255.0
inet6 addr: fe80::213:20ff:fe21:dfc1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:694410 errors:148 dropped:0 overruns:0 frame:148
TX packets:478168 errors:1 dropped:0 overruns:0 carrier:1
collisions:80299 txqueuelen:1000
RX bytes:574281356 (547.6 MiB) TX bytes:45053982 (42.9 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:7 errors:0 dropped:0 overruns:0 frame:0
TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1248 (1.2 KiB) TX bytes:1248 (1.2 KiB)
vif31.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet addr:172.18.45.97 Bcast:172.18.45.97 Mask:255.255.255.255
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:68 errors:0 dropped:0 overruns:0 frame:0
TX packets:96 errors:0 dropped:5 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4487 (4.3 KiB) TX bytes:14813 (14.4 KiB)
lalonde:~# route -vn
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.18.45.65 0.0.0.0 255.255.255.255 UH 0 0 0 vif31.0
172.18.45.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
0.0.0.0 172.18.45.11 0.0.0.0 UG 0 0 0 eth0
lalonde:~# brctl show
bridge name bridge id STP enabled interfaces
lalonde:~# iptables -L -v -n
Chain INPUT (policy ACCEPT 689K packets, 562M bytes)
pkts bytes target prot opt in out source destination
14 976 AdtXenIn all -- vif31.0 * 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT 1434 packets, 2046K bytes)
pkts bytes target prot opt in out source destination
16 1159 AdtXenIn all -- vif31.0 * 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 477K packets, 38M bytes)
pkts bytes target prot opt in out source destination
Chain AdtXenIn (2 references)
pkts bytes target prot opt in out source destination
740 40461 ACCEPT tcp -- * * 0.0.0.0/0 172.18.45.6
tcp dpt:80
0 0 ACCEPT tcp -- * * 0.0.0.0/0 172.18.45.6
tcp dpt:53
10 730 ACCEPT udp -- * * 0.0.0.0/0 172.18.45.6
udp dpt:53
16 1236 ACCEPT icmp -- * * 0.0.0.0/0 172.18.45.6
49 3284 ACCEPT tcp -- * * 0.0.0.0/0
172.18.45.97 tcp flags:!0x16/0x02
5 420 ACCEPT icmp -- * * 0.0.0.0/0
172.18.45.97
9 2090 ACCEPT tcp -- * * 0.0.0.0/0 172.18.45.6
tcp flags:!0x16/0x02
0 0 ACCEPT icmp -- * * 0.0.0.0/0 172.18.45.6
0 0 REJECT all -- * * 0.0.0.0/0
192.168.0.0/24 reject-with icmp-net-prohibited
14 1176 REJECT all -- * * 0.0.0.0/0
172.16.0.0/12 reject-with icmp-net-prohibited
0 0 REJECT all -- * * 0.0.0.0/0 10.0.0.0/8
reject-with icmp-net-prohibited
0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0
tcp dpt:80
0 0 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0
8 608 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0
reject-with icmp-admin-prohibited
lalonde:~# egrep . /proc/sys/net/ipv4/conf/*/proxy_arp
/proc/sys/net/ipv4/conf/all/proxy_arp:0
/proc/sys/net/ipv4/conf/default/proxy_arp:0
/proc/sys/net/ipv4/conf/eth0/proxy_arp:1
/proc/sys/net/ipv4/conf/lo/proxy_arp:0
/proc/sys/net/ipv4/conf/vif31.0/proxy_arp:1
lalonde:~# ethtool -k eth0
Offload parameters for eth0:
Cannot get device rx csum settings: Operation not supported
Cannot get device tx csum settings: Operation not supported
Cannot get device scatter-gather settings: Operation not supported
Cannot get device tcp segmentation offload settings: Operation not supported
no offload info available
lalonde:~#
And the command in dom0 I'm using to test:
lalonde:~# ssh -v root@xxxxxxxxxxxx
OpenSSH_3.8.1p1 Debian-8.sarge.4, OpenSSL 0.9.7e 25 Oct 2004
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to 172.18.45.65 [172.18.45.65] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/identity type 0
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_dsa type 2
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.2p1
Debian-7ubuntu1
debug1: match: OpenSSH_4.2p1 Debian-7ubuntu1 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_3.8.1p1 Debian-8.sarge.4
debug1: SSH2_MSG_KEXINIT sent
lalonde:~#
domU:
root@lalonde:~# tcpdump -vvs500 -lnieth0
device eth0 entered promiscuous mode
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 500 bytes
17:56:44.217660 IP (tos 0x0, ttl 64, id 19458, offset 0, flags [DF], proto:
TCP (6), length: 60) 172.18.45.97.37227 > 172.18.45.65.22: S, cksum 0x15dc
(correct), 1738938563:1738938563(0) win 5840 <mss 1460,sackOK,timestamp
31872193 0,nop,wscale 2>
17:56:44.218441 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto: TCP
(6), length: 60) 172.18.45.65.22 > 172.18.45.97.37227: S, cksum 0x8efa
(correct), 1245122588:1245122588(0) ack 1738938564 win 5792 <mss
1460,sackOK,timestamp 273577 31872193,nop,wscale 2>
17:56:44.219100 IP (tos 0x0, ttl 64, id 19460, offset 0, flags [DF], proto:
TCP (6), length: 52) 172.18.45.97.37227 > 172.18.45.65.22: ., cksum 0xcead
(correct), 1:1(0) ack 1 win 1460 <nop,nop,timestamp 31872193 273577>
17:56:44.231565 IP (tos 0x0, ttl 64, id 15543, offset 0, flags [DF], proto:
TCP (6), length: 90) 172.18.45.65.22 > 172.18.45.97.37227: P, cksum 0x229a
(correct), 1:39(38) ack 1 win 1448 <nop,nop,timestamp 273578 31872193>
17:56:44.233412 IP (tos 0x0, ttl 64, id 19462, offset 0, flags [DF], proto:
TCP (6), length: 52) 172.18.45.97.37227 > 172.18.45.65.22: ., cksum 0xce84
(correct), 1:1(0) ack 39 win 1460 <nop,nop,timestamp 31872195 273578>
17:56:44.233426 IP (tos 0x0, ttl 64, id 19464, offset 0, flags [DF], proto:
TCP (6), length: 93) 172.18.45.97.37227 > 172.18.45.65.22: P, cksum 0xb316
(incorrect (-> 0x4d05), 1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872195
273578>
17:56:44.437078 IP (tos 0x0, ttl 64, id 19466, offset 0, flags [DF], proto:
TCP (6), length: 93) 172.18.45.97.37227 > 172.18.45.65.22: P, cksum 0xb316
(incorrect (-> 0x4cf0), 1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872216
273578>
17:56:44.857091 IP (tos 0x0, ttl 64, id 19468, offset 0, flags [DF], proto:
TCP (6), length: 93) 172.18.45.97.37227 > 172.18.45.65.22: P, cksum 0xb316
(incorrect (-> 0x4cc6), 1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872258
273578>
17:56:45.697115 IP (tos 0x0, ttl 64, id 19470, offset 0, flags [DF], proto:
TCP (6), length: 93) 172.18.45.97.37227 > 172.18.45.65.22: P, cksum 0xb316
(incorrect (-> 0x4c72), 1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872342
273578>
17:56:47.377132 IP (tos 0x0, ttl 64, id 19472, offset 0, flags [DF], proto:
TCP (6), length: 93) 172.18.45.97.37227 > 172.18.45.65.22: P, cksum 0xb316
(incorrect (-> 0x4bca), 1:42(41) ack 39 win 1460 <nop,nop,timestamp 31872510
273578>
10 packets captured
20 packets received by filter
0 packets dropped by kernel
device eth0 left promiscuous mode
root@lalonde:~# route -vn
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.18.45.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
0.0.0.0 172.18.45.97 0.0.0.0 UG 0 0 0 eth0
root@lalonde:~# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:16:3E:7C:AA:7F
inet addr:172.18.45.65 Bcast:172.18.45.255 Mask:255.255.255.0
inet6 addr: fe80::216:3eff:fe7c:aa7f/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:90 errors:0 dropped:0 overruns:0 frame:0
TX packets:52 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:15213 (14.8 KiB) TX bytes:3332 (3.2 KiB)
root@lalonde:~# ethtool -k eth0
Offload parameters for eth0:
Cannot get device rx csum settings: Operation not supported
Cannot get device scatter-gather settings: Operation not supported
Cannot get device tcp segmentation offload settings: Operation not supported
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp segmentation offload: off
root@lalonde:~#
And just to prove it works:
-davenant:~> traceroute -n 172.18.45.65
traceroute to 172.18.45.65 (172.18.45.65), 30 hops max, 38 byte packets
1 172.18.45.97 0.334 ms 0.378 ms 0.214 ms
2 172.18.45.65 0.430 ms 0.258 ms 0.240 ms
-davenant:~> ssh -v root@xxxxxxxxxxxx
OpenSSH_3.8.1p1 Debian-8.sarge.4, OpenSSL 0.9.7e 25 Oct 2004
debug1: Reading configuration data /u/ian/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 23: Deprecated option "RhostsAuthentication"
debug1: Connecting to 172.18.45.65 [172.18.45.65] port 22.
debug1: Connection established.
debug1: identity file /u/ian/.ssh/identity type 0
debug1: identity file /u/ian/.ssh/id_rsa type -1
debug1: identity file /u/ian/.ssh/id_dsa type 2
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.2p1
Debian-7ubuntu1
debug1: match: OpenSSH_4.2p1 Debian-7ubuntu1 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_3.8.1p1 Debian-8.sarge.4
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-cbc hmac-md5 none
debug1: kex: client->server aes128-cbc hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
The authenticity of host '172.18.45.65 (172.18.45.65)' can't be established.
RSA key fingerprint is 78:9f:f9:40:72:4a:3b:66:33:0f:e1:4a:3b:1f:e3:7d.
Are you sure you want to continue connecting (yes/no)?
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home