[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] lots of dropped packets on vif interfaces



Xen Users,

We recently attempted an upgrade only to have to roll back because of very unusual network problems. Our software is supposed to setup 10 TCP connections and maintain them yet when I looked with a "watch netstat" showed the connections constantly being dropped and re-established. Looking at the dom0 I see lots and lots of dropped packets. Does anyone know under what conditions a vif interface would drop packets?

We started the upgrade with jumbo frames but fearing that this was the issue we rolled them back. They were implemented by having a modified vif-bridge script

73a78,79
>       do_without_error ifconfig "$vif" mtu 9000
>       (sleep 10; ifconfig "$vif" mtu 9000) &
74a81
>

and having a script which went through and periodically set the vif interfaces to mtu 9000. We changed the MTU's back to 1500 and disabled the periodic but still we saw lots of dropped TCP connections. Unfortunately it was only post-upgrade that we noticed the dropped packets on the dom0.

CentOS 5.3, Xen 3.03, Kernel 2.6.18-128.2.1.el5xen

ifconfig output:
"""
bond0     Link encap:Ethernet  HWaddr 00:15:17:90:55:A0 
          inet addr:192.168.12.11  Bcast:192.168.255.255  Mask:255.255.0.0
          inet6 addr: fe80::215:17ff:fe90:55a0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4127440 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2011946 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:3137102607 (2.9 GiB)  TX bytes:4043618659 (3.7 GiB)

eth0      Link encap:Ethernet  HWaddr 00:30:48:C5:5A:CE 
          inet addr:208.85.150.68  Bcast:208.85.150.127  Mask:255.255.255.192
          inet6 addr: fe80::230:48ff:fec5:5ace/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:40971777 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4868631 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:8325395107 (7.7 GiB)  TX bytes:1774099435 (1.6 GiB)

eth1      Link encap:Ethernet  HWaddr 00:30:48:C5:5A:CF 
          inet addr:10.12.1.11  Bcast:10.12.255.255  Mask:255.255.0.0
          inet6 addr: fe80::230:48ff:fec5:5acf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4811203 errors:0 dropped:0 overruns:0 frame:0
          TX packets:105483 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1893191654 (1.7 GiB)  TX bytes:19080833 (18.1 MiB)

eth2      Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:109015823 errors:0 dropped:0 overruns:0 frame:0
          TX packets:61527076 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:124155590545 (115.6 GiB)  TX bytes:7949776919 (7.4 GiB)
          Memory:da320000-da340000

eth3      Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:87295647 errors:0 dropped:0 overruns:0 frame:0
          TX packets:208743472 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:79482318567 (74.0 GiB)  TX bytes:131665787763 (122.6 GiB)
          Memory:da360000-da380000

eth4      Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:368798651 errors:0 dropped:0 overruns:0 frame:0
          TX packets:77141330 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:358699531637 (334.0 GiB)  TX bytes:11545699506 (10.7 GiB)
          Memory:da420000-da440000

eth5      Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:79557756 errors:0 dropped:0 overruns:0 frame:0
          TX packets:219450792 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:94309068673 (87.8 GiB)  TX bytes:251020520979 (233.7 GiB)
          Memory:da460000-da480000

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:10950 errors:0 dropped:0 overruns:0 frame:0
          TX packets:10950 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1322596 (1.2 MiB)  TX bytes:1322596 (1.2 MiB)

pbond0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP MASTER  MTU:1500  Metric:1
          RX packets:644667877 errors:0 dropped:0 overruns:0 frame:0
          TX packets:566862670 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:656646509422 (611.5 GiB)  TX bytes:402181785167 (374.5 GiB)

peth0     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:41399478 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5177320 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:8392686228 (7.8 GiB)  TX bytes:1794386410 (1.6 GiB)
          Memory:da020000-da040000

peth1     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:2471814 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3417446 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:362232436 (345.4 MiB)  TX bytes:3259230903 (3.0 GiB)
          Memory:da060000-da080000

vif0.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:4868631 errors:0 dropped:0 overruns:0 frame:0
          TX packets:40971795 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1774099435 (1.6 GiB)  TX bytes:8325397390 (7.7 GiB)

vif0.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:2011948 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4127418 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:4043620328 (3.7 GiB)  TX bytes:3137070612 (2.9 GiB)

vif0.2    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:105483 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4811203 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:19080833 (18.1 MiB)  TX bytes:1893191654 (1.7 GiB)

vif15.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16167343 errors:0 dropped:2605 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:496 (496.0 b)  TX bytes:3048611500 (2.8 GiB)

vif15.1   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:25545589 errors:0 dropped:0 overruns:0 frame:0
          TX packets:86572687 errors:0 dropped:14971 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:218968845538 (203.9 GiB)  TX bytes:12499004215 (11.6 GiB)

vif15.2   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:90628 errors:0 dropped:0 overruns:0 frame:0
          TX packets:148849 errors:0 dropped:582 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:43151873 (41.1 MiB)  TX bytes:19783310 (18.8 MiB)

vif41.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:67 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15133388 errors:0 dropped:2917 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:2092 (2.0 KiB)  TX bytes:2784566746 (2.5 GiB)

vif41.1   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:18750910 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5144760 errors:0 dropped:299090 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:2825180056 (2.6 GiB)  TX bytes:2525900266 (2.3 GiB)

vif41.2   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:152588 errors:0 dropped:0 overruns:0 frame:0
          TX packets:72101 errors:0 dropped:1194 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:88969564 (84.8 MiB)  TX bytes:9209610 (8.7 MiB)

vif43.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:30 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14733600 errors:0 dropped:1573 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:1056 (1.0 KiB)  TX bytes:2709705832 (2.5 GiB)

vif43.1   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:24540834 errors:0 dropped:0 overruns:0 frame:0
          TX packets:36136102 errors:0 dropped:11070 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:68309891084 (63.6 GiB)  TX bytes:4202123253 (3.9 GiB)

vif43.2   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:566939 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1134448 errors:0 dropped:579 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:2593043740 (2.4 GiB)  TX bytes:84875829 (80.9 MiB)

vif45.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12104256 errors:0 dropped:2097 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:496 (496.0 b)  TX bytes:2217362387 (2.0 GiB)

vif45.1   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:11496076 errors:0 dropped:0 overruns:0 frame:0
          TX packets:21208103 errors:0 dropped:76369 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:826925754 (788.6 MiB)  TX bytes:28052880670 (26.1 GiB)

vif45.2   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:36670 errors:0 dropped:0 overruns:0 frame:0
          TX packets:53919 errors:0 dropped:1042 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:3744887 (3.5 MiB)  TX bytes:5659141 (5.3 MiB)

vif46.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12046566 errors:0 dropped:1895 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:496 (496.0 b)  TX bytes:2206365310 (2.0 GiB)

vif46.1   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:15506324 errors:0 dropped:0 overruns:0 frame:0
          TX packets:24790384 errors:0 dropped:40576 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:1061414707 (1012.2 MiB)  TX bytes:27871247434 (25.9 GiB)

vif46.2   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:36491 errors:0 dropped:0 overruns:0 frame:0
          TX packets:52028 errors:0 dropped:1016 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:3696538 (3.5 MiB)  TX bytes:5193571 (4.9 MiB)

vif47.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11998949 errors:0 dropped:1581 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:496 (496.0 b)  TX bytes:2197379798 (2.0 GiB)

vif47.1   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:3120044 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4318924 errors:0 dropped:64900 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:220041974 (209.8 MiB)  TX bytes:4477173983 (4.1 GiB)

vif47.2   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:53782 errors:0 dropped:0 overruns:0 frame:0
          TX packets:67304 errors:0 dropped:564 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:5578127 (5.3 MiB)  TX bytes:6662880 (6.3 MiB)

vif48.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:10 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11977782 errors:0 dropped:1645 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:496 (496.0 b)  TX bytes:2193205238 (2.0 GiB)

vif48.1   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:58647 errors:0 dropped:0 overruns:0 frame:0
          TX packets:161069 errors:0 dropped:47969 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:11373930 (10.8 MiB)  TX bytes:78154482 (74.5 MiB)

vif48.2   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:38945 errors:0 dropped:0 overruns:0 frame:0
          TX packets:53491 errors:0 dropped:1092 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:4114153 (3.9 MiB)  TX bytes:5008162 (4.7 MiB)

virbr0    Link encap:Ethernet  HWaddr 00:00:00:00:00:00 
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:32 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:7273 (7.1 KiB)

xenbr0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:146301 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:7282456 (6.9 MiB)  TX bytes:0 (0.0 b)

xenbr1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:248368 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:147614308 (140.7 MiB)  TX bytes:0 (0.0 b)

xenbr2    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF 
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:37916 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2005907 (1.9 MiB)  TX bytes:0 (0.0 b)
"""

Joel
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.