WARNING - OLD ARCHIVES

This is an archived copy of the Xen.org mailing list, which we have preserved to ensure that existing links to archives are not broken. The live archive, which contains the latest emails, can be found at http://lists.xen.org/
   
 
 
Xen 
 
Home Products Support Community News
 
   
 

xen-devel

Re: [Xen-devel] pv_ops routing problem?

On Wed, Apr 28, 2010 at 03:06:25PM -0400, Michael D Labriola wrote:
> xen-devel-bounces@xxxxxxxxxxxxxxxxxxx wrote on 04/28/2010 09:21:50 AM:
> 
> *snip*>
> > > > > On 04/27/2010 02:22 PM, Michael D Labriola wrote:
> > > > > > Ok, I've got a problem.  My current setup is as follows:
> > > > > >
> > > > > > I have a pv_ops dom0 (xen/master from a few weeks ago) running 3 
> 
> > > > domUs. 
> > > > > > Each domU has an eth0 interface and one or more VLAN interfaces.
> > > > > >
> > > > > > vm1, vm2, and vm3 are all domUs running with the same pv_ops 
> > kernel 
> > > as 
> > > > 
> > > > > > dom0.  vm1's eth0.107 interface routes through vm3 to get to 
> vm2's 
> > 
> > > > > > eth0.1020 interface.
> > > > > >
> > > > > > dom0:
> > > > > >   eth0:     192.168.1.20/24
> > > > > >
> > > > > > vm1:
> > > > > >   eth0:     192.168.1.10/24
> > > > > >   eth0.107: 10.7.1.1/16
> > > > > >   gateway:  10.7.0.1
> > > > > >
> > > > > > vm2:
> > > > > >   eth0:      192.168.1.11/24
> > > > > >   eth0.1020: 10.20.1.1/16
> > > > > >   gateway:   10.20.0.1
> > > > > >
> > > > > > vm3:
> > > > > >   eth0:      192.168.1.12/24
> > > > > >   eth0.107:  10.7.0.1/16
> > > > > >   eth0.1020: 10.20.0.1/16
> > > > > >
> > > > > >
> > > > > > >From vm1, I can ping the gateway (10.7.0.1) and vm3's 1020 
> > > interface 
> > > > > > (10.20.0.1), but get no response if I try to ping vm2's 
> 10.20.1.1.
> > > > > >
> > > > > > vm3 can ping both 10.7.1.1 and 10.20.1.1.
> > > > > >
> > > > > > Should be a pretty simple packet forwarding example, and it used 
> 
> > to 
> > > > work 
> > > > > > using the old xen-linux 2.6.18 kernel we were using on our 
> > deployed 
> > > > > > systems.  But now it's broken...
> > > > > >
> > > > > > I have verified that the exact same setup running on 3 physical 
> > PCs 
> > > > > > bare-metal works w/out any problems.  I had the problem 
> initially 
> > on 
> > > 
> > > > Xen 
> > > > > > 3.4.2, but just upgraded to 4.0.0 and have the exact same 
> > symptoms.
> > > > > > 
> > > > > 
> > > > > Have you tried turning off tx checksum offload?  (ethtool -K <dev> 
> 
> > tx 
> > > > off)
> > > > > 
> > > > > Though for me that just fixes network access between dom0 and 
> domU;
> > > > > external traffic normally works fine.
> > > > 
> > > > Jeremy, I haven't tried that yet.  I'll try it this morning, 
> although 
> > I 
> > > > don't really expect that to fix the problem.  I'm also going to pull 
> 
> > > from 
> > > > the xen/master branch again to see if that makes a difference.  My 
> > last 
> > > > 2.6.31.12 kernel was pulled from your tree a few weeks ago.
> > > > 
> > > > By the way, the packet forwarding is definitely happening on my 
> router 
> > 
> > > > domU.  If I look at vm3's eth0 w/ wireshark, I can see the VLAN 
> > > > encapsulated ICMP echo request coming in on the 107 VLAN and I can 
> see 
> > 
> > > > that vm3 is forwarding it out over its 1020 VLAN interface. However, 
> 
> > if 
> > > I 
> > > > look at my dom0's vif3.0 interface with wireshark I only see the 
> VLAN 
> > > 107 
> > > > traffic...  no sign of the 1020 forwarded packet anywhere.  It's as 
> if 
> > 
> > > > forwarded packets aren't finding their way onto dom0's vif3.0.  I've 
> 
> > > never 
> > > > seen this before, and was under the impression that dom0's vif 
> > > interfaces 
> > > > _WERE_ the domU's interfaces just with a different name... 
> > > 
> > > Should the tx checksum offload only be disabled on dom0?  And does it 
> > have 
> > > to get disabled before xend starts?  I've tried disabling it on just 
> > dom0 
> > > and also on domUs with no difference.
> > > 
> > > Also, I updated to the current xen/master and it behaves the same.
> > > 
> > > I just tried adding 107 and 1020 VLAN interfaces to my dom0 (10.7.0.2 
> > and 
> > > 10.20.0.2) and it turns out that dom0 also cannot ping the VLAN 
> > interfaces 
> > > on any of the domUs...  not sure if that's a separate issue or if it's 
> 
> > all 
> > > related...
> > 
> > Once I lowered and re-raised dom0's VLAN interfaces dom0 could ping all 
> > the domU's VLAN interfaces just fine.  Not sure why I had to do that, 
> > though...
> > 
> > Still can't ping from vm1 to vm2 using v3 as the default gateway, 
> though.
> > 
> > I've also tried this now on xen/stable with the exact same results.
> > 
> > -Mike
> > 
> > PS - Jeremy, when I pulled from you this morning xen/stable was actually 
> 
> > one commit ahead of xen/stable-2.6.32.x... not sure if that was 
> > intentional.  ;-)
> 
> Can anyone confirm or deny that routing between VLAN interfaces on domUs 
> hosted on a xen/master dom0 is indeed broken?  I'm not sure what else to 
> do to troubleshoot this and it's behaving like this on all my hardware.

Adnan, is this the same problem as with the TX checksum issue?

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

<Prev in Thread] Current Thread [Next in Thread>