Nugraha wrote on Tue, 28 Apr 2009 19:58:26 +0700:
> Seriously?
Yes. I would love to give you all the details. The problem is that what
I'm working with is not the final setup but some "semblance". I had to
improvise to get something that *should* behave similar than the final
setup.
The final setup will be machine a4 with three NICs attached to two other
machines (say a1 and a2, both with more than one NIC) via crossover cable
and a4 pulling from the domUs on a1 and a2 data. I want to go thru the
crossover link and not the switch, via a private network where all
additional NICs and the domUs are on.
The problem is that a1 and a2 are not here in the office, but in
production in the datacenter. So, I have a4 for testing and maybe can
attach another box if need be. But most boxes I have here are in use and
have only one NIC. So I can use them only for a short time and have to
reconfigure them each time. I did this once and tested from another box
(name it a3) to domUs on a4 with the network setup I mentioned in my first
posting, so everythign done by the standard network-bridge script. I was
happy with that as long as I didn't reboot and noticed the initial running
of network-bridge breaks the eth0/peth0 bridge.
Now, as I cannot always attach another machine for testing (and I don't
have one with two NICs anyway) my base assumption is that what works from
a3 -> a4 -> domU should also work without a3. e.g. I don't ping from a3
but from a4 dom0 -> a4 domUs, but using the interface/IP number that is on
the private subnet ("ping 192.168.2.10" for instance, or "ping -I xenbr1
192.168.2.10" which would ping from the xenbr1 interface which is bridging
to eth1, where the external machine comes in).
I'm using this simplified test scenario for testing. ping the private IP
on domU from dom0. If it works, fine. If it doesn't, use -I xenbr1 for the
ping. If that works, fine, but not so comfortable. If that doesn't work:
bad :-(
Does this make sense so far?
Now I created two bridges as you suggested on a4 and two NICs on the dom0.
That works but only in one way (dom0 -> domU) and only if I attach the
ping to xenbr1. I just played around with routes and have found that
adding a static route (on dom0) solves this so I can ping from either side
and without specifying an interface. This solution is much more
comfortable. The private subnet in question is 192.168.2.0/27.
The output of this setup is now as follows:
dom0:
brctl show
bridge name bridge id STP enabled interfaces
xenbr0 8000.001ec9fefbab no eth0
vif14.0
xenbr1 8000.001ec9fefbac no eth1
vif14.1
ip addr list | grep "inet "
inet 127.0.0.1/8 scope host lo
inet 192.168.2.4/27 brd 192.168.2.31 scope global eth2
inet 192.168.1.24/24 brd 192.168.1.255 scope global xenbr0
inet 192.168.2.3/27 brd 192.168.2.31 scope global xenbr1
ip route
192.168.2.10 via 192.168.2.3 dev xenbr1 scope link
192.168.2.0/27 dev eth2 proto kernel scope link src 192.168.2.4
192.168.2.0/27 dev xenbr1 proto kernel scope link src 192.168.2.3
192.168.1.0/24 dev xenbr0 proto kernel scope link src 192.168.1.24
default via 192.168.1.1 dev xenbr0
domU is straight forward:
no bridge
ip addr list | grep "inet "
inet 127.0.0.1/8 scope host lo
inet 212.202.99.237/28 brd 212.202.99.239 scope global eth0
inet 192.168.1.237/24 brd 192.168.1.255 scope global eth0:1
inet 192.168.2.10/27 brd 192.168.2.31 scope global eth1
ip route
212.202.99.224/28 dev eth0 proto kernel scope link src 212.202.99.237
192.168.2.0/27 dev eth1 proto kernel scope link src 192.168.2.10
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.237
default via 192.168.1.1 dev eth0 src 192.168.1.237
default via 212.202.99.225 dev eth0
I had forwarding enabled in the meantime, but it works without it as well,
so I disabled that again.
So, just to make it clear, *this* setup with the additional route is
working now in all directions. I now realize that my best option is
probably to use a different subnet each for eth1 and eth2 and different
subnets on the two machines it goes out to (remember a4 will not be the
target, but the source). If I do that I can use a route for each subnet,
otherwise I have to use a route per single IP address.
Oh, and I just find that using different nets for eth1 and eth2 solves the
problem, anyway, without a static route. Like so:
ip route
192.168.3.0/27 dev eth2 proto kernel scope link src 192.168.3.1
192.168.2.0/27 dev xenbr1 proto kernel scope link src 192.168.2.3
192.168.1.0/24 dev xenbr0 proto kernel scope link src 192.168.1.24
default via 192.168.1.1 dev xenbr0
The big problem obviously was eth2 being on the same subnet as eth1/xenbr1
and thus catching the packets to nowhere.
Good. I had just tried to add the static route with sysconfig/route-xenbr1
and all variations I tried failed with obscure errors. I'm going to
follow-up on this on the CentOS list.
Thanks for your answers, they were all really very helpful!
I think I have found a solution now, although I still would like to know
why the original method works on some machines but not this one.
Interestingly, one of the errors I got when trying to use route-xenbr1 was
the same as when using the original setup and the network-bridge script.
If you are still interested in the original data I can revert this machine
to the original setup and grab the data from there or from the other
machine I mentioned where it works fine.
Kai
--
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home