Hi Adam,
Adam Tworkowski wrote:
> Hi,
>
> I seem to have made some progress. I add the ifconfig statements for
> the bridges in /etc/network/interfaces and they come up automatically.
>
Yes, but your dummies are not being added to the bridges with your config.
> I discovered a typo in my xen firewall config -- I was using the plural
> "vifs" instead of "vif".
>
> ifconfig is now reporting all bridges and vifs is per Marcus' example.
> route looks similar is well.
>
> brctl reports multiple vifs bound to the related bridges only when I
> remove the "backend=fw" statemnent from the vif parameter in the domU1
> xen config.
Is your firewall kernel configured for network backend?
>
> Should I be attempting to use the "backend" parameter?
I have used the dummy+bridge system in dom0 because when using
the backend from the Firewall to domUs a new vif with the next
sequential number is created at the firewall. I found that
Shorewall didn't like this when starting & stopping domUs,
as it will class the new interfaces as undefined.
If, however, you attach the firewall to a dummy+bridge dmz
exported from dom0 you can achieve much the same thing just
by not assigning an ip at the dom0 end.
Then, each domU can attach itself to the bridge as required
without creating new interfaces, and dom0 is isolated from
the dmz.
You either need to use the dummy+bridge method OR the network
backend to achieve your design. Not both.
DUMMY + BRIDGE METHOD:
+------------------------+
| +---------------+ |
Internet -------+ Firewall domU +--------- Intranet
| +------+--------+ |
| | |
| | DMZ |
| +-----+------+ |
| | : | |
| +--+--+ : +--+--+ |
| |domU1| : |domU2| |
| +-----+ : +-----+ |
| : |
| +-----+------+ |
| | dom0 | |
+----+------------+------+
NETWORK BACKEND METHOD:
+------------------------+
| physical machine, dom0 |
| +---------------+ |
Internet -------+ Firewall domU +--------- Intranet
| +------+--------+ |
| | |
| | DMZ |
| +-----+------+ |
| | | |
| +--+--+ +--+--+ |
| |domU1| |domU2| |
| +-----+ +-----+ |
+------------------------+
I'd have to think about how to automatically add
new vifs to a bridge using this method, which
may help my shorewall problem, but it will be a
while before I visit this issue again as I prefer
the first method.
>
> I still cannot "see" the firewall from domu1.
>
> Should the vifs show up in with arp? The fw (domU) reports its first
> hop in arp but does not report domU1.
>
> Looking at your brctl show ... I suspect I have an issue here:
>
> br1 vif1.2
> vif3.0
> br2 vif1.2
> br3 vif1.3
> xen-br0 dummy0
> vif1.0
>
> (note that I omitted the br ids /stp info.)
Right, so I'd guess that your firewall has something weird going on here.
- You have two vif1.2 interfaces?
- The only bridge that will work is xen-br0. br1-3 have no interface attached.
- Try starting your domU on xen-br0 rather than br1 and see if you've
got everything working. You can always move it to another bridge
later.
>
> I see that your bridges all have dummy interfaces reported as attached.
> As well you have multiples vifs attached to your xen-br0. Any ideas?
The dummy interface provides a network device for the bridge to attach to.
When a new domU is created, a vif is attached to the bridge specified in
the domU config.
Marcus.
>
> I will have to throw another NIC in this machine so I can actually get
> to this machine and provide copy and past logs, etc.
>
> Thanks
>
> Adam
>
> On Mon, 2005-08-29 at 18:15 +1000, Marcus Brown wrote:
>
>>Hi Adam,
>>I've added some of my configs and thoughts below:
>>
>>Adam Tworkowski wrote:
>>
>>>Hi Marcus,
>>>
>>>I have the dummy module loaded on dom0 and several dummy interfaces
>>>which show up with ifconfig. They are configured as follows:
>>>
>>>auto dummy0
>>>iface dummy0 inet static
>>> address 192.168.254.1
>>> netmask 255.255.255.248
>>> network 192.168.254.0
>>> broadcast 192.168.254.7
>>> gateway 192.168.254.6
>>> post-up brctl addbr xen-br0 || true
>>> post-up brctl addif xen-br0 dummy0 || true
>>> post-up brctl delif xen-br0 dummy0 || true
>>>
>>>Additional bridges configured in same manner.
>>>
>>>Without intervention dom0 boots with 4 dummy interfaces
>>>(dummy0,dummy1,etc.) and xen-br0. This is apparent from ifconfig.
>>>The additional bridges, however, do not load w/o intervention. I have
>>>to use the brctl command as per the above /etc/network/interfaces.
>>>
>>>In /etc/xen/scripts/network I have the bridges mapped to the dummy
>>>devices as so:
>>>
>>>bridge=${bridge:-xen-br0}
>>>netdev=${netdev:-dummy0}
>>>bridge=${bridge:-xen-br1}
>>>netdev=${netdev:-dummy1}
>>>...
>>>
>>
>>As xen brings up xen-br0 automatically, I haven't used post hooks for
>>dummy0:
>>/etc/network/interfaces includes:
>> auto dummy0
>> iface dummy0 inet static
>> address 192.168.254.1
>> netmask 255.255.255.248
>> network 192.168.254.0
>> broadcast 192.168.254.7
>> gateway 192.168.254.6
>> dns-nameservers 192.168.254.6
>>
>> auto dummy1
>> iface dummy1 inet static
>> address 192.168.254.9
>> netmask 255.255.255.248
>> network 192.168.254.8
>> broadcast 192.168.254.15
>> post-up brctl addbr br1 || true
>> post-up brctl addif br1 dummy1 || true
>> post-up ifconfig br1 192.168.254.9/29
>> post-down brctl delif br1 dummy1
>>
>> auto dummy2
>> iface dummy2 inet static
>> address 192.168.254.17
>> netmask 255.255.255.248
>> network 192.168.254.16
>> broadcast 192.168.254.23
>> post-up brctl addbr br2 || true
>> post-up brctl addif br2 dummy2 || true
>> post-up ifconfig br2 192.168.254.17/29
>> post-down brctl delif br2 dummy2
>>
>> ... and so on
>>
>>/etc/xen/scripts/network includes:
>> bridge=${bridge:-xen-br0}
>> netdev=${netdev:-dummy0}
>> antispoof=${antispoof:-yes}
>>
>>I'm not sure if multiple bridge & netdev statements work here,
>>but nice if it does.
>>
>>BTW:
>>The post hook configuring the ip of the bridge is wrong, as this
>>method only works if the bridge already has an ip.
>>It should read:
>> post-up ifconfig br2 192.168.254.17 netmask 255.255.255.248
>>Also, I don't think it's necessary to assign ip to the dummy, but
>>I'll look into that later.
>>
>>
>>>It is worth noting that the routing table shows something like the
>>>following (having only brought up xen-br0 and xen-br1):
>>>
>>>192.168.254.16 0.0.0.0 255.255.255.248 dummy2
>>>192.168.254.24 0.0.0.0 255.255.255.248 dummy3
>>>192.168.254.0 0.0.0.0 255.255.255.248 xen-br0
>>>192.168.254.8 0.0.0.0 255.255.255.248 xen-br1
>>>0.0.0.0 192.168.254.6 0.0.0.0 xen-br0
>>>
>>>Does this look sound?
>>>
>>
>>yep,
>># route -n
>>Kernel IP routing table
>>Destination Gateway Genmask Flags Metric Ref Use Iface
>>192.168.254.64 0.0.0.0 255.255.255.248 U 0 0 0 dummy6
>>192.168.254.72 0.0.0.0 255.255.255.248 U 0 0 0 dummy7
>>192.168.254.16 0.0.0.0 255.255.255.248 U 0 0 0 dummy2
>>192.168.254.24 0.0.0.0 255.255.255.248 U 0 0 0 dummy3
>>192.168.254.0 0.0.0.0 255.255.255.248 U 0 0 0
>>xen-br0
>>192.168.254.8 0.0.0.0 255.255.255.248 U 0 0 0 dummy1
>>192.168.254.96 0.0.0.0 255.255.255.240 U 0 0 0 dummy9
>>192.168.254.80 0.0.0.0 255.255.255.240 U 0 0 0 dummy8
>>192.168.254.48 0.0.0.0 255.255.255.240 U 0 0 0 dummy5
>>192.168.254.32 0.0.0.0 255.255.255.240 U 0 0 0 dummy4
>>192.168.253.0 0.0.0.0 255.255.255.0 U 0 0 0
>>dummy10
>>192.168.254.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
>>192.168.254.0 0.0.0.0 255.255.255.0 U 0 0 0 br2
>>192.168.254.0 0.0.0.0 255.255.255.0 U 0 0 0 br3
>>192.168.254.0 0.0.0.0 255.255.255.0 U 0 0 0 br4
>>192.168.254.0 0.0.0.0 255.255.255.0 U 0 0 0 br5
>>0.0.0.0 192.168.254.6 0.0.0.0 UG 0 0 0
>>xen-br0
>>
>># ifconfig | egrep "Link|addr"
>>br1 Link encap:Ethernet HWaddr 62:3F:39:21:9D:DB
>> inet addr:192.168.254.9 Bcast:192.168.254.255 Mask:255.255.255.0
>>br2 Link encap:Ethernet HWaddr 82:D1:F6:2A:55:B6
>> inet addr:192.168.254.17 Bcast:192.168.254.255 Mask:255.255.255.0
>>br3 Link encap:Ethernet HWaddr AE:DB:D9:29:16:B8
>> inet addr:192.168.254.25 Bcast:192.168.254.255 Mask:255.255.255.0
>>br4 Link encap:Ethernet HWaddr 62:50:E4:EF:72:A1
>> inet addr:192.168.254.33 Bcast:192.168.254.255 Mask:255.255.255.0
>>br5 Link encap:Ethernet HWaddr 9E:4A:BC:41:68:A8
>> inet addr:192.168.254.49 Bcast:192.168.254.255 Mask:255.255.255.0
>>dummy0 Link encap:Ethernet HWaddr FA:99:8D:02:7D:FE
>> inet addr:192.168.254.1 Bcast:192.168.254.7 Mask:255.255.255.248
>>dummy1 Link encap:Ethernet HWaddr 62:3F:39:21:9D:DB
>> inet addr:192.168.254.9 Bcast:192.168.254.15 Mask:255.255.255.248
>>dummy2 Link encap:Ethernet HWaddr 82:D1:F6:2A:55:B6
>> inet addr:192.168.254.17 Bcast:192.168.254.23 Mask:255.255.255.248
>>dummy3 Link encap:Ethernet HWaddr AE:DB:D9:29:16:B8
>> inet addr:192.168.254.25 Bcast:192.168.254.31 Mask:255.255.255.248
>>dummy4 Link encap:Ethernet HWaddr 62:50:E4:EF:72:A1
>> inet addr:192.168.254.33 Bcast:192.168.254.47 Mask:255.255.255.240
>>dummy5 Link encap:Ethernet HWaddr 9E:4A:BC:41:68:A8
>> inet addr:192.168.254.49 Bcast:192.168.254.63 Mask:255.255.255.240
>>dummy6 Link encap:Ethernet HWaddr 06:EB:9E:DF:25:2F
>> inet addr:192.168.254.65 Bcast:192.168.254.71 Mask:255.255.255.248
>>dummy7 Link encap:Ethernet HWaddr 6E:92:58:A4:1D:C2
>> inet addr:192.168.254.73 Bcast:192.168.254.79 Mask:255.255.255.248
>>dummy8 Link encap:Ethernet HWaddr 32:4E:B6:5D:1B:12
>> inet addr:192.168.254.81 Bcast:192.168.254.95 Mask:255.255.255.240
>>dummy9 Link encap:Ethernet HWaddr 06:1E:0C:F4:5C:09
>> inet addr:192.168.254.97 Bcast:192.168.254.111
>> Mask:255.255.255.240
>>dummy10 Link encap:Ethernet HWaddr E6:5B:DC:A9:F2:54
>> inet addr:192.168.253.1 Bcast:192.168.253.255 Mask:255.255.255.0
>>lo Link encap:Local Loopback
>> inet addr:127.0.0.1 Mask:255.0.0.0
>>vif1.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif1.1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif1.2 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif1.3 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif1.4 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif1.5 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif1.6 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif1.7 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif1.8 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif1.9 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif1.10 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif2.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif4.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>vif5.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
>>xen-br0 Link encap:Ethernet HWaddr FA:99:8D:02:7D:FE
>> inet addr:192.168.254.1 Bcast:192.168.254.7 Mask:255.255.255.255
>>
>># brctl show
>>bridge name bridge id STP enabled interfaces
>>br10 8000.e65bdca9f254 no dummy10
>> vif1.0
>>br1 8000.623f39219ddb no dummy1
>> vif1.2
>>br2 8000.82d1f62a55b6 no dummy2
>> vif1.3
>>br3 8000.aedbd92916b8 no dummy3
>> vif1.4
>>br4 8000.6250e4ef72a1 no dummy4
>> vif1.5
>>br5 8000.9e4abc4168a8 no dummy5
>> vif1.6
>> vif2.0
>> vif4.0
>>br6 8000.06eb9edf252f no dummy6
>> vif1.7
>>br7 8000.6e9258a41dc2 no dummy7
>> vif1.8
>>br8 8000.324eb65d1b12 no dummy8
>> vif1.9
>>br9 8000.061e0cf45c09 no dummy9
>> vif1.10
>>xen-br0 8000.fa998d027dfe no dummy0
>> vif1.1
>> vif5.0
>>
>>
>>
>>
>>>Also note that /proc/net/dev only shows network activity for dummy0 and
>>>xen-br0 -- none for xen-br1 even though I have pinged the associated
>>>address and attempt to have domU1 use xen-br1 (vif = 'backend=fw,
>>>brige=xen-br1' ].
>>>
>>
>>fw doesn't need to export the interface AFAIK, as domU1 can have the
>>bridge exported to it directly from dom0.
>>eg.
>>
>>/etc/xen/Firewall includes:
>> name = "Firewall"
>> pci = ['00,0b,0', '00,0a,0' ]
>> nics=11
>> vif = [ 'mac=aa:00:00:00:22:01, bridge=br10',
>> 'mac=aa:00:00:25:40:01, bridge=xen-br0',
>> 'mac=aa:00:00:25:40:09, bridge=br1',
>> 'mac=aa:00:00:25:40:17, bridge=br2',
>> 'mac=aa:00:00:25:40:25, bridge=br3',
>> 'mac=aa:00:00:25:40:33, bridge=br4',
>> 'mac=aa:00:00:25:40:49, bridge=br5',
>> 'mac=aa:00:00:25:40:45, bridge=br6',
>> 'mac=aa:00:00:25:40:73, bridge=br7',
>> 'mac=aa:00:00:25:40:81, bridge=br8',
>> 'mac=aa:00:00:25:40:97, bridge=br9' ]
>>
>>In the Firewall, the PCI NICs become eth0/1, and the bridges are eth2...
>>
>>An example domU:
>>/etc/xen/Mail includes:
>> name = "MailServer"
>> vif = [ 'mac=AA:00:00:25:40:51, bridge=br5' ]
>>
>>You can tell I've gone overboard with bridges. I'll be reducing this
>>number with the latest design, but they're great to have lying around
>>for testing :) . That many interfaces causes shorewall to think for
>>30-60 secs when recreating iptables rules !!!
>>
>>Marcus.
>>
>>
>>>What else do you need from me to help troubleshoot?
>>>
>>>Adam
>>>
>>>On Mon, 2005-08-29 at 02:24 -0400, Marcus Brown wrote:
>>>
>>>
>>>>Hi Adam,
>>>>
>>>>Perhaps you could try loading the dummy modules in dom0 instead?
>>>>Then attach your dom0 bridges to the dummy interfaces by replacing
>>>>the netdev argument for "eth0" in /etc/xen/scripts/network with "dummy",
>>>>and create auto scripts for the dummy interfaces that create bridges.
>>>>
>>>>Sorry to be so brief ... just caught me before going to bed.
>>>> (Trying to work on a quote :) )
>>>>If you get stuck perhaps you could provide more detail?
>>>>
>>>>Regards,
>>>>
>>>>Marcus.
>>>>
>>>>
>>>>Adam Tworkowski wrote:
>>>>
>>>>
>>>>
>>>>>Hi,
>>>>>
>>>>>Could someone please help me out with networking my firewall on xenU
>>>>>configuration. I have combed the list archives for posts on similar
>>>>>configurations and getting bridging working properly but I am stuck and
>>>>>generally confused. I am trying to achieve the scenario below (ascii
>>>>>borrowed from previous thread).
>>>>>
>>>>> +------------------------+
>>>>> | physical machine, dom0 |
>>>>> | +---------------+ |
>>>>>-- Internet -------+ Firewall domU +--------- Intranet
>>>>> | +------+--------+ |
>>>>> | | |
>>>>> | | DMZ |
>>>>> | +-----+------+ |
>>>>> | | | |
>>>>> | +--+--+ +--+--+ |
>>>>> | |domU1| |domU2| |
>>>>> | +-----+ +-----+ |
>>>>> +------------------------+
>>>>>
>>>>>So far:
>>>>>
>>>>>- physical ethernet device (will be two later) hidden from dom0 (no issue)
>>>>>- domU (fw) using ethernet device -- has access to internet on eth0 (no
>>>>>issue)
>>>>>
>>>>>- domU1 and domU2 can only see domU when using bridge=xen-br0 although it
>>>>>appears that only on of the two can be active as if both are pinging domU
>>>>>there is much
>>>>>packet. Using vif = ['mac=xx.xx..., bridge=xen-br0'"].
>>>>>
>>>>>Should domU1 and domU2 should be using xen-br0 at all or should this only
>>>>>be for domU and the former be using xen-br1 and xen-br2 respectively.
>>>>>
>>>>>- I have attempted to have domU1 use vif = ['backend=fw'] without success
>>>>>
>>>>>- I have also attempted to created xen-br1,xen-br2 and have the non-fw
>>>>>domUs use these: vif = ['bridge=xen-br1']
>>>>>
>>>>>- Having searched the list, I have attempted to create dummy0, dummy1,etc.
>>>>>on the domU as suggested.
>>>>>
>>>>>
>>>>>I am hoping that someone can lay out what they have done to get this
>>>>>working. My suspicion is that my bridging is not working correctly on
>>>>>dom0.
>>>>>
>>>>>Question: what devices should be showing from 'ifconfig' on respective
>>>>>nodes (dom0, domU, domU1...)?
>>>>>
>>>>>xen0 shows multiple vifs, xen-br0-4.
>>>>>xenU shows eth0 - should it show vifs?
>>>>>xenU1,2 shows eth0. Should it show more? vifs?
>>>>>
>>>>>Maybe someone can share the output of their ifconfigs to compare.
>>>>>
>>>>>All my instance are running on Debian Sarge.
>>>>>
>>>>>Thanks in advance!
>>>>>
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home