Re: [Xen-users] Xen 3.4.2 networking help

[Alexander Zherdev <azherdev@xxxxxxxxx>]

Issue resolved! Simple issue, 4 days gone. :( Thank you all for your help! Explanation below.

Kept everything stock in terms of xen bridge configuration. Using dnsmasq for MAC to IP DHCP mapping (to keep DomU config simple).

Booted DomU
  - DomU sees the internet, has 192.168.122.150 IP, /24 network, 192.168.122.1 GW and DNS
  - Dom0 can ping DomU
  - Internet can not see DomU on 1.2.3.70 IP
  - Internet CAN ping 1.2.3.70, but it's eth0 of Dom0
  - Can't RDP from public IP 1.2.3.70 to DomU Windows

Applied this:

iptables -t nat -A PREROUTING -i eth0 -d 1.2.3.70 -j DNAT --to-destination 192.168.122.150
iptables -t nat -A POSTROUTING -o eth0 -s 192.168.122.150 -j SNAT --to-source 1.2.3.70

Pinging 1.2.3.70 from the internet is now unreachable

Removed rule 4 and 5 from the default forward policy in iptables

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination
1    ACCEPT     all  --  0.0.0.0/0            192.168.122.0/24    state RELATED,ESTABLISHED
2    ACCEPT     all  --  192.168.122.0/24     0.0.0.0/0
3    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
4    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable
5    REJECT     all  --  0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable

Now I can ping AND rdp into my DomU from 1.2.3.70 public IP!

Current full iptables:

Table: nat
Chain PREROUTING (policy ACCEPT)
num  target     prot opt source               destination
1    DNAT       all  --  0.0.0.0/0            1.2.3.70        to:192.168.122.150

Chain POSTROUTING (policy ACCEPT)
num  target     prot opt source               destination
1    MASQUERADE  all  --  192.168.122.0/24    !192.168.122.0/24
2    SNAT       all  --  192.168.122.150      0.0.0.0/0           to:1.2.3.70

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination

Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination
1    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:53
2    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:53
3    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:67
4    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:67

Chain FORWARD (policy ACCEPT)
num  target     prot opt source               destination
1    ACCEPT     all  --  0.0.0.0/0            192.168.122.0/24    state RELATED,ESTABLISHED
2    ACCEPT     all  --  192.168.122.0/24     0.0.0.0/0
3    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination

Alexander Zherdev
azherdev@xxxxxxxxx

---

From: Alexander Zherdev <azherdev@xxxxxxxxx>
To: lists@xxxxxxxxx; xen-users@xxxxxxxxxxxxxxxxxxx
Sent: Wed, October 27, 2010 11:22:54 PM
Subject: Re: [Xen-users] Xen 3.4.2 networking help

Thomas,

Thank you for your explanation. Here is where I am right now.

I have the standard network bridge scripts fired off with xen:
    network-bridge
    vif-bridge

The DomU is DHCP and gets an ip of 192.168.122.150/24 with 192.168.122.1 as GW+DNS from the dnsmasq service running on Dom0.

Dom0 has the following network (CentOS xen):
    - 1.2.3.64/27 network
    - 1.2.3.65 gateway
    - 1.2.3.67 on eth0 which is what I use for Dom0 communication (ssh)
    - 1.2.3.70 is the 2nd IP tied to eth0:1 of Dom0 that I want to use as direct mapping to one of my DomU

DomU has the following network (Windows 2003 HVM):
    - 192.168.122.0/24
    - 192.168.122.1 gateway
    - 192.168.122.150 IP

When I boot DomU I can:
    - Ping from Dom0 to DomU 192.168.122.150
    - Ping from DomU to Dom0 192.168.122.1 as well as www.google.com, 1.2.3.67, etc.
    - Surf the web on DomU

So the setup that you have suggested appears to work using the default xen scripts.

I then ran the iptables commands that you suggested for the 1:1 NAT as follows:

iptables -t nat -A PREROUTING -d 1.2.3.70 -j DNAT --to-destination 192.168.122.150
iptables -t nat -A POSTROUTING -s 192.168.122.150 -j SNAT --to-source 1.2.3.70

But I can not access the system from outside. I did a tcpdump and I see the 1.2.3.70 being requested for the RDP port and it replies back as no port found. No forwarding of any sort.

Could this be because my Dom0 and DomU have different subnets? My Dom0 is on /27 and my DomU reside on /24. I feel like I'm a command line away from accomplishing this.

At the risk of being redundant, here is what I see with iptables and ip r s with the above setup:

ifconfig:

eth0 - 1.2.3.67/27
eth0:1 - 1.2.3.70/27
peth0 - noip
tap1.0 - noip
vif1.0 - noip
virbr0 - 192.168.122.1/24

iptables:
------------------------
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     udp  --  anywhere             anywhere            udp dpt:domain
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:domain
ACCEPT     udp  --  anywhere             anywhere            udp dpt:bootps
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:bootps

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     all  --  anywhere             192.168.122.0/24    state RELATED,ESTABLISHED
ACCEPT     all  --  192.168.122.0/24     anywhere
ACCEPT     all  --  anywhere             anywhere
REJECT     all  --  anywhere             anywhere            reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere            reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

ip r s
----------------------

96.44.171.64/27 dev eth0  proto kernel  scope link  src 96.44.171.67
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1
169.254.0.0/16 dev eth0  scope link
default via 96.44.171.65 dev eth0

Alexander Zherdev
azherdev@xxxxxxxxx

---

From: Thomas Halinka <lists@xxxxxxxxx>
To: Alexander Zherdev <azherdev@xxxxxxxxx>
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Sent: Wed, October 27, 2010 2:40:45 AM
Subject: Re: [Xen-users] Xen 3.4.2 networking help

Hi Again,

just a short step-by-step guide.

Am Dienstag, den 26.10.2010, 23:54 -0700 schrieb Alexander Zherdev:
> Pardon my long email below, I hope it will shed some light.
>
> I've googled and tried various things but nothing seem to work. I have
> upgraded to 3.4.3 of Xen and the kernel had an update too.

so u had a lot of fun ;-)

> My brain is fried right now. The only thing that seems to work is
> bridged mode. In bridged mode, my DomU gets the DHCP from dnsmasq and
> it can then surf the web. But I can't get to it from outside. In route
> or nat mode, the DomU can't even get out. Below is a test in NAT mode
> of xend.

Dont use NAT - its just MASQUERADING! Communication from internet would
be only possible through portforwarding....

> Below I have a pretty verbose output of iptables, ip r, and ifconfig
> right after I boot the physical server, then after I start the DomU,
> and then after I apply the SNAT and DNAT settings (only ip r changes
> then).
>
> I appreciate any help that you have.
>
> -----------------------------
>
> Kernel:  2.6.18-194.17.4.el5xen
> Xen: 3.4.3
> Source: www.gitco.de
>
> /etc/xen/xend-config.sxp
>    (network-nat)
>    (vif-nat)

Please do the following.

- Disable default Firewall (only to get ur setup running)
# service iptables off


- Write down a ugly script, something like:

#!/bin/bash
# i used /27 since your public-net was /27 too
# 192.168.128.65 is dom0-IP
brctl addbr xen-privatelan
ip a a 192.168.128.65/27 dev xen-privatelan 
ifconfig xen-privatelan up
echo 1 > /proc/sys/net/ipv4/ip_forward

- and save it e.g. to
/etc/xen/scripts/network-mynet

- make it executable
chmod +x /etc/xen/scripts/network-mynet


- change any kind of xen-networking-script to e.g.
...
(network-script network-mynet)
(vif-script vif-bridge)
.....



    ######## reboot ur dom0 #####################



After reboot setup your windows-box to use the bridge "xen-privatelan"

- change domU.cfg

...
vif = [ 'type=ioemu, bridge=xen-privatelan, mac=00:16:3e:00:01:02' ]
.....


- start ur domU
- setup nw-settings in domU (192.168.128.70/27 gw: 192.168.128.65)
                                                ^^^^  dom0-IP


- at this point u should be able to ping dom0 from ur domU!
  access to internet and from internet to domU should NOT work
  Otherwise triplecheck "brctl show", ip r s, and friends...

- Setup "1:1-NAT"

  iptables -t nat -A PREROUTING -d XXX.XXX.XXX.70 -j DNAT
--to-destination 192.168.128.70
 
  iptables -t nat -A POSTROUTING -s 192.168.128.70 -j SNAT --to-source
XXX.XXX.XXX.70

--> domU has internal IP 192.168.128.70 and is reachable via externalIP
XXX.XXX.XXX.70

--> domU should be able to ping the "internet"
--> domU should be available from "internet" trough XXX.XXX.XXX.70

Am i right? :-)

cu,

thomas

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users