KVM Guest with NAT + Bridged networking

Question

I currently have a few KVM Guests on a dedicated server with bridged networking (this works) and i can successfully ping the outside ips i assign via ifconfig (in the guest).

However, due to the fact i only have 5 public ipv4 ip addresses, i would like to port forward services like so:

hostip:port -> kvm_guest:port

UPDATE

I found out KVM comes with a "default" NAT interface, so added the virtual NIC to the Guest virsh configuration then configured it in the Guest, it has the ip address:

192.168.122.112

I can successfully ping 192.168.122.112 and access all ports on 192.168.122.112 from the KVM Host, so i tried to port forward like so:

iptables -t nat -I PREROUTING -p tcp --dport 5222 -j DNAT --to-destination 192.168.122.112:2521
iptables -I FORWARD -m state -d 192.168.122.0/24 --state NEW,RELATED,ESTABLISHED -j ACCEPT

telnet KVM_HOST_IP 5222 just hangs on "trying"

telnet 192.168.122.112 2521 works

[root@node1 ~]# tcpdump port 5222
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
23:43:47.216181 IP 1.152.245.247.51183 > null.xmpp-client: Flags [S], seq 1183303931, win 65535, options [mss 1400,nop,wscale 3,nop,nop,TS val 445777813 ecr 0,sackOK,eol], length 0
23:43:48.315747 IP 1.152.245.247.51183 > null.xmpp-client: Flags [S], seq 1183303931, win 65535, options [mss 1400,nop,wscale 3,nop,nop,TS val 445778912 ecr 0,sackOK,eol], length 0
23:43:49.415606 IP 1.152.245.247.51183 > null.xmpp-client: Flags [S], seq 1183303931, win 65535, options [mss 1400,nop,wscale 3,nop,nop,TS val 445780010 ecr 0,sackOK,eol], length 0

7 packets received by filter 0 packets dropped by kernel

[root@node1 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             192.168.122.0/24    state NEW,RELATED,ESTABLISHED 

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination   


[root@node1 ~]# iptables -nvL
Chain INPUT (policy ACCEPT 976 packets, 57008 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   11   640 ACCEPT     all  --  *      *       0.0.0.0/0            192.168.122.0/24    state NEW,RELATED,ESTABLISHED 

Chain OUTPUT (policy ACCEPT 673 packets, 40901 bytes)
 pkts bytes target     prot opt in     out     source               destination         
[root@node1 ~]# iptables -nvL -t nat
Chain PREROUTING (policy ACCEPT 549 packets, 34067 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    1    64 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:5222 to:192.168.122.112:2521 
    3   192 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:5222 to:192.168.122.112:2521 
    1    64 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:5225 to:192.168.122.112:2521 
    1    64 DNAT       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:5222 to:192.168.122.112:2521 

Chain POSTROUTING (policy ACCEPT 45 packets, 3169 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 44 packets, 3105 bytes)
 pkts bytes target     prot opt in     out     source               destination         

All help is appreciated.

Thanks.

Solution

if you run kvm in this way:

kvm -drive... -net nic -net tap .... 

you will become a new interface (in root host) named tapX (where X is a number). This interface is normally configured by default via a script located in somewhere in /etc (/etc/kvm/kvm-ifup, /etc/qemu-ifup or else) you could change them with: kvm -drive... -net nic -net tap,script=mynatbrigescript...

From there, this interface have to be configured on each points:

ifconfig tapX 192.168.124.1/30

and in the client os:

ifconfig eth0 192.168.124.2/30

so if you hit (on root host), you will see a new local network (assuming your public ip is 1.2.3.4):

ip r s
1.2.3.4 dev eth0 ...
192.168.124.0/30 dev tapX  proto kernel  scope link src 192.168.124.1

From there, you must be able to ping to 192.168.124.2 from the root host. You could then DNAT your incomings tcp packets with:

iptables -t nat -A PREROUTING -i eth0 -d 192.168.122.31 --dport 5222 -j DNAT --to-destination 192.168.124.2:2521

and SNAT the answer packets in the other direction:

iptables -t nat -A POSTROUTING -o eth0 -s 192.168.124.2 -j SNAT --to-source 192.168.122.31

now if for the root node, this work:

telnet 192.168.124.2 2521

Then from your localdomain, this must do the same:

telnet 1.2.3.4 5222

At all, your root host have to forward ip packets, this could be verified by:

cat /proc/sys/net/ipv4/ip_forward 
1

or

sysctl net.ipv4.conf.all.forwarding
net.ipv4.conf.all.forwarding = 1

This could be set by:

echo 1 > /proc/sys/net/ipv4/ip_forward

or

sysctl net.ipv4.conf.all.forwarding=1

Comments

[mgorven]

Please provide the output of iptables -nvL and iptables -nvL -t nat (these show packet counts so you can tell which rules are being enacted). Please also provide a tcpdump -n from inside the guest so we can see what's getting to the guest. Are you testing the connection from the host itself or from a different location?

[Daniel]

@mrgorven check updated answer, tested connection from host and local box.

[Michael Hampton]

Two things: Why are you doing this at all? You can just assign public IP addresses to the guests. Second, where's your IPv6?

[Daniel]

@michaelHampton Please re read the question "However, due to the fact i only have 5 public ipv4 ip addresses, i would like to port forward services like so:"

[Michael Hampton]

So you ran out of IP addresses? You should clarify this.

[Daniel]

@MichaelHampton They're expensive at my host

[teej]

Sorry for writing this as an answer; I'd comment but I can't yet. I second what F. Hauri commented on the previous answer. I'll phrase it in a different way: can you reach the Internet from your guest? Can you, say, ping an outside host, or run a telnet www.google.com 80 and get a successful connection?

[slm]

@Daniel Do you have to use KVM's default NAT? We use the bridge networking where I work and every guest get's a real IP address on our LAN. We then just point our external IP + port to a guest's IP + port via our firewall box.

[Daniel]

The server doesn't connect to a local LAN/NAT router, it's hosted in a D.C

[chutz]

Did you also check the ip_forward sysctl? sysctl net.ipv4.ip_forward should be set to 1.

[Daniel]

I'm not running KVM this way, i'm using virt-install/virsh configuration files

[F. Hauri]

@Daniel: Did you SNAT (or MASQUERADE) in the answer way, what you DNAT in request way?