How to configure a Linux network namespace that allows UDP broadcast

Question

I'm trying to use the ip netns family of commands in Linux to create a network namespace in which I can run a program that uses UDP broadcast. I do not need access to the Internet, or any interface on the root namespace (but if that's what's necessary to get things working, it's definitely acceptable).

Here's an example server and client in Ruby (tested with Ruby 1.9.3, but I expect it will work in other versions):

#! /usr/bin/env ruby

require 'socket'

PORT = 5000

case ARGV[0]
when 'server'
  soc = UDPSocket.open
  begin
    soc.bind('', PORT)
    puts "SERVER #{Process.pid} listening on #{PORT}"
    msg = soc.recv(1)
    puts "SERVER got msg: #{msg}"
  ensure
    soc.close
  end
when 'client'
  soc = UDPSocket.open
  begin
    soc.setsockopt(Socket::SOL_SOCKET, Socket::SO_BROADCAST, true)
    puts "CLIENT sending message"
    soc.send('m', 0, '<broadcast>', PORT)
  ensure
    soc.close
  end
else
  abort "usage: #{$0} {server | client}"
end

It creates either a server or client. The server listens on the 0.0.0.0 interface (soc.bind('', ...)). The client sends a message to the broadcast address (soc.send(..., ..., '<broadcast>', ...)).

When run within the root namespace, it seems to work correctly:

$ ./udp-broadcast.rb server & sleep 0.5 && sudo netstat --listen --udp -p | grep 5000 && ./udp-broadcast.rb client
SERVER 22981 listening on 5000
udp        0      0 *:5000                  *:*                                 22981/ruby
CLIENT sending message
SERVER got msg: m

Here is a script where I attempt to create a new network namespace and run the same commands:

#!

set -e

NS=udp-broadcast-test
nsexec="ip netns exec $NS"

ip netns add $NS

trap "ip netns delete $NS" EXIT

$nsexec ip link set lo up

# Can loopback have a broadcast address?
# $nsexec ip link set lo broadcast 255.255.255.255
# RTNETLINK answers: Invalid argument
# $nsexec ip addr add broadcast 255.255.255.255 dev lo
# RTNETLINK answers: Invalid argument

$nsexec ip link add veth0 type veth peer name veth1
$nsexec ifconfig veth0 192.168.99.1/24 up

$nsexec ip link
$nsexec ip route
$nsexec ifconfig

timeout 2s $nsexec ./udp-broadcast.rb server &
sleep 0.2
$nsexec netstat -n --udp --listen -p
timeout 2s $nsexec ./udp-broadcast.rb client
wait

When run, it produces the following output:

$ sudo ./netns.sh
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: veth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether e2:a1:c4:14:c4:5e brd ff:ff:ff:ff:ff:ff
3: veth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000
    link/ether a6:2f:84:9f:08:36 brd ff:ff:ff:ff:ff:ff
192.168.99.0/24 dev veth0  proto kernel  scope link  src 192.168.99.1
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

veth0     Link encap:Ethernet  HWaddr a6:2f:84:9f:08:36
          inet addr:192.168.99.1  Bcast:192.168.99.255  Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

SERVER 23320 listening on 5000
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
udp        0      0 0.0.0.0:5000            0.0.0.0:*                           23320/ruby
CLIENT sending message
./udp-broadcast.rb:23:in `send': Network is unreachable - sendto(2) (Errno::ENETUNREACH)
        from ./udp-broadcast.rb:23:in `<main>'

Now, if I change the address that the server is listening on, and that the client sends a message to, to 192.168.99.1, then the message gets through, so I know my veth0 at least partially works.

How can I configure things such that the broadcast message gets through? The server/client code is extracted from a larger codebase and not easily changed, so the only thing I can change is my network configuration.

Solution 1

This particular issue is solved by adding a default route to veth0:

$nsexec ip route add default via 192.168.99.1 dev veth0

Add that line immediately after the line that brings veth0 up, and the script runs successfully.

Solution 2

Well, theres a number of reasons it does not work.

1. You create a veth pair, then fail to add one side of it into the new network namespace.
2. One of the veths sides is not up.
3. Specifying the broadcast address as 255.255.255.255 as in your example cause a routing table lookup and the packet to be sent against the default route.
4. Consequently you dont use SO_BINDTODEVICE to specify which interface you actually want to send down to. Note that this requires root privileges, which in many cases is not ideal.

Also, you did not setup any routing relationship between the child namespace and the parent namespace so it wouldnt even work pinging the host directly.

In general, using the generic broadcast address for anything aside from provision of fundamental network services is not a good practice. You should use the broadcast address of the subnet you are aiming for.

I got all of what you mentioned working doing the following for the network namespace to prepare.

# ip netns add TEST
# ip link add veth0 type veth peer name veth1
# ip link set dev veth1 netns TEST
# ip link set dev veth0 up
# ip netns exec TEST ip link set dev veth1 up
# ip netns exec TEST ip addr add 10.10.10.10/32 dev veth1
# ip route add 10.10.10.10/32 dev veth0
# ip netns exec TEST ip route add 192.168.1.3/32 dev veth1
# ping -c1 10.10.10.10
PING 10.10.10.10 (10.10.10.10) 56(84) bytes of data.
64 bytes from 10.10.10.10: icmp_seq=1 ttl=64 time=0.202 ms

--- 10.10.10.10 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.202/0.202/0.202/0.000 ms

Here is the script used. Notice the SO_BINDTODEVICE call..

#!/usr/bin/python
import socket as sock
import sys, time, os

if __name__ == "__main__":
  if sys.argv[1] == "server":
    s = sock.socket(sock.AF_INET, sock.SOCK_DGRAM)
    s.bind(('0.0.0.0', 50000))
    data = s.recvfrom(50)
    print "Got {0}".format(data)

  elif sys.argv[1] == "client":
    s = sock.socket(sock.AF_INET, sock.SOCK_DGRAM)
    s.setsockopt(sock.SOL_SOCKET, sock.SO_BROADCAST, 1)
    s.setsockopt(sock.SOL_SOCKET, sock.SO_BINDTODEVICE, "veth0")
    s.connect(('255.255.255.255', 50000))
    s.send("hello world\n")

And then the result..

# ip netns exec TEST python test.py server &
[1] 24961
# python test.py client
Got ('hello world\n', ('192.168.1.3', 41971))

Comments

[Patrick]

I can't change the actual code to use SO_BINDTODEVICE (of course I can change my demo). At any rate, I think just adding a default route to veth0 is sufficient (and it doesn't matter that veth1 is down). I've not fully tested yet however.

[Patrick]

Yes, I have no need or desire for any routing relationship between the child namespace and the parent namespace.