traffic shaping on OSX 10.10 with pfctl and dnctl

Question

I am trying to do traffic shaping (throttling) on Mac OS X 10.10 via pfctl and dnctl.

I have implemented a simple test server and client with netcat (nc) and a synthetic random payload file to verify if my dummynet pipe throttling is working correctly. So far, attempting to configure the dummynet pipe using the murus firewall GUI app does not seem to correctly throttle traffic (64MB transfer completes in ~200ms).

Here's an OSX bash shell script (requires brew install coreutils for gdate) that is a complete example. If you run it in one terminal as:

./throttle-test.sh server

and another one as

./throttle-test.sh client

They will try to transfer a 64MB payload over your en0 interface (not using lo0 because its huge MTU is not analogous to WAN traffic).

I have also tested transferring the file to a remote linux laptop to see if the source and destination IP both being local IPs was bypassing the throttling, but even to a remote machine on my LAN/wifi the speed is much faster than the throttled limit.

My question is what would be a correct script to configure pfctl and dnctl to throttle this file transfer to a given bandwidth limit (say 8mbps for example). The scope of the throttling can be a specific TCP port.

Note OS X 10.10 no longer includes ipfw so I'm looking for something using pfctl and dnctl.

Here's my throttle-test.sh file:

#!/bin/bash
set -o errexit    # always exit on error
set -o errtrace   # trap errors in functions as well
set -o pipefail   # don't ignore exit codes when piping output
set -o posix      # more strict failures in subshells
# set -x          # enable debugging

IFS="$(printf "\n\t")"

setup_payload() {
  local payload_path="$1"
  local size_kbytes="$2"
  mkdir -p $(basename "${payload_path}")

  if [[ -f "${payload_path}" ]]; then
    local on_disk=$(wc -c < "${payload_path}")
  fi
  if [[ ${on_disk} -ne $((${size_kbytes} * 1024)) ]]; then
    echo "creating payload file ${payload_path}"
    dd if=/dev/urandom of="${payload_path}" \
      bs=1024 count="${size_kbytes}" &> /dev/null
  fi
}

start_server() {
  local payload_path="$1"
  local ip="$2"
  local port="$3"
  while true; do
    echo "Starting netcat server for ${payload_path} on ${ip}:${port}"
    nc -l "${ip}" "${port}" < "${payload_path}"
    sleep 1
  done
}

hash() {
  shasum -a 256 "$1" | cut -d " " -f 1
}

verify_hashes() {
  # Sanity check no funny business
  from_hash=$(hash "$1")
  to_hash=$(hash "$2")
  if [[ "${from_hash}" != "${to_hash}" ]]; then
    echo "checksums did not match ${from_hash} ${to_hash}" 1>&2
    exit 10
  fi
}

client() {
  local payload_path="$1"
  local ip="$2"
  local port="$3"

  # time how long it takes to transfer the payload file
  start=$(gdate +%s%3N)
  nc -d "${ip}" "${port}" > "${payload_path}.client"
  stop=$(gdate +%s%3N)

  verify_hashes "${payload_path}" "${payload_path}.client"

  local duration=$((${stop} - ${start}))
  echo "${duration}"
}

main() {
  local size_kbytes=$((64 * 1024)) # 64 MB
  local payload_path="/tmp/throttle-test/data-${size_kbytes}.bin"
  local port="${PORT-9112}"
  # You may need to change this if you are on linux
  local interface="${INTERFACE-en0}"
  local ip=$(ipconfig getifaddr "${interface}")

  setup_payload "${payload_path}" "${size_kbytes}"
  case "$1" in
    server)
      start_server "${payload_path}" "${ip}" "${port}"
    ;;
    client)
      local duration=$(client "${payload_path}" "${ip}" "${port}")
      echo "Transfered ${size_kbytes} kbytes in ${duration} ms"
    ;;
    *)
      echo "Usage: $0 <server|client>"
    ;;
  esac
}

main "$@"

Update

Here is what I have so far. This seems to work correctly for the download direction, but not throttle at all in the upload direction.

throttle_start() {
  local down_mbps="$1"
  local up_mbps="$2"
  local delay=$(($3 / 2))
  sudo dnctl pipe 1 config bw "${down_mbps}Mbit/s" delay "${delay}"
  sudo dnctl pipe 2 config bw "${up_mbps}Mbit/s" delay "${delay}"
  (cat /etc/pf.conf && \
    echo 'dummynet-anchor "throttle"' && \
    echo 'anchor "throttle"') | sudo pfctl -f -
  cat << EOF | sudo pfctl -a throttle -f -
dummynet in quick proto tcp from any port = 9112 to any pipe 1
dummynet out quick proto tcp from any to any port = 9112 pipe 2
EOF
  sudo pfctl -q -e
}

Solution

This is a script I used on El Capitan 10.11 with some success:

#!/bin/bash

# Reset dummynet to default config
dnctl -f flush

# Compose an addendum to the default config: creates a new anchor
(cat /etc/pf.conf &&
  echo 'dummynet-anchor "my_anchor"' &&
  echo 'anchor "my_anchor"') | pfctl -q -f -

# Configure the new anchor
cat <<EOF | pfctl -q -a my_anchor -f -
no dummynet quick on lo0 all
dummynet out proto tcp from any to any port 1:65535 pipe 1
EOF

# Create the dummynet queue
dnctl pipe 1 config bw 40000byte/s

# Activate PF
pfctl -E

# to check that dnctl is properly configured: sudo dnctl list

The only relevant difference seems like the no dummynet quick on lo0 all, which I don't really know what it does, found here: https://www.reddit.com/r/osx/comments/3g7dim/limiting_bandwidth_per_application/

Comments

[prasvin]

Hi. Does this not work now? I am on 10.10.5. I tried a lot of options, read the manpages, looked here and there, but could not throttle the bandwidth on a port (29719). I was able to create my anchor, and create pipes of smaller bandwidth, but somehow could not associate a port with the pipes.

[prasvin]

I was able to create my anchor, and create pipes of smaller bandwidth, but somehow could not associate a port with the pipes. It would be helpful to know the output of sudo dnctl list after running these commands.

[psmith]

The dnctl list output is not very useful unfortunately. Do you want to limit the outgoing traffic on one specific port, like 80? You can restrict the port by changing the "1:65535" with the port number you want. You can have multiple "dummynet" lines.