Simple way to simulate a slow network in python

Question

Scenario. I have a client with two network connections to a server. One connection uses a mobile phone, and the other is using a wlan connection.

The way that I have solved this is by having the server listen at two ports. But, the mobile connection should be slower than the wlan connection. The data that is sent, is usually just a line of text. I have solved the slower connection issue by allowing the mobile connection to send data in a five second interval, the wlan connection have an interval of one second.

But is there a better way of doing the slowdown? Perhaps by sending smaller packets, meaning I need to send more data?

Any ideas on a good way to slow down one of the connections?

Orjanp

A simple client example with only one connection.

def client():
    import sys, time, socket

    port = 11111
    host = '127.0.0.1'
    buf_size = 1024

    try:
        mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        mySocket.connect((host, port))
    except socket.error, (value, message):
        if mySocket:
            mySocket.close()
        print 'Could not open socket: ' + message
        sys.exit(1)
    mySocket.send('Hello, server')
    data = mySocket.recv(buf_size)
    print data
    time.sleep(5)

    mySocket.close()

client()

A simple server listening to one port.

def server():
    import sys, os, socket

    port = 11111
    host = ''
    backlog = 5
    buf_size = 1024

    try:
        listening_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
        listening_socket.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) 
        listening_socket.bind((host, port)) 
        listening_socket.listen(backlog)
    except socket.error, (value, message):
        if listening_socket:
            listening_socket.close()
        print 'Could not open socket: ' + message
        sys.exit(1)

    while True:
        accepted_socket, adress = listening_socket.accept()
        data = accepted_socket.recv(buf_size)
        if data:
            accepted_socket.send('Hello, and goodbye.')
        accepted_socket.close()

server()

Solution 1

Aside from using an external tool to simulate the kind of network you're interested in, one good approach is to use a substitute implementation of socket.

This involves making the socket construction a parameter to your function, instead of importing the socket module and using it directly. For normal operation, you will pass in the real socket type, but when you want to test various adverse network conditions, you can pass in an implementation that simulates those conditions. For example, you might create a socket type which parameterizes latency and bandwidth (untested code, beware):

import time, socket

class ControllableSocket:
    def __init__(self, latency, bandwidth):
        self._latency = latency
        self._bandwidth = bandwidth
        self._bytesSent = 0
        self._timeCreated = time.time()
        self._socket = socket.socket()

    def send(self, bytes):
        now = time.time()
        connectionDuration = now - self._timeCreated
        self._bytesSent += len(bytes)
        # How long should it have taken to send how many bytes we've sent with our
        # given bandwidth limitation?
        requiredDuration = self._bytesSent / self._bandwidth
        time.sleep(max(requiredDuration - connectionDuration, self._latency))
        return self._socket.send(bytes)

If you implement the other socket methods, connect, recv, etc, you can substitute an instance of this class for an instance of the real socket type. This leaves all the rest of your program completely free of any knowledge of your simulation, simplifying it, while also letting you try out many different network configurations by just implementing a new socket type that simulates them.

This idea is one of the reasons Twisted explicitly separates the idea of "protocols" - objects which know how to interpret bytes from the network and generate new bytes to send to the network - from "transports" - objects which know how to get bytes off the network and put bytes onto it. The separation eases testing and allows novel configurations like this one, where a simulation of some other network conditions (which may be difficult to produce for real) is provided by the transport.

Solution 2

At the risk of not answering the question you asked, I would look for software that does this at a lower level.

Netlimiter does this for Windows. I think BWMeter and Bandwidth Controller can do it too.

pyshaper is a similar tool for Linux. Open source. You might just be able to import it into your Python program.

(Another thing to consider is that you might already have a router capable of shaping traffic the way you want. That's a pretty big dependency to add to your software, though, and it might be more work to configure.)

Solution 3

Well, what makes a network connection "slower" than another, is due to latency and/or bandwidth. So if you want to have a realistic simulation, you need to find the bandwidth of your mobile phone connection, as well as its latency, and simulate that in your client program.

But you seem to imply that you send very little data, so bandwidth is probably not really going to affect your connection speed. So you can just simulate latency, and that's doing what you do: sleep(latency) between each packet sent. 5 second seems like a lot though.

But if you think bandwidth might be relevant, it's actually quite simple to simulate: bandwidth is the max number of bytes per second that you can send, and latency is the duration it's going to take to get to its destination.

How to do it: have a global timestamp "blockedUntil", representing the time until your connection becomes free again to send data. Initialise to 0 at the beginning of your program, as we assume it's not being used yet. Then, everytime you have a packet to send, If "_blockedUntil" is less than now(), set it to now(). Then calculate the time it would take to write to your fictitious "wire" by doing: packet.size() / bandwidth, that'll get you a time duration, add the latency, and add that to "blockedUntil".

Now compute dt = blockedUntil - now(), add the packet to the queue, and add a timer firing in "dt", which will pop the first packet in the queue, and send it.

There you go, you've simulated bandwidth and latency.

Edit: as someone mentioned there's the question of dropped packets too. You can simulate that by having a probability of dropping packets. Note: This is solely possible when one is manipulating packets from an unconnected protocol such as Ethernet or UDP. In the case of TCP for example, it won't work.

Solution 4

The simulation of slow connections can be achieved easily with the poorconn Python package:

pip install poorconn

For example, in the following code, the function client_socket()/server_socket() returns a client/server socket object that delay roughly 2 seconds for sending every 1024 bytes of messages.

from socket import socket
from poorconn import delay_before_sending, make_socket_patchable

def client_socket():
    s = socket()
    s = make_socket_patchable(s)
    delay_before_sending(s, 2, 1024)
    return s

def server_socket():
    s = socket()
    s = make_socket_patchable(s)
    delay_before_sending_upon_acceptance(s, 2, 1024)
    return s

Then, use the returned socket objects like normal socket objects.

Disclaimer: I'm the main author of poorconn. Feedback is welcome.

View more solutions

Comments

[ChristopheD]

Related: stackoverflow.com/questions/1094760/…

[Wim]

You can't really cause a packet to be dropped when you're working on the socket layer. You can model the effect though, which is an additional delay (timeout + retransmit), with the same probability as that of a dropped packet.

[Florian]

Well, if the packet is dropped, just don't send it. That's what I meant.

[Jean-Paul Calderone]

TCP doesn't work that way. Connections are "reliable" which means that if an IP datagram carrying some TCP payload is dropped, the data will be re-sent. From the application level, working with TCP sockets, you can't tell when this happens, nor simulate it. Dropping some bytes that are passed to socket.send to simulate packet loss is an incorrect simulation of packet loss.

[Florian]

You are a 100% right, I got carried away, last time I did that I was manipulating ethernet frames, so it would work fine. I was assuming the same for the OP, but it's obviously not the case, got carried away. My bad.

[Orjanp]

The simulation does not need to be realistic as long at the result is that it takes longer time to send data over the mobile connection compared to the wlan connection. So dividing the data in multiple packets and sleeping between each packet is a feasible approach. Just need to tell the server when all the data is sent. Seems simple enough.

[Orjanp]

I'll have a look into this approach since I also need a even slower connection than the mobile one. Here I don't have to worry about telling the server that all the data has been sent. Quite an elegant solution. Thanks. :)