Fast ping sweep in python

Question

So, I'm trying to get similar results using python as I do with a bash script.

Code for the bash script:

    #!/bin/bash

    for ip in $(seq 1 254); do
        ping -c 1 10.10.10.$ip | grep "bytes from" | cut -d " " -f 4 | cut -d ":" -f 1 &
    done

The thing that I would like to do is get the same results with similar speed. The issue that I've had with every version of the python script is that it takes a very long time to complete compared to the few seconds the batch script takes.

The batch file takes about 2 seconds to sweep a /24 network while the the best I can get with the python script is about 5-8 minutes.

Latest version of python script:

import subprocess

cmdping = "ping -c1 10.10.10."

for x in range (2,255):
    p = subprocess.Popen(cmdping+str(x), shell=True, stderr=subprocess.PIPE)

    while True:
        out = p.stderr.read(1)
        if out == '' and p.poll() != None:
            break
        if out != '':
            sys.stdout.write(out)
            sys.stdout.flush()

I've tried several different ways in python but can't get anywhere near the speed of the bash script.

Any suggestions?

Solution

Multiprocessing

#!/usr/bin/python2

import multiprocessing
import subprocess
import os

def pinger( job_q, results_q ):
    DEVNULL = open(os.devnull,'w')
    while True:
        ip = job_q.get()
        if ip is None: break

        try:
            subprocess.check_call(['ping','-c1',ip],
                                  stdout=DEVNULL)
            results_q.put(ip)
        except:
            pass

if __name__ == '__main__':
    pool_size = 255

    jobs = multiprocessing.Queue()
    results = multiprocessing.Queue()

    pool = [ multiprocessing.Process(target=pinger, args=(jobs,results))
             for i in range(pool_size) ]

    for p in pool:
        p.start()

    for i in range(1,255):
        jobs.put('192.168.1.{0}'.format(i))

    for p in pool:
        jobs.put(None)

    for p in pool:
        p.join()

    while not results.empty():
        ip = results.get()
        print(ip)

Comments

[Isaias]

What's the purpose of jobs.put(None)? @mojo

[mojo]

@Isaias Each None is a signal that there are no more jobs. You could perhaps end() the queue and get similar results if you modified the worker process code.