How to avoid defunct python processes in Linux?

Question

With python3 (3.4.3) on Ubuntu 14.04 I have created a Forker class which I use in two different processes to create child processes. Here is the Forker class:

class Forker(object):
    def __init__(self):     
        self.active_children = []            

    def reap_children(self):                              
        while self.active_children:                        
            pid,stat = os.waitpid(0, os.WNOHANG)       
            if not pid: break
            self.active_children.remove(pid)                

    def fork(self, function, *args, **kwargs):         
        self.reap_children()

        child_pid = os.fork()
        if child_pid == 0:
            function(*args, **kwargs)
            os._exit(0)
        else:                                               
            self.active_children.append(child_pid)

        return child_pid

In the two different Linux processes I derive from this class and then call e.g. something like:

self.fork(my_function, my_data)

However, I still get a couple of defunct python processes:

alexand+ 24777 24735  0 20:40 pts/29   00:00:00 [python3] <defunct>
alexand+ 24838 24733  0 20:40 pts/29   00:00:00 [python3] <defunct>

Maybe there is something I can change to avoid these defunct processes?

Solution 1

Process marked as <defunct> are called zombie processes. The system keeps them in that defunct state to allow their parent to read their status after they have finished running.

There are two ways to remove them:

• wait for them in the caller shortly after they have finished. The caller could setup a handler for SIGCHLD to be warned that one of its child have just ended, or simply poll them from time to time
• just ignore them. If the caller explicitely ignores SIGCHLD (signal(SIGCHLD, SIG_IGN);) at fork time, the childs will be immediately removed (no defunct state), but the parent has no more way to wait for them.

Solution 2

Children which exited are <defunct> until the parent waits for them. Since you are calling wait in reap_children() only when you fork() a new child, the exited children stay <defunct> until the next child is forked. To get rid of them earlier, you could call reap_children() (i. e. wait) at earlier times. Normally, this is done through a handler for the SIGCHLD signal.