Python equivalent of golang's defer statement

Question

How would one implement something that works like the defer statement from go in python?

Defer pushes a function call to a stack. When the function containing the defer statement returns, the defered function calls are popped and executed one by one, in the scope that the defer statement was inside in the first place. Defer statements look like function calls, but are not executed until they are popped.

Go example of how it works:

func main() {
    fmt.Println("counting")

    var a *int
    for i := 0; i < 10; i++ {
        a = &i
        defer fmt.Println(*a, i)
    }

    x := 42
    a = &x

    fmt.Println("done")
}

Outputs:

counting
done
9 9
8 8
7 7
6 6
5 5
4 4
3 3
2 2
1 1
0 0

Go example of a usecase:

var m sync.Mutex
func someFunction() {
    m.Lock()
    defer m.Unlock()
    // Whatever you want, with as many return statements as you want, wherever.
    // Simply forget that you ever locked a mutex, or that you have to remember to release it again.
}

Solution 1

To emulate defer fmt.Println(*a, i) example, you could use contextlib.ExitStack:

#!/usr/bin/env python3
from contextlib import ExitStack
from functools import partial

print("counting")
with ExitStack() as stack:
    for i in range(10):
        a = i
        stack.callback(partial(print, a, i))

    x = 42
    a = x
    print("done")

Output

counting
done
9 9
8 8
7 7
6 6
5 5
4 4
3 3
2 2
1 1
0 0

It is easy to emulate the mutex case:

def some_function(lock=Lock()):
    with lock:
        # whatever

Solution 2

Python's with statement serves a similar purpose to Go's defer.

The similar code in Python is:

mutex = Lock()

def someFunction():
    with mutex:
        # Whatever you want, with as many return statements
        # as you want, wherever. Simply forget that you ever
        # locked a mutex, or that you have to remember to 
        # release it again.

Solution 3

I've made one there (compatible with 2.x):

@defers_collector
def func():
    f = open('file.txt', 'w')
    defer(lambda: f.close())

    defer(lambda : print("Defer called!"))

    def my_defer():
    recover()

    defer(lambda: my_defer())

    print("Ok )")
    panic("WTF?")

    print("Never printed (((")


func()
print("Recovered!")

Source of defers_collector is:

# Go-style error handling

import inspect
import sys

def panic(x):
    raise Exception(x)

def defer(x):
    for f in inspect.stack():
    if '__defers__' in f[0].f_locals:
        f[0].f_locals['__defers__'].append(x)
        break

def recover():
    val = None
    for f in inspect.stack():
    loc = f[0].f_locals
    if f[3] == '__exit__' and '__suppress__' in loc:
        val = loc['exc_value']
        loc['__suppress__'].append(True)
        break
    return val

class DefersContainer(object):
    def __init__(self):
    # List for sustain refer in shallow clone
    self.defers = []

    def append(self, defer):
    self.defers.append(defer)

    def __enter__(self):
    pass

    def __exit__(self, exc_type, exc_value, traceback):
    __suppress__ = []
    for d in reversed(self.defers):
        try:
            d()
        except:
            __suppress__ = []
            exc_type, exc_value, traceback = sys.exc_info()
    return __suppress__


def defers_collector(func):
    def __wrap__(*args, **kwargs):
    __defers__ = DefersContainer()
    with __defers__:
        func(*args, **kwargs)
    return __wrap__

Solution 4

A defer implementation partly inspired by @DenisKolodin answer is available as part of pygolang, 2:

   wc = wcfs.join(zurl)    │     wc = wcfs.join(zurl)
   defer(wc.close)         │     try:
                           │        ...
   ...                     │        ...
   ...                     │        ...
   ...                     │     finally:
                           │        wc.close()

Solution 5

I've tried to make an equivalent for fun (only tested as a proof of concept)

Here is is:

import os
import inspect


class defer:
    """
    Proof of concept for a python equivalent of golang's defer statement

    Note that the callback order is probably not guaranteed

    """
    def __init__(self, callback, *args, **kwargs):
        self.callback = callback
        self.args = args
        self.kwargs = kwargs

        # Add a reference to self in the caller variables so our __del__
        # method will be called when the function goes out of scope
        caller = inspect.currentframe().f_back
        caller.f_locals[b'_' + os.urandom(48)] = self

    def __del__(self):
        self.callback(*self.args, **self.kwargs)

Usage example:

def main():
    first()
    second()

def first():
    print('- first')
    defer(lambda: print('   - deferred'))
    print('- first exit')

def second():
    print('- second')      

if __name__ == '__main__':
    main()

View more solutions

Comments

[a p]

One thing to remember here is that you need to have a context manager defined to use with. Lots of things (files, threading.Lock, for instance) come with this built-in, but if you want to do something specific like in the first example, you have to custom build it.

[kirr]

A defer implementation partly inspired by this answer is available as part of pygolang: stackoverflow.com/a/53069630/9456786

[Quinten Cabo]

I build this custom thing github.com/tintin10q/python_json_database_manager