python 'with' statement, should I use contextlib.closing?

Question

from contextlib import closing

def init_db():
    with closing(connect_db()) as db:
        with app.open_resource('schema.sql') as f:
            db.cursor().executescript(f.read())
        db.commit()

This is from flask tutorial Step 3(http://flask.pocoo.org/docs/tutorial/dbinit/#tutorial-dbinit). And I'm little curious about the line 4 of that.

Must I import and use that 'contextlib.closing()' method?

When I've learned about with statement, many articles said that it closes file automatically after process like below.(same as Finally: thing.close())

with open('filename','w') as f:
    f.write(someString);

Even though I don't use that contextlib.closing() like below, What's difference? It's from version 2.7.6, Thank you.

def init_db():
    with connect_db() as db:
        with app.open_resource('schema.sql') as f:
            db.cursor().executescript(f.read())
        db.commit()

Solution 1

Yes, you should be using context.closing(); your own version does something different entirely.

The with statement lets a context manager know when a block of code is entered and exited; on exit the context manager is also given access to the exception, if one occurred. File objects use this to automatically close the file when the block is exited.

The connect_db() function from the tutorial returns a sqlite3 connection object, which can indeed be used as a context manager. However, the connection.__exit__() method doesn't close the connection, it commits the transaction on a successful completion, or aborts it when there is an exception.

The contextlib.closing() context manager on the other hand, calls the connection.close() method on the connection. This is something entirely different.

So, your second snippet may work, but does something different. The tutorial code closes the connection, your version commits a transaction. You are already calling db.commit(), so the action is actually redundant provided no exceptions are raised.

You could use the connection as a context manager again to have the automatic transaction handling behaviour:

def init_db():
    with closing(connect_db()) as db:
        with app.open_resource('schema.sql') as f, db:
            db.cursor().executescript(f.read())

Note the , db on the second with line, ensuring that the db.__exit__() method is called when the block exits.

Solution 2

The only thing done by the with statement is to call __enter__ method before entering its block and __exit__ method before exiting it. If those methods are not defined the with statement won't work as you may expect. I don't know what is the return type of connect_db, but I guess that it could be many different things from different third-party libraries. So, your code without closing will probably work in many (all?) cases, but you never know what can be returned by connect_db.

Comments

[Dave B]

I don't really understand your last statement (regarding ", db"). Could you elaborate how this works? Isn't db closed when the outer context (with closing...) finishes? Why pull db into the inner, nested context?

[Martijn Pieters]

@DavidBarker: with closing(connect_db()) as db does two things: it creates a context manager that will call close() on the connect_db() result when exiting, and it returns that same object from the cm.__enter__() method, so with assigns that to the name db. Now, that object itself is also a context manager, so in the second with we ensure that with calls db.__enter__() and db.__exit__() too, because those control the transaction.

[Dave B]

So the closing() context manager only calls close() and doesn't call __enter__() or __exit__() (even if the object is itself a context manager) at the end of the block? That makes perfect sense - thanks for taking the time to reply to a comment on such an old post.