Why db.session.remove() must be called?

Question

I'm following a tutorial to learn flask web developing, and here is its unit testing file:

import unittest
from flask import current_app
from app import create_app, db

class BasicsTestCase(unittest.TestCase):
    def setUp(self):
        self.app = create_app('testing')
        self.app_context = self.app.app_context()
        self.app_context.push()
        db.create_all()

    def tearDown(self):
        db.session.remove()
        db.drop_all()
        self.app_context.pop()

    def test_foo(self):
        pass

Also, I've found these sentences in SQLAlchemy document:

Using the above flow, the process of integrating the Session with the web application has exactly two requirements:

• ......

• Ensure that scoped_session.remove() is called when the web request ends, usually by integrating with the web framework’s event system to establish an “on request end” event.

My question is: Why do I need to call db.session.remove()?

I think as long as db.session.commit() is not invoked, the database won't be modified. Also, when I comment out this line, the application will still be able to pass the unit test.

I've consulted the documents of both Flask-SQLAlchemy and SQLAlchemy, but the former doesn't even mention db.session.remove(), while the latter is too abstract for me to understand.

Solution 1

Using the above flow, the process of integrating the Session with the web application has exactly two requirements:

• ......
• Ensure that scoped_session.remove() is called when the web request ends, usually by integrating with the web framework’s event system to establish an “on request end” event.

In SQLAlchemy, above action is mentioned because sessions in web application should be scoped, meaning that each request handler creates and destroys its own session.

This is necessary because web servers can be multi-threaded, so multiple requests might be served at the same time, each working with a different database session.

This scenario is beautifully handled by Flask-SQLAlchemy, it creates a fresh or new scoped session for each request. If you dig further it you will find out here, it also installs a hook on app.teardown_appcontext (for Flask >=0.9), app.teardown_request (for Flask 0.7-0.8), app.after_request (for Flask <0.7) and here is where it calls db.session.remove().

The testing environment does not fully replicate the environment of a real request because it does not push/pop the application context. Because of that the session is never removed at the end of the request.

As a side note, keep in mind that functions registered with before_request and after_request are also not called when you call client.get().

You can force an application context automatically push and pop with a small change to your test instead of manually push in setUp() and pop in tearDown():

def test_foo(self):
  with app.app_context():
      client = app.test_client()
      # do testing here for your endpoints
  

with this change the test passes without manually writing db.session.remove().

The documentation for Flask-Testing seems to be wrong or more likely outdated. Maybe things worked like they describe at some point, but that isn't accurate for current Flask and Flask-SQLAlchemy versions.

I hope this helps!

Solution 2

I haven't understand why db.session.remove() is necessary until I inspected the whole project:

This is because in config.py, SQLALCHEMY_COMMIT_ON_TEARDOWN is set to True. As a result, changes made to db.session would be auto-commited if db.session isn't destroyed.

Comments

[nalzok]

It's very helpful, and I love your with app.app_context(): approach!

[nalzok]

Hello, sir. I just reviewed this answer, but now I can't understand why you said The testing environment does not fully replicate the environment of a real request because it does not push/pop the application context. What's the difference between (i) manually pushing in setUp() and poping in tearDown(), and (ii) with app.app_context():?

[Michael Ekoka]

my tests seem to indicate that after_request and before_request do get called during client.get(). However, it's true that triggering client.get() while within an AppContext the way you did will result in RequestContext not pushing an AppContext of its own (which is incidentally why there's no AppContext teardown and no triggering of the registered teardown_appcontext handlers). The doc does mention that app.test_request_context() does not trigger before_request and after_request.