Removing cached files after a pytest run

Question

I'm using a joblib.Memory to cache expensive computations when running tests with py.test. The code I'm using reduces to the following,

from joblib import Memory

memory = Memory(cachedir='/tmp/')

@memory.cache
def expensive_function(x):
    return x**2   # some computationally expensive operation here

def test_other_function():
    input_ds = expensive_function(x=10)
    ## run some tests with input_ds

which works fine. I'm aware this could be possibly more elegantly done with tmpdir_factory fixture but that's beside the point.

The issue I'm having is how to clean the cached files once all the tests run,

• is it possible to share a global variable among all tests (which would contains e.g. a list of path to the cached objects) ?
• is there a mechanism in py.test to call some command once all the tests are run (whether they succeed or not)?

Solution 1

is it possible to share a global variable among all tests (which would contains e.g. a list of path to the cached objects) ?

I wouldn't go down that path. Global mutable state is something best avoided, particularly in testing.

is there a mechanism in py.test to call some command once all the tests are run (whether they succeed or not)?

Yes, add an auto-used session-scoped fixture into your project-level conftest.py file:

# conftest.py
import pytest

@pytest.yield_fixture(autouse=True, scope='session')
def test_suite_cleanup_thing():
    # setup
    yield
    # teardown - put your command here

The code after the yield will be run - once - at the end of the test suite, regardless of pass or fail.

Solution 2

is it possible to share a global variable among all tests (which would contains e.g. a list of path to the cached objects) ?

There are actually a couple of ways to do that, each with pros and cons. I think this SO answer sums them up quite nice - https://stackoverflow.com/a/22793013/3023841 - but for example:

def pytest_namespace():
     return  {'my_global_variable': 0}

def test_namespace(self):
     assert pytest.my_global_variable == 0

is there a mechanism in py.test to call some command once all the tests are run (whether they succeed or not)?

Yes, py.test has teardown functions available:

def setup_module(module):
    """ setup any state specific to the execution of the given module."""

def teardown_module(module):
    """ teardown any state that was previously setup with a setup_module
    method.
    """

Comments

[rth]

Thanks, that's what I was looking for. Agree with you regarding global variables in test..

[rth]

Thanks for your response. I think I will go with the other solution, but this is definitely useful information. I didn't know about the module level teardown functions.