how to share a variable across modules for all tests in py.test
Solution 1
Update: pytest-namespace hook is deprecated/removed. Do not use. See #3735 for details.
You mention the obvious and least magical option: using a fixture. You can apply it to entire modules using pytestmark = pytest.mark.usefixtures('big_dict')
in your module, but then it won't be in your namespace so explicitly requesting it might be best.
Alternatively you can assign things into the pytest namespace using the hook:
# conftest.py
def pytest_namespace():
return {'my_big_dict': {'foo': 'bar'}}
And now you have pytest.my_big_dict
. The fixture is probably still nicer though.
Solution 2
There are tons of things I love about py.test, but one thing I absolutely HATE is how poorly it plays with code intelligence tools. I disagree that an autouse fixture to declare a variable is the "most clear" method in this case because not only does it completely baffle my linter, but also anyone else who is not familiar with how py.test works. There is a lot of magic there, imo.
So, one thing you can do that doesn't make your linter explode and doesn't require TestCase boilerplate is to create a module called globals. Inside this module, stub the names of the things you want global to {} or None and import the global module into your tests. Then in your conftest.py file, use the py.test hooks to set (or reset) your global variable(s) as appropriate. This has the advantage of giving you the stub to work with when building tests and the full data for the tests at runtime.
For example, you can use the pytest_configure()
hook to set your dict right when py.test starts up. Or, if you wanted to make sure the data was pristine between each test, you could autouse a fixture to assign your global variable to your known state before each test.
# globals.py
my_data = {} # Create a stub for your variable
# test_module.py
import globals as gbl
def test_foo():
assert gbl.my_data['foo'] == 'bar' # The global is in the namespace when creating tests
# conftest.py
import globals as gbl
my_data = {'foo': 'bar'} # Create the master copy in conftest
@pytest.fixture(autouse=True)
def populate_globals():
gbl.my_data = my_data # Assign the master value to the global before each test
One other advantage to this approach is you can use type hinting in your globals module to give you code completion on the global objects in your test, which probably isn't necessary for a dict but I find it handy when I am using an object (such as webdriver). :)
Solution 3
I'm suprised no answer mentioned caching yet: since version 2.8, pytest
has a powerful cache mechanism.
Usage example
@pytest.fixture(autouse=True)
def init_cache(request):
data = request.config.cache.get('my_data', None)
data = {'spam': 'eggs'}
request.config.cache.set('my_data', data)
Access the data dict in tests via builtin request
fixture:
def test_spam(request):
data = request.config.cache.get('my_data')
assert data['spam'] == 'eggs'
Sharing the data between test runs
The cool thing about request.cache
is that it is persisted on disk, so it can be even shared between test runs. This comes handy when you running tests distributed (pytest-xdist
) or have some long-running data generation which does not change once generated:
@pytest.fixture(autouse=True)
def generate_data(request):
data = request.config.cache.get('my_data', None)
if data is None:
data = long_running_generation_function()
request.config.cache.set('my_data', data)
Now the tests won't need to recalculate the value on different test runs unless you clear the cache on disk explicitly. Take a look what's currently in the cache:
$ pytest --cache-show
...
my_data contains:
{'spam': 'eggs'}
Rerun the tests with the --cache-clear
flag to delete the cache and force the data to be recalculated. Or just remove the .pytest_cache
directory in the project root dir.
Where to go from here
The related section in pytest
docs: Cache: working with cross-testrun state.
Solution 4
Having a big dictionary of globals that every test uses is probably a bad idea. If possible, I suggest refactoring your tests to avoid this sort of thing.
That said, here is how I would do it: define an autouse fixture that adds a reference to the dictionary in the global namespace of every function.
Here is some code. It's all in the same file, but you can move the fixture out to conftest.py
at the top level of your tests.
import pytest
my_big_global = {'key': 'value'}
@pytest.fixture(autouse=True)
def myglobal(request):
request.function.func_globals['foo'] = my_big_global
def test_foo():
assert foo['key'] == 'value'
def test_bar():
assert foo['key'] == 'bar'
Here is the output from when I run this code:
$ py.test test_global.py -vv
======================================= test session starts =======================================
platform darwin -- Python 2.7.5 -- py-1.4.20 -- pytest-2.5.2 -- env/bin/python
collected 2 items
test_global.py:9: test_foo PASSED
test_global.py:12: test_bar FAILED
============================================ FAILURES =============================================
____________________________________________ test_bar _____________________________________________
def test_bar():
> assert foo['key'] == 'bar'
E assert 'value' == 'bar'
E - value
E + bar
test_global.py:13: AssertionError
=============================== 1 failed, 1 passed in 0.01 seconds ===============================
Note that you can't use a session-scoped fixture because then you don't have access to each function object. Because of this, I'm making sure to define my big global dictionary once and use references to it -- if I defined the dictionary in that assignment statement, a new copy would be made each time.
In closing, doing anything like this is probably a bad idea. Good luck though :)
Solution 5
You can add your global variable as an option inside the pytest_addoption
hook.
It is possible to do it explicitly with addoption
or use set_defaults
method if you want your attribute be determined without any inspection of the command line, docs
When option was defined, you can paste it inside any fixture with request.config.getoption
and then pass it to the test explicitly or with autouse.
Alternatively, you can pass your option into almost any hook inside the config
object.
#conftest.py
def pytest_addoption(parser):
parser.addoption("--my_global_var", default="foo")
parser.set_defaults(my_hidden_var="bar")
@pytest.fixture()
def my_hidden_var(request):
return request.config.getoption("my_hidden_var")
#test.py
def test_my_hidden_var(my_hidden_var):
assert my_hidden_var == "bar"
Trevor
Updated on December 06, 2020Comments
-
Trevor over 3 years
I have multiple tests run by py.test that are located in multiple classes in multiple files.
What is the simplest way to share a large dictionary - which I do not want to duplicate - with every method of every class in every file to be used by py.test?
In short, I need to make a "global variable" for every test. Outside of py.test, I have no use for this variable, so I don't want to store it in the files being tested. I made frequent use of py.test's fixtures, but this seems overkill for this need. Maybe it's the only way?
-
Trevor about 10 yearsThanks for interesting possibility, Frank! It looks as though it would work, but manipulating the function's namespace is a bit too much for me too. :) ... Thanks!
-
Trevor about 10 yearsI was originally leaning this way, but PyDev (Eclipse) gets confused and complains of all usages as undeclared variables. ... If you had some way that obviously added this to the namespace (so that Eclipse-PyDev would be happy) and did not require adding the fixture to every method declaration, that would be the perfect answer for me. :)
-
Joe about 9 yearsI used a similar method and some collegues modified objects within their tests, and then other tests down the line failed. I highly recommend providing a copy, not the actual object if you can support the workload.
-
shelper about 8 yearsfor me it seems
request.function.func_globals
does not work, it should be:request.function.__globals__
, i am under python 3.5 btw. -
Maximiliano Guerra over 7 yearsWhat if I want to define a global variable based on a config? This hook doesn't seem to accept
config
orrequest
fixtures. ``` hook 'pytest_namespace' argument 'config' not available plugin definition: pytest_namespace(config) available hookargs: multicall ``` -
hoefling almost 6 yearsNote that
pytest_namespace
is deprecated since version 3.2 and currently scheduled for removal in 4.0;pytest
itself removed all its internal usages ofpytest_namespace
in version 3.3. Its replacement is the caching mechanism, introduced in version 2.8. -
user9074332 over 5 yearsThis appears to be the most simple and elegant solution I have seen for sharing among the namespace.
-
flub over 5 yearsNotice that more recent pytest versions are deprecating the pytest_namespace hook and it may/will be removed in the future.
-
wim over 5 yearsAlternative approaches are discussed in this thread: github.com/pytest-dev/pytest/issues/2639 TL;DR just use your own namespace, e.g. a module that you import.
-
wim over 5 yearsThis seems like too much spooky action at a distance. Explicit is better than implicit.
-
Joao Coelho about 5 yearsTried this and it works (very interesting solution), but the binding is at runtime and the IDE complains about unresolved reference, which just makes it confusing. BTW, needed to do
request.function.__globals__
like the previous commenter said (Python 3.6). -
Andry over 4 yearsThe cache can not store a module:
TypeError: Object of type module is not JSON serializable
, so I can not store an imported module there. And I can not import in the global context either, because it executed in the collect phase.