How do I unload (reload) a Python module?

Question

I have a long-running Python server and would like to be able to upgrade a service without restarting the server. What's the best way do do this?

if foo.py has changed:
    unimport foo  <-- How do I do this?
    import foo
    myfoo = foo.Foo()

Solution 1

You can reload a module when it has already been imported by using importlib.reload():

from importlib import reload  # Python 3.4+
import foo

while True:
    # Do some things.
    if is_changed(foo):
        foo = reload(foo)

In Python 2, reload was a builtin. In Python 3, it was moved to the imp module. In 3.4, imp was deprecated in favor of importlib. When targeting 3 or later, either reference the appropriate module when calling reload or import it.

I think that this is what you want. Web servers like Django's development server use this so that you can see the effects of your code changes without restarting the server process itself.

To quote from the docs:

• Python module’s code is recompiled and the module-level code re-executed, defining a new set of objects which are bound to names in the module’s dictionary by reusing the loader which originally loaded the module. The init function of extension modules is not called a second time.
• As with all other objects in Python the old objects are only reclaimed after their reference counts drop to zero.
• The names in the module namespace are updated to point to any new or changed objects.
• Other references to the old objects (such as names external to the module) are not rebound to refer to the new objects and must be updated in each namespace where they occur if that is desired.

As you noted in your question, you'll have to reconstruct Foo objects if the Foo class resides in the foo module.

Solution 2

In Python 3.0–3.3 you would use: imp.reload(module)

The BDFL has answered this question.

However, imp was deprecated in 3.4, in favour of importlib (thanks @Stefan!).

I think, therefore, you’d now use importlib.reload(module), although I’m not sure.

Solution 3

It can be especially difficult to delete a module if it is not pure Python.

Here is some information from: How do I really delete an imported module?

You can use sys.getrefcount() to find out the actual number of references.

>>> import sys, empty, os
>>> sys.getrefcount(sys)
9
>>> sys.getrefcount(os)
6
>>> sys.getrefcount(empty)
3

Numbers greater than 3 indicate that it will be hard to get rid of the module. The homegrown "empty" (containing nothing) module should be garbage collected after

>>> del sys.modules["empty"]
>>> del empty

as the third reference is an artifact of the getrefcount() function.

Solution 4

reload(module), but only if it's completely stand-alone. If anything else has a reference to the module (or any object belonging to the module), then you'll get subtle and curious errors caused by the old code hanging around longer than you expected, and things like isinstance not working across different versions of the same code.

If you have one-way dependencies, you must also reload all modules that depend on the reloaded module to get rid of all the references to the old code. And then reload modules that depend on the reloaded modules, recursively.

If you have circular dependencies, which is very common for example when you are dealing with reloading a package, you must unload all the modules in the group in one go. You can't do this with reload() because it will re-import each module before its dependencies have been refreshed, allowing old references to creep into new modules.

The only way to do it in this case is to hack sys.modules, which is kind of unsupported. You'd have to go through and delete each sys.modules entry you wanted to be reloaded on next import, and also delete entries whose values are None to deal with an implementation issue to do with caching failed relative imports. It's not terribly nice but as long as you have a fully self-contained set of dependencies that doesn't leave references outside its codebase, it's workable.

It's probably best to restart the server. :-)

Solution 5

For Python 2 use built-in function reload:

reload(module)

For Python 2 and Python 3.2—3.3 use reload from module imp:

import imp
imp.reload(module)

For Python ≥3.4, imp is deprecated in favor of importlib, so use this:

import importlib
importlib.reload(module)

or:

from importlib import reload
reload(module)

---

TL;DR:

Python ≥ 3.4: importlib.reload(module)
Python 3.2 — 3.3: imp.reload(module)
Python 2: reload(module)

View more solutions

Comments

[Kos]

Memo tip: "import" doesn't mean "load", it means "load if not loaded yet and then import into namespace".

[rjmoggach]

the question should not include 'unload' as that is not possible in python yet - reload is however a known paradigm as answered below

[Pritam Pan]

I had the same problem when using a dynamic module in py2exe app. As py2exe always keep bytecode in zip directory reload was not working. But I found a working solution using import_file module. Now my application is working fine.

[Darkgaze]

What if you want to "unload" because trying to delete a .pyc file is being used by the code?

[hasen]

actually, the django dev server restarts itself when you change a file .. (it restarts the server, not just reloads the module)

[cdleary]

Ah, you're right -- looks like they just reboot the server thread. Well, just know that they could use reload, I guess. :-)

[u0b34a0f6ae]

I just discovered that if the module is a part of a package, you have to delete it there as well: setattr(package, "empty", None)

[Francis Davey]

Actually that doesn't seem to work reliably (in 2.6) because not everything in sys.modules.values() is a module. For example: >>> type(sys.modules.values()[1]) <class 'email.LazyImporter'> So if I try to run that code it falls over (I Know its not meant as a practical solution, just pointing that out).

[Admin]

It doesn't even work in earlier pythons - as written. I had to exclude some names. I'll update the post when I move that code to my new computer.

[Smandoli]

The earnest newbie is grateful to learn about critical nuances between Python 2 and 3.

[Sam]

where is this "is_changed" function coming from? i see no documentation on it and it doesn't run in my Python 3.1.3 environment, nor does it run in 2.6.4.

[Joe Dargie]

@LoïcFaure-Lacroix: good question, I’ve no idea. (I would guess yes?) Might be worth asking as a question.

[hobs]

On the iPython command line reload doesn't appear to follow the import dependency chain, it only reloads the module(s) indicated by the argument.

[Yuri Astrakhan]

Python 3 changed it to import imp / imp.reload(foo)

[Yabi Abdou]

Isn't dreload specifically for that scenario?

[Amit Patil]

@Josh: nope, it's for reloading a package tree, and even then it only works as long as the package doesn't have outside/circular dependencies.

[JBernardo]

@LoïcFaure-Lacroix the same way reload(__builtins__) is valid in 2.x

[Peter D]

+1. My goal was to run nose tests within python. After I had loaded a module and renamed some functions, the old names remained when calling nose.run(), even after reload(my_module) %run my_module

[Czarek Tomczak]

Works fine in Python 2.7 after some modifications: if mod and mod.__name__ != "__main__": imp.reload(mod)

[raylu]

no cdleary, Django can't just use reload: pyunit.sourceforge.net/notes/reloading.html

[John M.]

FYI, in Python 2 at least, you can simply do reload(foo) instead of foo = reload(foo).

[schlamar]

Can you elaborate the part with None values because I'm running exactly into this issue: I'm deleting items from sys.modules and after re-import some imported dependencies are None.

[Amit Patil]

@shclamar: See stackoverflow.com/questions/1958417/… (and the links from there) for background. It's unclear to me (even looking at the import.c code) how the None entries managed to make their way back through the import mechanism when the 'real' entries were deleted, and I can't seem to make it happen on 2.7; in future certainly it's no longer a problem as implicit relative imports have gone away. In the meantime, deleting all entries with None value does seem to fix it.

[Joe Dargie]

@Tarrasch: it's the Python module you want to reload, like in the example in the question.

[fatuhoku]

Strange. The answer only deals with reloading. What about unloading?

[Devyn Collier Johnson]

@LoïcFaure-Lacroix yes, imp can reload itself.

[drevicko]

If your module imports it's own submodules, you may need to delete those too. Something like [del(sys.modules[mod] for mod in sys.modules.keys() if mod.startswith('myModule.')].

[nmz787]

How do I use reload for an import like "from a.b.c import someClass as abcClass"... reload(abcClass) complains that it isn't defined.

[Joseph Garvin]

Noticed an issue, globals() refers to the module you define this function in, so if you define it in a module different than the one you call it in this doesn't work.

[Bob Stein]

For interactive, after >>> from X import Y to reload do >>> __import__('X', fromlist='Y')

[Doug Bradshaw]

reload is not sufficient for modules with dependencies. See bobince below: stackoverflow.com/a/438845/456878. This has bitten me before and wasted a good 10 minutes.

[Christoph]

This works well for me: import imp [reload(m) for m in sys.modules.values() if m and not "" in m.__name and not imp.is_builtin(m.__name__)]

[Mike C]

@BobStein-VisiBone, is there a way to make that work when fromlist='*'?

[Bob Stein]

Good question, don't know @MikeC. By the way I'm trending to cease almost all use of from in import statements. Just stark import <package> and explicit package.symbol in the code. Realize this may not always be possible or desirable. (Here's one exception: from future import print_function.)

[Eliethesaiyan]

it would be nicer for newbies if someone could reference where or from which library the function came from,in the case of this answer 'reload(module)'

[Amit Patil]

@Eliethesaiyan: do you mean the reload function? It is built-in, you don't have to import any library.

[Eliethesaiyan]

@bobince,i think you are right for reload.. i previously used it from package imp

[SiHa]

This answer is a direct copy from here: ebanshi.cc/questions/1942/…

[rampion]

Mike C: what works for me is foo = reload(foo); from foo import *

[Uyghur Lives Matter]

The module will never get garbage collected because a global reference is held at least in sys.modules.

[ivan_pozdeev]

This isn't possible for C extensions in Windows because the .pyd file is in use and cannot be updated.

[Cerin]

This is the correct solution, especially if you have a package with nested modules. reload() only reloads the top-most module, and anything inside it will not be reloaded unless you first delete it from sys.modules.

[James Draper]

How is the top answer for this question? It doesn't work in my py27 and/or py34 @jedmao did you ever find out where "is_changed" is coming from?

[Sam]

@JamesDraper nope, never found out where is_changed came from... or I forgot. It's been so long ago.

[George Fisher]

Add import importlib and importlib.reload( ...

[Tiago Coutinho]

FYI, reload is a source of evil. For example, all instances of objects created with the first import will reference the old class: import foo; bar = foo.Bar(); reload(foo); assert isinstance(bar, foo.Bar) will raise AssertError

[Anentropic]

to handle any of these cases: from six import reload_module (need to pip install six first of course)

[x0s]

@Anentropic: It is good advice to recommend using six package, but the syntax is from six.moves import reload_module (doc)

[pault]

Or just do from importlib import reload. Then you can do reload(MODULE_NAME). There is no need for this function.

[aydow]

@PaulD.Waite, can confirm this works in Python 3.6.5

[Richie Bendall]

I believe modulereload(MODULE_NAME) is more self-explanatory than just reload(MODULE_NAME) and has a lower chance to conflict with other functions.

[mbdevpl]

@RichieBendall Sorry, but this answer is completely wrong. The reload() function takes module object, not module name... Read the docs: docs.python.org/3/library/importlib.html#importlib.reload And I agree with @ pault - this "as modulereload" is superflous.

[Richie Bendall]

I've changed my answer to reflect your opinion.

[m3nda]

Both this Ipython dreload and reload() from importlib does complain with reload() argument must be module. I'm using a custom function import and doesn't seem to work. Using built-in modules does work. :-( it's a waste of time being reloading iPython for every little change i made to my code ...

[James Mchugh]

@jedmao @JamesDraper I am pretty sure the is_changed function is just an arbitrary function that you would have to write; it is not a built-in. For example, it could possibly open the file corresponding to the module that you are importing and diff it with a cached version to see if it changed.

[ahmettolga]

Also a simple addition: If you want to reload changes in a sub module "importlib.reload(module.submodule)" command can bu issued..

[EZLearner]

I wrote a function below that does overrides the previous module's contents with the new one's and hence deals with the problem @bobince rightfully mentioned. See stackoverflow.com/a/61617169/2642356

[EZLearner]

This method might not override other modules' references to the reloaded module. See stackoverflow.com/a/61617169/2642356 for a solution to that.

[Seperman]

I don't think that unloads the module. On Python 3.8: import sys; import json; del sys.modules['json']; print(json.dumps([1])) and json module is still working even though it is not in the sys.modules anymore.

[Doyousketch2]

yea, I noticed an underscored sys.modules['_json'] entry, and it still prints out after taking that into account for mod in [ m for m in sys.modules if m.lstrip('_').startswith('json') ]: del sys.modules[mod]

[Doyousketch2]

yep. Even w/ more aggressive bookeeping, removing entries along with all their dependencies doesn't remove module's capability to function. before = [mod for mod in sys.modules] ; import json ; after = [mod for mod in sys.modules if mod not in before] ; for mod in [ m for m in sys.modules if m in after ]: del sys.modules[mod] ( code-block didn't keep newlines. ; denotes newline )

[Jdeep]

I have to ask. What will happen after the program has been compiled?