Using a function defined in an exec'ed string in Python 3

Question

Why does the following python3 code produces an error?

a='''
def x():
  print(42)
'''

class Test:
    def __init__(self):
        exec(a)
        x()

t = Test()

Results in this message:

Traceback (most recent call last):
  File "bug.py", line 11, in <module>
    t = Test()
  File "bug.py", line 9, in __init__
    x()
NameError: global name 'x' is not defined

Solution 1

Note: exec was just a Simple statement in Python 2.x, whereas it is a function in Python 3.x.

Python 2.7

Let us check the changes made by executing a.

class Test:
    def __init__(self):
        l, g = locals().copy(), globals().copy()
        exec a           # NOT a function call but a statement
        print locals() == l, globals() == g
        x()

t = Test()

Output

False True
42

It means that, it has changed something in the locals dictionary. If you print locals().keys() before and after the exec, you will see x, after exec. As per the documentation of exex,

In all cases, if the optional parts are omitted, the code is executed in the current scope.

So, it does exactly what the documentation says.

Python 3.x:

When we execute the same in Python 3.x, we get similar result, except we get that error.

class Test:
    def __init__(self):
        l, g = locals().copy(), globals().copy()
        exec(a)          # Function call, NOT a statement
        print(locals() == l, globals() == g)
        x()

Output

False True
NameError: name 'x' is not defined

Even the documentation of exec function says,

In all cases, if the optional parts are omitted, the code is executed in the current scope.

But it also includes a note at the bottom,

Note: The default locals act as described for function locals() below: modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after function exec() returns.

So, we curiously check the locals() documentation and find

Note: The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.

So, interpreter doesn't honor the changes made to the locals() object. That is why it is not recognizing x as defined in the local scope.

But when we do

def __init__(self):
    exec(a, globals())
    x()

it works, because we add it to the globals dictionary. Python tries to find x in local scope first and then in class scope and then in global scope and it finds it there. So it executes it without any problem.

Solution 2

I am assuming you are using Python3.x, since in Python2.7, your code is working fine for me. So for Python3.x, change the line

exec(a)

to

exec(a, globals())

in order to add x to the global namespace.

Documentation

Solution 3

Python3 exec also takes globals and locals optional arguments of mapping type, which act as a context for the given code execution:

exec(object[, globals[, locals]])

By default the local scope gets passed in for both. The executed code can use it, can also modify the dict, but it will have no effect on the actual local scope. See locals() and example:

a = '''
print(t)

def x():
    print(42)
'''

class Test:
    def __init__(self):
        t = 'locals are accessible'
        # same as calling exec(a, locals())
        exec(a)
        print(locals())
        x()

t = Test()

Output:

locals are accessible
{'x': <function x at 0x6ffffd09af0>,
'self': <__main__.Test object at 0x6ffffce3f90>,
't': 'locals are accessible'}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 13, in __init__
NameError: global name 'x' is not defined

If you want the x to be available after the exec call you either need to pass in a global or a custom scope:

# global scope
class Test:
    def __init__(self):
        exec(a, globals())
        x()

# custom scope
class Test:
    def __init__(self):
        scope = {}
        exec(a, scope)
        scope['x']()

Comments

[Martijn Pieters]

In Python 2, exec is a statement, not a function. The compiler disables the local namespace optimisation when you use exec, which is why it works there. Python 3 no longer does this.

[Cecil Curry]

Excellent commentary. In my case, "Pass an explicit locals dictionary if you need to see effects of the code on locals after function exec() returns." was exactly the obscure solution I was uselessly grepping about for. I grep no longer. Thanks!

[Rea Haas]

This is exactly what I was needed! A way to save locally the function from the string code and call it in the local scope without effect the global scope of the program. Excellent :)

[Glen Thompson]

Same, this is exactly what I need, Thank you!

[Rick]

Hi @amousgarkin, where did you know this "By default the local scope gets passed in for both." ? There's no default value shown in the exec function signature