Short description of the scoping rules?

Question

What exactly are the Python scoping rules?

If I have some code:

code1
class Foo:
   code2
   def spam.....
      code3
      for code4..:
       code5
       x()

Where is x found? Some possible choices include the list below:

1. In the enclosing source file
2. In the class namespace
3. In the function definition
4. In the for loop index variable
5. Inside the for loop

Also there is the context during execution, when the function spam is passed somewhere else. And maybe lambda functions pass a bit differently?

There must be a simple reference or algorithm somewhere. It's a confusing world for intermediate Python programmers.

Solution 1

Actually, a concise rule for Python Scope resolution, from Learning Python, 3rd. Ed.. (These rules are specific to variable names, not attributes. If you reference it without a period, these rules apply.)

LEGB Rule

• Local — Names assigned in any way within a function (def or lambda), and not declared global in that function

• Enclosing-function — Names assigned in the local scope of any and all statically enclosing functions (def or lambda), from inner to outer

• Global (module) — Names assigned at the top-level of a module file, or by executing a global statement in a def within the file

• Built-in (Python) — Names preassigned in the built-in names module: open, range, SyntaxError, etc

So, in the case of

code1
class Foo:
    code2
    def spam():
        code3
        for code4:
            code5
            x()

The for loop does not have its own namespace. In LEGB order, the scopes would be

• L: Local in def spam (in code3, code4, and code5)
• E: Any enclosing functions (if the whole example were in another def)
• G: Were there any x declared globally in the module (in code1)?
• B: Any builtin x in Python.

x will never be found in code2 (even in cases where you might expect it would, see Antti's answer or here).

Solution 2

Essentially, the only thing in Python that introduces a new scope is a function definition. Classes are a bit of a special case in that anything defined directly in the body is placed in the class's namespace, but they are not directly accessible from within the methods (or nested classes) they contain.

In your example there are only 3 scopes where x will be searched in:

• spam's scope - containing everything defined in code3 and code5 (as well as code4, your loop variable)

• The global scope - containing everything defined in code1, as well as Foo (and whatever changes after it)

• The builtins namespace. A bit of a special case - this contains the various Python builtin functions and types such as len() and str(). Generally this shouldn't be modified by any user code, so expect it to contain the standard functions and nothing else.

More scopes only appear when you introduce a nested function (or lambda) into the picture. These will behave pretty much as you'd expect however. The nested function can access everything in the local scope, as well as anything in the enclosing function's scope. eg.

def foo():
    x=4
    def bar():
        print x  # Accesses x from foo's scope
    bar()  # Prints 4
    x=5
    bar()  # Prints 5

Restrictions:

Variables in scopes other than the local function's variables can be accessed, but can't be rebound to new parameters without further syntax. Instead, assignment will create a new local variable instead of affecting the variable in the parent scope. For example:

global_var1 = []
global_var2 = 1

def func():
    # This is OK: It's just accessing, not rebinding
    global_var1.append(4) 

    # This won't affect global_var2. Instead it creates a new variable
    global_var2 = 2 

    local1 = 4
    def embedded_func():
        # Again, this doen't affect func's local1 variable.  It creates a 
        # new local variable also called local1 instead.
        local1 = 5
        print local1

    embedded_func() # Prints 5
    print local1    # Prints 4

In order to actually modify the bindings of global variables from within a function scope, you need to specify that the variable is global with the global keyword. Eg:

global_var = 4
def change_global():
    global global_var
    global_var = global_var + 1

Currently there is no way to do the same for variables in enclosing function scopes, but Python 3 introduces a new keyword, "nonlocal" which will act in a similar way to global, but for nested function scopes.

Solution 3

There was no thorough answer concerning Python3 time, so I made an answer here. Most of what is described here is detailed in the 4.2.2 Resolution of names of the Python 3 documentation.

As provided in other answers, there are 4 basic scopes, the LEGB, for Local, Enclosing, Global and Builtin. In addition to those, there is a special scope, the class body, which does not comprise an enclosing scope for methods defined within the class; any assignments within the class body make the variable from there on be bound in the class body.

Especially, no block statement, besides def and class, create a variable scope. In Python 2 a list comprehension does not create a variable scope, however in Python 3 the loop variable within list comprehensions is created in a new scope.

To demonstrate the peculiarities of the class body

x = 0
class X(object):
    y = x
    x = x + 1 # x is now a variable
    z = x

    def method(self):
        print(self.x) # -> 1
        print(x)      # -> 0, the global x
        print(y)      # -> NameError: global name 'y' is not defined

inst = X()
print(inst.x, inst.y, inst.z, x) # -> (1, 0, 1, 0)

Thus unlike in function body, you can reassign the variable to the same name in class body, to get a class variable with the same name; further lookups on this name resolve to the class variable instead.

---

One of the greater surprises to many newcomers to Python is that a for loop does not create a variable scope. In Python 2 the list comprehensions do not create a scope either (while generators and dict comprehensions do!) Instead they leak the value in the function or the global scope:

>>> [ i for i in range(5) ]
>>> i
4

The comprehensions can be used as a cunning (or awful if you will) way to make modifiable variables within lambda expressions in Python 2 - a lambda expression does create a variable scope, like the def statement would, but within lambda no statements are allowed. Assignment being a statement in Python means that no variable assignments in lambda are allowed, but a list comprehension is an expression...

This behaviour has been fixed in Python 3 - no comprehension expressions or generators leak variables.

---

The global really means the module scope; the main python module is the __main__; all imported modules are accessible through the sys.modules variable; to get access to __main__ one can use sys.modules['__main__'], or import __main__; it is perfectly acceptable to access and assign attributes there; they will show up as variables in the global scope of the main module.

---

If a name is ever assigned to in the current scope (except in the class scope), it will be considered belonging to that scope, otherwise it will be considered to belonging to any enclosing scope that assigns to the variable (it might not be assigned yet, or not at all), or finally the global scope. If the variable is considered local, but it is not set yet, or has been deleted, reading the variable value will result in UnboundLocalError, which is a subclass of NameError.

x = 5
def foobar():
    print(x)  # causes UnboundLocalError!
    x += 1    # because assignment here makes x a local variable within the function

# call the function
foobar()

The scope can declare that it explicitly wants to modify the global (module scope) variable, with the global keyword:

x = 5
def foobar():
    global x
    print(x)
    x += 1

foobar() # -> 5
print(x) # -> 6

This also is possible even if it was shadowed in enclosing scope:

x = 5
y = 13
def make_closure():
    x = 42
    y = 911
    def func():
        global x # sees the global value
        print(x, y)
        x += 1

    return func

func = make_closure()
func()      # -> 5 911
print(x, y) # -> 6 13

In python 2 there is no easy way to modify the value in the enclosing scope; usually this is simulated by having a mutable value, such as a list with length of 1:

def make_closure():
    value = [0]
    def get_next_value():
        value[0] += 1
        return value[0]

    return get_next_value

get_next = make_closure()
print(get_next()) # -> 1
print(get_next()) # -> 2

However in python 3, the nonlocal comes to rescue:

def make_closure():
    value = 0
    def get_next_value():
        nonlocal value
        value += 1
        return value
    return get_next_value

get_next = make_closure() # identical behavior to the previous example.

The nonlocal documentation says that

Names listed in a nonlocal statement, unlike those listed in a global statement, must refer to pre-existing bindings in an enclosing scope (the scope in which a new binding should be created cannot be determined unambiguously).

i.e. nonlocal always refers to the innermost outer non-global scope where the name has been bound (i.e. assigned to, including used as the for target variable, in the with clause, or as a function parameter).

---

Any variable that is not deemed to be local to the current scope, or any enclosing scope, is a global variable. A global name is looked up in the module global dictionary; if not found, the global is then looked up from the builtins module; the name of the module was changed from python 2 to python 3; in python 2 it was __builtin__ and in python 3 it is now called builtins. If you assign to an attribute of builtins module, it will be visible thereafter to any module as a readable global variable, unless that module shadows them with its own global variable with the same name.

---

Reading the builtin module can also be useful; suppose that you want the python 3 style print function in some parts of file, but other parts of file still use the print statement. In Python 2.6-2.7 you can get hold of the Python 3 print function with:

import __builtin__

print3 = __builtin__.__dict__['print']

The from __future__ import print_function actually does not import the print function anywhere in Python 2 - instead it just disables the parsing rules for print statement in the current module, handling print like any other variable identifier, and thus allowing the print the function be looked up in the builtins.

Solution 4

A slightly more complete example of scope:

from __future__ import print_function  # for python 2 support

x = 100
print("1. Global x:", x)
class Test(object):
    y = x
    print("2. Enclosed y:", y)
    x = x + 1
    print("3. Enclosed x:", x)

    def method(self):
        print("4. Enclosed self.x", self.x)
        print("5. Global x", x)
        try:
            print(y)
        except NameError as e:
            print("6.", e)

    def method_local_ref(self):
        try:
            print(x)
        except UnboundLocalError as e:
            print("7.", e)
        x = 200 # causing 7 because has same name
        print("8. Local x", x)

inst = Test()
inst.method()
inst.method_local_ref()

output:

1. Global x: 100
2. Enclosed y: 100
3. Enclosed x: 101
4. Enclosed self.x 101
5. Global x 100
6. global name 'y' is not defined
7. local variable 'x' referenced before assignment
8. Local x 200

Solution 5

The scoping rules for Python 2.x have been outlined already in other answers. The only thing I would add is that in Python 3.0, there is also the concept of a non-local scope (indicated by the 'nonlocal' keyword). This allows you to access outer scopes directly, and opens up the ability to do some neat tricks, including lexical closures (without ugly hacks involving mutable objects).

EDIT: Here's the PEP with more information on this.

View more solutions

Comments

[Brian]

This is out of date. Since 2.1 (7 years ago) there are more than two scopes, as nested functions introduce new scopes, so a function within a function will have access to its local scope, the enclosing functions scope, and global scope (also builtins).

[Rizwan Kassim]

I'm sorry, this is no longer the case. Python has two namespaces available. Global and local-to-something.

[Peter Gibson]

As a caveat to Global access - reading a global variable can happen without explicit declaration, but writing to it without declaring global(var_name) will instead create a new local instance.

[martineau]

Actually @Peter, global(var_name) is syntactically incorrect. The correct syntax would be global var_name without parentheses. You have a valid point though.

[Jonathan Mayer]

If so, then why isn't foo's "y" variable visible to "bar" below: >>> def foo(x): ... y = x ... def bar(z): ... y = z ... bar(5) ... print x,y ... >>> foo(3) 3 3

[martineau]

@Jonathan: Because each y is being written to and there are no global y declarations -- see @Peter's comment.

[Ctrl-C]

@LakshmanPrasad It falls into "E", but has one special behaviour that's worth mentioning: it is a class variable, so it is a "global" among it's objects. Assigning to it will lead to unpredicted and hard to debug issues if you don't know what you're doing.

[kiril]

This is great answer. However, I think that the differences between method and method_local_ref should be highlighted. method is able to access the global variable and print it as in 5. Global x. But method_local_ref cannot because later on it defines a local variable with that same name. You can test this by removing the x = 200 line and see the difference

[Malik A. Rumi]

@brianray: What about z?

[brianray]

@kiril I added a note about that

[brianray]

@MalikA.Rumi I removed z as it wasn't interesting

[not2qubit]

Surprisingly, this is the only clear explanation of Python scopes, that I could find on all of SO. Simply using a very basic example. Thanks!

[chepner]

@Ctrl-C Not really; there is nothing special about class attributes in terms of scope. They are shared in the sense that self.someClassAttribute will refer to the same object regardless of which instance self refers to, but the name itself does have to be used as an attribute on an instance or the class itself. The actual special behavior is that while evaluating the statements in the class body, the class attribute will shadow any variables existing in the containing scope. E.g. j = 0; class Foo: j = 3; print(j); # end of class; print(j) will output 3, then 0.

[martineau]

Glad to finally see an answer that mentions the special class body scope which isn't covered by the fairly well-known LEGB rule.

[John Henckel]

I think your answer is incomplete. The except..as statement will also create a new scope. For example if you put try: raise ValueError('x'); except ValueError as v: pass you cannot access v outside the scope of the except clause

[MisterMiyagi]

@JohnHenckel That is not a new scope. except deletes its target from its scope when done. The target follows the regular scoping rules, e.g. it can even be declared global and will be deleted from global scope in this case. Demonstrator code.

[John Henckel]

OMG this is strange. Thank you for explaining that to me.