python dictionary passed as an input to a function acts like a global in that function rather than a local

Question

I am very confused by the behaviour below. Cases 1, 3, and 4 perform as I would expect, but case 2 does not. Why does case 2 allow the function to change the value of the dictionary entry globally, even though the dictionary is never returned by the function? A main reason I am using functions is to isolate everything in the function from the rest of the code, but this does not seem to be possible if I choose to use the same variable names inside of the function. I was under the understanding that anything explicitly defined in a function is local to that function, but this does not seem to be the case if the dictionary is defined and passed as an input to the function.

Python 2.7.2+ (default, Oct  4 2011, 20:06:09) 
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.

=============Case 1===============

>>> def testfun1(a):
...     a=2
... 
>>> a=0
>>> testfun1(a)
>>> a
0

=============Case 2===============

>>> def testfun2(b):
...     b['test']=2
... 
>>> b={}
>>> testfun2(b)
>>> b
{'test': 2}

=============Case 3===============

>>> def testfun3():
...     c=2
... 
>>> c=0
>>> testfun3()
>>> c
0

=============Case 4=============== (explained by this question: Global dictionaries don't need keyword global to modify them?)

>>> def testfun4():
...     d['test']=10
... 
>>> d={}
>>> testfun4()
>>> d
{'test': 10}

Solution 1

Python's "parameter evaluation strategy" acts a bit different than the languages you're probably used to. Instead of having explicit call by value and call by reference semantics, python has call by sharing. You are essentially always passing the object itself, and the object's mutability determines whether or not it can be modified. Lists and Dicts are mutable objects. Numbers, Strings, and Tuples are not.

You are passing the dictionary to the function, not a copy. Thus when you modify it, you are also modifying the original copy.

To avoid this, you should first copy the dictionary before calling the function, or from within the function (passing the dictionary to the dict function should do it, i.e. testfun4(dict(d)) and defining the function as def testfun4(d):).

Solution 2

To support what @Casey Kuball said, every object in Python is passed by reference. Each function receives a reference to the actual object you passed. Modifying these objects depends on whether they are mutable data types.

In essence, one can say that mutable objects like dictionaries, sets, and lists are passed by reference. Immutable objects like int, str, tuple are passed by value.

You should also note that there are cases where mutable objects are overwritten in a function thereby losing reference to the actual object passed to the function.

>>> def testfun(b):
...     b = b or {}  # Creates a new object if b is false
...     b['test'] = 2
... 
>>> b = {}
>>> testfun(b)
>>> b
{}

Solution 3

When you pass a basic object like an integer or a string to a function, if you change it inside the function nothing occurs to the corresponding object outside the function because when you are leading with a basic object, python passes it by value.

However, if you pass a dictionary or a list to a function they are passed by reference, which means you will have that behaviour: the object outside the function is changed, as you have seen.

edit: In addition, there is a difference between passing by value or by reference: by value, a "copy" of the object is made in order to be used in the function; by reference, the exactly same object is passed through reference and modifications to it inside the function are visible outside. By definition python passes its immutable objects by value, and its mutable objects by reference.

Solution 4

The global keyword is required only for assignment (and likely del, I've never tried it). Object mutations are perfectly valid.

Solution 5

You have passed a dict object to the function and modified it inside the function, so of course it will be modified after the function return. The object is not copied so you modify the same object that you passed, and this question has nothing to do with naming, similar names, scopes etc. as you passed the object explicitly.

View more solutions

Comments

[Pavan Gupta]

for copying dictionary one should use deepcopy method of copy module copy.deepcopy(d) which Returns a deep copy of d. for more information about shallow vs deepcopy please refer docs.python.org/2/library/copy.html

[Casey Kuball]

@PavanGupta That all depends on how 'deep' of a copy you want. If you have a dict whose values are also dicts, do you want to create copies of those dicts, or point to them. This rule/question applies recursively.

[Scott Lundberg]

Answer could be improved by pointing out that strings are immutable and dictionaries are mutable. That's the real difference. A basic object is really not "defined" per se.

[sissi_luaty]

@ScottLundberg But in other languages such as c/c++ strings are mutable; the fact that strings in python are immutable (i.e. basic objects) is a characteristic of python language [they could by an array or a list of characters - and lists are mutable]. (more info in the updated answer...)

[Scott Lundberg]

I agree with you, which is why I made the suggestion to make it clearer.

[Casey Kuball]

I don't believe the example you're giving is anything being "overwritten" here. What's happening is the {} value passed in is being converted to a boolean by the or operator, and being evaluated as "falsy", and thus the or operator returns the next value. I recommend against using the or operator in this way, as it hides other possible errors (e.g. the user passes in a 0 or empty string or None, etc, all of which evaluate to false).

[Chuma Umenze]

"Overwritten" might not be the right word. I concur, there could be hidden errors when evaluating b as boolean. The example was to point out Line 3.

[Casey Kuball]

Wanted to point out that line 2 will be a very poor example for anyone (especially someone coming from JavaScript), as empty dictionaries and lists evaluate to False (bool({}) == False), thus the user of testfun would be unable to pass an empty dictionary and have it work like any other. Anyways, rant over. Wanted to say that your first two paragraphs are very easy to grasp for any C/C++ or similar veteran who would be new to Python. :)

[Ege]

In addition @CaseyKuball Using None as a default parameter is both safe and conventional in such cases. You can then override it in the function itself.

[wjandrea]

I prefer d.copy() since it's more explicit than dict(d), but either way works.

[Casey Kuball]

@wjandrea just want to highlight that as Python is a duck-typed language, if the type of d is not known, dict(d) will guarantee you end up with a dictionary, while d.copy() will create a copy by calling a copy method. In the former case, it's assuming d can somehow be converted to a dict (e.g. an iterable of (key, value) pairs), while the latter just assumes there's a copy method.