Defining private module functions in python

Question

According to http://www.faqs.org/docs/diveintopython/fileinfo_private.html:

Like most languages, Python has the concept of private elements:

• Private functions, which can't be called from outside their module

However, if I define two files:

#a.py
__num=1

and:

#b.py
import a
print a.__num

when i run b.py it prints out 1 without giving any exception. Is diveintopython wrong, or did I misunderstand something? And is there some way to do define a module's function as private?

Solution 1

In Python, "privacy" depends on "consenting adults'" levels of agreement - you can't force it (any more than you can in real life;-). A single leading underscore means you're not supposed to access it "from the outside" -- two leading underscores (w/o trailing underscores) carry the message even more forcefully... but, in the end, it still depends on social convention and consensus: Python's introspection is forceful enough that you can't handcuff every other programmer in the world to respect your wishes.

((Btw, though it's a closely held secret, much the same holds for C++: with most compilers, a simple #define private public line before #includeing your .h file is all it takes for wily coders to make hash of your "privacy"...!-))

Solution 2

There may be confusion between class privates and module privates.

A module private starts with one underscore
Such a element is not copied along when using the from <module_name> import * form of the import command; it is however imported if using the import <moudule_name> syntax (see Ben Wilhelm's answer)
Simply remove one underscore from the a.__num of the question's example and it won't show in modules that import a.py using the from a import * syntax.

A class private starts with two underscores (aka dunder i.e. d-ouble under-score)
Such a variable has its name "mangled" to include the classname etc.
It can still be accessed outside of the class logic, through the mangled name.
Although the name mangling can serve as a mild prevention device against unauthorized access, its main purpose is to prevent possible name collisions with class members of the ancestor classes. See Alex Martelli's funny but accurate reference to consenting adults as he describes the convention used in regards to these variables.

>>> class Foo(object):
...    __bar = 99
...    def PrintBar(self):
...        print(self.__bar)
...
>>> myFoo = Foo()
>>> myFoo.__bar  #direct attempt no go
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Foo' object has no attribute '__bar'
>>> myFoo.PrintBar()  # the class itself of course can access it
99
>>> dir(Foo)    # yet can see it
['PrintBar', '_Foo__bar', '__class__', '__delattr__', '__dict__', '__doc__', '__
format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__',
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__
', '__subclasshook__', '__weakref__']
>>> myFoo._Foo__bar  #and get to it by its mangled name !  (but I shouldn't!!!)
99
>>>

Solution 3

This question was not fully answered, since module privacy is not purely conventional, and since using import may or may not recognize module privacy, depending on how it is used.

If you define private names in a module, those names will be imported into any script that uses the syntax, 'import module_name'. Thus, assuming you had correctly defined in your example the module private, _num, in a.py, like so..

#a.py
_num=1

..you would be able to access it in b.py with the module name symbol:

#b.py
import a
...
foo = a._num # 1

To import only non-privates from a.py, you must use the from syntax:

#b.py
from a import *
...
foo = _num # throws NameError: name '_num' is not defined

For the sake of clarity, however, it is better to be explicit when importing names from modules, rather than importing them all with a '*':

#b.py
from a import name1 
from a import name2
...

Solution 4

Python allows for private class members with the double underscore prefix. This technique doesn't work at a module level so I am thinking this is a mistake in Dive Into Python.

Here is an example of private class functions:

class foo():
    def bar(self): pass
    def __bar(self): pass

f = foo()
f.bar()   # this call succeeds
f.__bar() # this call fails

Solution 5

You can add an inner function:

def public(self, args):
   def private(self.root, data):
       if (self.root != None):
          pass #do something with data

Something like that if you really need that level of privacy.

View more solutions

Comments

[Homero Esmeraldo]

It's not that diveintopython is wrong, but in their example: >>> import fileinfo >>> m = fileinfo.MP3FileInfo() >>> m.__parse("/music/_singles/kairo.mp3") 1 Traceback (innermost last): File "<interactive input>", line 1, in ? AttributeError: 'MP3FileInfo' instance has no attribute '__parse' fileinfo.MP3FileInfo() is an instance of class. Which gives this exception when you use double underscore. Whereas in your case, you didn't create a class, you just created a module. See also: stackoverflow.com/questions/70528/…

[Santa]

Well, TIL. Any reason why they don't enforce module-level __private_function, though? I ran into this and got into errors because of it.

[rhinoinrepose]

Your note on C++ is incorrect. By using #define private public you're changing the code that gets sent to the compiler, which is where the name mangling takes place.

[Prof. Falken]

Also the C++ mangling is obscure, but hardly secret. You can "introspect" a binary produced by C++ too. OT, sorry.

[SevakPrime]

I think the OP's intent is to write functions that are not accessible outside of, for example, a commercial package. In that regard, this answer isn't complete. The __bar() function is still accessible from outside through f._foo__bar(). Therefore, the double-leading underscores do not make it private.

[Cory Kramer]

As an update to @rhinoinrepose, it is not just incorrect, it is undefined behavior according to the standard to redefine a keyword with a preprocessor macro.

[Edgar Klerks]

You can use a closure to make a variable private and then return the variables you want to export.

[FistOfFury]

where do you specify which functions/libraries are imported? in the init.py?

[mjv]

Thank you for the kind words @Terrabits, but I gladly stand second to Alex (well behind!) on all things 'Python'. Furthermore his answers are typically more concise and humorous while retaining a high level of authoritativeness given Alex extensive contributions to the language and the community.

[user877329]

@AlexMartelli Isn't static void foo() as private as it gets. It is at least hidden to the linker, and the function may be removed entirely by inlining.

[Josiah Yoder]

There is no risk of name collisions when _names are invoked with import a -- they are accesses as a._names when using this style.

[Mike Williamson]

@FistOfFury Yes, you specify the functions imported in the __init__.py file. See here for some help on that.

[HFBrowning]

@mjv This was such a helpful explanation! Thank you! I have been quite puzzled about this behavior for a while. I do wish the choice had been to throw some kind of error other than an AttributeError if you tried to directly access the class private; perhaps a "PrivateAccessError" or something would have been more explicit/helpful. (Since getting the error that it doesn't have an attribute isn't really true).

[Lay González]

In real life people get prosecuted if they break the laws

[safay]

Why is this not the best answer?

[Pradyumna Das]

I guess its because the nested function is not reusable anywhere else in the module so there is no benefit to defining a function, unless it is to be used multiple times inside the outer function. In this case I feel it would be more readable to just inline the code.

[Douglas Figueroa]

This does not apply to python. Maybe you're referring to Java.

[Darren Ng]

PEP 8 -- Style Guide for Python Code _single_leading_underscore: weak "internal use" indicator. E.g. from M import * does not import objects whose names start with an underscore.

[SharksRule]

This is useful, I was looking for a function accessible from within a class but not outside the class.