How can a function access its own attributes?

Question

is it possible to access the python function object attributes from within the function scope?

e.g. let's have

def f():
    return SOMETHING

f._x = "foo"
f()           # -> "foo"

now, what SOMETHING has to be, if we want to have the _x attribute content "foo" returned? if it's even possible (simply)

thanks

UPDATE:

i'd like the following work also:

g = f
del f
g()          # -> "foo"

UPDATE 2:

Statement that it is not possible (if it is the case), and why, is more satisfying than providing a way how to fake it e.g. with a different object than a function

Solution 1

Solution

Make one of the function's default arguments be a reference to the function itself.

def f(self):
    return self.x
f.func_defaults = (f,)

Example usage:

>>> f.x = 17
>>> b = f
>>> del f
>>> b()
17

Explanation

The original poster wanted a solution that does not require a global name lookup. The simple solution

def f():
    return f.x

performs a lookup of the global variable f on each call, which does not meet the requirements. If f is deleted, then the function fails. The more complicated inspect proposal fails in the same way.

What we want is to perform early binding and store the bound reference within the object itself. The following is conceptually what we are doing:

def f(self=f):
    return self.x

In the above, self is a local variable, so no global lookup is performed. However, we can't write the code as-is, because f is not yet defined when we try to bind the default value of self to it. Instead, we set the default value after f is defined.

Decorator

Here's a simple decorator to do this for you. Note that the self argument must come last, unlike methods, where self comes first. This also means that you must give a default value if any of your other arguments take a default value.

def self_reference(f):
    f.func_defaults = f.func_defaults[:-1] + (f,)
    return f

@self_reference
def foo(verb, adverb='swiftly', self=None):
    return '%s %s %s' % (self.subject, verb, adverb)

Example:

>>> foo.subject = 'Fred'
>>> bar = foo
>>> del foo
>>> bar('runs')
'Fred runs swiftly'

Solution 2

You could just use a class to do this

>>> class F(object):
...     def __call__(self, *args, **kw):
...         return self._x
... 
>>> f=F()
>>> f._x = "foo"
>>> f()
'foo'
>>> g=f
>>> del f
>>> g()
'foo'

Solution 3

Well, let's look at what function is:

>>> def foo():
...     return x
... 
>>> foo.x = 777
>>> foo.x
777
>>> foo()
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
  File "<interactive input>", line 2, in foo
NameError: global name 'x' is not defined
>>> dir(foo)
['__call__', '__class__', '__delattr__', '__dict__', '__doc__', '__get__', 
'__getattribute__', '__hash__', '__init__', '__module__', '__name__', '__new__', 
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', 
'func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 
'func_globals', 'func_name', 'x']
>>> getattr(foo, 'x')
777

Aha! So the attribute was added to the function object but it won't see it because it is looking for global x instead.

We can try to grab the frame of the function execution and try to look what's there (essentially what Anthony Kong suggested but w/o inspect module):

>>> def foo():
...     import sys
...     return sys._getframe()
... 
>>> fr = foo()
>>> dir(fr)
['__class__', '__delattr__', '__doc__', '__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', 'f_back', 'f_builtins', 'f_code', 'f_exc_traceback', 'f_exc_type', 'f_exc_value', 'f_globals', 'f_lasti', 'f_lineno', 'f_locals', 'f_restricted', 'f_trace']
>>> fr.f_locals
{'sys': <module 'sys' (built-in)>}
>>> fr.f_code
<code object foo at 01753020, file "<interactive input>", line 1>
>>> fr.f_code.co_code
'd\x01\x00d\x00\x00k\x00\x00}\x00\x00|\x00\x00i\x01\x00\x83\x00\x00S'
>>> fr.f_code.co_name
'foo'

Aha! So maybe we can get the name of the function from the name of the code block and then look in round-about way for the attribute? Sure enough:

>>> getattr(fr.f_globals[fr.f_code.co_name], 'x')
777
>>> fr.f_globals[fr.f_code.co_name].x
777
>>> def foo():
...     import sys
...     frm = sys._getframe()
...     return frm.f_globals[frm.f_code.co_name].x
... 
>>> foo.x=777
>>> foo()
777

That's great! But would it stand the renaming and deletion of original function?

>>> g = foo
>>> g.func_name
'foo'
>>> g.func_code.co_name
'foo'

Ah, very doubtful. The function object and its code object still insist they are called foo. Sure enough, here is where it breaks:

>>> g.x
777
>>> g.x=888
>>> foo.x
888
>>> g()
888
>>> del foo
>>> g()
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
  File "<interactive input>", line 4, in foo
KeyError: 'foo'

Dang! So in general it can't be done through introspection via the execution frames. The problems seems to be that there is a difference between function object and code object - code objects are what is executed and is just one attribute func_code of the function-object and as such has no access to the func_dict attribute, where our attribute x is:

>>> g
<function foo at 0x0173AE30>
>>> type(g)
<type 'function'>
>>> g.func_code
<code object foo at 017532F0, file "<interactive input>", line 1>
>>> type(g.func_code)
<type 'code'>
>>> g.func_dict
{'x': 888}

There is of course other chicanery you can do so that it seems as function - in particular the trick with class definition... but that is not a function per se. It all depends on what do you really need to do with that.

Solution 4

As a workaround you could use a factory function to fix your scope:

def factory():
    def inner():
        print inner.x
    return inner


>>> foo=factory()
>>> foo.x=11
>>> foo()
11
>>> bar = foo
>>> del foo
>>> bar()
11

Solution 5

Here's a decorator that injects current_fun into the functions globals before executing the function. It's quite the hack, but also quite effective.

from functools import wraps


def introspective(f):
    @wraps(f)
    def wrapper(*args, **kwargs):
        exists = 'current_fun' in f.func_globals
        old = f.func_globals.get('current_fun',None)
        f.func_globals['current_fun'] = wrapper
        try:
            return f(*args, **kwargs)
        finally:
            if exists:
                f.func_globals['current_fun'] = old
            else:
                del f.func_globals['current_fun']
    return wrapper

@introspective
def f():
    print 'func_dict is ',current_fun.func_dict
    print '__dict__ is ',current_fun.__dict__
    print 'x is ',current_fun.x

Here's a usage example

In [41]: f.x = 'x'

In [42]: f()
func_dict is  {'x': 'x'}
__dict__ is  {'x': 'x'}
x is  x

In [43]: g = f

In [44]: del f

In [45]: g()
func_dict is  {'x': 'x'}
__dict__ is  {'x': 'x'}
x is  x

View more solutions

Comments

[Björn Pollex]

What prevents you from simply having a function with one parameter?

[mykhal]

Space_C0wb0y: function parameters are off topic, this is a question on theory, not the real-life pragmatic solutions

[Nas Banov]

+1 for making me explore (and learn in) that corner of python internals ;-)

[NeilG]

See also: stackoverflow.com/questions/852401/…

[mykhal]

hmm. but what if i can have it working for also "renamed" function, like: g = f; del f; print(g())? :)

[nkrkv]

@mykhal it will preserve x value of course, since g is just another reference to something initially referenced only by f

[mykhal]

nailxx: err, yes, but since we reference this with explicit function name, which is now deleted, NameError is raised

[mykhal]

interesting. looked promising, but unfortunately doesn't work well.. try f3=f2; del f2; f3()

[Nas Banov]

i thought inspect requires the source file to be available?

[Mark Lodato]

This does not offer any improvement over the simple return f2._x. The code's co_name is the name of the function when it was defined, so if the function is renamed, the lookup in the global dictionary will fail.

[mykhal]

i hoped for simpler solution, but only yours fulfills the question conditions, so i'm giving you the bounty.

[Peter Hansen]

Thanks @mykhal. It could likely be simplified with one of the libraries at pypi.python.org/… but it really does appear that the design of Python makes this information practically unreachable. The function object holds context for invoking the code object, but once inside you have only that context and no direct record of what function object it came from, other than to trace backwards through the frames and bytecode as this hack is doing so crudely. It's an interesting limitation, but it appears we're stuck with it.

[mykhal]

this is what i'd use in real scenario

[hobs]

This feels like a closure decorator. I like it.

[Steven Rumbalski]

I'm not a big fan of this approach but cannot explain why. For other approaches see this answer to "Static variable in python?".

[cfi]

Can we add that f.func_defaults got renamed to f.__defaults__ in Python 3 for consistency? Also the doubts are shared by others

[cfi]

@Steven, I don't like the decorator because in my little brain I cannot remember that this decorator must not be applied to arbitrary functions but only to those that have a predisposition to accept self as last argument. Guess I'll have to do the usually wrapping to get another closure

[cfi]

Nice! :) For Python 3 func_globals must be replaced with __globals__. I have extended the decorator to accept the name for the attribute: def introspective(f, attribute_name='self'): and actually renamed it to @add_self. All 'current_fun' occurrences must be replaced with attribute_name. By default the decorator wraps the attribute self into the function. Now I can do self.x inside functions and classes. I never use it but for self.logger.info() which can now be nicely added to all of functions, methods, and classes with decorators.

[Mad Physicist]

Useless as an immediate answer, but incredibly illuminating in all other ways. Really taught me a lot about the whys of scope. +1

[Mad Physicist]

Very very nice. +1

[Pablo]

Is this still correct? I tried it and now my function won't detect any imports.

[samwyse]

@mykhal: What you have asked for is contrary to the design of python. This explains why.