How does inheritance of __slots__ in subclasses actually work?

Question

In the Python data model reference section on slots there is a list of notes on using __slots__. I am thoroughly confused by the 1st and 6th items, because they seem to be contradicting each other.

First item:

• When inheriting from a class without __slots__, the __dict__ attribute of that class will always be accessible, so a __slots__ definition in the subclass is meaningless.

Sixth item:

• The action of a __slots__ declaration is limited to the class where it is defined. As a result, subclasses will have a __dict__ unless they also define __slots__ (which must only contain names of any additional slots).

It seems to me these items could be better worded or shown through code, but I have been trying to wrap my head around this and am still coming up confused. I do understand how __slots__ are supposed to be used, and I am trying to get a better grasp on how they work.

The Question:

Can someone please explain to me in plain language what the conditions are for inheritance of slots when subclassing?

(Simple code examples would be helpful but not necessary.)

Solution 1

As others have mentioned, the sole reason for defining __slots__ is to save some memory, when you have simple objects with a predefined set of attributes and don't want each to carry around a dictionary. This is meaningful only for classes of which you plan to have many instances, of course.

The savings may not be immediately obvious -- consider...:

>>> class NoSlots(object): pass
... 
>>> n = NoSlots()
>>> class WithSlots(object): __slots__ = 'a', 'b', 'c'
... 
>>> w = WithSlots()
>>> n.a = n.b = n.c = 23
>>> w.a = w.b = w.c = 23
>>> sys.getsizeof(n)
32
>>> sys.getsizeof(w)
36

From this, it would seem the with-slots size is larger than the no-slots size! But that's a mistake, because sys.getsizeof doesn't consider "object contents" such as the dictionary:

>>> sys.getsizeof(n.__dict__)
140

Since the dict alone takes 140 bytes, clearly the "32 bytes" object n is alleged to take are not considering all that's involved in each instance. You can do a better job with third-party extensions such as pympler:

>>> import pympler.asizeof
>>> pympler.asizeof.asizeof(w)
96
>>> pympler.asizeof.asizeof(n)
288

This shows much more clearly the memory footprint that's saved by __slots__: for a simple object such as this case, it's a bit less than 200 bytes, almost 2/3 of the object's overall footprint. Now, since these days a megabyte more or less doesn't really matter all that much to most applications, this also tells you that __slots__ is not worth the bother if you're going to have just a few thousand instances around at a time -- however, for millions of instances, it sure does make a very important difference. You can also get a microscopic speedup (partly due to better cache use for small objects with __slots__):

$ python -mtimeit -s'class S(object): __slots__="x","y"' -s's=S(); s.x=s.y=23' 's.x'
10000000 loops, best of 3: 0.37 usec per loop
$ python -mtimeit -s'class S(object): pass' -s's=S(); s.x=s.y=23' 's.x'
1000000 loops, best of 3: 0.604 usec per loop
$ python -mtimeit -s'class S(object): __slots__="x","y"' -s's=S(); s.x=s.y=23' 's.x=45'
1000000 loops, best of 3: 0.28 usec per loop
$ python -mtimeit -s'class S(object): pass' -s's=S(); s.x=s.y=23' 's.x=45'
1000000 loops, best of 3: 0.332 usec per loop

but this is somewhat dependent on Python version (these are the numbers I measure repeatably with 2.5; with 2.6, I see a larger relative advantage to __slots__ for setting an attribute, but none at all, indeed a tiny disadvantage, for getting it).

Now, regarding inheritance: for an instance to be dict-less, all classes up its inheritance chain must also have dict-less instances. Classes with dict-less instances are those which define __slots__, plus most built-in types (built-in types whose instances have dicts are those on whose instances you can set arbitrary attributes, such as functions). Overlaps in slot names are not forbidden, but they're useless and waste some memory, since slots are inherited:

>>> class A(object): __slots__='a'
... 
>>> class AB(A): __slots__='b'
... 
>>> ab=AB()
>>> ab.a = ab.b = 23
>>> 

as you see, you can set attribute a on an AB instance -- AB itself only defines slot b, but it inherits slot a from A. Repeating the inherited slot isn't forbidden:

>>> class ABRed(A): __slots__='a','b'
... 
>>> abr=ABRed()
>>> abr.a = abr.b = 23

but does waste a little memory:

>>> pympler.asizeof.asizeof(ab)
88
>>> pympler.asizeof.asizeof(abr)
96

so there's really no reason to do it.

Solution 2

class WithSlots(object):
    __slots__ = "a_slot"

class NoSlots(object):       # This class has __dict__
    pass

First Item

class A(NoSlots):            # even though A has __slots__, it inherits __dict__
    __slots__ = "a_slot"     # from NoSlots, therefore __slots__ has no effect

Sixth Item

class B(WithSlots):          # This class has no __dict__
    __slots__ = "some_slot"

class C(WithSlots):          # This class has __dict__, because it doesn't
    pass                     # specify __slots__ even though the superclass does.

You probably won't need to use __slots__ in the near future. It's only intended to save memory at the cost of some flexibility. Unless you have tens of thousands of objects it won't matter.

Solution 3

Python: How does inheritance of __slots__ in subclasses actually work?

I am thoroughly confused by the 1st and 6th items, because they seem to be contradicting each other.

Those items don't actually contradict each other. The first regards subclasses of classes that don't implement __slots__, the second regards subclasses of classes that do implement __slots__.

Subclasses of classes that don't implement __slots__

I am increasingly aware that as great as the Python docs are (rightly) reputed to be, they are not perfect, especially regarding the less used features of the language. I would alter the docs as follows:

When inheriting from a class without __slots__, the __dict__ attribute of that class will always be accessible, so a __slots__ definition in the subclass is meaningless .

__slots__ is still meaningful for such a class. It documents the expected names of attributes of the class. It also creates slots for those attributes - they will get the faster lookups and use less space. It just allows for other attributes, which will be assigned to the __dict__.

This change has been accepted and is now in the latest documentation.

Here's an example:

class Foo: 
    """instances have __dict__"""

class Bar(Foo):
    __slots__ = 'foo', 'bar'

Bar not only has the slots it declares, it also has Foo's slots - which include __dict__:

>>> b = Bar()
>>> b.foo = 'foo'
>>> b.quux = 'quux'
>>> vars(b)
{'quux': 'quux'}
>>> b.foo
'foo'

Subclasses of classes that do implement __slots__

The action of a __slots__ declaration is limited to the class where it is defined. As a result, subclasses will have a __dict__ unless they also define __slots__ (which must only contain names of any additional slots).

Well that's not quite right either. The action of a __slots__ declaration is not entirely limited to the class where it is defined. They can have implications for multiple inheritance, for example.

I would change that to:

For classes in an inheritance tree that defines __slots__, subclasses will have a __dict__ unless they also define __slots__ (which must only contain names of any additional slots).

I have actually updated it to read:

The action of a __slots__ declaration is not limited to the class where it is defined. __slots__ declared in parents are available in child classes. However, child subclasses will get a __dict__ and __weakref__ unless they also define __slots__ (which should only contain names of any additional slots).

Here's an example:

class Foo:
    __slots__ = 'foo'

class Bar(Foo):
    """instances get __dict__ and __weakref__"""

And we see that a subclass of a slotted class gets to use the slots:

>>> b = Bar()
>>> b.foo = 'foo'
>>> b.bar = 'bar'
>>> vars(b)
{'bar': 'bar'}
>>> b.foo
'foo'

(For more on __slots__, see my answer here.)

Solution 4

My understanding is as follows:

• class X has no __dict__ <-------> class X and its superclasses all have __slots__ specified

• in this case, the actual slots of the class are comprised from the union of __slots__ declarations for X and its superclasses; the behavior is undefined (and will become an error) if this union is not disjoint

View more solutions

Comments

[jfs]

union is not disjoint is not a phrase from a plain English. :) As @Alex Martelli've shown it is not an error if slots sets are not disjoint. Otherwise -- nice summary.

[Wlerin]

@J.F.Sebastian The "meaning of the program is undefined" if they are not disjoint. The docs explicitly state this.

[Mr_and_Mrs_D]

According to the docs docs.python.org/2/reference/datamodel.html#slots "If a class defines a slot also defined in a base class, the instance variable defined by the base class slot is inaccessible (except by retrieving its descriptor directly from the base class). This renders the meaning of the program undefined. In the future, a check may be added to prevent this." - this undefined behavior worries me a bit - I have asked about it ( stackoverflow.com/q/41159714/281545 ) but got no answer

[dugloon]

Awesome explanation and kudos for coining the term dict-less

[sshilovsky]

Slots are also good to make sure that objects of a class won't obtain unwanted attributes.

[c z]

I wouldn't say the sole reason - an IDE like PyCharm or a linter can use slots to find errors in your code before they are executed.

[Breno]

What's weird in your example of ab = AB() is that ab.__slots__ shows only 'b'. But dir(ab) shows that both 'a' and 'b' are there.