How can I dynamically create derived classes from a base class

Question

For example I have a base class as follows:

class BaseClass(object):
    def __init__(self, classtype):
        self._type = classtype

From this class I derive several other classes, e.g.

class TestClass(BaseClass):
    def __init__(self):
        super(TestClass, self).__init__('Test')

class SpecialClass(BaseClass):
    def __init__(self):
        super(TestClass, self).__init__('Special')

Is there a nice, pythonic way to create those classes dynamically by a function call that puts the new class into my current scope, like:

foo(BaseClass, "My")
a = MyClass()
...

As there will be comments and questions why I need this: The derived classes all have the exact same internal structure with the difference, that the constructor takes a number of previously undefined arguments. So, for example, MyClass takes the keywords a while the constructor of class TestClass takes b and c.

inst1 = MyClass(a=4)
inst2 = MyClass(a=5)
inst3 = TestClass(b=False, c = "test")

But they should NEVER use the type of the class as input argument like

inst1 = BaseClass(classtype = "My", a=4)

I got this to work but would prefer the other way, i.e. dynamically created class objects.

Solution 1

This bit of code allows you to create new classes with dynamic names and parameter names. The parameter verification in __init__ just does not allow unknown parameters, if you need other verifications, like type, or that they are mandatory, just add the logic there:

class BaseClass(object):
    def __init__(self, classtype):
        self._type = classtype

def ClassFactory(name, argnames, BaseClass=BaseClass):
    def __init__(self, **kwargs):
        for key, value in kwargs.items():
            # here, the argnames variable is the one passed to the
            # ClassFactory call
            if key not in argnames:
                raise TypeError("Argument %s not valid for %s" 
                    % (key, self.__class__.__name__))
            setattr(self, key, value)
        BaseClass.__init__(self, name[:-len("Class")])
    newclass = type(name, (BaseClass,),{"__init__": __init__})
    return newclass

And this works like this, for example:

>>> SpecialClass = ClassFactory("SpecialClass", "a b c".split())
>>> s = SpecialClass(a=2)
>>> s.a
2
>>> s2 = SpecialClass(d=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 8, in __init__
TypeError: Argument d not valid for SpecialClass

I see you are asking for inserting the dynamic names in the naming scope -- now, that is not considered a good practice in Python - you either have variable names, known at coding time, or data - and names learned in runtime are more "data" than "variables" -

So, you could just add your classes to a dictionary and use them from there:

name = "SpecialClass"
classes = {}
classes[name] = ClassFactory(name, params)
instance = classes[name](...)

And if your design absolutely needs the names to come in scope, just do the same, but use the dictionary returned by the globals() call instead of an arbitrary dictionary:

name = "SpecialClass"
globals()[name] = ClassFactory(name, params)
instance = SpecialClass(...)

(It indeed would be possible for the class factory function to insert the name dynamically on the global scope of the caller - but that is even worse practice, and is not compatible across Python implementations. The way to do that would be to get the caller's execution frame, through sys._getframe(1) and setting the class name in the frame's global dictionary in its f_globals attribute).

update, tl;dr: This answer had become popular, still its very specific to the question body. The general answer on how to "dynamically create derived classes from a base class" in Python is a simple call to type passing the new class name, a tuple with the baseclass(es) and the __dict__ body for the new class -like this:

>>> new_class = type("NewClassName", (BaseClass,), {"new_method": lambda self: ...})

update
Anyone needing this should also check the dill project - it claims to be able to pickle and unpickle classes just like pickle does to ordinary objects, and had lived to it in some of my tests.

Solution 2

type() is the function that creates classes and in particular sub-classes, like in the question:

def set_x(self, value):
    self.x = value

# type() takes as argument the new class name, its base
# classes, and its attributes:
SubClass = type('SubClass', (BaseClass,), {'set_x': set_x})
# (More methods can be put in SubClass, including __init__().)

obj = SubClass()
obj.set_x(42)
print obj.x  # Prints 42
print isinstance(obj, BaseClass)  # True

Comments

[acattle]

Just to be sure, you want the type of instance to change depending on the supplied arguments? Like if I give an a it will always be MyClass and TestClass will never take an a? Why not just declare all 3 arguments in BaseClass.__init__() but default them all to None? def __init__(self, a=None, b=None, C=None)?

[Alex]

I cannot declare anything in the base class, as i do not know all arguments I might use. I might have 30 different clases with 5 different arguments each, so declaring 150 arguments in the constructur is not a solution.

[Bruno Feroleto]

If I remember correctly, BaseClass.__init__() would be better as the more general super(self.__class__).__init__(), which plays more nicely when the new classes are subclassed. (Reference: rhettinger.wordpress.com/2011/05/26/super-considered-super)

[jsbueno]

@EOL: It would for statically declared classes - but since you don't have the actual class name to hardcode as the first parameter to Super, that would require a lot of dancing around. Try replacing it with super above and create a subclass of a dynamically created class to understand it; And, on the other hand, in this case you can have the baseclass as a general objectfrom which to call __init__.

[Alex]

Now I had some time to look at the suggested solution, but it is not quite what I want. First, it looks like __init__ of BaseClass is called with one argument, but in fact BaseClass.__init__ always takes an arbitrary list of keyword arguments. Second, the solution above sets all the allowed parameter names as attributes, which is not what I want. ANY argument HAS to go to BaseClass, but which one I know when creating the derived class. I probably will update the question or ask a more precise one to make it clearer.

[Bruno Feroleto]

@jsbueno: Right, using the super() I was mentioning gives TypeError: must be type, not SubSubClass. If I understand correctly, this comes from the first argument self of __init__(), which is a SubSubClass where a type object is expected: this seems related to the fact super(self.__class__) is a unbound super object. What is its __init__() method? I'm not sure which such method could require a first argument of type type. Could you explain? (Side note: my super() approach indeed does not make sense, here, because __init__() has a variable signature.)

[jsbueno]

@EOL: the major problem is actually if you create another subclass of the factorized class: self.__class__ will refer to that subclass, not the class in which "super" is called - and you get infinite recursion.

[jsbueno]

@Alex: both the argument setting and calling __init__ of the baseclasswith just one parameter are just two ordinary lines, and not part of the "solution" of creating the classes dynamically - At this point, it is not quite clear for me what you are needing - but you certainy can change these behaviors in the code above.

[DaveL17]

In trying to understand this example using Python 2.7, I got a TypeError that said __init__() takes exactly 2 arguments (1 given). I found that adding something (anything?) to fill the gap would suffice. For example, obj = SubClass('foo') runs without error.

[Bruno Feroleto]

This is normal, since SubClass is a sub-class of BaseClass in the question and BaseClass takes a parameter (classtype, which is 'foo' in your example).

[Tejas Tank]

In dynamic paradigam this does not best fit

[mikazz]

@EricOLebigot Can I make a call to init of BaseClass here somehow? like with super?

[Bruno Feroleto]

Yes, and still with super(), but the Python interpreter cannot work its magic anymore and interpret a bare super() directly, so you must use in set_x (or any other method of SubClass) the more explicit form super(SubClass, self).__init__().