Import subclass from a base class in Python

Question

I have a base class that has a method that creates an instance of a subclass that is called the same as the input string.

This worked before by putting the subclasses and the base class in the same file, and doing something like globals()[name].

Now, however, I've split up the subclasses into other files. They each have an import base statement at the top, so I can't just simply import the subclasses in my base class or there'll be a chain of circular importing.

Is there any workaround for this?

In base.py:

from basefactory import BaseFactory
class Base:
    def __init__(self, arg1, arg2):
        ...
    def resolve(self, element):
        className = typing.getClassName(element)
        return BaseFactory.getInstance(className, element, self)

In basefactory.py:

from file1 import *
from file2 import *
...
class BaseFactory:
    @staticmethod
    def getInstance(name, arg1, arg2):
       subclass = globals()[name]
       return subclass(arg1, arg2)

In file1.py:

from base import Base

class subclass1(Base):
    def foo(self):
        return self.arg1

Solution 1

You can move the import statement that is failing to the method that creates the subclass object.

Solution 2

From what I'm understanding, you have:

1. A base class
2. A series of derived classes from the base class
3. A factory method in the base class that instantiates the correct type of derived class
4. The derived classes have been split into files, and they depend on the base class, but the factory method in the base class depends on the derived classes

One solution would be to create a separate function / class for the factory method, and put it in a separate file from the base class. This file could import all the files for the base class and derived classes without the circular reference.

For example:

# base.py:
class baseClass():
   def __init__(self):
      self.name = "Base"

# sub1.py:
from base import baseClass
class sub1Class(baseClass):
   def __init__(self):
      self.name = "sub1"

# sub2.py:
from base import baseClass
class sub2Class(baseClass):
   def __init__(self):
      self.name = "sub2"

# factory.py:
from sub1 import sub1Class
from sub2 import sub2Class # should not create an error
mapping = {'sub1': sub1Class, 'sub2': sub2Class}

def create(baseType):
  return mapping[baseType]

Actually, a better method might be to use type:

type(name, bases, dict) returns a new type object. This is essentially a dynamic form of the class statement. The name string is the class name and becomes the __name__ attribute; the bases tuple itemizes the base classes and becomes the __bases__ attribute; and the dict dictionary is the namespace containing definitions for class body and becomes the __dict__ attribute. For example, the following two statements create identical type objects:

>>> class X(object):
...     a = 1
...
>>> X = type('X', (object,), dict(a=1))

Why not move resolve into a manager class? Have a look at the domain class in Class factory in Python. I'm not sure if resolve is needed... you can get the class name directly from self.__class__.__name__, and use python functions like type() and isinstance() to check if they're particular types.

Also check out:
Can you use a string to instantiate a class in python?
Does python have an equivalent to Java Class.forName()?

Comments

[Kevin Li]

I have over 60 subclasses. Before, when they were in the same file as my base class, the file was over 800 LOC. It was more or less impossible to find things.

[Kevin Li]

While this works, what is the performance impact of this? Wouldn't re-importing 60 subclasses every time the method is called (it's done recursively a lot) be expensive?

[jcollado]

Imported modules are cached in sys.modules, so I wouldn't expect any significant impact.

[Kevin Li]

Also, I have to do something along the lines of from package.filename import * for each file. In the future, I would have to remember to add more imports every time I add a new file. Is there a more compact syntax for importing all members of all modules of a package?

[Adam Morris]

You can do "import package", and then reference objects as package.filename.member1, package.filename2.member2, etc.

[juankysmith]

60 subclasses!! what are they for?

[Kevin Li]

This just propagates the importing error to the factory class. Since the factory class just does from sub1 import *, from sub2 import *..., and each subclass does from base import *, this will throw an ImportError after attempting to import sub2. Trying to catch the exception and pass it means that all the other imports will be skipped.

[Kevin Li]

Since my subclass instance creator is automatic (based on strings), I don't know in which file the subclass is in.

[Adam Morris]

Unless you put the factory class in it's own file.

[Adam Morris]

Another solution might be to keep better control of your subclasses, and create a dict that specifies the mapping between the string and your derived class, and import each of your 60 base classes into the factory class explicitly.

[Kevin Li]

I did this - my factory class imports everything from each subclass (which imports the base class). Meanwhile, my base class, which is in a separate file, imports the factory class. This still results in a circular import.

[Adam Morris]

The base class shouldn't need to know about the factory class. Perhaps you could post some code on how they're intertwined?

[Adam Morris]

This question had some interesting tips on class factories: stackoverflow.com/questions/456672/class-factory-in-python

[Kevin Li]

The method you gave does not work for the example code I posted, because the base class calls into the factory class.

[jcollado]

If you don't know in which module the subclass implementation will be located beforehand, then you'll need to implement some kind of discovery mechanism. In any case, if I understand correctly, since your classes aren't going to be provided by third party libraries, it could be enough to add them to __all__ in the package __init__.py and import using from package import * (that requires some maintenance, though).