Python dynamic inheritance: How to choose base class upon instance creation?
Solution 1
What about defining the ImageZIP
class on function-level ?
This will enable your dynamic inheritance
.
def image_factory(path):
# ...
if format == ".gz":
image = unpack_gz(path)
format = os.path.splitext(image)[1][1:]
if format == "jpg":
return MakeImageZip(ImageJPG, image)
elif format == "png":
return MakeImageZip(ImagePNG, image)
else: raise Exception('The format "' + format + '" is not supported.')
def MakeImageZIP(base, path):
'''`base` either ImageJPG or ImagePNG.'''
class ImageZIP(base):
# ...
return ImageZIP(path)
Edit: Without need to change image_factory
def ImageZIP(path):
path = unpack_gz(path)
format = os.path.splitext(image)[1][1:]
if format == "jpg": base = ImageJPG
elif format == "png": base = ImagePNG
else: raise_unsupported_format_error()
class ImageZIP(base): # would it be better to use ImageZip_.__name__ = "ImageZIP" ?
# ...
return ImageZIP(path)
Solution 2
I would favor composition over inheritance here. I think your current inheritance hierarchy seems wrong. Some things, like opening the file with or gzip have little to do with the actual image format and can be easily handled in one place while you want to separate the details of working with a specific format own classes. I think using composition you can delegate implementation specific details and have a simple common Image class without requiring metaclasses or multiple inheritance.
import gzip
import struct
class ImageFormat(object):
def __init__(self, fileobj):
self._fileobj = fileobj
@property
def name(self):
raise NotImplementedError
@property
def magic_bytes(self):
raise NotImplementedError
@property
def magic_bytes_format(self):
raise NotImplementedError
def check_format(self):
peek = self._fileobj.read(len(self.magic_bytes_format))
self._fileobj.seek(0)
bytes = struct.unpack_from(self.magic_bytes_format, peek)
if (bytes == self.magic_bytes):
return True
return False
def get_pixel(self, n):
# ...
pass
class JpegFormat(ImageFormat):
name = "JPEG"
magic_bytes = (255, 216, 255, 224, 0, 16, 'J', 'F', 'I', 'F')
magic_bytes_format = "BBBBBBcccc"
class PngFormat(ImageFormat):
name = "PNG"
magic_bytes = (137, 80, 78, 71, 13, 10, 26, 10)
magic_bytes_format = "BBBBBBBB"
class Image(object):
supported_formats = (JpegFormat, PngFormat)
def __init__(self, path):
self.path = path
self._file = self._open()
self._format = self._identify_format()
@property
def format(self):
return self._format.name
def get_pixel(self, n):
return self._format.get_pixel(n)
def _open(self):
opener = open
if self.path.endswith(".gz"):
opener = gzip.open
return opener(self.path, "rb")
def _identify_format(self):
for format in self.supported_formats:
f = format(self._file)
if f.check_format():
return f
else:
raise ValueError("Unsupported file format!")
if __name__=="__main__":
jpeg = Image("images/a.jpg")
png = Image("images/b.png.gz")
I only tested this on a few local png and jpeg files but hopefully it illustrates another way of thinking about this problem.
Solution 3
If you ever need “black magic”, first try to think about a solution that doesn't require it. You're likely to find something that works better and results in needs clearer code.
It may be better for the image class constructors to take an already opened file instead of a path. Then, you're not limited to files on the disk, but you can use file-like objects from urllib, gzip, and the like.
Also, since you can tell JPG from PNG by looking at the contents of the file, and for gzip file you need this detection anyway, I recommend not looking at the file extension at all.
class Image(object):
def __init__(self, fileobj):
self.fileobj = fileobj
def image_factory(path):
return(image_from_file(open(path, 'rb')))
def image_from_file(fileobj):
if looks_like_png(fileobj):
return ImagePNG(fileobj)
elif looks_like_jpg(fileobj):
return ImageJPG(fileobj)
elif looks_like_gzip(fileobj):
return image_from_file(gzip.GzipFile(fileobj=fileobj))
else:
raise Exception('The format "' + format + '" is not supported.')
def looks_like_png(fileobj):
fileobj.seek(0)
return fileobj.read(4) == '\x89PNG' # or, better, use a library
# etc.
For black magic, go to What is a metaclass in Python?, but think twice before using that, especially at work.
Solution 4
You should use composition in this case, not inheritance. Take a look at the decorator design pattern. The ImageZIP
class should decorate other image classes with the desired functionality.
With decorators, you get a very dynamic behavior depending on the composition that you create:
ImageZIP(ImageJPG(path))
It's more flexible also, you can have other decorators:
ImageDecrypt(password, ImageZIP(ImageJPG(path)))
Each decorator just encapsulates the functionality it adds and delegates to the composed class as needed.
Related videos on Youtube
![xApple](https://i.stack.imgur.com/Ihzmb.png?s=256&g=1)
xApple
Updated on June 06, 2022Comments
-
xApple about 2 years
Introduction
I have encountered an interesting case in my programming job that requires me to implement a mechanism of dynamic class inheritance in python. What I mean when using the term "dynamic inheritance" is a class that doesn't inherit from any base class in particular, but rather chooses to inherit from one of several base classes at instantiation, depending on some parameter.
My question is thus the following: in the case I will present, what would be the best, most standard and "pythonic" way of implementing the needed extra functionality via dynamic inheritance.
To summarize the case in point in a simple manner, I will give an example using two classes that represent two different image formats:
'jpg'
and'png'
images. I will then try to add the ability to support a third format: the'gz'
image. I realize my question isn't that simple, but I hope you are ready to bear with me for a few more lines.The two images example case
This script contains two classes:
ImageJPG
andImagePNG
, both inheriting from theImage
base class. To create an instance of an image object, the user is asked to call theimage_factory
function with a file path as the only parameter.This function then guesses the file format (
jpg
orpng
) from the path and returns an instance of the corresponding class.Both concrete image classes (
ImageJPG
andImagePNG
) are able to decode files via theirdata
property. Both do this in a different way. However, both ask theImage
base class for a file object in order to do this.import os #------------------------------------------------------------------------------# def image_factory(path): '''Guesses the file format from the file extension and returns a corresponding image instance.''' format = os.path.splitext(path)[1][1:] if format == 'jpg': return ImageJPG(path) if format == 'png': return ImagePNG(path) else: raise Exception('The format "' + format + '" is not supported.') #------------------------------------------------------------------------------# class Image(object): '''Fake 1D image object consisting of twelve pixels.''' def __init__(self, path): self.path = path def get_pixel(self, x): assert x < 12 return self.data[x] @property def file_obj(self): return open(self.path, 'r') #------------------------------------------------------------------------------# class ImageJPG(Image): '''Fake JPG image class that parses a file in a given way.''' @property def format(self): return 'Joint Photographic Experts Group' @property def data(self): with self.file_obj as f: f.seek(-50) return f.read(12) #------------------------------------------------------------------------------# class ImagePNG(Image): '''Fake PNG image class that parses a file in a different way.''' @property def format(self): return 'Portable Network Graphics' @property def data(self): with self.file_obj as f: f.seek(10) return f.read(12) ################################################################################ i = image_factory('images/lena.png') print i.format print i.get_pixel(5)
The compressed image example case
Building on the first image example case, one would like to add the following functionality:
An extra file format should be supported, the
gz
format. Instead of being a new image file format, it is simply a compression layer that, once decompressed, reveals either ajpg
image or apng
image.The
image_factory
function keeps its working mechanism and will simply try to create an instance of the concrete image classImageZIP
when it is given agz
file. Exactly in the same way it would create an instance ofImageJPG
when given ajpg
file.The
ImageZIP
class just wants to redefine thefile_obj
property. In no case does it want to redefine thedata
property. The crux of the problem is that, depending on what file format is hiding inside the zip archive, theImageZIP
classes needs to inherit either fromImageJPG
or fromImagePNG
dynamically. The correct class to inherit from can only be determined upon class creation when thepath
parameter is parsed.Hence, here is the same script with the extra
ImageZIP
class and a single added line to theimage_factory
function.Obviously, the
ImageZIP
class is non-functional in this example. This code requires Python 2.7.import os, gzip #------------------------------------------------------------------------------# def image_factory(path): '''Guesses the file format from the file extension and returns a corresponding image instance.''' format = os.path.splitext(path)[1][1:] if format == 'jpg': return ImageJPG(path) if format == 'png': return ImagePNG(path) if format == 'gz': return ImageZIP(path) else: raise Exception('The format "' + format + '" is not supported.') #------------------------------------------------------------------------------# class Image(object): '''Fake 1D image object consisting of twelve pixels.''' def __init__(self, path): self.path = path def get_pixel(self, x): assert x < 12 return self.data[x] @property def file_obj(self): return open(self.path, 'r') #------------------------------------------------------------------------------# class ImageJPG(Image): '''Fake JPG image class that parses a file in a given way.''' @property def format(self): return 'Joint Photographic Experts Group' @property def data(self): with self.file_obj as f: f.seek(-50) return f.read(12) #------------------------------------------------------------------------------# class ImagePNG(Image): '''Fake PNG image class that parses a file in a different way.''' @property def format(self): return 'Portable Network Graphics' @property def data(self): with self.file_obj as f: f.seek(10) return f.read(12) #------------------------------------------------------------------------------# class ImageZIP(### ImageJPG OR ImagePNG ? ###): '''Class representing a compressed file. Sometimes inherits from ImageJPG and at other times inherits from ImagePNG''' @property def format(self): return 'Compressed ' + super(ImageZIP, self).format @property def file_obj(self): return gzip.open(self.path, 'r') ################################################################################ i = image_factory('images/lena.png.gz') print i.format print i.get_pixel(5)
A possible solution
I have found a way of getting the wanted behavior by intercepting the
__new__
call in theImageZIP
class and using thetype
function. But it feels clumsy and I suspect there might be a better way using some Python techniques or design patterns I don't yet know about.import re class ImageZIP(object): '''Class representing a compressed file. Sometimes inherits from ImageJPG and at other times inherits from ImagePNG''' def __new__(cls, path): if cls is ImageZIP: format = re.findall('(...)\.gz', path)[-1] if format == 'jpg': return type("CompressedJPG", (ImageZIP,ImageJPG), {})(path) if format == 'png': return type("CompressedPNG", (ImageZIP,ImagePNG), {})(path) else: return object.__new__(cls) @property def format(self): return 'Compressed ' + super(ImageZIP, self).format @property def file_obj(self): return gzip.open(self.path, 'r')
Conclusion
Bear in mind if you want to propose a solution that the goal is not to change the behavior of the
image_factory
function. That function should remain untouched. The goal, ideally, is to build a dynamicImageZIP
class.I just don't really know what the best way to do this is. But this is a perfect occasion for me to learn more about some of Python's "black magic". Maybe my answer lies with strategies like modifying the
self.__cls__
attribute after creation or maybe using the__metaclass__
class attribute? Or maybe something to do with the specialabc
abstract base classes could help here? Or other unexplored Python territory?-
Thomas K almost 13 yearsI think you're imposing an artificial constraint that it must be a class inheriting from your existing types. I think a factory function or a class encapsulating one of your types is more Pythonic. For that matter, I think it would be better still to have a generic
Image
class, with functions or classmethods for loading from different formats. -
agf almost 13 yearsEverything @Thomas says is right. If you need this, you've structured your inheritance wrong. Call the
Image
constructor with a "datatype" argument is the obvious way; there are others. Also, keep in mind instead of type() you can just call the__new__
methods of the appropriate base classes in the right order. -
Jochen Ritzel almost 13 yearsI dont get the problem either, you can do your example quite easily with
ImagePNG, ImageJPG, CompressedFile
classes and stick them together with multiple inheritance ieclass CompressedPNG(ImagePNG, CompressedFile)
and write a simpleimage_from_path
function. -
Glenn Maynard almost 13 yearsIf you want help, please don't ask your questions as four-page-long essays.
-
mdeous almost 13 yearsalso, relying on the file extension to detect the mime-type is really a bad practice, a better solution would be to use the file's magic-bytes (can be done with the
magic
module) -
Rudy Garcia almost 13 yearsAre you actually working with images or is this just an example? There are better ways to confirm that an image file is a specific type and libraries like PIL (pythonware.com/products/pil) are worth investigating rather than rolling your own solution.
-
xApple almost 13 years@MatToufoutu The file type stuff is intended as an example. However I wouldn't use the magic library as it seams the PyPI version seams to be unmaintained. A new version exists somewhere else but is not being added PyPI...
-
xApple almost 13 years@Mike It's really just intended as an example to try and create a situation similar to the one I'm experiencing. I'm not working with images.
-
xApple almost 13 years@Glenn Sorry, it seamed hard to explain and I preferred going into all the details than to risk being misinterpreted.
-
xApple almost 13 years@Jochen That would mean writing a class for (every file type) X (every compression format). The idea is to be able to add new formats and compression formats without too much boiler plate code.
-
Niklas R almost 13 yearsBtw, how did you create the class diagrams ? I'd need such a program. Thanks !
-
xApple almost 13 years@Niklas I used this website: yuml.me
-
-
Kevin Little almost 13 years+1! Maybe I'm just not smart enough to handle "advanced" inheritance schemes, but in cases like this I always find composition easier to think about and extend/debug.
-
xApple almost 13 yearsI have looked at decorators design patterns. They don't seam to apply here as a decorator must have, as one of its attribute, an instance of the class it inherits from. What would my decorator inherit from here ?
-
Jordão almost 13 yearsYour decorator decorates images, so it would inherit from the base
Image
class. -
xApple almost 13 yearsBut then if my decorator inherits from Image instead of ImageJPEG (or ImagePNG for the matter) the functionality specific to the format is missing, and "i.data" becomes an undefined property ?
-
xApple almost 13 yearsWell the inheritance scheme is set in stone so to speak, so I can't compose like that now: the idea of adding compression support comes as an afterthought. Plus, now, for every method f(x) you want to add to your Image class you have to redirect f(x): self._format.f(x)
-
xApple almost 13 yearsOnce again, I could change the whole inheritance schema of the project. But at this point it is difficult. The format and image guessing was really just taken as a good example as it would call for a "image_factory" function. I'm not actually dealing with Images. I'm just looking for a way to get some dynamic inheritance to solve my problem without refactoring too much what is already in place.
-
Jordão almost 13 yearsThe decorator composes the specific class it decorates. You'll create it, e.g., like this in your factory:
ImageZIP(ImageJPG(path))
. You'll also define adata
method that just delegates to the composed instance. -
Petr Viktorin almost 13 yearsWell, you've described that kind of solution in “A possible solution” in your question. As you said, that solution is clumsy, and there is a better way: the better way is refactoring the code so it makes more sense. If you're specifically looking for dynamic classes, you can't get much better than type() calls (except maybe
class
definitions inside a function, but then if you want a meaningful class name you have to set__name__
afterwards, so it's not much nicer). Sorry, but you have a working solution; I can't help you more. Just beware that you can't subclass your ImageZIP meaningfully. -
xApple almost 13 yearsA code example could be interesting. But I have the impression that we are back at square one, and must include the composition functionality by refactoring the image_facotry function.
-
xApple almost 13 yearsI hadn't thought about defining a class inside a function. Nice idea. But for it to work without changing the image_factory function, the new function would have to be called "ImageZIP".
-
Rudy Garcia almost 13 yearsFunction call overhead does have a cost but I depending on how frequently you make calls to the underlying format there are ways to mitigate that cost. I'm sorry to hear that you inheritance scheme can't be changed.
-
Niklas R almost 13 years@xApple I've editet the answer. This should be what you are searching for.
-
mechanical_meat over 2 yearsyessir this helped me out today. many thanks.