Factory pattern in Python

Question

I'm currently implementing the Factory design pattern in Python and I have a few questions.

1. Is there any way to prevent the direct instantiation of the actual concrete classes? For example, if I have a VehicleFactory that spawns Vehicles, I want users to just use that factory, and prevent anyone from accidentally instantiating Car() or Truck() directly. I can throw an exception in init() perhaps, but that would also mean that the factory can't create an instance of it...

2. It seems to me now that factories are getting addictive. Seems like everything should become a factory so that when I change internal implementation, the client codes will not change. I'm interested to know when is there an actual need to use factories, and when is it not appropriate to use. For example, I might have a Window class and there's only one of this type now (no PlasticWindow, ReinforcedWindow or anything like that). In that case, should I use a factory for the client to generate the Window, just in case I might add more types of Windows in the future?

3. I'm just wondering if there is a usual way of calling the factories. For example, now I'm calling my Vehicle factory as Vehicles, so the codes will go something like Vehicles.create(...). I see a lot of tutorials doing it like VehicleFactory, but I find it too long and it sort of exposes the implementation as well.

EDIT: What I meant by "exposes the implementation" is that it lets people know that it's a factory. What I felt was that the client need not know that it's a factory, but rather as some class that can return objects for you (which is a factory of course but maybe there's no need to explicitly tell clients that?). I know that the soure codes are easily exposed, so I didn't mean "exposing the way the functionalities are implemented in the source codes".

Thanks!

Solution 1

1. Don't expose the class (for example make it private __MyClass, or obvious that you don't want it used directly _MyClass). This way it can only be instantiated via the factory function.
2. Perhaps you should review the use of keyword arguments, and inheritance. It sounds like you may be overlooking these, which will generally reduce your dependence on complex factories (To be honest, I've rarely needed factories).
3. In Python you cannot easily protect against exposing implementation, it goes against the Zen of Python. (It's the same in any language, a determined individual can get what they want eventually). At most you should try to ensure that a user of your code does not accidentally do the wrong thing, but never presume to know what the end-user may decide to achieve with your code. Don't make it obfuscated and difficult to work with.

Solution 2

Be Pythonic. Don't overcomplicate your code with "enterprise" language (like Java) solutions that add unnecessary levels of abstraction.

Your code should be simple, and intuitive. You shouldn't need to delegate to another class to instantiate another.

Solution 3

Is there any way to prevent the direct instantiation of the actual concrete classes?

Why? Are your programmers evil sociopaths who refuse to follow the rules? If you provide a factory -- and the factory does what people need -- then they'll use the factory.

You can't "prevent" anything. Remember. This is Python -- they have the source.

should I use a factory for the client to generate the Window, just in case I might add more types of Windows in the future?

Meh. Neither good nor bad. It can get cumbersome to manage all the class-hierarchy-and-factory details.

Adding a factory isn't hard. This is Python -- you have all the source at all times -- you can use grep to find a class constructor and replace it with a factory when you need to.

Since you can use grep to find and fix your mistakes, you don't need to pre-plan this kind of thing as much as you might in Java or C++.

I see a lot of tutorials doing it like VehicleFactory, but I find it too long and it sort of exposes the implementation as well.

"Too Long"? It's used so rarely that it barely matters. Use long names -- it helps other folks understand what you're doing. This is not Code Golf where fewest keystrokes wins.

"exposes the implementation"? First, It exposes nothing. Second, this is Python -- you have all the source at all times -- everything is already exposed.

Stop thinking so much about prevention and privacy. It isn't helpful.

Comments

[Simon Hibbs]

Your naming convention is very confusing. Vehicles sounds like a collection of vehicle objects, and Vehicles.create() sounds like youre making a Vehicles object, not a Vehicle object.

[chaindriver]

Yes, that's true. I'll rename it. Any suggestions on how I should name it? Should I really go for VehicleFactory?

[user1066101]

"lets people know that it's a factory". They're going to find out anyway. What secret are you trying to keep? Why is this secrecy so important?

[chaindriver]

I guess it's a lot of miscommunication and misunderstanding here. Think I'm not expressing the point very clearly. I'm just saying that the client need not think of it as a factory, I'm not saying that the client must not know know that it's a factory. It's not a secret. Anyway, no probs, I'll just name it VehicleFactory. Thanks for the replies!

[user1066101]

@chaindriver: "the client need not think of it as a factory". Ummm... If they don't understand that it's a factory, they're doomed. They'll never be able to use it unless the understand it.

[chaindriver]

@S.Lott: Ummm...How many APIs explicitly tell you that they are using Factories, or Proxies, or Facade etc? In Qt4, there are so many design patterns being used, yet most of the classes don't reflect that in their names. QSqlDatabase is a facade, so I guess the implementers were too secretive about it and should have called it QSqlDatabaseFacade? Umm. QWidget is a composite, and it should have been called QWidgetComposite as well, so that users can use it, because the only way for users to use it is to understand that it's a composite?

[chaindriver]

@S.Lott: QApplication is a singleton, but it should have been called QApplicationSingleton, or else users will " never be able to use it "? Umm... A lot of users must be unable to use Qt4 then, and they are all " doomed ". My point is that yes it's good that users know that the class is a factory, and they probably can read it up in the docs as well, but it's not a necessary criteria for them to use the class. It's an implementation decision that works under the hood.

[user1066101]

@chaindriver: The "Name" and the "Intent" should match, but don't have to. Many programmers figure out which objects are factories, and then use them correctly because they understand them. Not because they're forced to by strange uses of __, but because they understand the design pattern and how it works. It actually is necessary to understand what's going on under the hood. Having worked with programmers who don't "get" it (and the attendant quality problems), I have to emphasize that transparency trumps all. Clarity. Simplicity.

[chaindriver]

@S.Lott: "The "Name" and the "Intent" should match, but don't have to" That's my point here, isn't it? That the class name need not end with the implementation details, like VehicleFactory.

[user1066101]

The "exposure" is the name? That's a petty concern. Hardly worth discussing.

[chaindriver]

That's exactly my point as well. As I've mentioned, I think there are a lot of miscommunication and misunderstanding here.

[user1066101]

@chaindriver: Some people choose names poorly; they don't use "Factory". You do not have to repeat their mistake.

[chaindriver]

Sure. As I've mentioned above, I'll be using VechicleFactory instead.

[rds]

On a related topic, I have just asked whether overwriting __class__ to implement a factory in Python sounds like a reasonable idea

[extraneon]

I think point 3 is most important. Just make it easy and clear to do the right thing, and if someone has special needs you don't currently foresee, than that's possible to.

[Manoj Govindan]

+1. Patterns are often (though not always) a holdover from languages like Java. Ensure that you absolutely need them before using them in Python.

[Chinmay Kanchi]

Re: "Too long": If you're finding yourself typing long words often, perhaps you need an editor that supports autocomplete...

[Humphrey Bogart]

...Unless the code is compiled, in which case you can't read its source, but if you know how to introspect it let me know!

[user1066101]

@Beau Martínez: "Unless the code is compiled"? As in a .pyc file? They're trivial to disassemble. What point are you making?

[chaindriver]

@S.Lott: You asked "Why? Are your programmers evil sociopaths who refuse to follow the rules?". I was just concerned that the users may not be aware of the factory and might accidentally instantiate the classes directly, and that defeats the purpose of the factory. That's the only thing that I'm trying to prevent, not because I think the users are evil.

[chaindriver]

Oh ok, thanks for the advice! So I shouldn't be looking for ways to prevent client codes from breaking when the internal implementations change in the future e.g. because more functionalities are added? That's the part that I was concerned with. I guess I'm too paranoid?

[chaindriver]

Point 1 is exactly the answer that I need. Thanks!

[user1066101]

@chaindriver: Can they not read your documentation? If they can read your documentation on how to use the factory, I'm unclear on what you're trying to prevent.

[user1066101]

@chaindriver: Sadly, point 1 is the least useful part of this answer.

[user1066101]

@chaindriver: way, way too paranoid. They can read your source. What possible "prevention" can you put in place?

[Matt Joiner]

Yeah, I was really hoping point 2 would be the most useful :(

[chaindriver]

@S.Lott: It's the tendency of users to use a class when it's available. Say if I have an API documentation and the Vehicle class is listed. When the user sees it, it's logical to assume that they think that the class can be used. What you are suggesting is similar to saying there should be no error checking in any of part of the codes since the user is suppose to know how to use all the classes and methods correctly. If that's the case, it's hard to catch bugs, since we have not precluded certain cases that should not happen. Perhaps this is the Python way and I'm not used to it yet I guess.

[chaindriver]

So if I just implement my codes without planning ahead, then what happens if I realise that some of the codes need to be reorganized, but I can't restructure the classes because the client instantiates the classes directly i.e. their codes have to change. What would you do in this case? (I'm not being sarcastic, I'm just searching for a solution to this problem)

[chaindriver]

Sorry, I'm not expressing it properly again. What I meant was that for point 1, that's the exacty answer that I need for that particular question. For the other points, they are useful too and I'm looking into them now. Thanks!

[user1066101]

@chaindriver: "It's the tendency of users to use a class when it's available" What? Are you saying that the "users" are malicious, evil sociopaths who will actively refuse to read and follow your documented architecture? "it's hard to catch bugs" Not really. If they break your rules, things crash. Immediately. What are you worried about? Fighting against these sociopathic users who won't follow your documentation?

[user1066101]

@chaindriver: "but I can't restructure the classes because the client instantiates the classes directly". False, generally. I supposed you might be able to create a situation that's so pathological that there's no way forward. But it's hard. "their codes have to change". Ummm. You tell them? You provide backwards compatible alternative? You fork and provide a new version with support and an old version without support? These are simple, common situations. Look at open source projects like Jinja where they switched to Jinja2.

[user1066101]

@chaindriver: point 1 is a terrible thing to do. Technically, it's sound. But actually, you do not have a problem and don't need this. You're creating a long-term problem withs lots of mangled __ class names. Even the hidden _ class names are a complete waste of your time.

[chaindriver]

@S.Lott: I'm not exactly sure why you have used "evil sociopaths" so many times in your replies. At this point in time, you seem to be the one who is more eager to assume that people are "evil sociopaths" than I do (that phrase has never crossed my mind). I find that programmers in time-crunching production do not always have the time to read through everything in the docs. I know you are going to say that I think they are "evil sociopaths" again, but no, that's not what I think.

[chaindriver]

@S.Lott: People are just too busy to read the whole docs (yes I know they should). They may read a certain portion, but may not have time to read in details, or may skip certain portions that may seem irrelevant to what they are doing at that point of time (yes I know they should read the whole doc from first page to the last, every single word).

[chaindriver]

@S.Lott: When the users break the rules, you'll be lucky if the system just crashes at that point. The problem is when the system continues running with the bug, and it crashes at another point which makes no sense. For example, you can use [] to index into a list, and that works for a string as well. If the user is suppose to provide a list of strings but they give a single string by accident (yes, they should have read the docs), then using the [] to index will get them a single character instead. This is already not something that the API-developer expects, but there's no error yet.

[chaindriver]

@S.Lott: This may cause certain problems further down the codes, and when it finally crashes, it may end up at a part which should not occur. The API-developer then has to trace through the codes (yes I know about debuggers and this debugging step can probably be very easy in your perfect world). If the API-developer has taken a step to check that the input is a list of strings, then he would be absolutely sure that what goes down the rest of the code is a list, and it makes debugging much easier.

[chaindriver]

@S.Lott: You probably are not an advocate of these checking, since the users are not suppose to make any mistakes and they remember every single detail from the docs. Life would really be great if that's the case, and there won't be any bugs. I live in the non-perfect world unfortunately.

[chaindriver]

@S.Lott: "But it's hard" That's exactly my point, that's why I want to minimize it. I understand that there are methods to let client codes change as smoothly as possible. What I'm trying to do here is to minimize these client-code changes when the underlying implementation has to change, especially when the changes are small. I know it's not possible to have a framework that allows client codes not to change at all (if the underlying system totally changes, client code definitely has to change). What I'm trying to do here is just to minimize these changes.

[chaindriver]

I've already tried it and it seems to be working fine so far. I'll remove them though, since you are so sure about this. I'll write up proper docs (which will look weird because the class will be there but I'll say "Please don't instantiate this class directly"), and if any users somehow use them and things break, I'll probably have to get you to help persuade them to read the docs. Don't call them "evil sociopaths" though, because they may not like it. Thanks!

[user1066101]

"Please don't instantiate this class directly" Interestingly, that's often stated in a variety of API docs. "any users somehow use them and things break, I'll probably have to get you to help persuade them to read the docs" Correct. You keep claiming they won't cooperate. I can't understand who would not cooperate except an evil sociopath. Most programmers cooperate. Unless you know something about them that you're not saying.

[user1066101]

@chaindriver: "But it's hard" -- to create a pathologically complex class. You've missed my point entirely. And taken the quote entirely out of context. You aren't "minimizing" anything with all this "privacy". You'll still have changes. You'll still have to make changes. You won't have impossible changes unless you really work at creating something so complex that it cannot be repaired.

[Matt Joiner]

I can't imagine anyone wanting to go searching for classes to use from your code. Personally I hate the things, and avoid them in interfaces. I much prefer to read the documentation and use the capabilities your module(s) expose. If an object is returned, its name and internals are of very little interest. The few times I've needed to poke through code for a class is when it was expected that I inherit, and I wanted to make sure I didn't break anything.

[chaindriver]

@Matt: I didn't mean anyone searching for classes from the code (sigh, another misunderstanding). When I write the docs, I'll write them in docstrings, but if users do a dir(myModule), they'll see those classes with the docs, including those that they shouldn't instantiate. But yep, I'll make sure I write up proper docstrings to let people know of the usage of the classes.

[chaindriver]

@S.Lott: What I mean is to minimize changes in client-code, not my own codes. Anyway, I think the points are clear enough: I shouldn't try too hard to prevent anything and that programmers will read docs. I'll be coding according to these two ideas now. Thanks for the replies.

[chaindriver]

@S.Lott: There are certain ideas that we are thinking differently. I previously find that not all programmers read the docs and might accidentally use the wrong classes. I was wrong. You are making a lot of assumptions though, like I have secrets to hide. I'm not exactly sure why you keep talking about secrets and evil sociopaths. If anyone can assume anything, I can also assume that your secret is that you are the evil sociopath, which is not a good assumption at all. I've digested your idea that programmers will read the docs and cooperate, so I hope we are now on the same page.

[user1066101]

"programmers ... might accidentally use the wrong classes". Correct. "prevention" doesn't help much here, does it? They read the docs, they didn't follow them. Their code will crash. If you write a lot of clever "prevention" code, what changes? It still crashes at run time. This is Python. All crashes are at run time. Either from bad programming (them not following the rules) or from good programming (you making sure it crashes at run time because the didn't follow the rules.) Prevention doesn't enter into it, does it?

[Sam Stoelinga]

@chaindriver As you're working with the client, you can just tell them if they use any part which is not part of the public api, you wont be able to support as it may change. The public api wont change but the private classes may change, they can use them but on their own risk. This may makes both of you happy, in the rare case they just want to use it, they can use it, but probably will only happen in rare cases. If you make a decent factory they will probably use that. Proper communication is most important for your case I think. Your points are valid though in trying to protect the client.

[Sam Stoelinga]

@chaindriver: maybe you could use warning messages whenever they use any part of the private api to protect them and yourself.