Type hints: solve circular dependency

Question

The following produces NameError: name 'Client' is not defined. How can I solve it?

class Server():
    def register_client(self, client: Client)
        pass


class Client():
    def __init__(self, server: Server):
        server.register_client(self)

Solution 1

You can use a forward reference by using a string name for the not-yet-defined Client class:

class Server():
    def register_client(self, client: 'Client')
        pass

As of Python 3.7, you can also postpone all runtime parsing of annotations by adding the following __future__ import at the top of your module:

from __future__ import annotations

at which point the annotations are stored as string representations of the abstract syntax tree for the expression; you can use typing.get_type_hints() to resolve those (and resolve forward references as used above).

See PEP 563 -- Postponed Evaluation of Annotations for details; this behaviour will be the default in Python 4.0.

Solution 2

If you are on Python 3.7+, use from __future__ import annotations as mentioned in another answer. However, if you cannot use 3.7 yet due to OS limitation (like Cygwin as of 2019-06-03), you can use Forward References module to satisfy these types of forward/circular dependency issues.

Pardon the contrived example but this should illustrate the usefulness of this methodology.

class Server():
    clients: list = None

    def __init__(self):
        self.clients=[]

    def register_client(self, client: 'Client') -> None:
        self.clients.append(client)
        print('Client `%s` registered with server' % client.name)

    def print_clients(self) -> None:
        for i, client in enumerate(self.clients):
            print('client %i: %s' % (i, client.name))

    @staticmethod
    def build_clone(server: 'Server') -> 'Server':
        svr_new: Server = Server()
        for client in server.clients:
            svr_new.register_client(client)
        return svr_new

class Client():
    name: str = None
    def __init__(self, name: str, server: 'Server'):
        self.name = name
        server.register_client(self)


svr = Server()
cli = Client('foo', svr)
svr.print_clients()

svr_clone = Server.build_clone(svr)
svr_clone.print_clients()

Comments

[Michael0x2a]

As a note, using string forward references here will no longer be necessary as of Python 3.7 -- instead, add a from __future__ import annotations import to delay evaluation of annotations. This behavior will apparently be enabled by default once Python 4.0 rolls around.

[pasta_sauce]

@MartijnPieters I'm trying to confirm that regardless of class name or string literal, it doesn't matter to Python since it doesn't alter performance. Thus, type hinting is really aimed at 3rd party tools to get maximum benefit and resolutions, correct? (ignoring the self-documented code piece)

[Martijn Pieters]

@pasta_sauce: correct. Type-hints add a very small amount of time to the time it takes to load a module, basically. With the from __future__ import annotations compiler switch that time is further reduced still.

[Semisonic]

While I totally love the from __future__ import annotations solution, for some reason it doesn't work for me: NameError: name 'TypeName' is not defined. Python version is 3.7. Any idea why it might not work?

[Semisonic]

To elaborate my point, this solution does not work if you don't define the annotation type in-place but make a type alias instead: MyType1 = typing.Union[str, MyType2] MyType2 = typing.Mapping[str, MyType1] In this case you still get a NameError

[grokpot]

@Semisonic You still need to cast to string. This works for me: MyType = typing.Union[str, 'MyType2']?

[Martijn Pieters]

@Semisonic (sorry, this dropped of my radar somehow): type aliases are regular assignments and so need to be fully executable; they are not annotations. I put those in a if TYPE_HINTING: block, at which point they are not executed at runtime. Or put the troublesome forward reference in a string, of course.

[urban]

QQ: have you tested the above with pyre? This approach fits a lot better what I try to do, however, pyre seems very confused

[Timothy C. Quinn]

@urban - Never used pyre and just read about for the first time. Seems like an interesting project. I'll give it a go.

[Timothy C. Quinn]

@urban - I did review the pyre tool and found several issues in relation to using typing library. It seems that pyre is handling issues on a case by case basis. Anybody who wants to write up a simple test case and raise an issue with the pyre github page, they may get fixed. For me, I've found a way to get off of Python 3.6 so can now use the annotations library from future. Best of luck.

[user2357112]

The approach suggested in this answer is invalid, and mypy will report a whole bunch of errors if you try to type-check this code. There's no actual connection between T_Server and Server, or T_Client and Client.

[Timothy C. Quinn]

@Monica - Thanks for the info. You may be correct but this did get me out of a bind. I have since figured out how to get all my systems to Python 3.7+ and thus would never use or recommend this methodology except for some who may be stuck in a similar scenario where they don't require static type analysis in their workflows.

[user2357112]

@TimothyC.Quinn: But even pre-3.7, you could have just used 'Server' and 'Client'. You don't need 3.7 to annotate this correctly.

[Timothy C. Quinn]

@Monica - I made an update and tested. Is this what you were meaning?

[user2357112]

No, I mean literally using the strings 'Server' and 'Client' as annotations. Like, def register_client(self, client: 'Client') -> None:

[Timothy C. Quinn]

Thanks. I was not aware of PEP 484 (python.org/dev/peps/pep-0484/#forward-references). That is a much better solution! I will update accordingly.