Custom type hint annotation

Question

I just wrote a simple @autowired decorator for Python that instantiate classes based on type annotations.

To enable lazy initialization of the class, the package provides a lazy(type_annotation: (Type, str)) function so that the caller can use it like this:

@autowired
def foo(bla, *, dep: lazy(MyClass)):
   ...

This works very well, under the hood this lazy function just returns a function that returns the actual type and that has a lazy_init property set to True. Also this does not break IDEs' (e.g., PyCharm) code completion feature.

But I want to enable the use of a subscriptable Lazy type use instead of the lazy function.

Like this:

@autowired
def foo(bla, *, dep: Lazy[MyClass]):
   ...

This would behave very much like typing.Union. And while I'm able to implement the subscriptable type, IDEs' code completion feature will be rendered useless as it will present suggestions for attributes in the Lazy class, not MyClass.

I've been working with this code:

class LazyMetaclass(type):
    def __getitem__(lazy_type, type_annotation):
        return lazy_type(type_annotation)

class Lazy(metaclass=LazyMetaclass):
    def __init__(self, type_annotation):
        self.type_annotation = type_annotation

I tried redefining Lazy.__dict__ as a property to forward to the subscripted type's __dict__ but this seems to have no effect on the code completion feature of PyCharm.

I strongly believe that what I'm trying to achieve is possible as typing.Union works well with IDEs' code completion. I've been trying to decipher what in the source code of typing.Union makes it to behave well with code completion features but with no success so far.

Solution

For the Container[Type] notation to work you would want to create a user-defined generic type:

from typing import TypeVar, Generic

T = TypeVar('T')

class Lazy(Generic[T]):
    pass

You then use

def foo(bla, *, dep: Lazy[MyClass]):

and Lazy is seen as a container that holds the class.

Note: this still means the IDE sees dep as an object of type Lazy. Lazy is a container type here, holding an object of type MyClass. Your IDE won't auto-complete for the MyClass type, you can't use it that way.

The notation also doesn't create an instance of the Lazy class; it creates a subclass instead, via the GenericMeta metaclass. The subclass has a special attribute __args__ to let you introspect the subscription arguments:

>>> a = Lazy[str]
>>> issubclass(a, Lazy)
True
>>> a.__args__
(<class 'str'>,)

If all you wanted was to reach into the type annotations at runtime but resolve the name lazily, you could just support a string value:

def foo(bla, *, dep: 'MyClass'):

This is valid type annotation, and your decorator could resolve the name at runtime by using the typing.get_type_hints() function (at a deferred time, not at decoration time), or by wrapping strings in your lazy() callable at decoration time.

If lazy() is meant to flag a type to be treated differently from other type hints, then you are trying to overload the type hint annotations with some other meaning, and type hinting simply doesn't support such use cases, and using a Lazy[...] containing can't make it work.

Comments

[Rodrigo Martins de Oliveira]

This is much better way of defining custom type annotations. Though it will break code completion features as well.

[Martijn Pieters]

@RodrigoMartins: yes, I was in the process of noting this. I don't think you really want to use the Container[Contained] notation here, it doesn't apply to your situation.

[Rodrigo Martins de Oliveira]

First, to just get this out of the way: I don't intent to hack a way to fool the IDE into thinking that Lazy[A] is of type A. I the same that is observed when using Union[A, B]: IDE will code-complete attributes of A and B because Union behaves like a proxy. The decorator supports string annotations and lazily resolving the name, but it is not intended that annotating your typing with a string will automatically force lazy initialization of the dependency. The lazy initialization feature is meant to be used only when explicitly requested by the caller.

[Rodrigo Martins de Oliveira]

Second: As you noticed, my lazy() function is there only to flag a type to be lazy initialized. But this does not mean to be treated differently by the IDE. I disagree with "type hinting simply doesn't support such use cases", once my decorator has access to the Lazy[A] annotation it can apply special behavior based on the existence of a class member, say Lazy.__lazy_init__. I gave the context of lazy init to better justify why I need this. But in the end, regardless of whatever, I want to define a container type Container[Type] that will show code completions to Type.

[Martijn Pieters]

@RodrigoMartins: your code can treat Lazy differently, but type hinting consumers like your IDE won't. Union has specific meaning in the type hinting spec, so it is indeed treated as a proxy by the IDE. It does so because the spec tells it to. There is no provision in the specs to support user-defined proxies however.

[Rodrigo Martins de Oliveira]

Union has specific meaning in the type hinting spec, so it is indeed treated as a proxy by the IDE. It does so because the spec tells it to. There is no provision in the specs to support user-defined proxies however. Thanks! This is exactly what I needed to know. For reference, this is confirmed by looking into these commits at PyCharm's repository: github.com/JetBrains/intellij-community/…