The correct way to annotate a "file type" in Python

Question

In modern versions of Python one can have static type analysis using function annotations, according to PEP 484. This is made easy through the typing module.

Now I'm wondering how I would give a "type hint" towards a "filestream".

def myfunction(file: FILETYPE):
    pass

with open(fname) as file:
    myfunction(file)

What would I insert as FILETYPE?

Using print(type(file)) returns <class '_io.TextIOWrapper'> which isn't clear at all.

Isn't there a generic "file" type?

Solution 1

You can use typing.IO, typing.TextIO, and typing.BinaryIO to represent different types of I/O streams. To quote the documentation:

сlass typing.IO
class typing.TextIO
class typing.BinaryIO

Generic type IO[AnyStr] and its subclasses TextIO(IO[str]) and BinaryIO(IO[bytes]) represent the types of I/O streams such as returned by open().

Solution 2

I think you want io.IOBase, "[t]he abstract base class for all I/O classes, acting on streams of bytes."

Note that this includes also in-memory streams like io.StringIO and io.BytesIO. Read the documentation on the module io for details.

Solution 3

Either this:

from typing import TextIO # or IO or BinaryIO

def myfunction(file: TextIO ):
    pass

OR this

from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from typing import TextIO # or IO or BinaryIO

def myfunction(file: 'TextIO'):
    pass

The second approach would avoid to import the class during execution. Although python would still have to import TYPE_CHECKING during execution, it is a good practice to avoid importing classes for type hinting only: (1) doesn't get executed (just parsed), and (2) it could avoid cyclic imports.

Comments

[jwodder]

Must the annotation refer to a physical file on disk, or can it also include file-like objects like StringIO?

[djvg]

related: stackoverflow.com/q/38569401, stackoverflow.com/q/24501462

[paul23]

Just as a comment: while this is the "best" answer I might get. The problem is still not solved with this. A lot of things depend on _io._base and derivatives from that. There is no "generic" top layer for both _io._base and io.base though?

[Stop harming Monica]

@paul23 I don't understand what you mean. AFAIK io.IOBase is the best type hint towards a "bytestream" and every file-like object that can be created using the standard library is an instance of it. If IOBase does not match your idea of a bytestream or you have a use case where it's not a good type hint you might want to edit your question and explain why.

[paul23]

for example if you open a in memory byte stream with BytesIO; This derives from _BufferedIOBase which derives from _IOBase

[Stop harming Monica]

@paul23 How is that a problem? Just in case you didn't notice, io.BytesIO also inherits from io.IOBase.

[gerardw]

Given one of the goals on the type hinting PEP 484, I don't see why it's a good practice to avoid imports for type hints.

[lead-free]

typing.IO, typing.TextIO, and typing.BinaryIO is deprecated since version 3.8, will be removed in version 3.12 per typing documentation.

[Eugene Yarmash]

@lead-free I believe the deprecation note only applies to the typing.io namespace. See this and this.

[lead-free]

Though it seems that there is currently no alternative to typing.io that mypy would respect (io.IOBase and it's children).

[djvg]

@EugeneYarmash is right. Here's the actual deprecation message from the docs: "The typing.io namespace is deprecated and will be removed. These types should be directly imported from typing instead."

[pabouk - Ukraine stay strong]

I would say that a private module _typeshed should not be used. Is not the module intended only for developers of type checker tools?

[Kache]

are we not effectively developers of a custom type checker tool for ourselves, in this case? Python type checking is optional, hence the if TYPE_CHECKING.