Type hinting / annotation (PEP 484) for numpy.ndarray
Solution 1
Update
Check recent numpy versions for a new typing
module
https://numpy.org/doc/stable/reference/typing.html#module-numpy.typing
dated answer
It looks like typing
module was developed at:
https://github.com/python/typing
The main numpy
repository is at
https://github.com/numpy/numpy
Python bugs and commits can be tracked at
The usual way of adding a feature is to fork the main repository, develop the feature till it is bomb proof, and then submit a pull request. Obviously at various points in the process you want feedback from other developers. If you can't do the development yourself, then you have to convince someone else that it is a worthwhile project.
cython
has a form of annotations, which it uses to generate efficient C
code.
You referenced the array-like
paragraph in numpy
documentation. Note its typing
information:
A simple way to find out if the object can be converted to a numpy array using array() is simply to try it interactively and see if it works! (The Python Way).
In other words the numpy
developers refuse to be pinned down. They don't, or can't, describe in words what kinds of objects can or cannot be converted to np.ndarray
.
In [586]: np.array({'test':1}) # a dictionary
Out[586]: array({'test': 1}, dtype=object)
In [587]: np.array(['one','two']) # a list
Out[587]:
array(['one', 'two'],
dtype='<U3')
In [589]: np.array({'one','two'}) # a set
Out[589]: array({'one', 'two'}, dtype=object)
For your own functions, an annotation like
def foo(x: np.ndarray) -> np.ndarray:
works. Of course if your function ends up calling some numpy
function that passes its argument through asanyarray
(as many do), such an annotation would be incomplete, since your input could be a list
, or np.matrix
, etc.
When evaluating this question and answer, pay attention to the date. 484 was a relatively new PEP back then, and code to make use of it for standard Python still in development. But it looks like the links provided are still valid.
Solution 2
Numpy 1.21 includes a numpy.typing
module with an NDArray
generic type.
From the Numpy 1.21 docs:
numpy.typing.NDArray = numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]
A generic version of
np.ndarray[Any, np.dtype[+ScalarType]]
.Can be used during runtime for typing arrays with a given dtype and unspecified shape.
Examples:
>>> import numpy as np >>> import numpy.typing as npt >>> print(npt.NDArray) numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]] >>> print(npt.NDArray[np.float64]) numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] >>> NDArrayInt = npt.NDArray[np.int_] >>> a: NDArrayInt = np.arange(10) >>> def func(a: npt.ArrayLike) -> npt.NDArray[Any]: ... return np.array(a)
As of 11/10/2021, support for shapes is still a work in progress per numpy/numpy#16544.
Solution 3
At my company we've been using:
from typing import TypeVar, Generic, Tuple, Union, Optional
import numpy as np
Shape = TypeVar("Shape")
DType = TypeVar("DType")
class Array(np.ndarray, Generic[Shape, DType]):
"""
Use this to type-annotate numpy arrays, e.g.
image: Array['H,W,3', np.uint8]
xy_points: Array['N,2', float]
nd_mask: Array['...', bool]
"""
pass
def compute_l2_norm(arr: Array['N,2', float]) -> Array['N', float]:
return (arr**2).sum(axis=1)**.5
print(compute_l2_norm(arr = np.array([(1, 2), (3, 1.5), (0, 5.5)])))
We actually have a MyPy checker around this that checks that the shapes work out (which we should release at some point). Only thing is it doesn't make PyCharm happy (ie you still get the nasty warning lines):
Solution 4
nptyping adds lots of flexibility for specifying numpy type hints.
Solution 5
Check out DataShape. It uses the datatypes as well as some syntax for how big the input and output arrays should be.
Comments
-
Inon almost 2 years
Has anyone implemented type hinting for the specific
numpy.ndarray
class?Right now, I'm using
typing.Any
, but it would be nice to have something more specific.For instance if the NumPy people added a type alias for their array_like object class. Better yet, implement support at the dtype level, so that other objects would be supported, as well as ufunc.
-
hpaulj over 8 yearsI don't recall seeing any use of Python3 type annotation in SO
numpy
questions or answers. -
hpaulj over 8 yearspypi.python.org/pypi/plac can make use of Py3 annotations - to populate an
argparse
parser. For Py2, it uses decorators to create a similarannocation
database. -
hpaulj over 8 years
typing
is new to Py 3.5. Manynumpy
users still work with Py2. I have 3.5 on my system, but I don't havenumpy
installed for it.numpy
developers are not going to add features for the cutting edge of Python (with the exception of the@
operator) -
Inon over 8 years@hpaulj, can you cite your source for the last comment? I'm not sure where I should go to interact with the Numpy maintainers... it could very well be that integrating other 'advanced' Python features would be popular.
-
hpaulj over 8 years
numpy
is maintained on agithub
repository. Look at theissues
andpull requests
; sign up and submit your own issue. There may be another forum for discussing development issues, but most I look at thegithub
issues. -
Itamar Mushkin about 5 yearsFor anyone looking into the issue - it looks like there's a relevant solution here: stackoverflow.com/questions/52839427/…
-
Jasha almost 4 yearsThere is now an open issue in the numpy github repository regarding type hinting / annotation for numpy types.
-
Inon almost 4 years> There is now... @Jasha this ticket was opened by me, the OP, 4.5 years ago.
-
-
Inon over 8 yearsso DataShape is a Numpy alternative? Not quite what I had in mind, as I'm using SciPy, which requires Numpy, explicitly.
-
Back2Basics over 8 yearsDataShape is a description. There is no official function annotations currently but so far this is the best description that I've seen for Numpy types if you are going to build in function annotations. And yes, I would suggest creating a new module name and using it as a proof of concept before introducing function annotations into the numpy source.
-
hpaulj over 8 yearsWhat software, editor or interpreter are you using that makes use of
annotations
? As best I know, in plain Python 3, a function gets a__annotations__
dictionary, but the interpreter does nothing with it. -
hpaulj over 8 yearsDo you want
typing
annotations added to existingnumpy
functions (includingnp.array
), or just types that would make it easier to add annotations to your own functions? -
Inon over 8 yearsI've marked this answer as the accepted one, but just for completeness, I was going for the latter (type hinting in my own code, which uses Numpy). I'm all for Duck Typing, but when you can provide static type information, I don't see why you wouldn't, if only for static code analysis (PyCharm does warn about incompatible types). Thanks, @hpaulj!
-
Vitalis almost 6 yearsSince, typing module simply provides hints, I have created two helper labels purely for readability and note it doesn't pass mypy static type checks.
def Vector(np_arr): return np_arr.ndim == 1 def Matrix(np_arr): return np_arr.ndim > 1
. Hope, it helps someone. -
Jules G.M. over 4 yearsthis is worse than using
np.ndarray
as a type -
papercrane over 4 yearsHow does one use DataShape? The documentation says a lot about what DataShape can do but I'm not really finding any specific example for how it can be used for type hinting in Python.
-
Steve3p0 almost 4 yearsWhat about the shape? I can add hints like def blah() -> np.ndarray(785): But I can't can't add a second dimension like -> np.ndarray(785, 10). Having a shape hint is very helpful and brings clarity to multiple functions in my code that produce arrays of varying dimensionality.
-
Georgy almost 4 yearsThis is just a link-only answer. It would be preferable to add a usage example of this library here, and provide the link for reference.
-
Back2Basics almost 4 yearsThanks for the feedback. I can update this later this week.
-
FarisHijazi about 3 yearsany updates on the MyPy checker? would love to integrate it to my env
-
amka66 over 2 yearsThis is good stuff, thanks for sharing. It seems, however, that the nptyping package (github.com/ramonhagenaars/nptyping) considerably generalizes this.