pyrefly-shape-extensions 1.4.0.dev2__tar.gz → 1.4.0.dev3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: pyrefly-shape-extensions
3
- Version: 1.4.0.dev2
3
+ Version: 1.4.0.dev3
4
4
  Summary: Shape typing primitives and DSL for Pyrefly tensor shape tracking
5
5
  Project-URL: homepage, https://pyrefly.org
6
6
  Project-URL: documentation, https://pyrefly.org/en/docs/
@@ -44,6 +44,9 @@ typing primitives so annotations such as `Tensor[B, T]`, `IntVar("B")`, and
44
44
  `assert_shape(x.shape, (2, 3))` can be evaluated by Python while Pyrefly uses the
45
45
  corresponding stubs for static shape checking.
46
46
 
47
+ `NamedInts` and `CaptureNamedInts` carry named axis lengths from `**kwargs`
48
+ into shape rules for einops-like APIs.
49
+
47
50
  `RegularNestedList[Shape, Domain]` represents regular (non-jagged) nested list
48
51
  literals; for example, `[[1, 2], [3, 4]]` binds `Shape` to `[2, 2]`. Unsupported
49
52
  containers and irregular literals use ordinary typing and any fallback overload
@@ -55,3 +58,31 @@ binds `Values` to `IntTuple[2, 3]`; an existing or starred list remains gradual,
55
58
  while a direct literal containing a non-integer is rejected.
56
59
 
57
60
  The package is versioned in lockstep with Pyrefly.
61
+
62
+ ## Portable shape annotations
63
+
64
+ `Shaped[T, "..."]` is an alias for `typing.Annotated`, so other type checkers
65
+ read `T` and ignore the shape string. Inside a `@shape_vars` function or class,
66
+ Pyrefly also reads the string in annotations, casts, type aliases, and class
67
+ bases. Use `@shape_vars("")` for a literal shape with no declared dimensions.
68
+
69
+ Legacy type aliases cannot capture dimensions declared on an enclosing
70
+ `@shape_vars` function or class. For example, an `Alias: TypeAlias =
71
+ Shaped[Array, "[N]"]` inside a `@shape_vars("N")` definition reports that `N`
72
+ is not in scope for the alias. This is the same restriction that applies when
73
+ legacy type aliases capture ordinary enclosing type parameters. Use the
74
+ `Shaped` annotation directly in that scope; literal-only aliases are supported.
75
+
76
+ A defaulted `@shape_vars("N")` dimension after a `*Ts` class parameter is not
77
+ supported: Pyrefly reports the declaration and can mistake a trailing ordinary
78
+ type argument for the dimension. Pyrefly accepts
79
+ `@shape_vars("N", required=True) class Required[*Ts]` with an explicit shape,
80
+ as in `Required[int, str, 3]`; `Required[int, str]` still treats `str` as a
81
+ dimension. This explicit specialization is specific to Pyrefly, not a
82
+ portable workaround for other type checkers.
83
+
84
+ At runtime, `Shaped[Base, "..."]` can appear as a class base because
85
+ `Annotated` resolves to `Base`. Pyrefly reads its shape. In Pyright 1.1.414,
86
+ the base is rejected and inherited attributes and methods are inferred as
87
+ `Unknown` downstream. Do not rely on this base spelling when Pyright users
88
+ need inherited signatures.
@@ -0,0 +1,52 @@
1
+ # pyrefly-shape-extensions
2
+
3
+ Runtime helpers for Pyrefly tensor shape annotations.
4
+
5
+ This package provides the lightweight `shape_extensions` module used by
6
+ Pyrefly's tensor shape stubs. It defines runtime no-op versions of the shape
7
+ typing primitives so annotations such as `Tensor[B, T]`, `IntVar("B")`, and
8
+ `assert_shape(x.shape, (2, 3))` can be evaluated by Python while Pyrefly uses the
9
+ corresponding stubs for static shape checking.
10
+
11
+ `NamedInts` and `CaptureNamedInts` carry named axis lengths from `**kwargs`
12
+ into shape rules for einops-like APIs.
13
+
14
+ `RegularNestedList[Shape, Domain]` represents regular (non-jagged) nested list
15
+ literals; for example, `[[1, 2], [3, 4]]` binds `Shape` to `[2, 2]`. Unsupported
16
+ containers and irregular literals use ordinary typing and any fallback overload
17
+ supplied by the consumer.
18
+
19
+ `IntTupleOrList[Values]` is a stub-authoring parameter type for APIs that accept
20
+ integer tuples and lists. A direct, unstarred list literal such as `[2, 3]`
21
+ binds `Values` to `IntTuple[2, 3]`; an existing or starred list remains gradual,
22
+ while a direct literal containing a non-integer is rejected.
23
+
24
+ The package is versioned in lockstep with Pyrefly.
25
+
26
+ ## Portable shape annotations
27
+
28
+ `Shaped[T, "..."]` is an alias for `typing.Annotated`, so other type checkers
29
+ read `T` and ignore the shape string. Inside a `@shape_vars` function or class,
30
+ Pyrefly also reads the string in annotations, casts, type aliases, and class
31
+ bases. Use `@shape_vars("")` for a literal shape with no declared dimensions.
32
+
33
+ Legacy type aliases cannot capture dimensions declared on an enclosing
34
+ `@shape_vars` function or class. For example, an `Alias: TypeAlias =
35
+ Shaped[Array, "[N]"]` inside a `@shape_vars("N")` definition reports that `N`
36
+ is not in scope for the alias. This is the same restriction that applies when
37
+ legacy type aliases capture ordinary enclosing type parameters. Use the
38
+ `Shaped` annotation directly in that scope; literal-only aliases are supported.
39
+
40
+ A defaulted `@shape_vars("N")` dimension after a `*Ts` class parameter is not
41
+ supported: Pyrefly reports the declaration and can mistake a trailing ordinary
42
+ type argument for the dimension. Pyrefly accepts
43
+ `@shape_vars("N", required=True) class Required[*Ts]` with an explicit shape,
44
+ as in `Required[int, str, 3]`; `Required[int, str]` still treats `str` as a
45
+ dimension. This explicit specialization is specific to Pyrefly, not a
46
+ portable workaround for other type checkers.
47
+
48
+ At runtime, `Shaped[Base, "..."]` can appear as a class base because
49
+ `Annotated` resolves to `Base`. Pyrefly reads its shape. In Pyright 1.1.414,
50
+ the base is rejected and inherited attributes and methods are inferred as
51
+ `Unknown` downstream. Do not rely on this base spelling when Pyright users
52
+ need inherited signatures.
@@ -8,7 +8,7 @@ description = "Shape typing primitives and DSL for Pyrefly tensor shape tracking
8
8
  readme = "README.md"
9
9
  requires-python = ">=3.10"
10
10
  license = { file = "LICENSE" }
11
- version = "1.4.0.dev2"
11
+ version = "1.4.0.dev3"
12
12
  keywords = ["typechecker", "typechecking", "shapes", "tensor", "pyrefly"]
13
13
  classifiers = [
14
14
  "Development Status :: 4 - Beta",
@@ -16,6 +16,24 @@ import keyword
16
16
  import typing
17
17
  from dataclasses import dataclass
18
18
 
19
+ # A shape annotation carrying its shape as string metadata. To Pyrefly, the
20
+ # string supplies type arguments to a generic class, after any explicit ones:
21
+ # `Shaped[Tensor, "[M, N]"]` is `Tensor[[M, N]]`, and
22
+ # `Shaped[Encoder[Input], "Dim, Hidden"]` is `Encoder[Input, Dim, Hidden]`. On
23
+ # `int` and on an integer tuple, the string instead replaces the type:
24
+ # `Shaped[int, "N"]` is `Int[N]`, and `Shaped[tuple[int, int], "[M, N]"]` is
25
+ # `IntTuple[M, N]`. Every other type checker reads the first argument alone. At
26
+ # runtime it is an ordinary `Annotated` object. The names inside the string are
27
+ # declared by an enclosing `shape_vars`.
28
+ #
29
+ # This must stay an import alias: mypy rejects `Shaped = typing.Annotated` as
30
+ # "not valid as a type". `compatibility_tests` pins this.
31
+ #
32
+ # Ruff recognizes `Annotated` only under that name, so it reports the names in a
33
+ # `Shaped` string as undefined (F821). Code that uses `Shaped` should disable
34
+ # F821; a type checker already reports genuinely undefined names.
35
+ from typing import Annotated as Shaped
36
+
19
37
  __all__ = [
20
38
  "Elements",
21
39
  "Int",
@@ -26,8 +44,11 @@ __all__ = [
26
44
  "IntVar",
27
45
  "Index",
28
46
  "MapIntTuples",
47
+ "NamedInts",
48
+ "CaptureNamedInts",
29
49
  "ProxyMethod",
30
50
  "RegularNestedList",
51
+ "Shaped",
31
52
  "SymbolicArithExpr",
32
53
  "assert_shape",
33
54
  "assert_raises",
@@ -35,6 +56,7 @@ __all__ = [
35
56
  "defines_assert_shape",
36
57
  "gufunc_broadcast",
37
58
  "index_shape",
59
+ "shape_vars",
38
60
  "static_jaxtyping",
39
61
  "type_shape_dsl_function",
40
62
  ]
@@ -179,6 +201,21 @@ class Index:
179
201
  pass
180
202
 
181
203
 
204
+ class NamedInts:
205
+ """Marker bound for named integer ``**kwargs`` in einops-like APIs."""
206
+
207
+ pass
208
+
209
+
210
+ _NamedIntsT = typing.TypeVar("_NamedIntsT")
211
+
212
+
213
+ class CaptureNamedInts(int, typing.Generic[_NamedIntsT]):
214
+ """Capture einops-like axis lengths in a ``NamedInts`` type parameter."""
215
+
216
+ pass
217
+
218
+
182
219
  class ProxyMethod(typing.Generic[_T]):
183
220
  """Type-checker marker for method forwarding annotations."""
184
221
 
@@ -405,24 +442,16 @@ def type_shape_dsl_function(fn: _F) -> _F:
405
442
  return fn
406
443
 
407
444
 
408
- def static_jaxtyping(declaration: str) -> typing.Callable[[_F], _F]:
409
- """Declare the dimension names a function or class may use with jaxtyping.
445
+ def _shape_declaration(tokens: typing.Iterable[str]) -> typing.Callable[[_F], _F]:
446
+ """Build the decorator shared by `static_jaxtyping` and `shape_vars`.
410
447
 
411
- ``declaration`` is a space-separated list of dimension names, where a
412
- leading ``*`` marks a variadic shape::
413
-
414
- @static_jaxtyping("batch channels *rest")
415
- def f(x: Float[Tensor, "batch channels"]) -> Float[Tensor, "*rest"]: ...
416
-
417
- Pyrefly reads the declaration to scope the dimensions and check the shape
418
- strings. Without it, jaxtyping annotations keep their ordinary ``Annotated``
419
- meaning and the array shape stays gradual. At runtime functions are unchanged,
420
- while classes become subscriptable so their static shape arguments can appear
421
- in evaluated annotations.
448
+ In both, a bare token names a dimension and a ``*``-prefixed token names a
449
+ variadic shape. They differ only in how the declaration string is split into
450
+ tokens.
422
451
  """
423
452
 
424
453
  declared_names: list[str] = []
425
- for token in declaration.split():
454
+ for token in tokens:
426
455
  name = token.removeprefix("*")
427
456
  if (
428
457
  name != "_"
@@ -462,7 +491,7 @@ def static_jaxtyping(declaration: str) -> typing.Callable[[_F], _F]:
462
491
  if candidate is None:
463
492
  continue
464
493
  function = getattr(candidate, "__func__", candidate)
465
- if getattr(function, "__static_jaxtyping__", False):
494
+ if getattr(function, "__shape_declaration__", False):
466
495
  continue
467
496
  delegate_descriptor = candidate
468
497
  break
@@ -473,13 +502,57 @@ def static_jaxtyping(declaration: str) -> typing.Callable[[_F], _F]:
473
502
  delegate = delegate_descriptor.__get__(None, cls)
474
503
  return delegate(ordinary_params)
475
504
 
476
- class_getitem.__static_jaxtyping__ = True
505
+ class_getitem.__shape_declaration__ = True
477
506
  value.__class_getitem__ = classmethod(class_getitem)
478
507
  return value
479
508
 
480
509
  return decorate
481
510
 
482
511
 
512
+ def static_jaxtyping(declaration: str) -> typing.Callable[[_F], _F]:
513
+ """Declare the dimension names a function or class may use with jaxtyping.
514
+
515
+ ``declaration`` is a space-separated list of dimension names, where a
516
+ leading ``*`` marks a variadic shape::
517
+
518
+ @static_jaxtyping("batch channels *rest")
519
+ def f(x: Float[Tensor, "batch channels"]) -> Float[Tensor, "*rest"]: ...
520
+
521
+ Pyrefly reads the declaration to scope the dimensions and check the shape
522
+ strings. Without it, jaxtyping annotations keep their ordinary ``Annotated``
523
+ meaning and the array shape stays gradual. At runtime functions are unchanged,
524
+ while classes become subscriptable so their static shape arguments can appear
525
+ in evaluated annotations.
526
+ """
527
+
528
+ return _shape_declaration(declaration.split())
529
+
530
+
531
+ def shape_vars(
532
+ declaration: str, *, required: bool = False
533
+ ) -> typing.Callable[[_F], _F]:
534
+ """Declare the dimension names a function or class may use with `Shaped`.
535
+
536
+ ``declaration`` is a comma-separated list of dimension names, where a
537
+ leading ``*`` marks a variadic shape::
538
+
539
+ @shape_vars("batch, channels, *rest")
540
+ def f(
541
+ x: Shaped[Tensor, "[batch, channels, *Elements[rest]]"],
542
+ ) -> Shaped[Tensor, "[*Elements[rest]]"]: ...
543
+
544
+ On a class, the declared names become type parameters that follow the
545
+ class's own, so Pyrefly reads `Shaped[Encoder, "4, 8"]` as `Encoder[4, 8]`.
546
+ They default to gradual dimensions, so a consumer's `Encoder` or
547
+ `GenericEncoder[Input]` still checks. With ``required=True`` they have no
548
+ defaults, and Pyrefly rejects a specialization that leaves them out. At
549
+ runtime, functions are unchanged, while classes become subscriptable so that
550
+ such annotations can be evaluated.
551
+ """
552
+
553
+ return _shape_declaration(token.strip() for token in declaration.split(","))
554
+
555
+
483
556
  # `dsl` imports the public schema classes above, so defer this import until they exist.
484
557
  from . import dsl as _dsl
485
558
 
@@ -89,17 +89,23 @@ def einops_einsum(spec: str, shapes: _IntTuplesSchema, /) -> _IntTupleSchema:
89
89
  ...
90
90
 
91
91
 
92
- def rearrange(spec: str, shape: _IntTupleSchema, /) -> _IntTupleSchema:
92
+ def rearrange(
93
+ spec: str, shape: _IntTupleSchema, axes: object | None = None, /
94
+ ) -> _IntTupleSchema:
93
95
  """Compute the output shape described by an einops rearrange pattern."""
94
96
  ...
95
97
 
96
98
 
97
- def reduce(spec: str, shape: _IntTupleSchema, /) -> _IntTupleSchema:
99
+ def reduce(
100
+ spec: str, shape: _IntTupleSchema, axes: object | None = None, /
101
+ ) -> _IntTupleSchema:
98
102
  """Compute the output shape described by an einops reduction pattern."""
99
103
  ...
100
104
 
101
105
 
102
- def repeat(spec: str, shape: _IntTupleSchema, /) -> _IntTupleSchema:
106
+ def repeat(
107
+ spec: str, shape: _IntTupleSchema, axes: object | None = None, /
108
+ ) -> _IntTupleSchema:
103
109
  """Compute the output shape described by an einops repeat pattern."""
104
110
  ...
105
111
 
@@ -1,21 +0,0 @@
1
- # pyrefly-shape-extensions
2
-
3
- Runtime helpers for Pyrefly tensor shape annotations.
4
-
5
- This package provides the lightweight `shape_extensions` module used by
6
- Pyrefly's tensor shape stubs. It defines runtime no-op versions of the shape
7
- typing primitives so annotations such as `Tensor[B, T]`, `IntVar("B")`, and
8
- `assert_shape(x.shape, (2, 3))` can be evaluated by Python while Pyrefly uses the
9
- corresponding stubs for static shape checking.
10
-
11
- `RegularNestedList[Shape, Domain]` represents regular (non-jagged) nested list
12
- literals; for example, `[[1, 2], [3, 4]]` binds `Shape` to `[2, 2]`. Unsupported
13
- containers and irregular literals use ordinary typing and any fallback overload
14
- supplied by the consumer.
15
-
16
- `IntTupleOrList[Values]` is a stub-authoring parameter type for APIs that accept
17
- integer tuples and lists. A direct, unstarred list literal such as `[2, 3]`
18
- binds `Values` to `IntTuple[2, 3]`; an existing or starred list remains gradual,
19
- while a direct literal containing a non-integer is rejected.
20
-
21
- The package is versioned in lockstep with Pyrefly.