ommlds 0.0.0.dev499__py3-none-any.whl → 0.0.0.dev503__py3-none-any.whl

ommlds/minichain/services/README.md

@@ -0,0 +1,154 @@
+The core service abstraction. In general, Services are intended to encapsulate non-trivial, resourceful, effectful
+operations, which are likely to have various implementations, each having their own specific capabilities in addition
+to their common interface, and where wrapping, adapting, and transforming them in a uniform manner is desirable.
+
+For example:
+ - A ChatService is passed a Request with a Chat value and returns a Response with an AiChat value.
+ - A ChatChoicesService is passed a Request with a Chat and returns a Response with a list of AiChoices.
+ - A ChatChoicesServiceChatService is a simple adapter taking a ChatChoicesService which it will invoke with its given
+   Request, expect a single AiChoice value in that Response, and return that single AiChat as its Response - thus acting
+   as a ChatService.
+   - Thus, all chat backends that return choices can be adapted to code expecting a single chat as output.
+ - A ChatStreamService is passed a Request with a Chat value and returns a Response with a value from which AiDeltas
+   may be streamed.
+ - A ChatChoicesStreamService is passed a Request with a Chat value and returns a Response with a value from which
+   AiChoicesDeltas may be streamed.
+ - A ChatChoicesStreamServiceChatChoicesService is an adapter taking a ChatChoicesStreamService and aggregating the
+   AiChoicesDeltas into joined, non-delta AiChoices.
+   - This may then be wrapped in a ChatChoicesServiceChatService to act as a ChatService.
+   - In practice however there are usually dedicated streaming and non-streaming implementations if possible as
+     non-streaming will usually have less overhead.
+ - An OpenaiChatChoicesService can act as a ChatChoicesService, and will accept all generic ChatOptions, in addition to
+   any OpenaiChatOptions inapplicable to any other backend. It may also produce all generic ChatOutputs, in addition to
+   OpenaiChatOutputs that will not be produced by other backends.
+ - Beyond chat, a VectorSearchService is passed a Request with a VectorSearch value and returns a Response with a
+   VectorHits value.
+ - A RetryService wraps any other Service and will attempt to re-invoke it on failure.
+ - A FirstInWinsService wraps any number of other Services and will return the first non-error Response it receives.
+
+The service abstraction consists of 3 interrelated generic types:
+ - Request, an immutable final generic class containing a single value and any number of options.
+ - Response, an immutable final generic class containing a single value and any number of outputs.
+ - Service, a generic protocol consisting of a single async method `invoke`, taking a request and returning a response.
+
+There are 2 related abstract base classes in the parent package:
+ - Option, a non-generic abstract class representing a service option.
+ - Output, a non-generic abstract class representing a service output.
+
+The purpose of this arrangement is to provide the following:
+ - There is only one method - `Service.invoke` - to deal with.
+ - There is no base `Service` class - service types are distinguished only by the requests they accept and responses
+   they return.
+ - It facilitates a clear, introspectable, generally type-safe means for handling 'less-specific' and 'more-specific'
+   service types.
+ - It facilitates generic wrapper and transformation machinery.
+
+The variance of the type parameters of the 3 classes is central:
+ - `Request[V_co, OptionT_co]`
+ - `Response[V_co, OutputT_contra]`
+ - `Service[RequestT_contra, ResponseT_co]`
+
+And to understand this, it's important to understand how Option and Output subtypes are intended to be arranged:
+ - These types are *not* intended to form a deep type hierarchy:
+   - A RemoteChatOption is *not* intended to inherit from a ChatOption: a ChatOption (be it a base class or union alias)
+     represents an option that *any* ChatService can accept, whereas a RemoteChatOption represents an option that *only*
+     applies to a RemoteChatService.
+   - If RemoteChatOption inherited from a base ChatOption, then it would have to apply to *all* ChatService
+     implementations.
+   - For example: were ApiKey to inherit from ChatOption, then it would have to apply to all ChatServices, including
+     LocalChatService, which has no concept of an api key.
+ - Similarly, a RemoteChatOutput is *not* intended to inherit from a ChatOutput: a ChatOutput represents an output that
+   *any* ChatService can produce, whereas a RemoteChatOutput represents an output that *only* applies to a
+   RemoteChatService.
+ - These 2 types are intended to form flat, disjoint, unrelated families of subtypes, and Request and Response are
+   intended to be parameterized by the unions of all such families they may contain.
+ - Because of this, one's visual intuition regarding types and subtypes may be reversed: `int` is effectively a subtype
+   of `int | str` despite `int` being a visually shorter, less complex type.
+   - `int` is a *MORE SPECIFIC* / *STRICT SUBSET* subtype of `int | str`, the *LESS SPECIFIC* / *STRICT SUPERSET*
+     supertype.
+
+Regarding type variance:
+ - Service has the classic setup of contravariant input and covariant output:
+   - A RemoteChatService *is a* ChatService.
+   - A RemoteChatService may accept less specific requests than a ChatService.
+   - A RemoteChatService may return more specific responses than a ChatService.
+ - Request is covariant on its options:
+   - Recall, a RemoteChatOption *is not a* ChatOption.
+   - A ChatRequest *is a* RemoteChatRequest as it will not contain options RemoteChatService cannot accept.
+ - Response is contravariant on its outputs:
+   - Recall, a RemoteChatOutput *is not a* ChatOutput.
+   - A RemoteChatResponse *is a* ChatResponse even though it may contain additional output variants not produced by
+     every ChatService.
+   - Code that calls a ChatService and is given a ChatResponse must be prepared to handle (usually by simply ignoring)
+     outputs not necessarily produced by a base ChatService.
+
+Finally, in addition to a value and either options or outputs, a Request and Response each also contain a collection of
+metadata. Very much unlike the Options and Outputs, the elements of these collections are simply of the types
+`RequestMetadata | CommonMetadata` and `ResponseMetadata | CommonMetadata`, and are not otherwise parameterized. These
+are intended for looser inputs and outputs: a generic unique id, timestamps, metrics, etc., and in general should
+neither affect the behavior of services nor be depended upon by callers.
+
+Below is a representative illustration of these types and their relationships. Note how:
+ - There is no subclassing of Request, Response, or Service - just type aliasing.
+ - There is no deep, shared subclassing of Option or Output.
+ - The type args passed to Request and Response are unions of all the Option and Output subtypes they may contain.
+   - These unions are kept in pluralized type aliases for convenience.
+ - There is no base ChatOption or ChatOutput class - were there, it would not be included in the base classes of any
+   local or remote only option or output.
+ - The local and remote sections take different but equivalent approaches:
+   - There are no base LocalChatOption or LocalChatOutput classes, but there *are* base RemoteChatOption and
+     RemoteChatOutput classes.
+   - Without any common base classes (besides the lowest level Output and Option classes), the local section treats them
+     as each distinct and bespoke, and the pluralized LocalChatOptions and LocalChatOutputs type aliases aggregate them
+     by explicitly listing them.
+   - With the common RemoteChatOption and RemoteChatOutput base classes, the remote section treats them as a related
+     family that any 'RemoteChat'-like service should accept and produce.
+
+```python
+# Common chat
+
+class MaxTokens(Option, tv.UniqueScalarTypedValue[int]): pass
+class Temperature(Option, tv.UniqueScalarTypedValue[float]): pass
+
+ChatOptions: ta.TypeAlias = MaxTokens | Temperature
+ChatRequest: ta.TypeAlias = Request[Chat, ChatOptions]
+
+class TokenUsage(Output, tv.UniqueScalarTypedValue[int]): pass
+class ElapsedTime(Output, tv.UniqueScalarTypedValue[float]): pass
+
+ChatOutputs: ta.TypeAlias = TokenUsage | ElapsedTime
+ChatResponse: ta.TypeAlias = Response[Message, ChatOutputs]
+
+ChatService: ta.TypeAlias = Service[ChatRequest, ChatResponse]
+
+# Local chat
+
+class ModelPath(Option, tv.ScalarTypedValue[str]): pass
+
+LocalChatOptions: ta.TypeAlias = ChatOptions | ModelPath
+LocalChatRequest: ta.TypeAlias = Request[Chat, LocalChatOptions]
+
+class LogPath(Output, tv.ScalarTypedValue[str]): pass
+
+LocalChatOutputs: ta.TypeAlias = ChatOutputs | LogPath
+LocalChatResponse: ta.TypeAlias = Response[Message, LocalChatOutputs]
+
+LocalChatService: ta.TypeAlias = Service[LocalChatRequest, LocalChatResponse]
+
+# Remote chat
+
+class RemoteChatOption(Option, lang.Abstract): pass
+class ApiKey(RemoteChatOption, tv.ScalarTypedValue[str]): pass
+
+RemoteChatOptions: ta.TypeAlias = ChatOptions | RemoteChatOption
+RemoteChatRequest: ta.TypeAlias = Request[Chat, RemoteChatOptions]
+
+class RemoteChatOutput(Output, lang.Abstract): pass
+class BilledCostInUsd(RemoteChatOutput, tv.UniqueScalarTypedValue[float]): pass
+
+RemoteChatOutputs: ta.TypeAlias = ChatOutputs | RemoteChatOutput
+RemoteChatResponse: ta.TypeAlias = Response[Message, RemoteChatOutputs]
+
+RemoteChatService: ta.TypeAlias = Service[RemoteChatRequest, RemoteChatResponse]
+```
+

ommlds/minichain/services/__init__.py

@@ -1,6 +1,4 @@
 # ruff: noqa: I001
-
-
 from .facades import (  # noqa
     ServiceFacade,
 
@@ -8,10 +6,16 @@ from .facades import (  # noqa
 )
 
 from .requests import (  # noqa
+    RequestMetadata,
+    RequestMetadatas,
+
     Request,
 )
 
 from .responses import (  # noqa
+    ResponseMetadata,
+    ResponseMetadatas,
+
     Response,
 )
 

ommlds/minichain/services/_marshal.py

@@ -1,3 +1,8 @@
+"""
+FIXME:
+ - everything lol
+ - can this just do what metadata does
+"""
 import typing as ta
 
 from omlish import check
@@ -22,10 +27,21 @@ def _is_rr_rty(rty: rfl.Type) -> bool:
     )
 
 
-def _get_tv_fld(rty: rfl.Type) -> dc.Field:
+class _RrFlds(ta.NamedTuple):
+    v: dc.Field
+    tv: dc.Field
+    md: dc.Field
+
+
+def _get_rr_flds(rty: rfl.Type) -> _RrFlds:
     flds = col.make_map_by(lambda f: f.name, dc.fields(check.not_none(rfl.get_concrete_type(rty))), strict=True)
-    flds.pop('v')
-    return check.single(flds.values())
+    v_fld = flds.pop('v')
+    md_fld = flds.pop('_metadata')
+    return _RrFlds(
+        v=v_fld,
+        tv=check.single(flds.values()),
+        md=md_fld,
+    )
 
 
 ##
@@ -34,7 +50,7 @@ def _get_tv_fld(rty: rfl.Type) -> dc.Field:
 @dc.dataclass(frozen=True)
 class _RequestResponseMarshaler(msh.Marshaler):
     rty: rfl.Type
-    tv_fld: dc.Field
+    rr_flds: _RrFlds
     v_m: msh.Marshaler | None
 
     def marshal(self, ctx: msh.MarshalContext, o: ta.Any) -> msh.Value:
@@ -51,9 +67,14 @@ class _RequestResponseMarshaler(msh.Marshaler):
         else:
             v_v = self.v_m.marshal(ctx, o.v)
 
+        md_fmd = self.rr_flds.md.metadata[msh.FieldMetadata]
+        md_m = md_fmd.marshaler_factory.make_marshaler(ctx.marshal_factory_context, self.rr_flds.md.type)()  # FIXME
+        md_v = md_m.marshal(ctx, o._metadata)  # noqa
+
         return {
             'v': v_v,
-            **({lang.strip_prefix(self.tv_fld.name, '_'): tv_v} if tv_v else {}),
+            **({lang.strip_prefix(self.rr_flds.tv.name, '_'): tv_v} if tv_v else {}),
+            **({'metadata': md_v} if md_v else {}),
         }
 
 
@@ -76,7 +97,7 @@ class _RequestResponseMarshalerFactory(msh.MarshalerFactory):
                 v_m = ctx.make_marshaler(v_rty)
             return _RequestResponseMarshaler(
                 rty,
-                _get_tv_fld(rty),
+                _get_rr_flds(rty),
                 v_m,
             )
 
@@ -89,9 +110,10 @@ class _RequestResponseMarshalerFactory(msh.MarshalerFactory):
 @dc.dataclass(frozen=True)
 class _RequestResponseUnmarshaler(msh.Unmarshaler):
     rty: rfl.Type
-    tv_fld: dc.Field
+    rr_flds: _RrFlds
     v_u: msh.Unmarshaler
     tv_u: msh.Unmarshaler
+    md_u: msh.Unmarshaler
 
     def unmarshal(self, ctx: msh.UnmarshalContext, v: msh.Value) -> ta.Any:
         dct = dict(check.isinstance(v, ta.Mapping))
@@ -99,9 +121,14 @@ class _RequestResponseUnmarshaler(msh.Unmarshaler):
         v_v = dct.pop('v')
         v = self.v_u.unmarshal(ctx, v_v)
 
+        if md_v := dct.pop('metadata', None):
+            md = self.md_u.unmarshal(ctx, md_v)
+        else:
+            md = []
+
         tvs: ta.Any
         if dct:
-            tv_vs = dct.pop(lang.strip_prefix(self.tv_fld.name, '_'))
+            tv_vs = dct.pop(lang.strip_prefix(self.rr_flds.tv.name, '_'))
             tvs = self.tv_u.unmarshal(ctx, tv_vs)
         else:
             tvs = []
@@ -109,7 +136,7 @@ class _RequestResponseUnmarshaler(msh.Unmarshaler):
         check.empty(dct)
 
         cty = rfl.get_concrete_type(self.rty)
-        return cty(v, tvs)  # type: ignore
+        return cty(v, tvs, _metadata=md)  # type: ignore
 
 
 class _RequestResponseUnmarshalerFactory(msh.UnmarshalerFactory):
@@ -123,15 +150,24 @@ class _RequestResponseUnmarshalerFactory(msh.UnmarshalerFactory):
             else:
                 # FIXME: ...
                 raise TypeError(rty)
+
+            rr_flds = _get_rr_flds(rty)
+
             tv_types_set = check.isinstance(tv_rty, rfl.Union).args
             tv_ta = tv.TypedValues[ta.Union[*tv_types_set]]  # type: ignore
             tv_u = ctx.make_unmarshaler(tv_ta)
+
             v_u = ctx.make_unmarshaler(v_rty)
+
+            md_fmd = rr_flds.md.metadata[msh.FieldMetadata]
+            md_u = md_fmd.unmarshaler_factory.make_unmarshaler(ctx, rr_flds.md.type)()  # FIXME
+
             return _RequestResponseUnmarshaler(
                 rty,
-                _get_tv_fld(rty),
+                _get_rr_flds(rty),
                 v_u,
                 tv_u,
+                md_u,
             )
 
         return inner

ommlds/minichain/services/_origclasses.py

@@ -43,3 +43,14 @@ class _OrigClassCapture:
         """Enforces that __orig_class__ has only been set once."""
 
         return check.single(object.__getattribute__(self, '__captured_orig_classes__'))
+
+
+def confer_orig_class(src, dst):
+    if src is not dst:
+        try:
+            oc = src.__orig_class__
+        except AttributeError:
+            pass
+        else:
+            dst.__orig_class__ = oc
+    return dst

ommlds/minichain/services/_typedvalues.py

@@ -45,14 +45,19 @@ class _TypedValues(
     lang.Abstract,
     ta.Generic[TypedValueT],
 ):
-    __typed_values_class__: ta.ClassVar[type[tv.TypedValue]]
+    """
+    The reason this is so complicated compared to any other TypedValues field (like metadata) is that the real set of
+    TypedValue types it accepts is known only via __orig_class__.
+    """
+
+    __typed_values_base__: ta.ClassVar[type[tv.TypedValue]]
 
     def __init_subclass__(cls, **kwargs: ta.Any) -> None:
         super().__init_subclass__(**kwargs)
 
         tvt = _get_typed_values_type_arg(cls)
         tvct = rfl.get_concrete_type(tvt, use_type_var_bound=True)
-        cls.__typed_values_class__ = check.issubclass(check.isinstance(tvct, type), tv.TypedValue)
+        cls.__typed_values_base__ = check.issubclass(check.isinstance(tvct, type), tv.TypedValue)
 
     #
 
@@ -77,7 +82,7 @@ class _TypedValues(
 
         tv_types_set = frozenset(tv.reflect_typed_values_impls(tvt))
         tv_types = tuple(sorted(
-            [check.issubclass(c, self.__typed_values_class__) for c in tv_types_set],
+            [check.issubclass(c, self.__typed_values_base__) for c in tv_types_set],
             key=lambda c: c.__qualname__,
         ))
 

ommlds/minichain/services/requests.py

@@ -6,8 +6,12 @@ from omlish import lang
 from omlish import typedvalues as tv
 
 from .._typedvalues import _tv_field_metadata
+from ..metadata import CommonMetadata
+from ..metadata import Metadata
+from ..metadata import MetadataContainerDataclass
 from ..types import Option
 from ..types import OptionT_co
+from ._origclasses import confer_orig_class
 from ._typedvalues import _TypedValues
 
 
@@ -19,6 +23,17 @@ OptionU = ta.TypeVar('OptionU', bound=Option)
 ##
 
 
+class RequestMetadata(Metadata, lang.Abstract):
+    pass
+
+
+RequestMetadatas: ta.TypeAlias = RequestMetadata | CommonMetadata
+
+
+##
+
+
+@ta.final
 @dc.dataclass(frozen=True)
 @dc.extra_class_params(
     allow_dynamic_dunder_attrs=True,
@@ -26,11 +41,32 @@ OptionU = ta.TypeVar('OptionU', bound=Option)
 )
 class Request(  # type: ignore[type-var]  # FIXME: _TypedValues param is invariant
     _TypedValues[OptionT_co],
+    MetadataContainerDataclass[RequestMetadatas],
     lang.Final,
     ta.Generic[V_co, OptionT_co],
 ):
+    """
+    Universal service request, comprised of:
+     - a value of type `V_co`
+     - a sequence of options of type `OptionT_co`
+     - metadata of type `RequestMetadatas`
+
+    Refer to the package README.md for an explanation of its type var variance.
+
+    This class is final, but each instance's `__orig_class__` (if present) is significant. It is encouraged to construct
+    these through a pre-parameterized type alias, and the provided `with_` methods should be used rather than
+    `dc.replace` (as they will propagate `__orig_class__`).
+    """
+
+    #
+
     v: V_co  # type: ignore[misc]  # FIXME: Cannot use a covariant type variable as a parameter
 
+    def with_v(self, v: V_co) -> ta.Self:  # type: ignore[misc]
+        return confer_orig_class(self, dc.replace(self, v=v))
+
+    #
+
     _options: ta.Sequence[OptionT_co] = dc.field(
         default=(),
         metadata=_tv_field_metadata(
@@ -43,13 +79,47 @@ class Request(  # type: ignore[type-var]  # FIXME: _TypedValues param is invaria
     def options(self) -> tv.TypedValues[OptionT_co]:
         return check.isinstance(self._options, tv.TypedValues)
 
-    def with_options(self, *options: OptionU, override: bool = False) -> 'Request[V_co, OptionT_co | OptionU]':
-        return dc.replace(self, _options=self.options.update(*options, override=override))
-
     @property
     def _typed_values(self) -> tv.TypedValues[OptionT_co]:
         return check.isinstance(self._options, tv.TypedValues)
 
+    def with_options(
+            self,
+            *add: OptionU,
+            discard: ta.Iterable[type] | None = None,
+            override: bool = False,
+    ) -> 'Request[V_co, OptionT_co | OptionU]':
+        new = (old := self.options).update(
+            *add,
+            discard=discard,
+            override=override,
+        )
+
+        if new is old:
+            return self
+
+        return confer_orig_class(self, dc.replace(self, _options=new))
+
+    #
+
+    _metadata: ta.Sequence[RequestMetadatas] = dc.field(
+        default=(),
+        kw_only=True,
+        repr=False,
+    )
+
+    MetadataContainerDataclass._configure_metadata_field(_metadata, RequestMetadatas)  # noqa
+
+    def with_metadata(
+            self,
+            *add: RequestMetadatas,
+            discard: ta.Iterable[type] | None = None,
+            override: bool = False,
+    ) -> ta.Self:
+        return confer_orig_class(self, super().with_metadata(*add, discard=discard, override=override))
+
+    #
+
     def validate(self) -> ta.Self:
         self._check_typed_values()
         return self

ommlds/minichain/services/responses.py

@@ -6,8 +6,12 @@ from omlish import lang
 from omlish import typedvalues as tv
 
 from .._typedvalues import _tv_field_metadata
+from ..metadata import CommonMetadata
+from ..metadata import Metadata
+from ..metadata import MetadataContainerDataclass
 from ..types import Output
 from ..types import OutputT_contra
+from ._origclasses import confer_orig_class
 from ._typedvalues import _TypedValues
 
 
@@ -17,6 +21,17 @@ V_co = ta.TypeVar('V_co', covariant=True)
 ##
 
 
+class ResponseMetadata(Metadata, lang.Abstract):
+    pass
+
+
+ResponseMetadatas: ta.TypeAlias = ResponseMetadata | CommonMetadata
+
+
+##
+
+
+@ta.final
 @dc.dataclass(frozen=True)
 @dc.extra_class_params(
     allow_dynamic_dunder_attrs=True,
@@ -24,11 +39,32 @@ V_co = ta.TypeVar('V_co', covariant=True)
 )
 class Response(  # type: ignore[type-var]  # FIXME: _TypedValues param is invariant
     _TypedValues[OutputT_contra],
+    MetadataContainerDataclass[ResponseMetadatas],
     lang.Final,
     ta.Generic[V_co, OutputT_contra],
 ):
+    """
+    Universal service response, comprised of:
+     - a value of type `V_co`
+     - a sequence of outputs of type `OutputT_contra`
+     - metadata of type `ResponseMetadatas`
+
+    Refer to the package README.md for an explanation of its type var variance.
+
+    This class is final, but each instance's `__orig_class__` (if present) is significant. It is encouraged to construct
+    these through a pre-parameterized type alias, and the provided `with_` methods should be used rather than
+    `dc.replace` (as they will propagate `__orig_class__`).
+    """
+
+    #
+
     v: V_co  # type: ignore[misc]  # FIXME: Cannot use a covariant type variable as a parameter
 
+    def with_v(self, v: V_co) -> ta.Self:  # type: ignore[misc]
+        return confer_orig_class(self, dc.replace(self, v=v))
+
+    #
+
     _outputs: ta.Sequence[OutputT_contra] = dc.field(
         default=(),
         metadata=_tv_field_metadata(
@@ -41,13 +77,47 @@ class Response(  # type: ignore[type-var]  # FIXME: _TypedValues param is invari
     def outputs(self) -> tv.TypedValues[OutputT_contra]:
         return check.isinstance(self._outputs, tv.TypedValues)
 
-    def with_outputs(self, *outputs: OutputT_contra, override: bool = False) -> ta.Self:
-        return dc.replace(self, _outputs=self.outputs.update(*outputs, override=override))
-
     @property
     def _typed_values(self) -> tv.TypedValues[OutputT_contra]:
         return check.isinstance(self._outputs, tv.TypedValues)
 
+    def with_outputs(
+            self,
+            *add: OutputT_contra,
+            discard: ta.Iterable[type] | None = None,
+            override: bool = False,
+    ) -> ta.Self:
+        new = (old := self.outputs).update(
+            *add,
+            discard=discard,
+            override=override,
+        )
+
+        if new is old:
+            return self
+
+        return confer_orig_class(self, dc.replace(self, _outputs=new))
+
+    #
+
+    _metadata: ta.Sequence[ResponseMetadatas] = dc.field(
+        default=(),
+        kw_only=True,
+        repr=False,
+    )
+
+    MetadataContainerDataclass._configure_metadata_field(_metadata, ResponseMetadatas)  # noqa
+
+    def with_metadata(
+            self,
+            *add: ResponseMetadatas,
+            discard: ta.Iterable[type] | None = None,
+            override: bool = False,
+    ) -> ta.Self:
+        return confer_orig_class(self, super().with_metadata(*add, discard=discard, override=override))
+
+    #
+
     def validate(self) -> ta.Self:
         self._check_typed_values()
         return self

ommlds/minichain/services/services.py

@@ -11,4 +11,13 @@ from .responses import ResponseT_co
 
 @ta.runtime_checkable
 class Service(lang.ProtocolForbiddenAsBaseClass, ta.Protocol[RequestT_contra, ResponseT_co]):
+    """
+    Universal service protocol, comprised of a single method `invoke`, accepting a request of type `RequestT_contra` and
+    returning a response of type `ResponseT_co`.
+
+    Refer to the package README.md for an explanation of its type var variance.
+
+    This class is final, but each instance's `__orig_class__` (if present) is significant.
+    """
+
     def invoke(self, request: RequestT_contra) -> ta.Awaitable[ResponseT_co]: ...

ommlds/minichain/stream/services.py

@@ -1,3 +1,7 @@
+"""
+TODO:
+ - new_stream_response carry TypeAlias __orig_type__ somehow
+"""
 import abc
 import itertools
 import types
@@ -104,17 +108,23 @@ class _StreamServiceResponse(StreamResponseIterator[V, OutputT]):
             return _StreamServiceResponse._Emit(self._ssr, item)
 
     _state: ta.Literal['new', 'running', 'closed'] = 'new'
+
     _sink: _Sink[V]
-    _a: ta.Any
+
     _cr: ta.Any
+    _a: ta.Any
+    _g: ta.Any
 
     async def __aenter__(self) -> ta.Self:
         check.state(self._state == 'new')
         self._state = 'running'
+
         self._sink = _StreamServiceResponse._Sink(self)
+
         self._cr = self._fn(self._sink)
         self._a = self._cr.__await__()
         self._g = iter(self._a)
+
         return self
 
     @types.coroutine
@@ -123,8 +133,10 @@ class _StreamServiceResponse(StreamResponseIterator[V, OutputT]):
         self._state = 'closed'
         if old_state != 'running':
             return
+
         if self._cr.cr_running or self._cr.cr_suspended:
             cex = StreamServiceCancelledError()
+
             i = None
             for n in itertools.count():
                 try:
@@ -132,13 +144,18 @@ class _StreamServiceResponse(StreamResponseIterator[V, OutputT]):
                         x = self._g.throw(cex)
                     else:
                         x = self._g.send(i)
+
                 except StreamServiceCancelledError as cex2:
                     if cex2 is cex:
                         break
+
                     raise
+
                 i = yield x
+
         if self._cr.cr_running:
             raise RuntimeError(f'Coroutine {self._cr!r} not terminated')
+
         if self._g is not self._a:
             self._g.close()
         self._a.close()
@@ -156,15 +173,18 @@ class _StreamServiceResponse(StreamResponseIterator[V, OutputT]):
     @types.coroutine
     def _anext(self):
         check.state(self._state == 'running')
+
         i = None
         while True:
             try:
                 x = self._g.send(i)
+
             except StopIteration as e:
                 if e.value is not None:
                     self._outputs = tv.TypedValues(*check.isinstance(e.value, ta.Sequence))
                 else:
                     self._outputs = tv.TypedValues()
+
                 raise StopAsyncIteration from None
 
             if isinstance(x, _StreamServiceResponse._Emit) and x.ssr is self:
@@ -200,11 +220,14 @@ async def new_stream_response(
     ssr = _StreamServiceResponse(fn)
 
     v = rs.new_managed(await rs.enter_async_context(ssr))
+
     try:
         return StreamResponse(v, outputs or [])
+
     except BaseException:  # noqa
         # The StreamResponse ctor can raise - for example in `_tv_field_coercer` - in which case we need to clean up the
         # resources ref we have already allocated before reraising.
         async with v:
             pass
+
         raise