dex-python-sdk 0.2.9__tar.gz → 0.2.11__tar.gz

{dex_python_sdk-0.2.9 → dex_python_sdk-0.2.11}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dex-python-sdk
-Version: 0.2.9
+Version: 0.2.11
 Requires-Dist: grpcio>=1.83.0
 Requires-Dist: grpcio-status>=1.83.0
 Requires-Dist: protobuf>=7.35.1
@@ -30,6 +30,35 @@ RPC handlers can stage `channel.delete(context, message_id)`. Declare the RPC as
 Attribute locks already select transactional execution, but Channel deletion
 itself does not.
 
+## RPC state loading
+
+RPCs receive ordinary Attributes and all Channel size metadata by default.
+AttributeMap entries and pending Channel messages are opt-in:
+
+```python
+@dex.rpc(
+    load_attribute_map_instances=(items.load("tenant-a"),),
+    load_channels=(queued,),
+    load_channel_maps=(by_tenant,),
+)
+def snapshot(self, context: dex.Context) -> dex.RPCResult[Snapshot]:
+    messages = queued.pending_messages(context)
+    return dex.RPCResult(Snapshot(messages=messages))
+```
+
+Put an AttributeMap or ChannelMap directly in its plural load option when every
+current instance is needed. Use the singular instance options for the less common
+exact-instance case. A selected empty queue returns an empty tuple; reading an
+unselected map entry raises **AttributeMapNotLoadedError**. An unselected pending-message
+snapshot raises **ChannelMessagesNotLoadedError**.
+Pending messages preserve FIFO order and include the server-assigned message ID.
+The snapshot does not change after the handler stages a publish or deletion.
+
+Loading controls which data reaches the Worker. **is_transactional** controls
+atomic commit and Channel deletion validation. Attribute locks add isolation
+only among cooperating Steps and RPCs using the same lock. Write-only publish
+and delete operations do not require loading.
+
 Python SDK for [Dex workflow engine](https://github.com/superdurable/dex)
 
 ## New user contracts
@@ -284,14 +313,17 @@ instead of raw Flow, Step, or RPC strings.
 
 ### Waiting and map inspection
 
-`Wait.all_of` and `Wait.any_of` may use unnamed Conditions. Every Condition in
-`Wait.any_combination_of` must have a non-empty user ID; the same Condition
-instance may appear in multiple combinations.
+`Wait.until`, `Wait.all_of`, and `Wait.any_of` use unnamed Conditions by
+default. Do not add condition IDs merely because a Condition is nested in one
+of these waits. Every Condition in `Wait.any_combination_of` must have a
+non-empty user ID; the same Condition instance may appear in multiple
+combinations.
 
 Both `Client` and `AsyncClient` provide singleton and AttributeMap-instance
 overloads of `wait_for_attribute_equal`. They target the current run and accept
 only string, bool, int, or float wire values. JSON objects, bytes, and null fail
-before transport. `AttributeMap.get_map_size/get_all_instance_keys` include
+before transport. Every AttributeMap and ChannelMap instance must be non-empty
+and must not contain `/`. `AttributeMap.get_map_size/get_all_instance_keys` include
 buffered sets and deletes. The matching `ChannelMap` methods are RPC-only,
 include buffered publishes, and omit empty instances. Keys are decoded and
 sorted. Use `force_complete_if_channels_empty(...)` for conditional completion.

{dex_python_sdk-0.2.9 → dex_python_sdk-0.2.11}/README.md

@@ -14,6 +14,35 @@ RPC handlers can stage `channel.delete(context, message_id)`. Declare the RPC as
 Attribute locks already select transactional execution, but Channel deletion
 itself does not.
 
+## RPC state loading
+
+RPCs receive ordinary Attributes and all Channel size metadata by default.
+AttributeMap entries and pending Channel messages are opt-in:
+
+```python
+@dex.rpc(
+    load_attribute_map_instances=(items.load("tenant-a"),),
+    load_channels=(queued,),
+    load_channel_maps=(by_tenant,),
+)
+def snapshot(self, context: dex.Context) -> dex.RPCResult[Snapshot]:
+    messages = queued.pending_messages(context)
+    return dex.RPCResult(Snapshot(messages=messages))
+```
+
+Put an AttributeMap or ChannelMap directly in its plural load option when every
+current instance is needed. Use the singular instance options for the less common
+exact-instance case. A selected empty queue returns an empty tuple; reading an
+unselected map entry raises **AttributeMapNotLoadedError**. An unselected pending-message
+snapshot raises **ChannelMessagesNotLoadedError**.
+Pending messages preserve FIFO order and include the server-assigned message ID.
+The snapshot does not change after the handler stages a publish or deletion.
+
+Loading controls which data reaches the Worker. **is_transactional** controls
+atomic commit and Channel deletion validation. Attribute locks add isolation
+only among cooperating Steps and RPCs using the same lock. Write-only publish
+and delete operations do not require loading.
+
 Python SDK for [Dex workflow engine](https://github.com/superdurable/dex)
 
 ## New user contracts
@@ -268,14 +297,17 @@ instead of raw Flow, Step, or RPC strings.
 
 ### Waiting and map inspection
 
-`Wait.all_of` and `Wait.any_of` may use unnamed Conditions. Every Condition in
-`Wait.any_combination_of` must have a non-empty user ID; the same Condition
-instance may appear in multiple combinations.
+`Wait.until`, `Wait.all_of`, and `Wait.any_of` use unnamed Conditions by
+default. Do not add condition IDs merely because a Condition is nested in one
+of these waits. Every Condition in `Wait.any_combination_of` must have a
+non-empty user ID; the same Condition instance may appear in multiple
+combinations.
 
 Both `Client` and `AsyncClient` provide singleton and AttributeMap-instance
 overloads of `wait_for_attribute_equal`. They target the current run and accept
 only string, bool, int, or float wire values. JSON objects, bytes, and null fail
-before transport. `AttributeMap.get_map_size/get_all_instance_keys` include
+before transport. Every AttributeMap and ChannelMap instance must be non-empty
+and must not contain `/`. `AttributeMap.get_map_size/get_all_instance_keys` include
 buffered sets and deletes. The matching `ChannelMap` methods are RPC-only,
 include buffered publishes, and omit empty instances. Keys are decoded and
 sorted. Use `force_complete_if_channels_empty(...)` for conditional completion.

{dex_python_sdk-0.2.9 → dex_python_sdk-0.2.11}/dex/__init__.py

@@ -16,10 +16,11 @@ from dex.attribute import (
     AttributeIndex,
     AttributeLock,
     AttributeMap,
+    AttributeMapLoad,
     IndexType,
 )
 from dex.blob_cache import BlobCache, BlobCacheConfig, open_blob_cache
-from dex.channel import Channel, ChannelMap, ChannelMessage
+from dex.channel import Channel, ChannelMap, ChannelMapLoad, ChannelMessage
 from dex.client import Client
 from dex.client_options import ClientOptions
 from dex.codec import (
@@ -61,6 +62,8 @@ from dex.flow_result import FlowResult, StepCompletion
 from dex.runtime_errors import (
     DexServiceError,
     ErrorSubStatus,
+    AttributeMapNotLoadedError,
+    ChannelMessagesNotLoadedError,
     FlowAlreadyStartedError,
     FlowDefinitionError,
     FlowErrorType,
@@ -116,6 +119,8 @@ __all__ = [
     "AttributeIndex",
     "AttributeLock",
     "AttributeMap",
+    "AttributeMapLoad",
+    "AttributeMapNotLoadedError",
     "AsyncContext",
     "AsyncClient",
     "AsyncBufferedTextStream",
@@ -125,8 +130,10 @@ __all__ = [
     "BufferedTextStream",
     "Channel",
     "ChannelMap",
+    "ChannelMapLoad",
     "ChannelMessage",
     "ChannelMessageNotFoundError",
+    "ChannelMessagesNotLoadedError",
     "Client",
     "ClientOptions",
     "Codec",

{dex_python_sdk-0.2.9 → dex_python_sdk-0.2.11}/dex/_async_value_hydrator.py

@@ -124,10 +124,24 @@ class AsyncValueHydrator:
         result = pb.InvokeWorkerRPCRequest()
         result.CopyFrom(request)
         values = [request.input, *(entry.value for entry in request.attributes)]
+        channel_messages = [
+            message
+            for channel_values in request.loaded_channel_messages.values()
+            for message in channel_values.messages
+        ]
+        values.extend(message.value for message in channel_messages)
         hydrated = await self.hydrate_all(values)
         result.input.CopyFrom(hydrated[0])
-        for entry, value in zip(result.attributes, hydrated[1:]):
-            entry.value.CopyFrom(value)
+        hydrated_entries = iter(hydrated[1:])
+        for entry in result.attributes:
+            entry.value.CopyFrom(next(hydrated_entries))
+        result_messages = [
+            message
+            for channel_values in result.loaded_channel_messages.values()
+            for message in channel_values.messages
+        ]
+        for message in result_messages:
+            message.value.CopyFrom(next(hydrated_entries))
         return result
 
     async def step_outputs(

{dex_python_sdk-0.2.9 → dex_python_sdk-0.2.11}/dex/_async_worker_dispatcher.py

@@ -294,6 +294,10 @@ class AsyncWorkerDispatcher(WorkerDispatcher):
             self._values,
             request.attributes,
             channel_infos=dict(request.channel_infos),
+            loaded_channel_messages=dict(request.loaded_channel_messages),
+            loaded_attribute_map_instances=request.loaded_attribute_map_instances,
+            loaded_channel_names=request.loaded_channel_names,
+            loaded_channel_map_instances=request.loaded_channel_map_instances,
             is_active=is_active,
         )
         arguments: list[object] = [context]

{dex_python_sdk-0.2.9 → dex_python_sdk-0.2.11}/dex/_invocation_context.py

@@ -12,14 +12,18 @@ from enum import Enum
 from typing import Any, Callable, Protocol, Sequence, TypeVar, cast
 from urllib.parse import unquote
 
-from dex._utils import require_name
+from dex._utils import require_map_instance, require_name
 from dex._value_mapper import ValueMapper
 from dex.attribute import Attribute, AttributeMap, _apply_attribute_store_sync
-from dex.channel import Channel, ChannelMap
+from dex.channel import Channel, ChannelMap, ChannelMessage
 from dex.codec import Codec
 from dex.dexpb import dex_pb2 as pb
 from dex.flow import Registry, _RegisteredFlow
 from dex.flow_result import FlowResult, flow_result_from_proto
+from dex.runtime_errors import (
+    AttributeMapNotLoadedError,
+    ChannelMessagesNotLoadedError,
+)
 from dex.step import (
     _NO_HEARTBEAT_VALUE,
     StepOutput,
@@ -66,6 +70,10 @@ class InvocationContext:
         locals: Sequence[pb.KV] = (),
         condition_results: pb.ConditionResults | None = None,
         channel_infos: dict[str, pb.ChannelInfo] | None = None,
+        loaded_channel_messages: dict[str, pb.ChannelValues] | None = None,
+        loaded_attribute_map_instances: Sequence[str] = (),
+        loaded_channel_names: Sequence[str] = (),
+        loaded_channel_map_instances: Sequence[str] = (),
         is_active: Callable[[], bool] | None = None,
         output_emitter: _StepOutputEmitter | None = None,
     ) -> None:
@@ -77,6 +85,10 @@ class InvocationContext:
         self._locals = self._map_values("step-execution local", locals)
         self._condition_results = condition_results
         self._channel_infos = dict(channel_infos or {})
+        self._loaded_channel_messages = dict(loaded_channel_messages or {})
+        self._loaded_attribute_map_instances = frozenset(loaded_attribute_map_instances)
+        self._loaded_channel_names = frozenset(loaded_channel_names)
+        self._loaded_channel_map_instances = frozenset(loaded_channel_map_instances)
         self._is_active = is_active or _always_active
         self._output_emitter = output_emitter
         self.attribute_writes: dict[str, pb.AttributeWrite] = {}
@@ -276,6 +288,7 @@ class InvocationContext:
         instance: str | None,
     ) -> ValueT:
         self._require_registered(definition)
+        self._require_attribute_map_instance_loaded(definition, instance)
         key = self._physical_name(definition, instance)
         write = self.attribute_writes.get(key)
         if write is not None:
@@ -375,6 +388,7 @@ class InvocationContext:
         definition: AttributeMap[object],
     ) -> tuple[str, ...]:
         self._require_registered(definition)
+        self._require_attribute_map_all_loaded(definition)
         prefix = f"{definition.name}/"
         physical_keys = {key for key in self._attributes if key.startswith(prefix)}
         for key, write in self.attribute_writes.items():
@@ -384,7 +398,12 @@ class InvocationContext:
                 physical_keys.discard(key)
             else:
                 physical_keys.add(key)
-        return tuple(sorted(unquote(key[len(prefix) :]) for key in physical_keys))
+        return tuple(
+            sorted(
+                require_map_instance(unquote(key[len(prefix) :]))
+                for key in physical_keys
+            )
+        )
 
     def _channel_map_keys(
         self,
@@ -396,7 +415,7 @@ class InvocationContext:
         prefix = f"{definition.name}/"
         return tuple(
             sorted(
-                unquote(key[len(prefix) :])
+                require_map_instance(unquote(key[len(prefix) :]))
                 for key, info in self._channel_infos.items()
                 if key.startswith(prefix) and info.size > 0
             )
@@ -411,6 +430,39 @@ class InvocationContext:
         info = self._channel_infos.get(self._physical_name(definition, instance))
         return info.size if info is not None else 0
 
+    def _pending_channel_messages(
+        self,
+        definition: Channel[ValueT] | ChannelMap[ValueT],
+        instance: str | None,
+    ) -> tuple[ChannelMessage[ValueT], ...]:
+        if self._method is not InvocationMethod.RPC:
+            raise ValueError("pending Channel messages require an RPC Context")
+        self._require_registered(definition)
+        if isinstance(definition, ChannelMap):
+            channel_name = self._physical_name(definition, instance)
+            is_loaded = (
+                f"{definition.name}/" in self._loaded_channel_map_instances
+                or channel_name in self._loaded_channel_map_instances
+            )
+        else:
+            channel_name = self._physical_name(definition, instance)
+            is_loaded = definition.name in self._loaded_channel_names
+        if not is_loaded:
+            raise ChannelMessagesNotLoadedError(
+                f"Channel messages were not loaded for RPC: {definition.name}"
+            )
+        values = self._loaded_channel_messages.get(channel_name)
+        if values is None:
+            return ()
+        codec = self._flow_codec(definition.value_type)
+        return tuple(
+            ChannelMessage(
+                message_id=message.message_id,
+                value=cast(ValueT, self._values.decode(message.value, codec)),
+            )
+            for message in values.messages
+        )
+
     def _channel_results(
         self,
         definition: Channel[ValueT] | ChannelMap[ValueT],
@@ -432,6 +484,36 @@ class InvocationContext:
     def _flow_codec(self, value_type: object) -> Codec[Any]:
         return self._values.codec(value_type)
 
+    def _require_attribute_map_instance_loaded(
+        self,
+        definition: Attribute[Any] | AttributeMap[Any],
+        instance: str | None,
+    ) -> None:
+        if self._method is not InvocationMethod.RPC or not isinstance(
+            definition, AttributeMap
+        ):
+            return
+        physical_name = self._physical_name(definition, instance)
+        if (
+            f"{definition.name}/" not in self._loaded_attribute_map_instances
+            and physical_name not in self._loaded_attribute_map_instances
+        ):
+            raise AttributeMapNotLoadedError(
+                f"AttributeMap instance was not loaded for RPC: {physical_name}"
+            )
+
+    def _require_attribute_map_all_loaded(
+        self,
+        definition: AttributeMap[object],
+    ) -> None:
+        if (
+            self._method is InvocationMethod.RPC
+            and f"{definition.name}/" not in self._loaded_attribute_map_instances
+        ):
+            raise AttributeMapNotLoadedError(
+                f"all AttributeMap instances were not loaded for RPC: {definition.name}"
+            )
+
     def _require_registered(self, definition: _Definition) -> None:
         registered = self._flow.persistence.get(definition.name)
         if registered is not definition:

{dex_python_sdk-0.2.9 → dex_python_sdk-0.2.11}/dex/_utils.py

@@ -20,6 +20,13 @@ def require_persistence_definition_name(name: str) -> str:
     return name
 
 
+def require_map_instance(instance: str) -> str:
+    require_name(instance)
+    if "/" in instance:
+        raise ValueError("map instances must not contain '/'")
+    return instance
+
+
 def validate_condition_id(condition_id: str | None) -> None:
     if condition_id is not None and not condition_id:
         raise ValueError("condition ID must not be empty")

{dex_python_sdk-0.2.9 → dex_python_sdk-0.2.11}/dex/_value_hydrator.py

@@ -131,10 +131,24 @@ class ValueHydrator:
         result = pb.InvokeWorkerRPCRequest()
         result.CopyFrom(request)
         values = [request.input, *(entry.value for entry in request.attributes)]
+        channel_messages = [
+            message
+            for channel_values in request.loaded_channel_messages.values()
+            for message in channel_values.messages
+        ]
+        values.extend(message.value for message in channel_messages)
         hydrated = self.hydrate_all(values)
         result.input.CopyFrom(hydrated[0])
-        for entry, value in zip(result.attributes, hydrated[1:]):
-            entry.value.CopyFrom(value)
+        hydrated_entries = iter(hydrated[1:])
+        for entry in result.attributes:
+            entry.value.CopyFrom(next(hydrated_entries))
+        result_messages = [
+            message
+            for channel_values in result.loaded_channel_messages.values()
+            for message in channel_values.messages
+        ]
+        for message in result_messages:
+            message.value.CopyFrom(next(hydrated_entries))
         return result
 
     def step_outputs(

{dex_python_sdk-0.2.9 → dex_python_sdk-0.2.11}/dex/_worker_dispatcher.py

@@ -219,6 +219,10 @@ class WorkerDispatcher:
             self._values,
             request.attributes,
             channel_infos=dict(request.channel_infos),
+            loaded_channel_messages=dict(request.loaded_channel_messages),
+            loaded_attribute_map_instances=request.loaded_attribute_map_instances,
+            loaded_channel_names=request.loaded_channel_names,
+            loaded_channel_map_instances=request.loaded_channel_map_instances,
             is_active=is_active,
         )
         arguments: list[object] = [context]

{dex_python_sdk-0.2.9 → dex_python_sdk-0.2.11}/dex/async_client.py

@@ -274,6 +274,9 @@ class AsyncClient:
                     lock_attribute_keys=rpc.locks,
                     request_id=str(uuid4()),
                     is_transactional=rpc.options.is_transactional,
+                    load_attribute_map_instances=rpc.load_attribute_map_instances,
+                    load_channel_names=rpc.load_channel_names,
+                    load_channel_map_instances=rpc.load_channel_map_instances,
                 ),
                 "invoke_rpc",
                 flow_id,
@@ -319,7 +322,7 @@ class AsyncClient:
         Args:
             flow_id: The non-empty target Flow ID.
             attribute: A typed singleton Attribute or AttributeMap definition.
-            instance: The non-empty map key; omit for a singleton Attribute.
+            instance: The map instance. Omit it for a singleton Attribute. Slash is prohibited because it is a reserved character.
             run_id: Optional exact run; ``""`` targets the current run.
 
         Returns:
@@ -1068,7 +1071,7 @@ class AsyncClient:
         Args:
             flow_id: The non-empty active Flow ID.
             attribute: The registered AttributeMap to observe.
-            instance: The non-empty logical map key to observe.
+            instance: The map instance to observe. Slash is prohibited because it is a reserved character.
             expected: The string, bool, int, or float value to await.
             timeout: The non-negative server-side wait duration.
 

{dex_python_sdk-0.2.9 → dex_python_sdk-0.2.11}/dex/attribute.py

@@ -13,14 +13,41 @@ from __future__ import annotations
 from dataclasses import dataclass
 from enum import Enum
 from typing import Any, Generic, TypeVar, cast
+from urllib.parse import quote
 
-from dex._utils import require_name, require_persistence_definition_name
+from dex._utils import require_map_instance, require_persistence_definition_name
 from dex.context import Context
 from dex.dexpb import dex_pb2 as pb
 
 ValueT = TypeVar("ValueT")
 
 
+@dataclass(frozen=True)
+class AttributeMapLoad:
+    """Load one AttributeMap instance for an RPC snapshot.
+
+    Create a load with :meth:`AttributeMap.load`. Loading provides a point-in-time
+    value snapshot; it does not lock the map against concurrent writers.
+
+    Attributes:
+        attribute_map: The exact AttributeMap definition registered with the Flow.
+        instance: One logical instance key.
+    """
+
+    attribute_map: AttributeMap[Any]
+    instance: str
+
+    @property
+    def physical_name(self) -> str:
+        """Return the physical instance name for this typed load.
+
+        Returns:
+            The encoded physical instance name.
+        """
+        instance = require_map_instance(self.instance)
+        return f"{self.attribute_map.name}/{quote(instance, safe='')}"
+
+
 class IndexType(Enum):
     """Select how Dex indexes an Attribute for Flow search.
 
@@ -145,6 +172,7 @@ class AttributeMap(Generic[ValueT]):
 
     AttributeMap instances share one schema definition while keeping independent
     values and locks. Declare the map in ``PersistenceSchema`` before using it.
+    Instance keys must be non-empty and must not contain ``/``.
     Synced instances use their physical Attribute names as target columns. Projection
     is asynchronous and latest-state only, deletion projects SQL NULL, and failures
     do not roll back Flow Attributes.
@@ -171,19 +199,35 @@ class AttributeMap(Generic[ValueT]):
     def __post_init__(self) -> None:
         require_persistence_definition_name(self.name)
 
+    def load(self, instance: str) -> AttributeMapLoad:
+        """Select one instance for an RPC snapshot.
+
+        Pass the result through ``@rpc(load_attribute_map_instances=(...))``.
+
+        Args:
+            instance: The non-empty, slash-free logical map key to load.
+
+        Returns:
+            A load for this one instance.
+
+        Raises:
+            ValueError: If ``instance`` is empty or contains ``/``.
+        """
+        return AttributeMapLoad(self, require_map_instance(instance))
+
     def get(self, context: Context, instance: str) -> ValueT:
         """Return one map instance from a Step or RPC Context.
 
         Args:
             context: The current handler Context.
-            instance: The non-empty logical map key.
+            instance: The map instance. Slash is prohibited because it is a reserved character.
 
         Returns:
             The decoded instance value.
 
         Raises:
             KeyError: If the instance has no value.
-            ValueError: If ``instance`` is empty.
+            ValueError: If ``instance`` is empty or contains ``/``.
         """
         return context._get_attribute(self, instance)
 
@@ -192,7 +236,7 @@ class AttributeMap(Generic[ValueT]):
 
         Args:
             context: The current handler Context.
-            instance: The non-empty logical map key.
+            instance: The map instance. Slash is prohibited because it is a reserved character.
             value: A value compatible with ``value_type``.
         """
         context._set_attribute(self, instance, value)
@@ -202,7 +246,7 @@ class AttributeMap(Generic[ValueT]):
 
         Args:
             context: The current handler Context.
-            instance: The non-empty logical map key.
+            instance: The map instance. Slash is prohibited because it is a reserved character.
         """
         context._delete_attribute(cast(AttributeMap[object], self), instance)
 
@@ -232,15 +276,15 @@ class AttributeMap(Generic[ValueT]):
         """Return a lock request for one map instance.
 
         Args:
-            instance: The non-empty logical map key.
+            instance: The map instance. Slash is prohibited because it is a reserved character.
 
         Returns:
             A lock for ``instance``.
 
         Raises:
-            ValueError: If ``instance`` is empty.
+            ValueError: If ``instance`` is empty or contains ``/``.
         """
-        require_name(instance)
+        require_map_instance(instance)
         return AttributeLock(self, instance)
 
 
@@ -256,6 +300,10 @@ class AttributeLock:
     attribute: Attribute[Any] | AttributeMap[Any]
     instance: str | None = None
 
+    def __post_init__(self) -> None:
+        if self.instance is not None:
+            require_map_instance(self.instance)
+
 
 def _apply_attribute_store_sync(
     write: pb.AttributeWrite,