dex-python-sdk 1.32.0__tar.gz → 1.35.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/PKG-INFO +114 -91
  2. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/README.md +113 -90
  3. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/__init__.py +2 -1
  4. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_invocation_context.py +8 -1
  5. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_pull_worker.py +252 -176
  6. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_server_protocol.py +40 -8
  7. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/async_client.py +137 -23
  8. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/async_worker.py +41 -20
  9. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/attribute.py +5 -10
  10. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/client.py +133 -22
  11. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/context.py +33 -1
  12. dex_python_sdk-1.35.0/dex/dexpb/dex_pb2.py +443 -0
  13. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/dexpb/dex_pb2.pyi +80 -10
  14. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/dexpb/dex_pb2_grpc.py +16 -7
  15. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/pull_registration.py +28 -13
  16. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/step.py +14 -11
  17. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/wait_options.py +4 -0
  18. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/worker.py +41 -20
  19. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/worker_options.py +18 -28
  20. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/pyproject.toml +1 -1
  21. dex_python_sdk-1.32.0/dex/dexpb/dex_pb2.py +0 -433
  22. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/LEGACY_NOTICES.md +0 -0
  23. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/LICENSE +0 -0
  24. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_async_value_hydrator.py +0 -0
  25. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_async_worker_dispatcher.py +0 -0
  26. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_attribute_reads.py +0 -0
  27. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_credential_transport.py +0 -0
  28. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_grpc_errors.py +0 -0
  29. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_native.pyi +0 -0
  30. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_utils.py +0 -0
  31. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_value_hydrator.py +0 -0
  32. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_value_mapper.py +0 -0
  33. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/_worker_dispatcher.py +0 -0
  34. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/attribute_match.py +0 -0
  35. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/attribute_values.py +0 -0
  36. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/blob_cache.py +0 -0
  37. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/channel.py +0 -0
  38. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/client_options.py +0 -0
  39. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/codec.py +0 -0
  40. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/condition.py +0 -0
  41. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/credential.py +0 -0
  42. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/dexpb/__init__.py +0 -0
  43. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/flow.py +0 -0
  44. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/flow_config.py +0 -0
  45. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/flow_info.py +0 -0
  46. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/flow_options.py +0 -0
  47. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/flow_result.py +0 -0
  48. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/py.typed +0 -0
  49. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/rpc_invoke_options.py +0 -0
  50. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/runtime_errors.py +0 -0
  51. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/step_execution.py +0 -0
  52. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/stream.py +0 -0
  53. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/subflow.py +0 -0
  54. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/timer.py +0 -0
  55. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/dex/wait.py +0 -0
  56. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/Cargo.lock +0 -0
  57. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/Cargo.toml +0 -0
  58. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache/Cargo.toml +0 -0
  59. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache/LICENSE +0 -0
  60. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache/README.md +0 -0
  61. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache/src/config.rs +0 -0
  62. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache/src/entry.rs +0 -0
  63. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache/src/error.rs +0 -0
  64. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache/src/format.rs +0 -0
  65. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache/src/lib.rs +0 -0
  66. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache/src/policy.rs +0 -0
  67. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache/src/store.rs +0 -0
  68. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache/tests/blob_cache_integration.rs +0 -0
  69. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache-python/Cargo.toml +0 -0
  70. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache-python/LICENSE +0 -0
  71. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache-python/README.md +0 -0
  72. {dex_python_sdk-1.32.0 → dex_python_sdk-1.35.0}/sdk-rust/crates/dex-blob-cache-python/src/lib.rs +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: dex-python-sdk
3
- Version: 1.32.0
3
+ Version: 1.35.0
4
4
  Requires-Dist: grpcio>=1.83.0
5
5
  Requires-Dist: grpcio-status>=1.83.0
6
6
  Requires-Dist: protobuf>=7.35.1
@@ -116,7 +116,7 @@ worker = Worker(
116
116
  cache,
117
117
  WorkerOptions(
118
118
  server_address="dex.example:8801",
119
- pull=PullWorkerOptions(max_concurrent_tasks=8, idle_streams_per_queue=1),
119
+ pull=PullWorkerOptions(max_concurrent_tasks=8),
120
120
  ),
121
121
  )
122
122
  worker_thread = threading.Thread(target=worker.start)
@@ -135,30 +135,26 @@ finally:
135
135
  worker_thread.join()
136
136
  ```
137
137
 
138
- The Worker polls its sticky queue and every registered Flow type queue. Flow
139
- type names must be unique within a Dex server. `max_concurrent_tasks` defaults
140
- to the CPU count, and `idle_streams_per_queue` defaults to one. At capacity,
141
- the Worker rejects an assignment so Dex can reassign it. Each stream runs one
142
- task at a time; Step heartbeat and stream writes use the same stream. After
143
- tasks finish, the Worker gracefully closes surplus idle streams while preserving
144
- readiness. The SDK sends an upstream Pong every 20 seconds while a task runs and
145
- stops before its final frame. This interval is internal and needs no Worker
146
- option. After load drops,
147
- the steady-state bound is `max_concurrent_tasks + (1 + number of Flow types) *
148
- idle_streams_per_queue` streams.
149
- Allow up to one more stream per queue for 30 seconds after reserve declines,
150
- or up to `1 + number of Flow types` additional streams. The transient peak
151
- during a burst remains bounded by the matching caps. Size these options for
152
- `matching.maxConcurrentTasksCap` per app Worker ID per matching owner (256 by
153
- default) and `matching.maxStreams` per Matching instance (100,000 by default).
154
-
155
- The Worker rejects an unregistered Step or RPC by name. Dex retries that task
156
- on another pull Worker without consuming a Step attempt, including when the
157
- Flow was sticky to the rejecting Worker. If no capable Worker accepts it within
158
- the server's dispatch wait (10 seconds by default, or the local Step attempt),
159
- the Step follows its failure and retry policy; an RPC caller receives the named
160
- error. This reassignment requires a Dex Server that supports it. Against older
161
- Servers, the Worker returns the not-registered error as before.
138
+ The Worker opens exactly `max_concurrent_tasks` long-lived streams. Each stream
139
+ is one execution slot serving the sticky queue and every registered Flow type.
140
+ Flow type names must be unique within a Dex server. Concurrency defaults to the
141
+ CPU count. Start fails if the configured concurrency exceeds the Server's
142
+ reported `matching.maxConcurrentTasksCap`; zero means no limit was reported.
143
+ The Worker checks the limit on its first registration only. It then half-closes
144
+ its streams, drops any task they receive, and never reports readiness;
145
+ registration waiters raise the limit error. A lower limit reported on a later
146
+ reconnect is logged as a warning, and the Worker keeps running.
147
+ `idle_streams_per_queue` is deprecated, remains accepted, and has no effect.
148
+ Each slot executes every task it receives and never rejects a task for capacity.
149
+ Step heartbeats and stream writes use the same stream. Busy streams send a Pong
150
+ every 20 seconds until the final frame.
151
+
152
+ The Worker rejects an unregistered Step or RPC by name before its handler runs.
153
+ Dex tries another Worker without consuming a Step attempt, including when the
154
+ Flow was sticky to the rejecting Worker. If no connected Worker supports the
155
+ Step type, the Step stays queued until a Worker that supports it connects, the
156
+ Flow times out, or the Flow is stopped. An RPC caller receives the named error
157
+ after the Server's dispatch wait.
162
158
 
163
159
  The generated `app_worker_id` uses the first nonempty `DEX_WORKER_INSTANCE`,
164
160
  `POD_NAME`, `HOSTNAME`, or system hostname, plus eight random base32 characters.
@@ -166,41 +162,67 @@ An override must uniquely identify one Worker process and change after restart.
166
162
  It must contain 1-63 ASCII letters, digits, dots, underscores, or hyphens and
167
163
  must not equal a registered Flow type.
168
164
 
169
- Pull streams reconnect after failures. A server-directed reconnect waits for
170
- an active task to finish, then waits up to `stop_drain_timeout` for the server
171
- to close the stream after the final frame. An idle stream reconnects
172
- immediately. An older server without pull support produces one explicit error
173
- log and is retried after one minute. `start()` reports startup
174
- synchronization errors, while later stream errors are logged and retried by default.
175
- `stop()` drains for at most
176
- `stop_drain_timeout`, which defaults to 30 seconds. Afterward it requests
177
- cancellation and waits for handlers to return. Synchronous handlers should
178
- check `Context.is_cancellation_requested()` during long operations. Streams
179
- that sent a final frame wait for server EOF until the remaining Stop drain
180
- deadline before the channel closes.
165
+ Pull streams reconnect after failures. A broken stream cancels its handler;
166
+ its output is never sent on a replacement connection. Server-directed reconnect
167
+ lets an active task finish before sending its final output and half-closing.
168
+ If the Server has not confirmed that final output by EOF when the reconnect wait
169
+ of `stop_drain_timeout` ends, the stream gets up to 5 more seconds before it is
170
+ reset.
171
+ `start()` reports protocol and startup synchronization failures. Later transport
172
+ errors are logged and retried by default.
173
+
174
+ `stop()` starts no new tasks. Idle streams half-close immediately; busy streams
175
+ finish their handlers and send final output before half-closing. Task frames
176
+ arriving after stop begins are dropped without calling a handler. Streams keep
177
+ reading until server EOF or the remaining `stop_drain_timeout`, which defaults
178
+ to 30 seconds. On timeout, a busy stream is reset and its handler is canceled;
179
+ stop waits for the handler to return. Synchronous handlers should check
180
+ `Context.is_cancellation_requested()` during long operations. Any stream still
181
+ open at the deadline is reset, even one that is half-closed and still waiting
182
+ for server EOF, so stop returns once handlers return. Dex treats a task it sent
183
+ in that window as an attempt that may have run.
184
+
185
+ A stream whose final output was queued or written, but not yet confirmed by
186
+ server EOF or by the next assignment on that stream, is not reset at the drain
187
+ deadline. This holds even if its handler has finished or the stream has
188
+ half-closed, because a completed local write does not mean the Server read the
189
+ bytes. The stream gets up to 5 more seconds to send the output, half-close, and
190
+ receive server EOF, and is reset after that. `stop()` can therefore return up
191
+ to 5 seconds after the drain deadline.
192
+
193
+ A stream that fails locally, such as a read error, an undecodable or oversized
194
+ frame, a task frame with an invalid deadline or no request, a failed final
195
+ output write, or an unexpected server EOF, is always reset. It never
196
+ half-closes cleanly, because a clean half-close tells Dex the assigned task did
197
+ not run. A malformed task is never admitted and its handler never runs. Poll
198
+ streams accept only uncompressed responses.
199
+
200
+ After a slot sends a task's final output, Dex may assign its next task
201
+ immediately. The slot finishes local cleanup for the previous task first, then
202
+ starts the next one, unless stop has begun.
181
203
 
182
204
  Call `worker.wait_pull_registered(timeout=30)` after starting a synchronous
183
205
  Worker on another thread. For AsyncWorker, start it in another task and use
184
206
  `await worker.wait_pull_registered(timeout=30)`. The timeout is in seconds;
185
- `None` waits indefinitely. Each waits for every Flow type queue and the app
186
- Worker ID queue to register at least once. Timeout raises `TimeoutError`;
207
+ `None` waits indefinitely. Each waits for one `STEP_AND_RPC` stream and, when
208
+ RPC reservation is configured, one `RPC_ONLY` stream to register. Timeout raises `TimeoutError`;
187
209
  async cancellation cancels only that wait. Startup failures reach waiters.
188
210
  A permanent rejection before readiness raises `PullRegistrationError`, whose
189
- `queue` and `error` identify the rejected queue and original transport error.
211
+ `queue` and `error` identify the rejected slot kind and original transport error.
190
212
  Stopping before readiness raises `RuntimeError`.
191
213
 
192
214
  Use `worker.pull_registration_state()` for subsequent readiness probes. Its
193
215
  immutable `queues` expose `queue`, `has_registered_stream`, and
194
216
  `last_permanent_error`. After initial registration, `is_ready` stays true across
195
- transient failures, reconnects, and stop. It becomes false only when a queue
217
+ transient failures, reconnects, and stop. It becomes false only when a required slot kind
196
218
  has a permanent error and no registered stream. Re-registration clears the
197
219
  error. The startup wait keeps succeeding after initial registration, including
198
220
  after stop. Multiple callers may wait independently before or after start.
199
221
 
200
222
  `PullWorkerOptions(stop_on_permanent_stream_rejection=True)` makes the first
201
- permanent rejection leaving a queue without a registered stream drain within
223
+ permanent rejection leaving a slot kind without a registered stream drain within
202
224
  `stop_drain_timeout`, then raise `PullRegistrationError` from `start()`. This
203
- applies before and after readiness. Another registered stream for that queue
225
+ applies before and after readiness. Another registered stream for that slot kind
204
226
  prevents shutdown. The default is False and retries indefinitely. Always call
205
227
  `stop()` or `close()` afterward; both remain safe after `start()` returns.
206
228
 
@@ -215,32 +237,18 @@ long-running Steps; use a limit of at least two or set the reserve explicitly.
215
237
  Flows must stay responsive because RPCs share Worker task capacity with Steps.
216
238
  The value must be nonnegative and less than `max_concurrent_tasks` after its
217
239
  CPU-count default is resolved; invalid values fail construction. RPC and Step
218
- handlers count together. WaitFor and Execute tasks, including local attempts,
219
- are declined with `NO_CAPACITY` when the busy
220
- count reaches the task limit minus the reserve. API-direct and transactional
221
- InvokeWorkerRPC tasks can use the full limit.
240
+ handlers count together. Reserved slots register as `RPC_ONLY`; the remaining
241
+ slots register as `STEP_AND_RPC`. The Server assigns Steps only to the latter.
242
+ The Worker never declines a Step for capacity.
222
243
 
223
244
  ```python
224
245
  pull = PullWorkerOptions(max_concurrent_tasks=8, reserved_rpc_tasks=2)
225
246
  ```
226
247
 
227
- A decline lets Dex try another Worker without consuming an attempt. If all
228
- Workers remain at their Step limit, a Step can wait until the dispatch wait
229
- expires, then receive `NO_WORKER_AVAILABLE`, consume an attempt, and follow its
230
- retry policy, as when all Workers are full. Local attempts wait the local
231
- dispatch wait, then fall back according to their policy. A queue that declines
232
- a Step for reserved RPC capacity keeps one additional idle stream until 30
233
- seconds after its last decline. Other queues retain their configured count.
234
- The Worker does not open the extra stream while all task slots are busy. Set
235
- `idle_streams_per_queue` to at least two for heavy Step backlogs. A reserve decline
236
- does not close a stream or change registration readiness.
237
-
238
- Each reserve decline records the Flow type through the dex._pull_worker logger
239
- at DEBUG level. Export the cumulative `pull_reserved_rpc_declines` Worker value
240
- to distinguish reserve pressure from full-capacity rejections. The count excludes
241
- full-capacity rejections and unknown names. It counts decline events, including
242
- repeated declines of the same Step dispatch while Dex retries it. Read its rate
243
- to assess reserve pressure. It never resets and remains available after shutdown.
248
+ The `pull_reserved_rpc_declines` counter is deprecated and always zero. It stays
249
+ available for source compatibility. Registration snapshot `queues` and error
250
+ `queue` fields retain their names but now hold slot kind labels, `STEP_AND_RPC`
251
+ and `RPC_ONLY`, rather than individual queue names.
244
252
 
245
253
  ## Flow state I/O
246
254
 
@@ -362,7 +370,7 @@ class TimeoutRecoveryStep(Step[None]):
362
370
  return force_fail(failure.detail)
363
371
  ```
364
372
 
365
- Python SDK for [Dex workflow engine](https://github.com/superdurable/dex)
373
+ Python SDK for [Dex durable workflows](https://github.com/superdurable/dex)
366
374
 
367
375
  ## New user contracts
368
376
 
@@ -471,12 +479,10 @@ option cannot be combined with an index or `sync_to_attribute_store`, and
471
479
  the stored value. Handlers and `get_attributes` receive the value inline, and
472
480
  the SDK never stores it in the local blob cache.
473
481
 
474
- Hard-delete writes require a Server with an enabled blob store and Server
475
- protocol version 3 or later. Workers still start against older Servers.
476
- There, a Step, RPC, or timeout handler result that writes a hard-delete
477
- Attribute fails that handler attempt with `RuntimeError`, as does a `wait_for`
478
- result whose SubFlow start initializes one. A Client checks the Server protocol
479
- once, before its first `start_flow` that initializes one.
482
+ Hard-delete writes require a Server with an enabled blob store. This SDK
483
+ requires protocol 4. Workers negotiate before startup and Clients negotiate
484
+ lazily before their first service request. Incompatible Servers fail with
485
+ `RuntimeError` before values are sent.
480
486
 
481
487
  Each Flow keeps one stored copy per Attribute key, overwritten in place. A Step
482
488
  or RPC attempt writes or deletes the stored value before its result commits, so
@@ -598,19 +604,18 @@ integer and string instances and routes to both by instance and StepRef.
598
604
  `StepOptions.wait_for_method_timeout` and `execute_method_timeout` bound the
599
605
  two handler calls. Timer and channel conditions determine how long a Step waits.
600
606
 
601
- `wait_for_retry` and `execute_retry` limit one logical handler execution. With
602
- `StepDurability.ASYNC`, local and fallback regular activities share maximum
603
- attempts, total duration, and 1-based attempt numbers. Fallback starts
604
- immediately; later regular retries continue the backoff sequence at the
605
- cumulative attempt.
607
+ `wait_for_retry` and `execute_retry` limit one logical handler execution.
608
+ Attempts use one-based numbers. The retry total duration starts when Dex first
609
+ hands the Step to a Worker. After an attempt fails, Dex checks that budget to
610
+ decide whether to schedule another retry; it does not end a running attempt.
606
611
 
607
612
  The default Step durability is synchronous. A Flow configuration can select
608
613
  asynchronous durability, and a Step method override has highest precedence. The
609
- default retry total duration is four hours. Regular attempts default to a two-hour
614
+ default retry total duration is four hours. Attempts default to a two-hour
610
615
  method timeout and one-minute heartbeat timeout; an explicit heartbeat timeout must
611
616
  meet the server minimum, which defaults to ten seconds. Asynchronous durability
612
- first allows at most three local attempts in seven seconds. The local phase ignores
613
- method timeouts and heartbeat frames before falling back to a regular activity.
617
+ gives lower latency and faster single Flows. Sustained throughput is about the
618
+ same as synchronous durability.
614
619
 
615
620
  ### Flow run start time
616
621
 
@@ -627,8 +632,7 @@ with `datetime.now(timezone.utc)` to measure the run's age. It matches
627
632
  current `wait_for`, `execute`, or Flow timeout handler attempt. It is a
628
633
  timezone-aware UTC `datetime` with microsecond precision, not a duration.
629
634
  Subtract `datetime.now(timezone.utc)` and call `total_seconds()` to budget
630
- remaining work after checking for `None`. A shorter remaining
631
- `RetryPolicy.total_duration` limits this deadline; each retry receives its own
635
+ remaining work after checking for `None`. Each retry receives its own
632
636
  effective deadline. RPC handlers and tasks without a supplied deadline return
633
637
  `None`.
634
638
 
@@ -993,15 +997,34 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#releases-monorepo-tags) for monorepo ta
993
997
 
994
998
  ## Server protocol compatibility
995
999
 
996
- Both `Worker` and `AsyncWorker` call `GetServerInfo`, negotiate the highest
997
- common protocol, synchronize Attribute indexes, and only then open outbound
998
- task streams. The initial Python SDK interval is `[1,1]`. Missing information,
999
- invalid or disjoint intervals, and RPC failures stop startup before polling.
1000
- The current Python SDK interval is `[1,3]`. Protocol 3 adds hard-delete
1001
- Attributes. Workers record the negotiated version at startup and reject each
1002
- handler result that contains a hard-delete Attribute write when it is below 3.
1003
- Clients negotiate lazily, once, before the first `start_flow` that initializes
1004
- a hard-delete Attribute.
1000
+ Workers call `GetServerInfo`, require protocol 4, synchronize Attribute indexes,
1001
+ and then open execution slot streams. The Python SDK protocol interval is
1002
+ `[4,4]`. Older Servers fail startup with this error, substituting the protocol
1003
+ range and Server version actually reported:
1004
+
1005
+ ```text
1006
+ This SDK requires a Dex Server with protocol version 4 or later (server-zd newer than v0.1.16). The Server reports protocol versions <min>-<current> (server version <server_version>). Upgrade the Server before upgrading the SDK.
1007
+ ```
1008
+
1009
+ A Server whose minimum protocol is newer than 4 fails with an error that asks
1010
+ you to upgrade the SDK. Missing information, invalid intervals, and RPC
1011
+ failures stop startup before polling.
1012
+
1013
+ Clients perform no network I/O during construction. Before the first request of
1014
+ any method, they negotiate protocol 4. Success and incompatibility are cached;
1015
+ later requests against an incompatible Server raise the same clear
1016
+ `RuntimeError`. If `GetServerInfo` itself fails, for example because the Server
1017
+ is unreachable, the request raises the usual `DexServiceError`, and the next
1018
+ call negotiates again. Negotiation counts against a request's timeout, including
1019
+ the `request_timeout` of durable waits.
1020
+
1021
+ `RetryPolicy.total_duration` counts from the first hand-out to a Worker. After
1022
+ an attempt fails, Dex checks it to decide whether to retry. It does not end a
1023
+ running attempt. Recovery handlers can inspect `RecoveryErrorInfo.attempt_progress`:
1024
+ `StepAttemptProgress.NOT_STARTED` guarantees no handler ran;
1025
+ `NOT_COMPLETED` means some attempt may have run but none completed;
1026
+ `UNSPECIFIED` covers older Servers or failures recorded before upgrade and must
1027
+ be treated like `NOT_COMPLETED`.
1005
1028
 
1006
1029
  The diagnostic SDK version comes from Python distribution metadata for
1007
1030
  `dex-python-sdk` and falls back to `dev` when the distribution is unavailable.
@@ -100,7 +100,7 @@ worker = Worker(
100
100
  cache,
101
101
  WorkerOptions(
102
102
  server_address="dex.example:8801",
103
- pull=PullWorkerOptions(max_concurrent_tasks=8, idle_streams_per_queue=1),
103
+ pull=PullWorkerOptions(max_concurrent_tasks=8),
104
104
  ),
105
105
  )
106
106
  worker_thread = threading.Thread(target=worker.start)
@@ -119,30 +119,26 @@ finally:
119
119
  worker_thread.join()
120
120
  ```
121
121
 
122
- The Worker polls its sticky queue and every registered Flow type queue. Flow
123
- type names must be unique within a Dex server. `max_concurrent_tasks` defaults
124
- to the CPU count, and `idle_streams_per_queue` defaults to one. At capacity,
125
- the Worker rejects an assignment so Dex can reassign it. Each stream runs one
126
- task at a time; Step heartbeat and stream writes use the same stream. After
127
- tasks finish, the Worker gracefully closes surplus idle streams while preserving
128
- readiness. The SDK sends an upstream Pong every 20 seconds while a task runs and
129
- stops before its final frame. This interval is internal and needs no Worker
130
- option. After load drops,
131
- the steady-state bound is `max_concurrent_tasks + (1 + number of Flow types) *
132
- idle_streams_per_queue` streams.
133
- Allow up to one more stream per queue for 30 seconds after reserve declines,
134
- or up to `1 + number of Flow types` additional streams. The transient peak
135
- during a burst remains bounded by the matching caps. Size these options for
136
- `matching.maxConcurrentTasksCap` per app Worker ID per matching owner (256 by
137
- default) and `matching.maxStreams` per Matching instance (100,000 by default).
138
-
139
- The Worker rejects an unregistered Step or RPC by name. Dex retries that task
140
- on another pull Worker without consuming a Step attempt, including when the
141
- Flow was sticky to the rejecting Worker. If no capable Worker accepts it within
142
- the server's dispatch wait (10 seconds by default, or the local Step attempt),
143
- the Step follows its failure and retry policy; an RPC caller receives the named
144
- error. This reassignment requires a Dex Server that supports it. Against older
145
- Servers, the Worker returns the not-registered error as before.
122
+ The Worker opens exactly `max_concurrent_tasks` long-lived streams. Each stream
123
+ is one execution slot serving the sticky queue and every registered Flow type.
124
+ Flow type names must be unique within a Dex server. Concurrency defaults to the
125
+ CPU count. Start fails if the configured concurrency exceeds the Server's
126
+ reported `matching.maxConcurrentTasksCap`; zero means no limit was reported.
127
+ The Worker checks the limit on its first registration only. It then half-closes
128
+ its streams, drops any task they receive, and never reports readiness;
129
+ registration waiters raise the limit error. A lower limit reported on a later
130
+ reconnect is logged as a warning, and the Worker keeps running.
131
+ `idle_streams_per_queue` is deprecated, remains accepted, and has no effect.
132
+ Each slot executes every task it receives and never rejects a task for capacity.
133
+ Step heartbeats and stream writes use the same stream. Busy streams send a Pong
134
+ every 20 seconds until the final frame.
135
+
136
+ The Worker rejects an unregistered Step or RPC by name before its handler runs.
137
+ Dex tries another Worker without consuming a Step attempt, including when the
138
+ Flow was sticky to the rejecting Worker. If no connected Worker supports the
139
+ Step type, the Step stays queued until a Worker that supports it connects, the
140
+ Flow times out, or the Flow is stopped. An RPC caller receives the named error
141
+ after the Server's dispatch wait.
146
142
 
147
143
  The generated `app_worker_id` uses the first nonempty `DEX_WORKER_INSTANCE`,
148
144
  `POD_NAME`, `HOSTNAME`, or system hostname, plus eight random base32 characters.
@@ -150,41 +146,67 @@ An override must uniquely identify one Worker process and change after restart.
150
146
  It must contain 1-63 ASCII letters, digits, dots, underscores, or hyphens and
151
147
  must not equal a registered Flow type.
152
148
 
153
- Pull streams reconnect after failures. A server-directed reconnect waits for
154
- an active task to finish, then waits up to `stop_drain_timeout` for the server
155
- to close the stream after the final frame. An idle stream reconnects
156
- immediately. An older server without pull support produces one explicit error
157
- log and is retried after one minute. `start()` reports startup
158
- synchronization errors, while later stream errors are logged and retried by default.
159
- `stop()` drains for at most
160
- `stop_drain_timeout`, which defaults to 30 seconds. Afterward it requests
161
- cancellation and waits for handlers to return. Synchronous handlers should
162
- check `Context.is_cancellation_requested()` during long operations. Streams
163
- that sent a final frame wait for server EOF until the remaining Stop drain
164
- deadline before the channel closes.
149
+ Pull streams reconnect after failures. A broken stream cancels its handler;
150
+ its output is never sent on a replacement connection. Server-directed reconnect
151
+ lets an active task finish before sending its final output and half-closing.
152
+ If the Server has not confirmed that final output by EOF when the reconnect wait
153
+ of `stop_drain_timeout` ends, the stream gets up to 5 more seconds before it is
154
+ reset.
155
+ `start()` reports protocol and startup synchronization failures. Later transport
156
+ errors are logged and retried by default.
157
+
158
+ `stop()` starts no new tasks. Idle streams half-close immediately; busy streams
159
+ finish their handlers and send final output before half-closing. Task frames
160
+ arriving after stop begins are dropped without calling a handler. Streams keep
161
+ reading until server EOF or the remaining `stop_drain_timeout`, which defaults
162
+ to 30 seconds. On timeout, a busy stream is reset and its handler is canceled;
163
+ stop waits for the handler to return. Synchronous handlers should check
164
+ `Context.is_cancellation_requested()` during long operations. Any stream still
165
+ open at the deadline is reset, even one that is half-closed and still waiting
166
+ for server EOF, so stop returns once handlers return. Dex treats a task it sent
167
+ in that window as an attempt that may have run.
168
+
169
+ A stream whose final output was queued or written, but not yet confirmed by
170
+ server EOF or by the next assignment on that stream, is not reset at the drain
171
+ deadline. This holds even if its handler has finished or the stream has
172
+ half-closed, because a completed local write does not mean the Server read the
173
+ bytes. The stream gets up to 5 more seconds to send the output, half-close, and
174
+ receive server EOF, and is reset after that. `stop()` can therefore return up
175
+ to 5 seconds after the drain deadline.
176
+
177
+ A stream that fails locally, such as a read error, an undecodable or oversized
178
+ frame, a task frame with an invalid deadline or no request, a failed final
179
+ output write, or an unexpected server EOF, is always reset. It never
180
+ half-closes cleanly, because a clean half-close tells Dex the assigned task did
181
+ not run. A malformed task is never admitted and its handler never runs. Poll
182
+ streams accept only uncompressed responses.
183
+
184
+ After a slot sends a task's final output, Dex may assign its next task
185
+ immediately. The slot finishes local cleanup for the previous task first, then
186
+ starts the next one, unless stop has begun.
165
187
 
166
188
  Call `worker.wait_pull_registered(timeout=30)` after starting a synchronous
167
189
  Worker on another thread. For AsyncWorker, start it in another task and use
168
190
  `await worker.wait_pull_registered(timeout=30)`. The timeout is in seconds;
169
- `None` waits indefinitely. Each waits for every Flow type queue and the app
170
- Worker ID queue to register at least once. Timeout raises `TimeoutError`;
191
+ `None` waits indefinitely. Each waits for one `STEP_AND_RPC` stream and, when
192
+ RPC reservation is configured, one `RPC_ONLY` stream to register. Timeout raises `TimeoutError`;
171
193
  async cancellation cancels only that wait. Startup failures reach waiters.
172
194
  A permanent rejection before readiness raises `PullRegistrationError`, whose
173
- `queue` and `error` identify the rejected queue and original transport error.
195
+ `queue` and `error` identify the rejected slot kind and original transport error.
174
196
  Stopping before readiness raises `RuntimeError`.
175
197
 
176
198
  Use `worker.pull_registration_state()` for subsequent readiness probes. Its
177
199
  immutable `queues` expose `queue`, `has_registered_stream`, and
178
200
  `last_permanent_error`. After initial registration, `is_ready` stays true across
179
- transient failures, reconnects, and stop. It becomes false only when a queue
201
+ transient failures, reconnects, and stop. It becomes false only when a required slot kind
180
202
  has a permanent error and no registered stream. Re-registration clears the
181
203
  error. The startup wait keeps succeeding after initial registration, including
182
204
  after stop. Multiple callers may wait independently before or after start.
183
205
 
184
206
  `PullWorkerOptions(stop_on_permanent_stream_rejection=True)` makes the first
185
- permanent rejection leaving a queue without a registered stream drain within
207
+ permanent rejection leaving a slot kind without a registered stream drain within
186
208
  `stop_drain_timeout`, then raise `PullRegistrationError` from `start()`. This
187
- applies before and after readiness. Another registered stream for that queue
209
+ applies before and after readiness. Another registered stream for that slot kind
188
210
  prevents shutdown. The default is False and retries indefinitely. Always call
189
211
  `stop()` or `close()` afterward; both remain safe after `start()` returns.
190
212
 
@@ -199,32 +221,18 @@ long-running Steps; use a limit of at least two or set the reserve explicitly.
199
221
  Flows must stay responsive because RPCs share Worker task capacity with Steps.
200
222
  The value must be nonnegative and less than `max_concurrent_tasks` after its
201
223
  CPU-count default is resolved; invalid values fail construction. RPC and Step
202
- handlers count together. WaitFor and Execute tasks, including local attempts,
203
- are declined with `NO_CAPACITY` when the busy
204
- count reaches the task limit minus the reserve. API-direct and transactional
205
- InvokeWorkerRPC tasks can use the full limit.
224
+ handlers count together. Reserved slots register as `RPC_ONLY`; the remaining
225
+ slots register as `STEP_AND_RPC`. The Server assigns Steps only to the latter.
226
+ The Worker never declines a Step for capacity.
206
227
 
207
228
  ```python
208
229
  pull = PullWorkerOptions(max_concurrent_tasks=8, reserved_rpc_tasks=2)
209
230
  ```
210
231
 
211
- A decline lets Dex try another Worker without consuming an attempt. If all
212
- Workers remain at their Step limit, a Step can wait until the dispatch wait
213
- expires, then receive `NO_WORKER_AVAILABLE`, consume an attempt, and follow its
214
- retry policy, as when all Workers are full. Local attempts wait the local
215
- dispatch wait, then fall back according to their policy. A queue that declines
216
- a Step for reserved RPC capacity keeps one additional idle stream until 30
217
- seconds after its last decline. Other queues retain their configured count.
218
- The Worker does not open the extra stream while all task slots are busy. Set
219
- `idle_streams_per_queue` to at least two for heavy Step backlogs. A reserve decline
220
- does not close a stream or change registration readiness.
221
-
222
- Each reserve decline records the Flow type through the dex._pull_worker logger
223
- at DEBUG level. Export the cumulative `pull_reserved_rpc_declines` Worker value
224
- to distinguish reserve pressure from full-capacity rejections. The count excludes
225
- full-capacity rejections and unknown names. It counts decline events, including
226
- repeated declines of the same Step dispatch while Dex retries it. Read its rate
227
- to assess reserve pressure. It never resets and remains available after shutdown.
232
+ The `pull_reserved_rpc_declines` counter is deprecated and always zero. It stays
233
+ available for source compatibility. Registration snapshot `queues` and error
234
+ `queue` fields retain their names but now hold slot kind labels, `STEP_AND_RPC`
235
+ and `RPC_ONLY`, rather than individual queue names.
228
236
 
229
237
  ## Flow state I/O
230
238
 
@@ -346,7 +354,7 @@ class TimeoutRecoveryStep(Step[None]):
346
354
  return force_fail(failure.detail)
347
355
  ```
348
356
 
349
- Python SDK for [Dex workflow engine](https://github.com/superdurable/dex)
357
+ Python SDK for [Dex durable workflows](https://github.com/superdurable/dex)
350
358
 
351
359
  ## New user contracts
352
360
 
@@ -455,12 +463,10 @@ option cannot be combined with an index or `sync_to_attribute_store`, and
455
463
  the stored value. Handlers and `get_attributes` receive the value inline, and
456
464
  the SDK never stores it in the local blob cache.
457
465
 
458
- Hard-delete writes require a Server with an enabled blob store and Server
459
- protocol version 3 or later. Workers still start against older Servers.
460
- There, a Step, RPC, or timeout handler result that writes a hard-delete
461
- Attribute fails that handler attempt with `RuntimeError`, as does a `wait_for`
462
- result whose SubFlow start initializes one. A Client checks the Server protocol
463
- once, before its first `start_flow` that initializes one.
466
+ Hard-delete writes require a Server with an enabled blob store. This SDK
467
+ requires protocol 4. Workers negotiate before startup and Clients negotiate
468
+ lazily before their first service request. Incompatible Servers fail with
469
+ `RuntimeError` before values are sent.
464
470
 
465
471
  Each Flow keeps one stored copy per Attribute key, overwritten in place. A Step
466
472
  or RPC attempt writes or deletes the stored value before its result commits, so
@@ -582,19 +588,18 @@ integer and string instances and routes to both by instance and StepRef.
582
588
  `StepOptions.wait_for_method_timeout` and `execute_method_timeout` bound the
583
589
  two handler calls. Timer and channel conditions determine how long a Step waits.
584
590
 
585
- `wait_for_retry` and `execute_retry` limit one logical handler execution. With
586
- `StepDurability.ASYNC`, local and fallback regular activities share maximum
587
- attempts, total duration, and 1-based attempt numbers. Fallback starts
588
- immediately; later regular retries continue the backoff sequence at the
589
- cumulative attempt.
591
+ `wait_for_retry` and `execute_retry` limit one logical handler execution.
592
+ Attempts use one-based numbers. The retry total duration starts when Dex first
593
+ hands the Step to a Worker. After an attempt fails, Dex checks that budget to
594
+ decide whether to schedule another retry; it does not end a running attempt.
590
595
 
591
596
  The default Step durability is synchronous. A Flow configuration can select
592
597
  asynchronous durability, and a Step method override has highest precedence. The
593
- default retry total duration is four hours. Regular attempts default to a two-hour
598
+ default retry total duration is four hours. Attempts default to a two-hour
594
599
  method timeout and one-minute heartbeat timeout; an explicit heartbeat timeout must
595
600
  meet the server minimum, which defaults to ten seconds. Asynchronous durability
596
- first allows at most three local attempts in seven seconds. The local phase ignores
597
- method timeouts and heartbeat frames before falling back to a regular activity.
601
+ gives lower latency and faster single Flows. Sustained throughput is about the
602
+ same as synchronous durability.
598
603
 
599
604
  ### Flow run start time
600
605
 
@@ -611,8 +616,7 @@ with `datetime.now(timezone.utc)` to measure the run's age. It matches
611
616
  current `wait_for`, `execute`, or Flow timeout handler attempt. It is a
612
617
  timezone-aware UTC `datetime` with microsecond precision, not a duration.
613
618
  Subtract `datetime.now(timezone.utc)` and call `total_seconds()` to budget
614
- remaining work after checking for `None`. A shorter remaining
615
- `RetryPolicy.total_duration` limits this deadline; each retry receives its own
619
+ remaining work after checking for `None`. Each retry receives its own
616
620
  effective deadline. RPC handlers and tasks without a supplied deadline return
617
621
  `None`.
618
622
 
@@ -977,15 +981,34 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#releases-monorepo-tags) for monorepo ta
977
981
 
978
982
  ## Server protocol compatibility
979
983
 
980
- Both `Worker` and `AsyncWorker` call `GetServerInfo`, negotiate the highest
981
- common protocol, synchronize Attribute indexes, and only then open outbound
982
- task streams. The initial Python SDK interval is `[1,1]`. Missing information,
983
- invalid or disjoint intervals, and RPC failures stop startup before polling.
984
- The current Python SDK interval is `[1,3]`. Protocol 3 adds hard-delete
985
- Attributes. Workers record the negotiated version at startup and reject each
986
- handler result that contains a hard-delete Attribute write when it is below 3.
987
- Clients negotiate lazily, once, before the first `start_flow` that initializes
988
- a hard-delete Attribute.
984
+ Workers call `GetServerInfo`, require protocol 4, synchronize Attribute indexes,
985
+ and then open execution slot streams. The Python SDK protocol interval is
986
+ `[4,4]`. Older Servers fail startup with this error, substituting the protocol
987
+ range and Server version actually reported:
988
+
989
+ ```text
990
+ This SDK requires a Dex Server with protocol version 4 or later (server-zd newer than v0.1.16). The Server reports protocol versions <min>-<current> (server version <server_version>). Upgrade the Server before upgrading the SDK.
991
+ ```
992
+
993
+ A Server whose minimum protocol is newer than 4 fails with an error that asks
994
+ you to upgrade the SDK. Missing information, invalid intervals, and RPC
995
+ failures stop startup before polling.
996
+
997
+ Clients perform no network I/O during construction. Before the first request of
998
+ any method, they negotiate protocol 4. Success and incompatibility are cached;
999
+ later requests against an incompatible Server raise the same clear
1000
+ `RuntimeError`. If `GetServerInfo` itself fails, for example because the Server
1001
+ is unreachable, the request raises the usual `DexServiceError`, and the next
1002
+ call negotiates again. Negotiation counts against a request's timeout, including
1003
+ the `request_timeout` of durable waits.
1004
+
1005
+ `RetryPolicy.total_duration` counts from the first hand-out to a Worker. After
1006
+ an attempt fails, Dex checks it to decide whether to retry. It does not end a
1007
+ running attempt. Recovery handlers can inspect `RecoveryErrorInfo.attempt_progress`:
1008
+ `StepAttemptProgress.NOT_STARTED` guarantees no handler ran;
1009
+ `NOT_COMPLETED` means some attempt may have run but none completed;
1010
+ `UNSPECIFIED` covers older Servers or failures recorded before upgrade and must
1011
+ be treated like `NOT_COMPLETED`.
989
1012
 
990
1013
  The diagnostic SDK version comes from Python distribution metadata for
991
1014
  `dex-python-sdk` and falls back to `dev` when the distribution is unavailable.