flo-python 0.1.5__tar.gz → 0.1.7__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {flo_python-0.1.5 → flo_python-0.1.7}/.github/flo-ref.env +1 -1
- {flo_python-0.1.5 → flo_python-0.1.7}/CHANGELOG.md +119 -1
- {flo_python-0.1.5 → flo_python-0.1.7}/PKG-INFO +230 -9
- {flo_python-0.1.5 → flo_python-0.1.7}/README.md +229 -8
- {flo_python-0.1.5 → flo_python-0.1.7}/pyproject.toml +1 -1
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/__init__.py +63 -1
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/client.py +2 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/exceptions.py +26 -7
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/kv.py +5 -14
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/kv_body.py +266 -2
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/processing.py +16 -3
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/queue.py +67 -72
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/retry.py +13 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/streams.py +200 -1
- flo_python-0.1.7/src/flo/ts.py +465 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/types.py +397 -29
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/wire.py +7 -3
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/worker.py +103 -21
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_action_ops.py +0 -57
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_action_status.py +12 -3
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_actions_live.py +1 -1
- flo_python-0.1.7/tests/test_dead_letters.py +361 -0
- flo_python-0.1.7/tests/test_dead_letters_live.py +307 -0
- flo_python-0.1.7/tests/test_not_found_answers.py +113 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_ops_drift.py +23 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_processing_decode.py +29 -1
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_processing_live.py +36 -1
- flo_python-0.1.7/tests/test_read_not_found_live.py +78 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_result_resend.py +95 -2
- flo_python-0.1.7/tests/test_stream_settings.py +73 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_stream_trim.py +4 -2
- flo_python-0.1.7/tests/test_ts_live.py +259 -0
- flo_python-0.1.7/tests/test_ts_ops.py +467 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_wire.py +1 -1
- flo_python-0.1.7/tests/test_worker_close.py +278 -0
- flo_python-0.1.7/tests/test_worker_close_live.py +97 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_worker_drain.py +6 -2
- flo_python-0.1.5/tests/test_not_found_answers.py +0 -38
- {flo_python-0.1.5 → flo_python-0.1.7}/.github/workflows/ci.yml +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/.github/workflows/release.yml +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/.gitignore +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/LICENSE +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/examples/basic.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/examples/stream_worker.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/examples/test_decorator.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/examples/worker.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/actions.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/kv_txn.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/src/flo/workflows.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/__init__.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/conftest.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/order-workflow.yaml +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_auto_touch.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_block_ms.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_blocking_deadline.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_empty_poll_backoff.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_group_ops.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_groups_live.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_halts.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_kv_live.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_kv_ops.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_kv_table.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_kv_ttl.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_list_paging.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_list_paging_live.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_missing_answers_live.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_op_codes.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_poll_loops.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_protocol_version.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_queue_live.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_queue_ops.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_reconnect_cancel.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_retry.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_status_codes.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_stream_delete.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_stream_delete_live.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_stream_tier.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_stream_worker_ack.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_worker_info.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_worker_info_live.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_worker_list.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_worker_live.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_worker_liveness.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_workflow_list_runs.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_workflow_runs_live.py +0 -0
- {flo_python-0.1.5 → flo_python-0.1.7}/tests/test_workflows.py +0 -0
|
@@ -1,4 +1,4 @@
|
|
|
1
1
|
# The flo commit the live tests run against: CI checks floruntime/flo out at
|
|
2
2
|
# this commit, builds it and starts it. Bump it in the same commit as
|
|
3
3
|
# TABLE_HASH, so the SDK and the server agree.
|
|
4
|
-
FLO_REF=
|
|
4
|
+
FLO_REF=1fd555dcec49985f3bb533ca0a6fb4b95e749341
|
|
@@ -5,7 +5,125 @@ All notable changes to the Flo Python SDK will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
-
## [
|
|
8
|
+
## [0.1.7] - 2026-11-18
|
|
9
|
+
|
|
10
|
+
Works with flo v0.1.0-dev.19.
|
|
11
|
+
|
|
12
|
+
The server's op table hash is now `0xac07f1f7338c66bd` (`TABLE_HASH`).
|
|
13
|
+
|
|
14
|
+
### Added
|
|
15
|
+
|
|
16
|
+
- `stream.group_steal(stream, group, consumer, records, min_idle_ms)`: take
|
|
17
|
+
records another consumer holds and has left idle that long, each with a new
|
|
18
|
+
delivery and lease, its failures kept. It answers a `GroupStealResult`, a
|
|
19
|
+
delivery acked, nacked or touched as a read's, with `refused` for the
|
|
20
|
+
records it didn't take (`NOT_IDLE` for one its holder touched within the
|
|
21
|
+
idle time)
|
|
22
|
+
- A consumer group's dead letters: `stream.group_dlq_list` (a page, oldest
|
|
23
|
+
dead first, with each one's deliveries, failures, `dead_at_ms` and last
|
|
24
|
+
error; page by the opaque `cursor`), `group_dlq_requeue` (ready again,
|
|
25
|
+
failures reset) and `group_dlq_delete` (with `dry_run`). Requeue and delete
|
|
26
|
+
name up to 100 records by id, or take the oldest with `all`, a batch of up to
|
|
27
|
+
`limit`; they answer `applied`, `refused` (with why and the server's words)
|
|
28
|
+
and `more`
|
|
29
|
+
- `queue.dlq_requeue` and `queue.dlq_delete` (with `dry_run`), by seq or
|
|
30
|
+
`all`, answering a `DlqResult` as the group's do
|
|
31
|
+
- `QueueRefusalReason.NOT_DEAD` (6) and `NOT_IDLE` (7): a requeue or delete
|
|
32
|
+
of one that isn't dead, and a steal of one that isn't idle.
|
|
33
|
+
`GroupRefusal.reason` carries the server's words where it gives them
|
|
34
|
+
- `ActionRunStatus.held`: why a run's lease, past due, isn't ended yet (its
|
|
35
|
+
end waits for room on the calling workflow's shard)
|
|
36
|
+
- `stream.create` (partitions, retention by age and count) and `stream.alter`
|
|
37
|
+
(retention); `StreamInfo.retention_age_s` and `retention_count`
|
|
38
|
+
|
|
39
|
+
### Changed
|
|
40
|
+
|
|
41
|
+
- A worker's complete, fail, ack or nack refused overloaded without being
|
|
42
|
+
run (after the client's own retries) is sent again with a capped backoff
|
|
43
|
+
until the server takes it or answers otherwise, and logged; it used to be
|
|
44
|
+
dropped. A complete or fail that ends a run a workflow step called is
|
|
45
|
+
refused so while the server's writes to the workflow's shard are queued
|
|
46
|
+
full, and the run's lease doesn't run out meanwhile; an ack's or nack's
|
|
47
|
+
lease keeps running, so a long overload can deliver the records again. A
|
|
48
|
+
stopping worker gives up after a couple of seconds, a backoff wait
|
|
49
|
+
included
|
|
50
|
+
- An alter of a stream an append made succeeds; it used to be not found.
|
|
51
|
+
After an alter a create must name the stream's settings as altered, or it
|
|
52
|
+
raises `ConflictError`; a create whose name hashes as another stream's does
|
|
53
|
+
too
|
|
54
|
+
- A workflow step whose action isn't registered waits for one, up to the
|
|
55
|
+
server's missing-action wait, instead of failing at once
|
|
56
|
+
|
|
57
|
+
### Breaking
|
|
58
|
+
|
|
59
|
+
- `queue.dlq_list` sends the new op: `DlqPage` is `messages` and `cursor`
|
|
60
|
+
(a `DlqCursor`, `None` on the last page), replacing `total`, `has_more` and
|
|
61
|
+
`last_seq`; `DlqListOptions.after` takes the cursor, and `limit` defaults
|
|
62
|
+
to 20. `DeadLetter` gains `failures`, `dead_at_ms` and `error`, and loses
|
|
63
|
+
`moved_at_ms`. A queue that doesn't exist raises `NotFoundError`
|
|
64
|
+
- The old dead letter op codes (0x176..0x178) are gone; `OpCode.QUEUE_DLQ_*`
|
|
65
|
+
are the server's 0x185..0x187
|
|
66
|
+
|
|
67
|
+
## [0.1.6] - 2026-11-16
|
|
68
|
+
|
|
69
|
+
The op table hash moves with the server in this release: this SDK talks only
|
|
70
|
+
to the flo it pins, and the pin lands with the release.
|
|
71
|
+
The server's op table hash is now `0x17f3632347cd6dd9` (`TABLE_HASH`).
|
|
72
|
+
|
|
73
|
+
### Added
|
|
74
|
+
|
|
75
|
+
- `client.ts`: `write`, `read` (one page), `iter_points` (every page) and
|
|
76
|
+
`floql`. A read takes `field`, `tags`, `from_ns`/`to_ns` (inclusive),
|
|
77
|
+
`limit` (1..4096) and `order` (`TSOrder.OLDEST_FIRST` or `NEWEST_FIRST`),
|
|
78
|
+
and answers the points of every matching series merged by time, then
|
|
79
|
+
write order, with an opaque cursor that is `None` on the last page. A
|
|
80
|
+
page goes on strictly past the cursor, so no point is read twice or
|
|
81
|
+
skipped; page until the cursor is `None`, not until a page is short
|
|
82
|
+
- Times are Unix ns `int`s: `TSWriteOptions.time_ns` (`OptionTag.TS_TIMESTAMP_NS`,
|
|
83
|
+
0x67, an i64; none or 0 is the server's time), `TSWriteResult.time_ns`,
|
|
84
|
+
`TSPoint.time_ns`, `FloQLPoint.time_ns`. `ns_from_datetime` and
|
|
85
|
+
`datetime_from_ns` (and `TSPoint.time`) convert at the microsecond
|
|
86
|
+
- `ts.query` (`TSQueryOptions`: `from_ns`, `to_ns`, a `window` as ns or a
|
|
87
|
+
`timedelta`, and `TSAggregation`) answers `TSBucket`s with `start_ns`;
|
|
88
|
+
`ts.info` answers a `TSInfo` with `oldest_ns` and `newest_ns`.
|
|
89
|
+
`ns_from_timedelta` converts exactly
|
|
90
|
+
- `ProcessingRuntime.late_edge_ms` and `late_points`: at least that many of
|
|
91
|
+
a job's TS source points came in later than the edge at or above its
|
|
92
|
+
lateness window, so its sources never read them; both `None` for a job
|
|
93
|
+
with no TS source
|
|
94
|
+
- `ActionWorker.close(timeout=30.0)` and `StreamWorker.close(timeout=30.0)`
|
|
95
|
+
wait up to `timeout` for `start()` to return, so in-flight completes,
|
|
96
|
+
fails, acks, nacks and hand-backs are sent before the connections close
|
|
97
|
+
|
|
98
|
+
### Changed
|
|
99
|
+
|
|
100
|
+
- `stream.read` of a stream that doesn't exist (never created, or deleted)
|
|
101
|
+
raises `NotFoundError`, as the server now answers; an existing stream
|
|
102
|
+
with nothing to read is still empty, and a blocking read of a stream not
|
|
103
|
+
created yet still waits for its first append. `ts.read` and `iter_points`
|
|
104
|
+
raise `NotFoundError` when no series of the field matches. A consumer
|
|
105
|
+
that may start before its stream or series exists catches it and starts
|
|
106
|
+
again. A not found answered stale (by a node that may be behind) is
|
|
107
|
+
`StaleNotFoundError`, as for `kv.get`: the stream or series may exist, so
|
|
108
|
+
read again
|
|
109
|
+
- A stream worker whose read is answered not found fails `start()` with
|
|
110
|
+
`NotFoundError` instead of retrying as if the stream were empty
|
|
111
|
+
- A worker task cancelled while its handler runs waits (up to the handler
|
|
112
|
+
timeout) for the handler to end before handing the work back; a cancel
|
|
113
|
+
aimed at the task while a timed-out handler ends is no longer swallowed
|
|
114
|
+
- A FloQL answer of more than 4096 raw points is refused by the server
|
|
115
|
+
(`BadRequestError`, with its text); aggregate, narrow, or page with
|
|
116
|
+
`ts.read`
|
|
117
|
+
|
|
118
|
+
### Breaking
|
|
119
|
+
|
|
120
|
+
- `OptionTag.TS_TIMESTAMP` is `TS_TIMESTAMP_NS`: the server reads it as ns,
|
|
121
|
+
so a time sent as ms lands near 1970. A time before 1970 is refused
|
|
122
|
+
- `OptionTag.TS_FROM_MS`, `TS_TO_MS` and `TS_WINDOW_MS` are `TS_FROM_NS`,
|
|
123
|
+
`TS_TO_NS` and `TS_WINDOW_NS`, i64s of ns. Every time-series time and
|
|
124
|
+
window is ns; no ms remains in the time-series API
|
|
125
|
+
|
|
126
|
+
## [0.1.5] - 2026-11-15
|
|
9
127
|
|
|
10
128
|
The op table hash moves with the server in this release: this SDK talks only
|
|
11
129
|
to the flo it pins, and the pin lands with the release.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: flo-python
|
|
3
|
-
Version: 0.1.
|
|
3
|
+
Version: 0.1.7
|
|
4
4
|
Summary: Python SDK for the Flo distributed systems platform
|
|
5
5
|
Project-URL: Homepage, https://github.com/floruntime/flo-python
|
|
6
6
|
Project-URL: Documentation, https://github.com/floruntime/flo-python#readme
|
|
@@ -65,6 +65,7 @@ asyncio.run(main())
|
|
|
65
65
|
- **KV Store**: Versioned key-value storage with MVCC, TTL, and optimistic locking
|
|
66
66
|
- **Queues**: Priority task queues, at-least-once: a dequeue leases its messages until they're completed
|
|
67
67
|
- **Streams**: Append-only logs with consumer groups for distributed processing
|
|
68
|
+
- **Time series**: Points in nanoseconds, read a page at a time either way, and FloQL
|
|
68
69
|
- **Actions**: Registered tasks with configurable timeouts and retries
|
|
69
70
|
- **Workers**: Distributed task execution with lease management and heartbeats
|
|
70
71
|
- **Async/await**: Native asyncio support for high-performance applications
|
|
@@ -305,15 +306,32 @@ result = await client.queue.dequeue("tasks", 10, DequeueOptions(ack=True))
|
|
|
305
306
|
`queue.purge` removes every message of a queue, ready, leased and dead, and
|
|
306
307
|
returns how many.
|
|
307
308
|
|
|
308
|
-
`queue.dlq_list` reads a page of dead letters, oldest first (`limit`
|
|
309
|
+
`queue.dlq_list` reads a page of dead letters, oldest dead first (`limit`
|
|
310
|
+
1..100, default 20), each with up to 4 KiB of its payload (`size` is its
|
|
311
|
+
length), its deliveries and failures, when it died and its last error. Page
|
|
312
|
+
with the answer's opaque `cursor` until it is `None`.
|
|
313
|
+
|
|
314
|
+
`queue.dlq_requeue` makes dead letters ready again at once, failures reset;
|
|
315
|
+
`queue.dlq_delete` deletes them for good (`dry_run` says how many it would).
|
|
316
|
+
Each names up to 100 by seq, or takes the oldest dead with `all`, a batch of
|
|
317
|
+
up to `limit`: `more` says others remain. One that isn't dead comes back in
|
|
318
|
+
`refused` as `NOT_DEAD`, one not there as `GONE`; the call itself succeeds.
|
|
319
|
+
A queue that doesn't exist raises `NotFoundError`.
|
|
309
320
|
|
|
310
321
|
```python
|
|
311
|
-
from flo import DlqListOptions
|
|
322
|
+
from flo import DlqDeleteOptions, DlqListOptions, DlqRequeueOptions
|
|
312
323
|
|
|
313
324
|
page = await client.queue.dlq_list("tasks", DlqListOptions(limit=100))
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
325
|
+
while True:
|
|
326
|
+
for msg in page.messages:
|
|
327
|
+
print(msg.seq, msg.failures, msg.error)
|
|
328
|
+
if page.cursor is None:
|
|
329
|
+
break
|
|
330
|
+
page = await client.queue.dlq_list("tasks", DlqListOptions(after=page.cursor))
|
|
331
|
+
|
|
332
|
+
await client.queue.dlq_requeue("tasks", [msg.seq])
|
|
333
|
+
while (await client.queue.dlq_delete("tasks", options=DlqDeleteOptions(all=True))).more:
|
|
334
|
+
pass
|
|
317
335
|
```
|
|
318
336
|
|
|
319
337
|
### Peek
|
|
@@ -374,6 +392,15 @@ result = await client.stream.read("events", StreamReadOptions(tail=True, count=1
|
|
|
374
392
|
result = await client.stream.read("events", StreamReadOptions(start=last, block_ms=30000))
|
|
375
393
|
```
|
|
376
394
|
|
|
395
|
+
A read of a stream that doesn't exist (never created or appended to, or
|
|
396
|
+
deleted) raises `NotFoundError`, while one that exists with nothing to read is
|
|
397
|
+
empty. A blocking read (`block_ms` > 0) of a stream not created yet waits for
|
|
398
|
+
its first append instead. A consumer that may start before its stream exists
|
|
399
|
+
catches `NotFoundError` and starts again. `StaleNotFoundError` (a
|
|
400
|
+
`NotFoundError`) is a not found from a node that may be behind: the stream may
|
|
401
|
+
exist but this node hasn't seen it yet, so read again rather than treat it as
|
|
402
|
+
gone.
|
|
403
|
+
|
|
377
404
|
### Consumer Groups
|
|
378
405
|
|
|
379
406
|
Consumer groups let many consumers work through a stream, each record leased
|
|
@@ -391,8 +418,10 @@ read that can take nothing new for either raises `GroupHeldFullError` (an
|
|
|
391
418
|
`OverloadedError`, reason `GROUP_HELD_FULL`; the message says which cap).
|
|
392
419
|
Reading again won't help until room is made, so the client doesn't retry it.
|
|
393
420
|
At the group's cap, acks and nacks make room, but dead records count too and
|
|
394
|
-
stay until the group is deleted or a trim of
|
|
395
|
-
|
|
421
|
+
stay until they are requeued or deleted, the group is deleted, or a trim of
|
|
422
|
+
the stream cuts them. At the
|
|
423
|
+
shard's cap, this group's own acks free room, as do other groups and queues on
|
|
424
|
+
the shard draining.
|
|
396
425
|
|
|
397
426
|
A group's lease (`ack_timeout_ms`, 5 s to 12 h, 30 s by default) and
|
|
398
427
|
`max_deliver` (1 to 255) are fixed when it is created; the first read of a
|
|
@@ -452,6 +481,183 @@ members = await client.stream.group_members("events", "processors") # consumers
|
|
|
452
481
|
await client.stream.group_delete("events", "processors")
|
|
453
482
|
```
|
|
454
483
|
|
|
484
|
+
A group's dead letters list a page at a time, oldest dead first, each with its
|
|
485
|
+
deliveries, failures, when it died and its last error; requeue and delete
|
|
486
|
+
take them by id or `all`, as a queue's do (`NOT_DEAD` for one that isn't).
|
|
487
|
+
|
|
488
|
+
```python
|
|
489
|
+
from flo import GroupDlqDeleteOptions, GroupDlqListOptions
|
|
490
|
+
|
|
491
|
+
page = await client.stream.group_dlq_list("events", "processors", GroupDlqListOptions(limit=50))
|
|
492
|
+
await client.stream.group_dlq_requeue("events", "processors", [d.id for d in page.dead])
|
|
493
|
+
await client.stream.group_dlq_delete(
|
|
494
|
+
"events", "processors", options=GroupDlqDeleteOptions(all=True, dry_run=True)
|
|
495
|
+
)
|
|
496
|
+
```
|
|
497
|
+
|
|
498
|
+
`group_steal` is an operator's command: it takes records another consumer
|
|
499
|
+
holds and has left idle for at least `min_idle_ms` (idle is the time since
|
|
500
|
+
its holder last read, stole or touched it), each with a new delivery and
|
|
501
|
+
lease, its failures kept. Name each record with the deliveries its holder has
|
|
502
|
+
(a read's records, or `group_pending`'s entries). The result is a delivery,
|
|
503
|
+
acked, nacked or touched as a read's; records it didn't take come back in
|
|
504
|
+
`refused` (`NOT_IDLE` for one touched within the idle time). A read takes
|
|
505
|
+
expired records by itself, so a steal is for records whose holder is stuck
|
|
506
|
+
but still touching them, or to move work sooner than the lease.
|
|
507
|
+
|
|
508
|
+
```python
|
|
509
|
+
pending = await client.stream.group_pending("events", "processors", "worker-1")
|
|
510
|
+
stolen = await client.stream.group_steal("events", "processors", "worker-2", pending, 60_000)
|
|
511
|
+
await client.stream.group_ack("events", "processors", stolen)
|
|
512
|
+
```
|
|
513
|
+
|
|
514
|
+
### Stream Settings
|
|
515
|
+
|
|
516
|
+
An append creates a stream with no settings. `create` gives one its
|
|
517
|
+
partitions and retention (records older than `retention_age_s`, or past
|
|
518
|
+
`retention_count`, are trimmed); `alter` changes its retention, each setting
|
|
519
|
+
given replacing the stream's, on a stream an append made as on one a create
|
|
520
|
+
made. `info` answers the settings.
|
|
521
|
+
|
|
522
|
+
Once a stream has settings, from a create or an alter, a create that names
|
|
523
|
+
the same ones succeeds (a retry), and one that names others raises
|
|
524
|
+
`ConflictError` ("the stream exists with other settings"): after an alter, a
|
|
525
|
+
create must name the settings as altered. A create of a stream an append made
|
|
526
|
+
and nothing altered gives it the settings named. A create whose name hashes
|
|
527
|
+
as another stream's raises `ConflictError` too; choose another name.
|
|
528
|
+
|
|
529
|
+
```python
|
|
530
|
+
from flo import StreamAlterOptions, StreamCreateOptions
|
|
531
|
+
|
|
532
|
+
await client.stream.create("events", StreamCreateOptions(retention_age_s=86_400))
|
|
533
|
+
await client.stream.alter("events", StreamAlterOptions(retention_count=1_000_000))
|
|
534
|
+
info = await client.stream.info("events")
|
|
535
|
+
print(info.retention_age_s, info.retention_count)
|
|
536
|
+
```
|
|
537
|
+
|
|
538
|
+
## Time-Series Operations
|
|
539
|
+
|
|
540
|
+
Every time and window is Unix nanoseconds, a Python `int`. A point's
|
|
541
|
+
measurement is the request key; its field defaults to `value`, and its tags
|
|
542
|
+
(`"k=v,k2=v2"` or a dict) name the series.
|
|
543
|
+
|
|
544
|
+
### Write
|
|
545
|
+
|
|
546
|
+
```python
|
|
547
|
+
import time
|
|
548
|
+
from flo import TSWriteOptions
|
|
549
|
+
|
|
550
|
+
r = await client.ts.write("cpu", 0.42, TSWriteOptions(tags={"host": "a"}, time_ns=time.time_ns()))
|
|
551
|
+
print(r.time_ns) # the point's time, in ns
|
|
552
|
+
|
|
553
|
+
# No time (or 0): the server's time, answered in r.time_ns.
|
|
554
|
+
await client.ts.write("cpu", 0.40, TSWriteOptions(tags={"host": "a"}))
|
|
555
|
+
```
|
|
556
|
+
|
|
557
|
+
A time before 1970 is refused (`BadRequestError`), and a value that isn't
|
|
558
|
+
finite is refused before it is sent.
|
|
559
|
+
|
|
560
|
+
### Read
|
|
561
|
+
|
|
562
|
+
`ts.read` answers one page: the points of every series that matches the
|
|
563
|
+
field and tag filter, merged by time and then by write order, oldest first or
|
|
564
|
+
`TSOrder.NEWEST_FIRST`, at most `limit` (1..4096, 4096 by default), within
|
|
565
|
+
`from_ns`..`to_ns` (inclusive) when given. Ask again with the page's cursor
|
|
566
|
+
until it is `None`; a short page doesn't mean the last one. A page goes on
|
|
567
|
+
strictly after the cursor (before it, newest first), so a point there for the
|
|
568
|
+
whole read is read exactly once. The cursor is opaque bytes.
|
|
569
|
+
|
|
570
|
+
```python
|
|
571
|
+
from flo import TSOrder, TSReadOptions
|
|
572
|
+
|
|
573
|
+
opts = TSReadOptions(tags="host=a", limit=1000)
|
|
574
|
+
while True:
|
|
575
|
+
page = await client.ts.read("cpu", opts)
|
|
576
|
+
for p in page.points:
|
|
577
|
+
print(p.time_ns, p.value)
|
|
578
|
+
if page.cursor is None:
|
|
579
|
+
break
|
|
580
|
+
opts.cursor = page.cursor
|
|
581
|
+
|
|
582
|
+
# Or let the iterator page: every point, newest first.
|
|
583
|
+
async for p in client.ts.iter_points("cpu", TSReadOptions(order=TSOrder.NEWEST_FIRST)):
|
|
584
|
+
print(p.time_ns, p.value)
|
|
585
|
+
|
|
586
|
+
# The latest point.
|
|
587
|
+
latest = await client.ts.read("cpu", TSReadOptions(limit=1, order=TSOrder.NEWEST_FIRST))
|
|
588
|
+
```
|
|
589
|
+
|
|
590
|
+
A tag filter that isn't `key=value` pairs, or a cursor that isn't one a page
|
|
591
|
+
answered, is refused (`BadRequestError`), as is an empty measurement name. A
|
|
592
|
+
read that matches no series of the field (a measurement never written, or a
|
|
593
|
+
tag filter no series has) raises `NotFoundError`; a series with no points in
|
|
594
|
+
the range is an empty page. A refusal on any page, `NotFoundError` included,
|
|
595
|
+
is raised from `iter_points` rather than ending it; a reader that may start
|
|
596
|
+
before its series exists catches `NotFoundError` and starts again. As for
|
|
597
|
+
streams, `StaleNotFoundError` means the series may exist but this node hasn't
|
|
598
|
+
seen it yet: read again.
|
|
599
|
+
|
|
600
|
+
`datetime` helpers convert at the edges, to the microsecond (`datetime` holds
|
|
601
|
+
no finer): `ns_from_datetime(dt)` takes an aware `datetime`, and
|
|
602
|
+
`datetime_from_ns(ns)` and `TSPoint.time` give one in UTC. The ns `int` stays
|
|
603
|
+
the exact value.
|
|
604
|
+
|
|
605
|
+
### Query and info
|
|
606
|
+
|
|
607
|
+
```python
|
|
608
|
+
from datetime import timedelta
|
|
609
|
+
from flo import TSAggregation, TSQueryOptions
|
|
610
|
+
|
|
611
|
+
# One value per 5-minute window; each bucket's start is Unix ns.
|
|
612
|
+
buckets = await client.ts.query("cpu", TSQueryOptions(
|
|
613
|
+
from_ns=start_ns, to_ns=end_ns, window=timedelta(minutes=5), aggregation=TSAggregation.MAX))
|
|
614
|
+
for b in buckets:
|
|
615
|
+
print(b.start_ns, b.value)
|
|
616
|
+
|
|
617
|
+
info = await client.ts.info("cpu")
|
|
618
|
+
print(info.series, info.points, info.oldest_ns, info.newest_ns)
|
|
619
|
+
```
|
|
620
|
+
|
|
621
|
+
A query's window is an `int` of ns or a `timedelta` (exact to its
|
|
622
|
+
microsecond); without one, the server's minute. Windows start at multiples of
|
|
623
|
+
the width from the epoch, and only windows holding a point are answered. A
|
|
624
|
+
range holding points in more than 1024 windows is refused (`BadRequestError`,
|
|
625
|
+
with the server's text): widen the window or narrow the range. `ts.info` of a
|
|
626
|
+
measurement with no series raises `NotFoundError`.
|
|
627
|
+
|
|
628
|
+
### FloQL
|
|
629
|
+
|
|
630
|
+
```python
|
|
631
|
+
series = await client.ts.floql("cpu[1h] | avg()")
|
|
632
|
+
for s in series:
|
|
633
|
+
for p in s.points:
|
|
634
|
+
print(s.key, p.time_ns, p.value)
|
|
635
|
+
```
|
|
636
|
+
|
|
637
|
+
In the query text, a bare number in a range is epoch milliseconds and
|
|
638
|
+
durations (`5m`, `30s`) are as FloQL writes them; the answer's times are ns.
|
|
639
|
+
FloQL doesn't page: an answer of more than 4096 raw points is refused
|
|
640
|
+
(`BadRequestError`, with the server's text). Aggregate them, narrow the
|
|
641
|
+
query, or page through them with `ts.read`.
|
|
642
|
+
|
|
643
|
+
## Processing
|
|
644
|
+
|
|
645
|
+
```python
|
|
646
|
+
job_id = await client.processing.submit(open("pipeline.yaml").read())
|
|
647
|
+
status = await client.processing.status(job_id)
|
|
648
|
+
print(status.status, status.runtime.records_in, status.runtime.late_points)
|
|
649
|
+
```
|
|
650
|
+
|
|
651
|
+
A TS source reads its field up to its `lateness_ms` window behind the
|
|
652
|
+
field's newest point (30 s by default, at most a day; 0 reads with no
|
|
653
|
+
window), so it lags by up to the window. A point that arrives later than
|
|
654
|
+
that is never read by the source, and is counted: `runtime.late_points`
|
|
655
|
+
says at least that many of the job's source points came in more than
|
|
656
|
+
`runtime.late_edge_ms` behind their field's newest. The server counts at
|
|
657
|
+
fixed edges (0 s, 1 s, 5 s, 15 s, 30 s, 1 m, 5 m, 1 h), against the smallest
|
|
658
|
+
at or above the job's largest window. Both are `None` for a job with no TS
|
|
659
|
+
source, or a window past an hour.
|
|
660
|
+
|
|
455
661
|
## Action Operations
|
|
456
662
|
|
|
457
663
|
Actions are registered tasks that can be invoked and executed by workers.
|
|
@@ -504,6 +710,8 @@ if status.error_message is not None:
|
|
|
504
710
|
print(f"Error: {status.error_message}")
|
|
505
711
|
if status.outcome is not None: # None for success
|
|
506
712
|
print(f"Outcome: {status.outcome}")
|
|
713
|
+
if status.held is not None: # why a past-due lease isn't ended yet
|
|
714
|
+
print(f"Held: {status.held}")
|
|
507
715
|
|
|
508
716
|
# An action's definition, its runs by status and its completed runs' latency
|
|
509
717
|
info = await client.action.info("process-image")
|
|
@@ -649,6 +857,10 @@ Both workers register in the worker registry and heartbeat every 30 s, reporting
|
|
|
649
857
|
|
|
650
858
|
`worker.drain()` stops taking work and lets the running handlers finish, heartbeats going on; the server is told, so it assigns the worker nothing more, and `start()` returns once the last handler has reported. A drain started on the server does the same. Neither waits out `block_ms`: `drain()` interrupts the worker's own poll connection, so an action worker's held await and a stream worker's held read end at once. A drain started on the server answers an action worker's held await at once, refused as `WORKER_DRAINING`; it doesn't wake a stream worker's held group read, so a stream worker learns of it from its next heartbeat (every 30 s) and only then interrupts its read and leaves. `worker.stop()` ends polling at once, also mid-reconnect or mid-drain; running handlers still finish, and a stream worker hands back (abandons) the records of its last batch it hasn't started, as it does if its task is cancelled: they are ready again at once, and spend none of their attempts. A record whose handler is cancelled while the worker stops (or whose task the worker's cancellation cuts short) is abandoned the same way, so a rolling restart doesn't dead-letter it at `max_deliver` 1; a handler that raises, `CancelledError` included, while the worker isn't stopping is nacked. Either one before or during `start()` ends that run, and a worker can be started again afterwards.
|
|
651
859
|
|
|
860
|
+
`await worker.close()` stops the worker. close() waits up to 30 s (timeout=) for the running loop to send its completes, acks and hand-backs, then closes the client; anything unsent is redelivered when its lease expires.
|
|
861
|
+
|
|
862
|
+
A stream worker whose read is answered not found fails `start()` with `NotFoundError` rather than reading on as if the stream were empty. A worker that may start before its stream exists catches it and starts again.
|
|
863
|
+
|
|
652
864
|
### StreamWorker Example
|
|
653
865
|
|
|
654
866
|
```python
|
|
@@ -676,6 +888,15 @@ async def run_stream_worker():
|
|
|
676
888
|
asyncio.run(run_stream_worker())
|
|
677
889
|
```
|
|
678
890
|
|
|
891
|
+
A worker's complete, fail, ack or nack refused overloaded without being run
|
|
892
|
+
(once the client's own retries are spent) is sent again with a capped backoff
|
|
893
|
+
until the server takes it or answers otherwise, and logged each time. A
|
|
894
|
+
complete or fail that ends a run a workflow step called is refused so while
|
|
895
|
+
the server's queue of writes to the workflow's shard is full; meanwhile the
|
|
896
|
+
run's lease doesn't run out, and `action.status` says why in `held`. An ack's
|
|
897
|
+
or nack's lease keeps running, so a long overload can deliver the records
|
|
898
|
+
again. A stopping worker gives up on it after a couple of seconds.
|
|
899
|
+
|
|
679
900
|
A stream worker acks each record its handler returns from and nacks each one
|
|
680
901
|
its handler raises on, naming the read's delivery. A record whose handler runs
|
|
681
902
|
past the group's lease is read again elsewhere unless `auto_touch=True`, which
|
|
@@ -761,7 +982,7 @@ A call that times out (`RequestTimeoutError`, also an `asyncio.TimeoutError`) or
|
|
|
761
982
|
| `QueueFullError` | An `OverloadedError` with reason `QUEUE_FULL`: an enqueue found the queue, or its shard's queues and groups, at their caps, and the client's retries ran out |
|
|
762
983
|
| `GroupHeldFullError` | An `OverloadedError` with reason `GROUP_HELD_FULL`: a group read took nothing new because the group is at its cap (records out and dead together) or the shard's queues and groups are at theirs. Never retried |
|
|
763
984
|
| `UnavailableError` | No leader, or the shard isn't taking writes or is offline, and the client's retries ran out (the message says why, and whether an operator must act first) |
|
|
764
|
-
| `StaleNotFoundError` | `kv.get` or `kv.json_get` found no key,
|
|
985
|
+
| `StaleNotFoundError` | A `NotFoundError` from a node that may be behind (a halted or read-blocked shard, or a follower that couldn't reach the leader): `kv.get` or `kv.json_get` found no key, or `stream.read`, `ts.read` or `ts.iter_points` no stream or series, but it may exist. Read again rather than treat it as gone |
|
|
765
986
|
| `LimitError` | Status `LIMIT` (15): the node or shard is at one of its limits; `reason` says which (`TOO_MANY_ACTIONS`, `MEMORY_BUDGET`, `TOO_MANY_WORKFLOW_DEFINITIONS`, `TOO_MANY_WORKFLOW_RUNS`). It didn't run. Terminal, never retried (see below) |
|
|
766
987
|
| `OutcomePendingError` | The write committed but hasn't applied: its shard halted at or before it (reason `SHARD_HALTED`, ran `UNKNOWN`; the message says whether it applies when the shard resumes). Never retried |
|
|
767
988
|
| `InternalServerError` | Server internal error; not retryable (may mean committed but not applied) |
|