xmemory-temporal 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- xmemory_temporal-1.0.0/LICENSE +21 -0
- xmemory_temporal-1.0.0/MANIFEST.in +9 -0
- xmemory_temporal-1.0.0/NOTICE +27 -0
- xmemory_temporal-1.0.0/PKG-INFO +313 -0
- xmemory_temporal-1.0.0/README.md +293 -0
- xmemory_temporal-1.0.0/pyproject.toml +81 -0
- xmemory_temporal-1.0.0/setup.cfg +4 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal/__init__.py +61 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal/activities.py +186 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal/client_factory.py +43 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal/config.py +96 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal/dto.py +160 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal/errors.py +235 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal/interceptor.py +182 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal/plugin.py +89 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal/protocol.py +49 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal/py.typed +0 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal/workflow_api.py +320 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal.egg-info/PKG-INFO +313 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal.egg-info/SOURCES.txt +21 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal.egg-info/dependency_links.txt +1 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal.egg-info/requires.txt +2 -0
- xmemory_temporal-1.0.0/src/xmemory_temporal.egg-info/top_level.txt +1 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) xmemory Inc.
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
# The sdist is the package, not the suite. setuptools auto-includes files matching
|
|
2
|
+
# `test*`, which shipped `tests/test_*.py` without their conftest, fakes, and
|
|
3
|
+
# workflow fixtures -- an artifact whose tests could only fail to collect. The
|
|
4
|
+
# suite lives in git, where it has everything it needs.
|
|
5
|
+
prune tests
|
|
6
|
+
prune examples
|
|
7
|
+
prune .github
|
|
8
|
+
prune docs
|
|
9
|
+
exclude .gitignore
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
xmemory Temporal integration
|
|
2
|
+
Copyright (c) xmemory Inc.
|
|
3
|
+
|
|
4
|
+
This product is licensed under the MIT License (see LICENSE). The following
|
|
5
|
+
notices supplement — but do not modify — that license.
|
|
6
|
+
|
|
7
|
+
SCOPE. The MIT license covers only this Temporal integration — the source code
|
|
8
|
+
in this directory (the `xmemory-temporal` and `@xmemory/temporal` plugin
|
|
9
|
+
packages), which is a thin client over the xmemory API wired into Temporal.
|
|
10
|
+
|
|
11
|
+
PROPRIETARY SERVICE. The xmemory service that this integration talks to, and the
|
|
12
|
+
underlying technology behind it — including the xmemory backend, its memory
|
|
13
|
+
engine, schemas, extraction and reader models, APIs, and hosted
|
|
14
|
+
infrastructure — are proprietary to xmemory Inc. and are NOT licensed under the
|
|
15
|
+
MIT license. Nothing in this license grants any right to access, copy, reverse
|
|
16
|
+
engineer, reimplement, or create derivative works of the xmemory service or its
|
|
17
|
+
underlying technology. Use of the service is governed by the Terms & Conditions
|
|
18
|
+
at https://xmemory.ai/terms-and-conditions.html and the Privacy Policy at
|
|
19
|
+
https://xmemory.ai/privacy-policy.html, and requires valid credentials issued by
|
|
20
|
+
xmemory Inc.
|
|
21
|
+
|
|
22
|
+
TRADEMARKS. The "xmemory" name and logo are trademarks of xmemory Inc. The MIT
|
|
23
|
+
license does not grant permission to use them, except as required for reasonable
|
|
24
|
+
and customary use in describing the origin of the Software. "Temporal" is a
|
|
25
|
+
trademark of Temporal Technologies, Inc.; this is an independent, unofficial
|
|
26
|
+
integration and is not affiliated with, endorsed by, or sponsored by Temporal
|
|
27
|
+
Technologies, Inc.
|
|
@@ -0,0 +1,313 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: xmemory-temporal
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Temporal plugin for xmemory — durable agent memory as Temporal Activities
|
|
5
|
+
License-Expression: MIT
|
|
6
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
7
|
+
Classifier: Intended Audience :: Developers
|
|
8
|
+
Classifier: Programming Language :: Python :: 3
|
|
9
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
10
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
11
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
13
|
+
Requires-Python: >=3.10
|
|
14
|
+
Description-Content-Type: text/markdown
|
|
15
|
+
License-File: LICENSE
|
|
16
|
+
License-File: NOTICE
|
|
17
|
+
Requires-Dist: temporalio<2,>=1.30
|
|
18
|
+
Requires-Dist: xmemory-ai>=0.12
|
|
19
|
+
Dynamic: license-file
|
|
20
|
+
|
|
21
|
+
# xmemory-temporal
|
|
22
|
+
|
|
23
|
+
Durable agent memory for [Temporal](https://temporal.io) — add
|
|
24
|
+
[xmemory](https://xmemory.ai) reads and writes to your workflows as replay-safe
|
|
25
|
+
Temporal Activities, with one plugin line on your Worker.
|
|
26
|
+
|
|
27
|
+
> An agent's memory is exactly the state you don't want to lose when a worker
|
|
28
|
+
> crashes mid-turn. Putting xmemory behind Temporal makes a memory write a
|
|
29
|
+
> durable step: it survives process death, redeploys, and rolling upgrades, and
|
|
30
|
+
> Temporal — not your code — owns its retries and timeouts.
|
|
31
|
+
|
|
32
|
+
A TypeScript port with the same API ships as
|
|
33
|
+
[`@xmemory/temporal`](https://github.com/xmemory-ai/xmemory-temporal-ts).
|
|
34
|
+
|
|
35
|
+
## What you get
|
|
36
|
+
|
|
37
|
+
- **Memory as Activities.** `read`, `write`, `write_async` + `write_status` run
|
|
38
|
+
as Activities (all I/O stays out of workflow code, so workflows replay
|
|
39
|
+
deterministically).
|
|
40
|
+
- **A durable deep write.** `write_durable(text)` enqueues a write and polls it
|
|
41
|
+
to completion from the workflow, so a multi-minute extraction survives worker
|
|
42
|
+
restarts — the poll state lives in workflow history, not a worker process.
|
|
43
|
+
- **A near-zero-diff migration.** The workflow-side handle mirrors the plain
|
|
44
|
+
xmemory client's methods, so agent code that already calls `inst.read(...)` /
|
|
45
|
+
`inst.write(...)` keeps working — it just dispatches to an Activity.
|
|
46
|
+
- **Temporal-owned retries and timeouts.** xmemory errors map to typed
|
|
47
|
+
`ApplicationError`s with retryable/non-retryable verdicts, so you can tune
|
|
48
|
+
`RetryPolicy` against stable error-type strings.
|
|
49
|
+
- **Opt-in auto-capture** of activity results into memory, via an Activity
|
|
50
|
+
interceptor that never touches the replay path.
|
|
51
|
+
|
|
52
|
+
## Install
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
pip install xmemory-temporal
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Requires Python 3.10+ and `temporalio` 1.30+.
|
|
59
|
+
|
|
60
|
+
## Quickstart
|
|
61
|
+
|
|
62
|
+
Register the plugin on your **Client**; the Worker inherits it automatically:
|
|
63
|
+
|
|
64
|
+
```python
|
|
65
|
+
from temporalio.client import Client
|
|
66
|
+
from temporalio.worker import Worker
|
|
67
|
+
from xmemory_temporal import XmemoryConfig, XmemoryPlugin
|
|
68
|
+
|
|
69
|
+
config = XmemoryConfig(instance_id="<your-instance-id>") # reads XMEM_API_KEY from the env
|
|
70
|
+
plugin = XmemoryPlugin(config)
|
|
71
|
+
|
|
72
|
+
client = await Client.connect("localhost:7233", plugins=[plugin])
|
|
73
|
+
# The Worker inherits the client's plugins automatically — do NOT pass it again here.
|
|
74
|
+
worker = Worker(client, task_queue="my-agent", workflows=[MyWorkflow])
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Then call memory from inside a workflow:
|
|
78
|
+
|
|
79
|
+
```python
|
|
80
|
+
from temporalio import workflow
|
|
81
|
+
|
|
82
|
+
with workflow.unsafe.imports_passed_through():
|
|
83
|
+
from xmemory_temporal import xmemory_for_workflow
|
|
84
|
+
|
|
85
|
+
@workflow.defn
|
|
86
|
+
class MyWorkflow:
|
|
87
|
+
@workflow.run
|
|
88
|
+
async def run(self, user_name: str, user_message: str) -> str:
|
|
89
|
+
mem = xmemory_for_workflow()
|
|
90
|
+
# A memory store has no ambient "current user" — name whom the fact is
|
|
91
|
+
# about, then recall by that name (or pass scope= to bind to a record).
|
|
92
|
+
await mem.write_durable(f"{user_name}: {user_message}") # durable, survives restarts
|
|
93
|
+
answer = await mem.read(f"what do we know about {user_name}?")
|
|
94
|
+
return str(answer.reader_result) # reader_result is Any
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
> **Register the plugin once, never twice.** Put it on the Client
|
|
98
|
+
> (`Client.connect(plugins=[plugin])`); the Worker inherits its client's plugins,
|
|
99
|
+
> so do *not* also pass it to `Worker(...)`, which registers the activities twice
|
|
100
|
+
> and fails with "More than one activity named xmemory_read". (Worker-only also
|
|
101
|
+
> works; just never both.)
|
|
102
|
+
|
|
103
|
+
Runnable end-to-end scripts live in [`examples/`](./examples): create an instance
|
|
104
|
+
with a schema, run a worker, and drive a support-agent workflow. They call
|
|
105
|
+
`worker.run()` directly to stay readable. In production, install SIGINT/SIGTERM
|
|
106
|
+
handlers so a deploy drains the worker instead of killing it mid-activity; see
|
|
107
|
+
[Temporal's worker shutdown guidance](https://docs.temporal.io/encyclopedia/workers/worker-shutdown#graceful-shutdown).
|
|
108
|
+
|
|
109
|
+
## Timeouts
|
|
110
|
+
|
|
111
|
+
**The workflow owns every activity budget.** `xmemory_for_workflow()` sets each
|
|
112
|
+
call's `start_to_close_timeout`, and the activity derives its xmemory client
|
|
113
|
+
timeout from the deadline Temporal actually assigned it, always a margin below,
|
|
114
|
+
so the client gives up first and you get an attributable xmemory error instead of
|
|
115
|
+
an opaque Temporal activity timeout.
|
|
116
|
+
|
|
117
|
+
```python
|
|
118
|
+
from datetime import timedelta
|
|
119
|
+
|
|
120
|
+
mem = xmemory_for_workflow(
|
|
121
|
+
read_timeout=timedelta(seconds=60), # a deep read on a large instance
|
|
122
|
+
write_timeout=timedelta(minutes=5),
|
|
123
|
+
)
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Because the client timeout is *derived* rather than configured separately, the
|
|
127
|
+
two can never disagree: lowering a workflow's budget lowers the client's with it.
|
|
128
|
+
`XmemoryTimeouts` supplies the defaults for any budget you do not set;
|
|
129
|
+
`XmemoryConfig(client_margin_seconds=...)` tunes the gap between the two.
|
|
130
|
+
|
|
131
|
+
## Durable writes
|
|
132
|
+
|
|
133
|
+
`write_durable(text)` enqueues a deep write and polls it to completion from the
|
|
134
|
+
workflow, so the wait is a Temporal timer in server-side history rather than a
|
|
135
|
+
blocked activity slot. Redeploy the worker fleet mid-write and nothing is lost:
|
|
136
|
+
the poll loop resumes on the new worker and completes.
|
|
137
|
+
|
|
138
|
+
```python
|
|
139
|
+
status = await mem.write_durable(text, max_wait=timedelta(minutes=15))
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Each poll adds an activity and a timer to workflow history: roughly 105 events at
|
|
143
|
+
the defaults (15 minutes), about 1,500 for a four-hour wait. A long `max_wait`
|
|
144
|
+
with a short `max_poll_interval` can approach Temporal's per-workflow event
|
|
145
|
+
limit, and the loop logs a warning once Temporal itself suggests continuing as
|
|
146
|
+
new. This helper cannot call `continue_as_new` for you, since it runs inside
|
|
147
|
+
*your* workflow and restarting that would discard your state. For multi-hour
|
|
148
|
+
waits, run `write_durable` in a child workflow, where continue-as-new is yours to
|
|
149
|
+
use.
|
|
150
|
+
|
|
151
|
+
For the fire-and-forget pattern (kick off several writes, keep working, join
|
|
152
|
+
before the turn ends), `write_async_start()` and `write_status()` are public too.
|
|
153
|
+
|
|
154
|
+
## Credentials never reach workflow history
|
|
155
|
+
|
|
156
|
+
The config holds the **name** of the environment variable that supplies the API
|
|
157
|
+
key (`XMEM_API_KEY` by default), never the key itself — so nothing secret is ever
|
|
158
|
+
serialized into activity arguments, which Temporal persists in the clear. Pass
|
|
159
|
+
the key in-process instead with `XmemoryPlugin(config, api_key=...)` if you
|
|
160
|
+
prefer.
|
|
161
|
+
|
|
162
|
+
**Your memory text and queries, however, *are* in history.** The query you `read`
|
|
163
|
+
and the text you `write` are activity inputs, and the error mapping keeps raw
|
|
164
|
+
transport strings out of failure *messages* (a failed durable write carries the
|
|
165
|
+
server's reason in the error `details`, not the cleartext history title) — but the
|
|
166
|
+
inputs themselves, and the `reader_result`, are persisted to cleartext Temporal
|
|
167
|
+
history and shown in the Web UI. `include_content_in_summary=False` (the default) only keeps content out of
|
|
168
|
+
the one-line activity *summary*; it does not remove it from the payload. If your
|
|
169
|
+
memory text is sensitive, install a Temporal **Payload Codec** to encrypt
|
|
170
|
+
payloads at the edge — this plugin deliberately does not impose one, since a
|
|
171
|
+
codec applies namespace-wide to every payload, not just xmemory's.
|
|
172
|
+
|
|
173
|
+
## Replay safety and idempotency
|
|
174
|
+
|
|
175
|
+
Two things keep memory operations correct under retries and replay:
|
|
176
|
+
|
|
177
|
+
- **Replay never re-issues an operation.** All I/O is in Activities; workflow
|
|
178
|
+
code only schedules Activities and sleeps. Temporal replays workflow code but
|
|
179
|
+
never re-runs a completed Activity, so a replay never repeats a memory read or
|
|
180
|
+
write. The suite proves this with a forced-replay (`max_cached_workflows=0`)
|
|
181
|
+
side-effects test.
|
|
182
|
+
- **Writes default to at-most-once.** It is tempting to lean on xmemory's
|
|
183
|
+
primary-key dedup to make retries safe — a re-write of the same fact should
|
|
184
|
+
update the same record. But **PK extraction is non-deterministic**: xmemory
|
|
185
|
+
authors primary keys with a model that can normalize the same value differently
|
|
186
|
+
across runs (e.g. `Dr. Robert Kim` vs `Robert Kim`), and a disagreement forks
|
|
187
|
+
the entity into a **new** row. So a lost-response retry can duplicate. Rather
|
|
188
|
+
than risk that silently, write Activities default to `maximum_attempts=1`: a
|
|
189
|
+
failed write is surfaced to your workflow, which decides to retry, compensate,
|
|
190
|
+
or fail. Reads and status-polls (idempotent) retry generously.
|
|
191
|
+
|
|
192
|
+
**Structured writes are the reliable way to make a write retryable.** Pass
|
|
193
|
+
explicit mutations instead of free text and the primary key is one you supply, so
|
|
194
|
+
nothing is extracted and re-applying the write is deterministic:
|
|
195
|
+
|
|
196
|
+
```python
|
|
197
|
+
mem = xmemory_for_workflow(write_retry_policy=RetryPolicy(maximum_attempts=3))
|
|
198
|
+
await mem.write(
|
|
199
|
+
structured_mutations=[
|
|
200
|
+
{
|
|
201
|
+
"object_mutation": {
|
|
202
|
+
"object_type": "Customer",
|
|
203
|
+
"update": {"key": {"customer_id": "c-1"}, "values": {"tier": "gold"}},
|
|
204
|
+
}
|
|
205
|
+
}
|
|
206
|
+
]
|
|
207
|
+
)
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
A mutation is a `create`, `update`, or `delete` on one object or relation, and
|
|
211
|
+
it carries the key explicitly, so a retry addresses the same row instead of
|
|
212
|
+
forking a new one. An `update` in particular re-applies identically.
|
|
213
|
+
|
|
214
|
+
For text writes, opt into retries only when your primary keys are literal
|
|
215
|
+
identifiers that appear verbatim in the text, such as a `customer_id` you supply,
|
|
216
|
+
so the extractor has no room to normalise them differently on a second pass. That
|
|
217
|
+
is a convention you have to keep, not something the API enforces.
|
|
218
|
+
|
|
219
|
+
**Scoped writes**, which xmemory is adding in the near future, will bind a text
|
|
220
|
+
write to a known record and guarantee a stable primary key, closing the gap for
|
|
221
|
+
text writes too.
|
|
222
|
+
|
|
223
|
+
[`examples/setup_memory.py`](./examples/setup_memory.py) shows creating an
|
|
224
|
+
instance with a schema.
|
|
225
|
+
|
|
226
|
+
## Error handling
|
|
227
|
+
|
|
228
|
+
xmemory errors become `ApplicationError`s with stable `type` strings you can
|
|
229
|
+
match in a `RetryPolicy` (`non_retryable_error_types=[...]`). The mapping is
|
|
230
|
+
derived from the server's error codes:
|
|
231
|
+
|
|
232
|
+
| xmemory condition | `type` | Retryable? |
|
|
233
|
+
|---|---|---|
|
|
234
|
+
| transport error / timeout / HTTP ≥ 500 / 408 | `XmemoryServerError` / `XmemoryUnavailable` | yes |
|
|
235
|
+
| `RATE_LIMITED` (429) | `XmemoryRateLimited` | yes — honors `Retry-After` |
|
|
236
|
+
| `QUOTA_EXCEEDED` + `daily_quota_exceeded` | `XmemoryDailyQuotaExceeded` | yes (long backoff) |
|
|
237
|
+
| `QUOTA_EXCEEDED` + `monthly_quota_exceeded` | `XmemoryMonthlyQuotaExceeded` | no |
|
|
238
|
+
| `QUOTA_EXCEEDED` (kind unknown) | `XmemoryQuotaExceeded` | no |
|
|
239
|
+
| `UNAUTHORIZED` / `FORBIDDEN` | `XmemoryAuthFailed` | no |
|
|
240
|
+
| `NOT_FOUND` | `XmemoryNotFound` | no |
|
|
241
|
+
| validation / conflict / schema-evolution rejections | `XmemoryBadRequest` / `XmemorySchemaRejected` | no |
|
|
242
|
+
| activities registered without the plugin | `XmemoryNotBound` | no |
|
|
243
|
+
| an activity scheduled with neither close timeout | `XmemoryNoDeadline` | no |
|
|
244
|
+
| an unrecognized code | `XmemoryUnknown` | yes (never fatal) |
|
|
245
|
+
|
|
246
|
+
Plus three raised by the durable write loop (`write_durable`), from a polled
|
|
247
|
+
`write_status` — all non-retryable:
|
|
248
|
+
|
|
249
|
+
| durable-write outcome | `type` |
|
|
250
|
+
|---|---|
|
|
251
|
+
| the queued write reported `failed` | `XmemoryWriteFailed` |
|
|
252
|
+
| the queued write id was `not_found` | `XmemoryWriteNotFound` |
|
|
253
|
+
| polling exceeded `max_wait` | `XmemoryWriteTimeout` |
|
|
254
|
+
|
|
255
|
+
An unrecognized error code stays retryable and never raises — a stricter client
|
|
256
|
+
that crashed on a newer server's code would break during rolling deploys.
|
|
257
|
+
|
|
258
|
+
> **Note.** 402 means `QUOTA_EXCEEDED` only. `TRIAL_ENDED` was removed from the
|
|
259
|
+
> xmemory contract when trials were retired end-to-end; do not rely on it.
|
|
260
|
+
|
|
261
|
+
## Auto-capture (opt-in)
|
|
262
|
+
|
|
263
|
+
```python
|
|
264
|
+
from xmemory_temporal import AutoCaptureConfig, XmemoryPlugin
|
|
265
|
+
|
|
266
|
+
plugin = XmemoryPlugin(
|
|
267
|
+
config,
|
|
268
|
+
auto_capture=AutoCaptureConfig(
|
|
269
|
+
project=lambda activity_name, result: summarize(result), # return None to skip
|
|
270
|
+
sample_rate=0.25,
|
|
271
|
+
),
|
|
272
|
+
)
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
Off by default. It runs as an **Activity** interceptor (outside the replay
|
|
276
|
+
path), requires a `project` function that decides what — if anything — to
|
|
277
|
+
remember, samples to bound fan-out, and never fails the wrapped activity if a
|
|
278
|
+
capture write errors. Capture is an **enqueue** (`write_async`), and because it
|
|
279
|
+
runs inside the wrapped activity it is clamped to whatever that activity has
|
|
280
|
+
left of its own deadline — and skipped outright when nothing is left — so it
|
|
281
|
+
cannot push the activity past its `start_to_close` and get it retried.
|
|
282
|
+
|
|
283
|
+
> **Naming caveat.** Auto-capture skips any activity whose name starts with
|
|
284
|
+
> `xmemory_` (to avoid capturing its own writes). If you name one of *your* own
|
|
285
|
+
> activities `xmemory_...`, it will be silently skipped. It also never captures
|
|
286
|
+
> Queries.
|
|
287
|
+
|
|
288
|
+
## Testing
|
|
289
|
+
|
|
290
|
+
```bash
|
|
291
|
+
uv sync --dev
|
|
292
|
+
uv run pytest # everything except the live e2e (it self-skips)
|
|
293
|
+
uv run ruff check src tests examples
|
|
294
|
+
uv run pyright src tests examples
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
The suite runs with no live backend (a fake instance is injected), except a
|
|
298
|
+
`live`-marked end-to-end test that needs `XMEM_API_KEY` + `XMEM_INSTANCE_ID`.
|
|
299
|
+
See [`TESTING.md`](./TESTING.md) for the full strategy.
|
|
300
|
+
|
|
301
|
+
## Legal
|
|
302
|
+
|
|
303
|
+
- Privacy policy: <https://xmemory.ai/privacy-policy.html>
|
|
304
|
+
- Terms: <https://xmemory.ai/terms-and-conditions.html>
|
|
305
|
+
|
|
306
|
+
**MIT licensed** — see [`LICENSE`](./LICENSE). The MIT grant covers only this
|
|
307
|
+
integration's own code (a thin client over the xmemory API). The xmemory service
|
|
308
|
+
and its underlying technology — the backend, memory engine, schemas,
|
|
309
|
+
extraction/reader models, and hosted infrastructure — remain **proprietary to
|
|
310
|
+
xmemory Inc.** and are not licensed here; use of the service requires valid
|
|
311
|
+
credentials and is governed by the Terms above. These supplemental scope /
|
|
312
|
+
proprietary-service / trademark notices live in [`NOTICE`](./NOTICE), kept
|
|
313
|
+
separate from `LICENSE` so the package classifies cleanly as MIT.
|
|
@@ -0,0 +1,293 @@
|
|
|
1
|
+
# xmemory-temporal
|
|
2
|
+
|
|
3
|
+
Durable agent memory for [Temporal](https://temporal.io) — add
|
|
4
|
+
[xmemory](https://xmemory.ai) reads and writes to your workflows as replay-safe
|
|
5
|
+
Temporal Activities, with one plugin line on your Worker.
|
|
6
|
+
|
|
7
|
+
> An agent's memory is exactly the state you don't want to lose when a worker
|
|
8
|
+
> crashes mid-turn. Putting xmemory behind Temporal makes a memory write a
|
|
9
|
+
> durable step: it survives process death, redeploys, and rolling upgrades, and
|
|
10
|
+
> Temporal — not your code — owns its retries and timeouts.
|
|
11
|
+
|
|
12
|
+
A TypeScript port with the same API ships as
|
|
13
|
+
[`@xmemory/temporal`](https://github.com/xmemory-ai/xmemory-temporal-ts).
|
|
14
|
+
|
|
15
|
+
## What you get
|
|
16
|
+
|
|
17
|
+
- **Memory as Activities.** `read`, `write`, `write_async` + `write_status` run
|
|
18
|
+
as Activities (all I/O stays out of workflow code, so workflows replay
|
|
19
|
+
deterministically).
|
|
20
|
+
- **A durable deep write.** `write_durable(text)` enqueues a write and polls it
|
|
21
|
+
to completion from the workflow, so a multi-minute extraction survives worker
|
|
22
|
+
restarts — the poll state lives in workflow history, not a worker process.
|
|
23
|
+
- **A near-zero-diff migration.** The workflow-side handle mirrors the plain
|
|
24
|
+
xmemory client's methods, so agent code that already calls `inst.read(...)` /
|
|
25
|
+
`inst.write(...)` keeps working — it just dispatches to an Activity.
|
|
26
|
+
- **Temporal-owned retries and timeouts.** xmemory errors map to typed
|
|
27
|
+
`ApplicationError`s with retryable/non-retryable verdicts, so you can tune
|
|
28
|
+
`RetryPolicy` against stable error-type strings.
|
|
29
|
+
- **Opt-in auto-capture** of activity results into memory, via an Activity
|
|
30
|
+
interceptor that never touches the replay path.
|
|
31
|
+
|
|
32
|
+
## Install
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
pip install xmemory-temporal
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Requires Python 3.10+ and `temporalio` 1.30+.
|
|
39
|
+
|
|
40
|
+
## Quickstart
|
|
41
|
+
|
|
42
|
+
Register the plugin on your **Client**; the Worker inherits it automatically:
|
|
43
|
+
|
|
44
|
+
```python
|
|
45
|
+
from temporalio.client import Client
|
|
46
|
+
from temporalio.worker import Worker
|
|
47
|
+
from xmemory_temporal import XmemoryConfig, XmemoryPlugin
|
|
48
|
+
|
|
49
|
+
config = XmemoryConfig(instance_id="<your-instance-id>") # reads XMEM_API_KEY from the env
|
|
50
|
+
plugin = XmemoryPlugin(config)
|
|
51
|
+
|
|
52
|
+
client = await Client.connect("localhost:7233", plugins=[plugin])
|
|
53
|
+
# The Worker inherits the client's plugins automatically — do NOT pass it again here.
|
|
54
|
+
worker = Worker(client, task_queue="my-agent", workflows=[MyWorkflow])
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Then call memory from inside a workflow:
|
|
58
|
+
|
|
59
|
+
```python
|
|
60
|
+
from temporalio import workflow
|
|
61
|
+
|
|
62
|
+
with workflow.unsafe.imports_passed_through():
|
|
63
|
+
from xmemory_temporal import xmemory_for_workflow
|
|
64
|
+
|
|
65
|
+
@workflow.defn
|
|
66
|
+
class MyWorkflow:
|
|
67
|
+
@workflow.run
|
|
68
|
+
async def run(self, user_name: str, user_message: str) -> str:
|
|
69
|
+
mem = xmemory_for_workflow()
|
|
70
|
+
# A memory store has no ambient "current user" — name whom the fact is
|
|
71
|
+
# about, then recall by that name (or pass scope= to bind to a record).
|
|
72
|
+
await mem.write_durable(f"{user_name}: {user_message}") # durable, survives restarts
|
|
73
|
+
answer = await mem.read(f"what do we know about {user_name}?")
|
|
74
|
+
return str(answer.reader_result) # reader_result is Any
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
> **Register the plugin once, never twice.** Put it on the Client
|
|
78
|
+
> (`Client.connect(plugins=[plugin])`); the Worker inherits its client's plugins,
|
|
79
|
+
> so do *not* also pass it to `Worker(...)`, which registers the activities twice
|
|
80
|
+
> and fails with "More than one activity named xmemory_read". (Worker-only also
|
|
81
|
+
> works; just never both.)
|
|
82
|
+
|
|
83
|
+
Runnable end-to-end scripts live in [`examples/`](./examples): create an instance
|
|
84
|
+
with a schema, run a worker, and drive a support-agent workflow. They call
|
|
85
|
+
`worker.run()` directly to stay readable. In production, install SIGINT/SIGTERM
|
|
86
|
+
handlers so a deploy drains the worker instead of killing it mid-activity; see
|
|
87
|
+
[Temporal's worker shutdown guidance](https://docs.temporal.io/encyclopedia/workers/worker-shutdown#graceful-shutdown).
|
|
88
|
+
|
|
89
|
+
## Timeouts
|
|
90
|
+
|
|
91
|
+
**The workflow owns every activity budget.** `xmemory_for_workflow()` sets each
|
|
92
|
+
call's `start_to_close_timeout`, and the activity derives its xmemory client
|
|
93
|
+
timeout from the deadline Temporal actually assigned it, always a margin below,
|
|
94
|
+
so the client gives up first and you get an attributable xmemory error instead of
|
|
95
|
+
an opaque Temporal activity timeout.
|
|
96
|
+
|
|
97
|
+
```python
|
|
98
|
+
from datetime import timedelta
|
|
99
|
+
|
|
100
|
+
mem = xmemory_for_workflow(
|
|
101
|
+
read_timeout=timedelta(seconds=60), # a deep read on a large instance
|
|
102
|
+
write_timeout=timedelta(minutes=5),
|
|
103
|
+
)
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Because the client timeout is *derived* rather than configured separately, the
|
|
107
|
+
two can never disagree: lowering a workflow's budget lowers the client's with it.
|
|
108
|
+
`XmemoryTimeouts` supplies the defaults for any budget you do not set;
|
|
109
|
+
`XmemoryConfig(client_margin_seconds=...)` tunes the gap between the two.
|
|
110
|
+
|
|
111
|
+
## Durable writes
|
|
112
|
+
|
|
113
|
+
`write_durable(text)` enqueues a deep write and polls it to completion from the
|
|
114
|
+
workflow, so the wait is a Temporal timer in server-side history rather than a
|
|
115
|
+
blocked activity slot. Redeploy the worker fleet mid-write and nothing is lost:
|
|
116
|
+
the poll loop resumes on the new worker and completes.
|
|
117
|
+
|
|
118
|
+
```python
|
|
119
|
+
status = await mem.write_durable(text, max_wait=timedelta(minutes=15))
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
Each poll adds an activity and a timer to workflow history: roughly 105 events at
|
|
123
|
+
the defaults (15 minutes), about 1,500 for a four-hour wait. A long `max_wait`
|
|
124
|
+
with a short `max_poll_interval` can approach Temporal's per-workflow event
|
|
125
|
+
limit, and the loop logs a warning once Temporal itself suggests continuing as
|
|
126
|
+
new. This helper cannot call `continue_as_new` for you, since it runs inside
|
|
127
|
+
*your* workflow and restarting that would discard your state. For multi-hour
|
|
128
|
+
waits, run `write_durable` in a child workflow, where continue-as-new is yours to
|
|
129
|
+
use.
|
|
130
|
+
|
|
131
|
+
For the fire-and-forget pattern (kick off several writes, keep working, join
|
|
132
|
+
before the turn ends), `write_async_start()` and `write_status()` are public too.
|
|
133
|
+
|
|
134
|
+
## Credentials never reach workflow history
|
|
135
|
+
|
|
136
|
+
The config holds the **name** of the environment variable that supplies the API
|
|
137
|
+
key (`XMEM_API_KEY` by default), never the key itself — so nothing secret is ever
|
|
138
|
+
serialized into activity arguments, which Temporal persists in the clear. Pass
|
|
139
|
+
the key in-process instead with `XmemoryPlugin(config, api_key=...)` if you
|
|
140
|
+
prefer.
|
|
141
|
+
|
|
142
|
+
**Your memory text and queries, however, *are* in history.** The query you `read`
|
|
143
|
+
and the text you `write` are activity inputs, and the error mapping keeps raw
|
|
144
|
+
transport strings out of failure *messages* (a failed durable write carries the
|
|
145
|
+
server's reason in the error `details`, not the cleartext history title) — but the
|
|
146
|
+
inputs themselves, and the `reader_result`, are persisted to cleartext Temporal
|
|
147
|
+
history and shown in the Web UI. `include_content_in_summary=False` (the default) only keeps content out of
|
|
148
|
+
the one-line activity *summary*; it does not remove it from the payload. If your
|
|
149
|
+
memory text is sensitive, install a Temporal **Payload Codec** to encrypt
|
|
150
|
+
payloads at the edge — this plugin deliberately does not impose one, since a
|
|
151
|
+
codec applies namespace-wide to every payload, not just xmemory's.
|
|
152
|
+
|
|
153
|
+
## Replay safety and idempotency
|
|
154
|
+
|
|
155
|
+
Two things keep memory operations correct under retries and replay:
|
|
156
|
+
|
|
157
|
+
- **Replay never re-issues an operation.** All I/O is in Activities; workflow
|
|
158
|
+
code only schedules Activities and sleeps. Temporal replays workflow code but
|
|
159
|
+
never re-runs a completed Activity, so a replay never repeats a memory read or
|
|
160
|
+
write. The suite proves this with a forced-replay (`max_cached_workflows=0`)
|
|
161
|
+
side-effects test.
|
|
162
|
+
- **Writes default to at-most-once.** It is tempting to lean on xmemory's
|
|
163
|
+
primary-key dedup to make retries safe — a re-write of the same fact should
|
|
164
|
+
update the same record. But **PK extraction is non-deterministic**: xmemory
|
|
165
|
+
authors primary keys with a model that can normalize the same value differently
|
|
166
|
+
across runs (e.g. `Dr. Robert Kim` vs `Robert Kim`), and a disagreement forks
|
|
167
|
+
the entity into a **new** row. So a lost-response retry can duplicate. Rather
|
|
168
|
+
than risk that silently, write Activities default to `maximum_attempts=1`: a
|
|
169
|
+
failed write is surfaced to your workflow, which decides to retry, compensate,
|
|
170
|
+
or fail. Reads and status-polls (idempotent) retry generously.
|
|
171
|
+
|
|
172
|
+
**Structured writes are the reliable way to make a write retryable.** Pass
|
|
173
|
+
explicit mutations instead of free text and the primary key is one you supply, so
|
|
174
|
+
nothing is extracted and re-applying the write is deterministic:
|
|
175
|
+
|
|
176
|
+
```python
|
|
177
|
+
mem = xmemory_for_workflow(write_retry_policy=RetryPolicy(maximum_attempts=3))
|
|
178
|
+
await mem.write(
|
|
179
|
+
structured_mutations=[
|
|
180
|
+
{
|
|
181
|
+
"object_mutation": {
|
|
182
|
+
"object_type": "Customer",
|
|
183
|
+
"update": {"key": {"customer_id": "c-1"}, "values": {"tier": "gold"}},
|
|
184
|
+
}
|
|
185
|
+
}
|
|
186
|
+
]
|
|
187
|
+
)
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
A mutation is a `create`, `update`, or `delete` on one object or relation, and
|
|
191
|
+
it carries the key explicitly, so a retry addresses the same row instead of
|
|
192
|
+
forking a new one. An `update` in particular re-applies identically.
|
|
193
|
+
|
|
194
|
+
For text writes, opt into retries only when your primary keys are literal
|
|
195
|
+
identifiers that appear verbatim in the text, such as a `customer_id` you supply,
|
|
196
|
+
so the extractor has no room to normalise them differently on a second pass. That
|
|
197
|
+
is a convention you have to keep, not something the API enforces.
|
|
198
|
+
|
|
199
|
+
**Scoped writes**, which xmemory is adding in the near future, will bind a text
|
|
200
|
+
write to a known record and guarantee a stable primary key, closing the gap for
|
|
201
|
+
text writes too.
|
|
202
|
+
|
|
203
|
+
[`examples/setup_memory.py`](./examples/setup_memory.py) shows creating an
|
|
204
|
+
instance with a schema.
|
|
205
|
+
|
|
206
|
+
## Error handling
|
|
207
|
+
|
|
208
|
+
xmemory errors become `ApplicationError`s with stable `type` strings you can
|
|
209
|
+
match in a `RetryPolicy` (`non_retryable_error_types=[...]`). The mapping is
|
|
210
|
+
derived from the server's error codes:
|
|
211
|
+
|
|
212
|
+
| xmemory condition | `type` | Retryable? |
|
|
213
|
+
|---|---|---|
|
|
214
|
+
| transport error / timeout / HTTP ≥ 500 / 408 | `XmemoryServerError` / `XmemoryUnavailable` | yes |
|
|
215
|
+
| `RATE_LIMITED` (429) | `XmemoryRateLimited` | yes — honors `Retry-After` |
|
|
216
|
+
| `QUOTA_EXCEEDED` + `daily_quota_exceeded` | `XmemoryDailyQuotaExceeded` | yes (long backoff) |
|
|
217
|
+
| `QUOTA_EXCEEDED` + `monthly_quota_exceeded` | `XmemoryMonthlyQuotaExceeded` | no |
|
|
218
|
+
| `QUOTA_EXCEEDED` (kind unknown) | `XmemoryQuotaExceeded` | no |
|
|
219
|
+
| `UNAUTHORIZED` / `FORBIDDEN` | `XmemoryAuthFailed` | no |
|
|
220
|
+
| `NOT_FOUND` | `XmemoryNotFound` | no |
|
|
221
|
+
| validation / conflict / schema-evolution rejections | `XmemoryBadRequest` / `XmemorySchemaRejected` | no |
|
|
222
|
+
| activities registered without the plugin | `XmemoryNotBound` | no |
|
|
223
|
+
| an activity scheduled with neither close timeout | `XmemoryNoDeadline` | no |
|
|
224
|
+
| an unrecognized code | `XmemoryUnknown` | yes (never fatal) |
|
|
225
|
+
|
|
226
|
+
Plus three raised by the durable write loop (`write_durable`), from a polled
|
|
227
|
+
`write_status` — all non-retryable:
|
|
228
|
+
|
|
229
|
+
| durable-write outcome | `type` |
|
|
230
|
+
|---|---|
|
|
231
|
+
| the queued write reported `failed` | `XmemoryWriteFailed` |
|
|
232
|
+
| the queued write id was `not_found` | `XmemoryWriteNotFound` |
|
|
233
|
+
| polling exceeded `max_wait` | `XmemoryWriteTimeout` |
|
|
234
|
+
|
|
235
|
+
An unrecognized error code stays retryable and never raises — a stricter client
|
|
236
|
+
that crashed on a newer server's code would break during rolling deploys.
|
|
237
|
+
|
|
238
|
+
> **Note.** 402 means `QUOTA_EXCEEDED` only. `TRIAL_ENDED` was removed from the
|
|
239
|
+
> xmemory contract when trials were retired end-to-end; do not rely on it.
|
|
240
|
+
|
|
241
|
+
## Auto-capture (opt-in)
|
|
242
|
+
|
|
243
|
+
```python
|
|
244
|
+
from xmemory_temporal import AutoCaptureConfig, XmemoryPlugin
|
|
245
|
+
|
|
246
|
+
plugin = XmemoryPlugin(
|
|
247
|
+
config,
|
|
248
|
+
auto_capture=AutoCaptureConfig(
|
|
249
|
+
project=lambda activity_name, result: summarize(result), # return None to skip
|
|
250
|
+
sample_rate=0.25,
|
|
251
|
+
),
|
|
252
|
+
)
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
Off by default. It runs as an **Activity** interceptor (outside the replay
|
|
256
|
+
path), requires a `project` function that decides what — if anything — to
|
|
257
|
+
remember, samples to bound fan-out, and never fails the wrapped activity if a
|
|
258
|
+
capture write errors. Capture is an **enqueue** (`write_async`), and because it
|
|
259
|
+
runs inside the wrapped activity it is clamped to whatever that activity has
|
|
260
|
+
left of its own deadline — and skipped outright when nothing is left — so it
|
|
261
|
+
cannot push the activity past its `start_to_close` and get it retried.
|
|
262
|
+
|
|
263
|
+
> **Naming caveat.** Auto-capture skips any activity whose name starts with
|
|
264
|
+
> `xmemory_` (to avoid capturing its own writes). If you name one of *your* own
|
|
265
|
+
> activities `xmemory_...`, it will be silently skipped. It also never captures
|
|
266
|
+
> Queries.
|
|
267
|
+
|
|
268
|
+
## Testing
|
|
269
|
+
|
|
270
|
+
```bash
|
|
271
|
+
uv sync --dev
|
|
272
|
+
uv run pytest # everything except the live e2e (it self-skips)
|
|
273
|
+
uv run ruff check src tests examples
|
|
274
|
+
uv run pyright src tests examples
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
The suite runs with no live backend (a fake instance is injected), except a
|
|
278
|
+
`live`-marked end-to-end test that needs `XMEM_API_KEY` + `XMEM_INSTANCE_ID`.
|
|
279
|
+
See [`TESTING.md`](./TESTING.md) for the full strategy.
|
|
280
|
+
|
|
281
|
+
## Legal
|
|
282
|
+
|
|
283
|
+
- Privacy policy: <https://xmemory.ai/privacy-policy.html>
|
|
284
|
+
- Terms: <https://xmemory.ai/terms-and-conditions.html>
|
|
285
|
+
|
|
286
|
+
**MIT licensed** — see [`LICENSE`](./LICENSE). The MIT grant covers only this
|
|
287
|
+
integration's own code (a thin client over the xmemory API). The xmemory service
|
|
288
|
+
and its underlying technology — the backend, memory engine, schemas,
|
|
289
|
+
extraction/reader models, and hosted infrastructure — remain **proprietary to
|
|
290
|
+
xmemory Inc.** and are not licensed here; use of the service requires valid
|
|
291
|
+
credentials and is governed by the Terms above. These supplemental scope /
|
|
292
|
+
proprietary-service / trademark notices live in [`NOTICE`](./NOTICE), kept
|
|
293
|
+
separate from `LICENSE` so the package classifies cleanly as MIT.
|