pipecat-memcode 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- pipecat_memcode-0.1.0/CHANGELOG.md +18 -0
- pipecat_memcode-0.1.0/LICENSE +25 -0
- pipecat_memcode-0.1.0/MANIFEST.in +5 -0
- pipecat_memcode-0.1.0/PKG-INFO +309 -0
- pipecat_memcode-0.1.0/README.md +281 -0
- pipecat_memcode-0.1.0/examples/foundational/memcode_memory.py +311 -0
- pipecat_memcode-0.1.0/pyproject.toml +64 -0
- pipecat_memcode-0.1.0/setup.cfg +4 -0
- pipecat_memcode-0.1.0/src/pipecat_memcode/__init__.py +19 -0
- pipecat_memcode-0.1.0/src/pipecat_memcode/memory.py +786 -0
- pipecat_memcode-0.1.0/src/pipecat_memcode/py.typed +1 -0
- pipecat_memcode-0.1.0/src/pipecat_memcode.egg-info/PKG-INFO +309 -0
- pipecat_memcode-0.1.0/src/pipecat_memcode.egg-info/SOURCES.txt +15 -0
- pipecat_memcode-0.1.0/src/pipecat_memcode.egg-info/dependency_links.txt +1 -0
- pipecat_memcode-0.1.0/src/pipecat_memcode.egg-info/requires.txt +6 -0
- pipecat_memcode-0.1.0/src/pipecat_memcode.egg-info/top_level.txt +1 -0
- pipecat_memcode-0.1.0/tests/test_memory.py +852 -0
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to `pipecat-memcode` will be documented here.
|
|
4
|
+
|
|
5
|
+
## 0.1.0 - Unreleased
|
|
6
|
+
|
|
7
|
+
### Added
|
|
8
|
+
|
|
9
|
+
- Coordinated recall and capture processors for the tested Pipecat 1.10 line.
|
|
10
|
+
- Personal Memcode v2 search and durable background ingest.
|
|
11
|
+
- OAuth access-token-provider support through `memcode-sdk`.
|
|
12
|
+
- Bounded, fail-open recall and bounded terminal cleanup.
|
|
13
|
+
- Deterministic per-turn idempotency keys and injected-context filtering.
|
|
14
|
+
- Tool preamble and post-tool assistant segments are staged into one ingest.
|
|
15
|
+
- Urgent cancellation and interrupted partial turns are discarded without ingest.
|
|
16
|
+
- Cleanup fallback writes and owned-client closure share a bounded, cancellation-safe lifecycle.
|
|
17
|
+
- Runnable WebRTC voice example with separate OAuth registration and account-connection modes.
|
|
18
|
+
- Encrypted local token persistence for the foundational example.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
BSD 2-Clause License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026, Memcode
|
|
4
|
+
All rights reserved.
|
|
5
|
+
|
|
6
|
+
Redistribution and use in source and binary forms, with or without
|
|
7
|
+
modification, are permitted provided that the following conditions are met:
|
|
8
|
+
|
|
9
|
+
1. Redistributions of source code must retain the above copyright notice, this
|
|
10
|
+
list of conditions and the following disclaimer.
|
|
11
|
+
|
|
12
|
+
2. Redistributions in binary form must reproduce the above copyright notice,
|
|
13
|
+
this list of conditions and the following disclaimer in the documentation
|
|
14
|
+
and/or other materials provided with the distribution.
|
|
15
|
+
|
|
16
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
|
17
|
+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
18
|
+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
|
19
|
+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
|
|
20
|
+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
21
|
+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
|
|
22
|
+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
|
|
23
|
+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
|
24
|
+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
|
25
|
+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
@@ -0,0 +1,309 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: pipecat-memcode
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Memcode long-term memory processors for Pipecat
|
|
5
|
+
Author: Memcode
|
|
6
|
+
License-Expression: BSD-2-Clause
|
|
7
|
+
Project-URL: Homepage, https://memcode.in
|
|
8
|
+
Project-URL: Documentation, https://github.com/vivekguptaxmemcode/pipecat-memcode#readme
|
|
9
|
+
Project-URL: Repository, https://github.com/vivekguptaxmemcode/pipecat-memcode
|
|
10
|
+
Project-URL: Issues, https://github.com/vivekguptaxmemcode/pipecat-memcode/issues
|
|
11
|
+
Keywords: pipecat,memcode,memory,voice-ai,oauth
|
|
12
|
+
Classifier: Development Status :: 3 - Alpha
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
18
|
+
Classifier: Typing :: Typed
|
|
19
|
+
Requires-Python: >=3.11
|
|
20
|
+
Description-Content-Type: text/markdown
|
|
21
|
+
License-File: LICENSE
|
|
22
|
+
Requires-Dist: memcode-sdk<3,>=2.4.0
|
|
23
|
+
Requires-Dist: pipecat-ai<1.11,>=1.10
|
|
24
|
+
Provides-Extra: example
|
|
25
|
+
Requires-Dist: cryptography<47,>=45; extra == "example"
|
|
26
|
+
Requires-Dist: pipecat-ai[deepgram,runner,webrtc]<1.11,>=1.10; extra == "example"
|
|
27
|
+
Dynamic: license-file
|
|
28
|
+
|
|
29
|
+
# Pipecat Memcode
|
|
30
|
+
|
|
31
|
+
`pipecat-memcode` gives a Pipecat voice agent durable, personal long-term
|
|
32
|
+
memory backed by [Memcode](https://memcode.in). It retrieves relevant memories
|
|
33
|
+
before inference and stores only finalized user/assistant turns after assistant
|
|
34
|
+
aggregation.
|
|
35
|
+
|
|
36
|
+
This is a community-maintained Pipecat integration, maintained by Memcode. It
|
|
37
|
+
does not modify or ship as part of Pipecat core.
|
|
38
|
+
|
|
39
|
+
## Why there are two processors
|
|
40
|
+
|
|
41
|
+
One `MemcodeMemoryService` exposes two coordinated processors:
|
|
42
|
+
|
|
43
|
+
```text
|
|
44
|
+
transport.input() -> STT -> user_aggregator
|
|
45
|
+
-> memory.recall_processor()
|
|
46
|
+
-> LLM -> TTS -> transport.output()
|
|
47
|
+
-> assistant_aggregator
|
|
48
|
+
-> memory.capture_processor()
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
- Recall belongs **after the user aggregator and before the LLM**. It sees a
|
|
52
|
+
finalized user message and can enrich that inference.
|
|
53
|
+
- Capture belongs **after the assistant aggregator**. It receives Pipecat's
|
|
54
|
+
finalized `LLMContextAssistantTurnFrame`, pairs it with the finalized user
|
|
55
|
+
turn, and queues exactly that delta for ingestion.
|
|
56
|
+
|
|
57
|
+
Interim transcripts, speculative contexts, raw TTS text frames, old history,
|
|
58
|
+
and Memcode's injected context are never ingested.
|
|
59
|
+
|
|
60
|
+
## Installation
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
uv add pipecat-memcode
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Install the optional dependencies used by the runnable WebRTC example with:
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
uv add "pipecat-memcode[example]"
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
The first release targets Python 3.11-3.14, `pipecat-ai>=1.10,<1.11`, and
|
|
73
|
+
`memcode-sdk` 2.4.x. Pipecat releases outside the 1.10 line are not yet claimed
|
|
74
|
+
compatible. Source, issues, and release history live in the
|
|
75
|
+
[`pipecat-memcode` repository](https://github.com/vivekguptaxmemcode/pipecat-memcode).
|
|
76
|
+
|
|
77
|
+
## OAuth 2.1 connection
|
|
78
|
+
|
|
79
|
+
Production applications should connect each participant to Memcode using
|
|
80
|
+
Authorization Code with S256 PKCE and dynamic client registration:
|
|
81
|
+
|
|
82
|
+
1. Discover Memcode's authorization-server and protected-resource metadata.
|
|
83
|
+
2. Register the Pipecat application's exact callback URI once per deployment.
|
|
84
|
+
3. Generate a new `state`, PKCE verifier, and S256 challenge for each account
|
|
85
|
+
connection.
|
|
86
|
+
4. Send the user to Memcode's authorization and consent page, requesting the
|
|
87
|
+
Memory API resource and `memory:read memory:write` scopes.
|
|
88
|
+
5. Validate `state`, exchange the code with the verifier, and store the access
|
|
89
|
+
and rotating refresh tokens encrypted under the application's user record.
|
|
90
|
+
6. Give this package that user's `AsyncAccessTokenProvider`. The SDK resolves a
|
|
91
|
+
token for every request and performs one coordinated refresh/retry after a
|
|
92
|
+
401.
|
|
93
|
+
|
|
94
|
+
Dynamic registration is deployment setup, not per-call or per-conversation
|
|
95
|
+
work. A `MemcodeMemoryService` instance is per authenticated participant. The
|
|
96
|
+
OAuth token subject selects the personal memory scope, so this integration
|
|
97
|
+
never accepts or transmits a `user_id`.
|
|
98
|
+
|
|
99
|
+
`AsyncMemcodeOAuthClient` from `memcode-sdk>=2.4.0` implements discovery,
|
|
100
|
+
dynamic registration, PKCE, token exchange, rotation, and the access-token
|
|
101
|
+
provider interface consumed here.
|
|
102
|
+
|
|
103
|
+
Never put a refresh token, authorization code, or PKCE verifier in frontend
|
|
104
|
+
storage, logs, frame metadata, or LLM context.
|
|
105
|
+
|
|
106
|
+
## Run the foundational example
|
|
107
|
+
|
|
108
|
+
The [single-file example](examples/foundational/memcode_memory.py) is a complete
|
|
109
|
+
Small WebRTC voice bot using Deepgram STT, OpenAI, Cartesia TTS, and Memcode. Its
|
|
110
|
+
account-connection commands are separate from the real-time bot command, and it
|
|
111
|
+
never opens a browser automatically.
|
|
112
|
+
|
|
113
|
+
For a source checkout, install the package, development tools, and example
|
|
114
|
+
dependencies, then create the local environment file:
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
uv sync --group dev --extra example
|
|
118
|
+
cp .env.example .env
|
|
119
|
+
uv run python -c 'from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())'
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
Put the generated value in `MEMCODE_TOKEN_ENCRYPTION_KEY` in `.env`, then fill
|
|
123
|
+
the three voice-provider keys. These are the example variables:
|
|
124
|
+
|
|
125
|
+
| Variable | Required | Purpose |
|
|
126
|
+
|---|---|---|
|
|
127
|
+
| `DEEPGRAM_API_KEY` | bot | Speech-to-text |
|
|
128
|
+
| `OPENAI_API_KEY` | bot | Language model |
|
|
129
|
+
| `OPENAI_MODEL` | no | Defaults to `gpt-4.1-mini` |
|
|
130
|
+
| `CARTESIA_API_KEY` | bot | Text-to-speech |
|
|
131
|
+
| `CARTESIA_VOICE_ID` | no | Defaults to the voice in `.env.example` |
|
|
132
|
+
| `MEMCODE_CLIENT_ID` | connect and bot | Public client ID created during registration |
|
|
133
|
+
| `MEMCODE_REDIRECT_URI` | no | Defaults to `http://127.0.0.1:8765/callback` |
|
|
134
|
+
| `MEMCODE_TOKEN_KEY` | no | Stable local grant lookup key |
|
|
135
|
+
| `MEMCODE_TOKEN_ENCRYPTION_KEY` | connect and bot | Fernet key protecting the local token file |
|
|
136
|
+
| `MEMCODE_TOKEN_PATH` | no | Defaults to `.memcode-oauth.enc` |
|
|
137
|
+
|
|
138
|
+
Register the local public client once:
|
|
139
|
+
|
|
140
|
+
```bash
|
|
141
|
+
uv run python examples/foundational/memcode_memory.py --register
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
Copy the printed, non-secret client ID into `MEMCODE_CLIENT_ID` in `.env`. Then
|
|
145
|
+
start an explicit account connection:
|
|
146
|
+
|
|
147
|
+
```bash
|
|
148
|
+
uv run python examples/foundational/memcode_memory.py --connect
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
Open the printed authorization URL yourself, approve access, and paste the full
|
|
152
|
+
redirected callback URL into the hidden terminal prompt. A browser may show an
|
|
153
|
+
unreachable loopback page; the address bar still contains the callback URL.
|
|
154
|
+
The example validates OAuth state and writes access and rotating refresh tokens
|
|
155
|
+
only to the encrypted, gitignored token file.
|
|
156
|
+
|
|
157
|
+
After connection, run the bot:
|
|
158
|
+
|
|
159
|
+
```bash
|
|
160
|
+
uv run python examples/foundational/memcode_memory.py -t webrtc
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
Open the Pipecat runner URL printed in the terminal and connect your microphone.
|
|
164
|
+
The local encrypted store and its process-local refresh lock are intentionally
|
|
165
|
+
limited to this one-process example. Production deployments must use an
|
|
166
|
+
encrypted server-side `AsyncOAuthTokenStore` with an atomic save and a
|
|
167
|
+
distributed refresh lease covering every worker.
|
|
168
|
+
|
|
169
|
+
## Pipeline usage
|
|
170
|
+
|
|
171
|
+
```python
|
|
172
|
+
from pipecat.pipeline.pipeline import Pipeline
|
|
173
|
+
from pipecat.processors.aggregators.llm_context import LLMContext
|
|
174
|
+
from pipecat.processors.aggregators.llm_response_universal import (
|
|
175
|
+
LLMContextAggregatorPair,
|
|
176
|
+
)
|
|
177
|
+
from pipecat_memcode import MemcodeMemoryConfig, MemcodeMemoryService
|
|
178
|
+
|
|
179
|
+
# `token_provider` is scoped to the signed-in participant and implements
|
|
180
|
+
# memcode_sdk.AsyncAccessTokenProvider.
|
|
181
|
+
memory = MemcodeMemoryService(
|
|
182
|
+
access_token_provider=token_provider,
|
|
183
|
+
api_url="https://memory.memcode.in",
|
|
184
|
+
session_id=call_id, # use a stable room/call ID for retry idempotency
|
|
185
|
+
config=MemcodeMemoryConfig(
|
|
186
|
+
search_top_k=5,
|
|
187
|
+
search_timeout_seconds=1.5,
|
|
188
|
+
),
|
|
189
|
+
)
|
|
190
|
+
|
|
191
|
+
context = LLMContext([{"role": "developer", "content": "You are a concise, helpful assistant."}])
|
|
192
|
+
user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context)
|
|
193
|
+
|
|
194
|
+
pipeline = Pipeline(
|
|
195
|
+
[
|
|
196
|
+
transport.input(),
|
|
197
|
+
stt,
|
|
198
|
+
user_aggregator,
|
|
199
|
+
memory.recall_processor(),
|
|
200
|
+
llm,
|
|
201
|
+
tts,
|
|
202
|
+
transport.output(),
|
|
203
|
+
assistant_aggregator,
|
|
204
|
+
memory.capture_processor(),
|
|
205
|
+
]
|
|
206
|
+
)
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
Applications that already own a per-user SDK client can inject it instead:
|
|
210
|
+
|
|
211
|
+
```python
|
|
212
|
+
from memcode_sdk import AsyncMemcodeClient
|
|
213
|
+
from pipecat_memcode import MemcodeMemoryService
|
|
214
|
+
|
|
215
|
+
client = AsyncMemcodeClient(
|
|
216
|
+
api_url="https://memory.memcode.in",
|
|
217
|
+
access_token_provider=token_provider,
|
|
218
|
+
)
|
|
219
|
+
memory = MemcodeMemoryService(
|
|
220
|
+
client=client,
|
|
221
|
+
session_id=call_id,
|
|
222
|
+
close_client=False, # the application retains lifecycle ownership
|
|
223
|
+
)
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
When the service constructs the SDK client, it closes that client during
|
|
227
|
+
processor cleanup. The application still owns the injected token provider and
|
|
228
|
+
must close an `AsyncMemcodeOAuthClient` or `DelegatingMemoryTokenProvider` from
|
|
229
|
+
its own connection/session lifecycle. When an existing client is injected, the
|
|
230
|
+
application owns it unless `close_client=True` is explicitly requested.
|
|
231
|
+
|
|
232
|
+
See [`examples/foundational/memcode_memory.py`](examples/foundational/memcode_memory.py)
|
|
233
|
+
for the complete runnable integration.
|
|
234
|
+
|
|
235
|
+
## Runtime contract
|
|
236
|
+
|
|
237
|
+
Recall uses `AsyncMemcodeClient.search_v2`, not `retrieve_v2`. It always asks
|
|
238
|
+
for extracted memories only (`mode="memories"`,
|
|
239
|
+
`include_original_chunks=False`), bounds the resulting block, marks it as
|
|
240
|
+
reference-only data, and fails open if Memcode is slow or unavailable.
|
|
241
|
+
|
|
242
|
+
Capture uses `AsyncMemcodeClient.ingest_v2` in a Pipecat-managed background
|
|
243
|
+
task. Assistant-turn frames are staged because Pipecat can emit one at both a
|
|
244
|
+
tool preamble and the post-tool answer. All segments remain attached to the
|
|
245
|
+
same user turn and are written once when the next finalized user turn arrives,
|
|
246
|
+
or when graceful `EndFrame` or cleanup finalizes the session. An
|
|
247
|
+
`InterruptionFrame` discards the interrupted partial assistant turn so it
|
|
248
|
+
cannot be captured by the next user turn. Urgent `CancelFrame` discards the
|
|
249
|
+
active turn, signals any owned writes to stop, and propagates immediately
|
|
250
|
+
without waiting on Memcode. Each write carries a deterministic SHA-256
|
|
251
|
+
idempotency key derived from the stable session ID and combined finalized turn.
|
|
252
|
+
Graceful shutdown work is bounded by `shutdown_timeout_seconds`; cleanup and
|
|
253
|
+
client closing are cancellation-safe and idempotent.
|
|
254
|
+
|
|
255
|
+
The ingestion call returns a durable receipt. Memory extraction continues in
|
|
256
|
+
Memcode asynchronously; this package intentionally does not hold up the voice
|
|
257
|
+
pipeline by polling that job.
|
|
258
|
+
|
|
259
|
+
## Configuration
|
|
260
|
+
|
|
261
|
+
| Field | Default | Meaning |
|
|
262
|
+
|---|---:|---|
|
|
263
|
+
| `search_top_k` | `5` | Maximum memories requested per user turn |
|
|
264
|
+
| `search_minimum_score` | `0.0` | Minimum relevance score |
|
|
265
|
+
| `search_mode` | `"default"` | Memcode routing mode (`default` or `global`) |
|
|
266
|
+
| `search_timeout_seconds` | `1.5` | Recall latency budget before fail-open |
|
|
267
|
+
| `ingest_timeout_seconds` | `5.0` | Budget for a durable ingest receipt |
|
|
268
|
+
| `shutdown_timeout_seconds` | `2.0` | Graceful EndFrame and cleanup budget |
|
|
269
|
+
| `max_context_characters` | `4000` | Maximum complete injected context block |
|
|
270
|
+
| `context_role` | `"developer"` | Injected universal-context role |
|
|
271
|
+
| `context_header` | reference-only warning | Boundary between data and instructions |
|
|
272
|
+
| `effort_level` | `"low"` | Memcode ingest effort (`low` or `high`) |
|
|
273
|
+
|
|
274
|
+
Use a stable, non-secret `session_id` from the Pipecat call or room. If omitted,
|
|
275
|
+
the service creates a random ID, which preserves in-process retry safety but
|
|
276
|
+
cannot deduplicate the same turn after a process restart.
|
|
277
|
+
|
|
278
|
+
## Failure behavior
|
|
279
|
+
|
|
280
|
+
- Search timeout or error: the unchanged context continues to the LLM.
|
|
281
|
+
- Partial search response: available results are used; failed domains do not
|
|
282
|
+
erase valid hits.
|
|
283
|
+
- Ingest timeout or error: the voice response is never blocked or failed.
|
|
284
|
+
- Interruption or cancellation: partial active turns are discarded; urgent
|
|
285
|
+
cancellation never waits for Memcode.
|
|
286
|
+
- Duplicate context frames: recall is cached per finalized conversational
|
|
287
|
+
prefix and only one pending capture turn is created.
|
|
288
|
+
- Duplicate write attempt: the same finalized turn receives the same
|
|
289
|
+
idempotency key.
|
|
290
|
+
|
|
291
|
+
## Development
|
|
292
|
+
|
|
293
|
+
```bash
|
|
294
|
+
uv sync --group dev --extra example
|
|
295
|
+
uv run ruff check .
|
|
296
|
+
uv run ruff format --check .
|
|
297
|
+
uv run pytest
|
|
298
|
+
uv build
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
No browser is required for the unit suite. A release candidate should also be
|
|
302
|
+
verified against a real OAuth account in an explicitly authorized staging run,
|
|
303
|
+
including a session long enough to rotate an access token.
|
|
304
|
+
|
|
305
|
+
## License and attribution
|
|
306
|
+
|
|
307
|
+
This integration is released under the BSD 2-Clause License. Pipecat is an
|
|
308
|
+
open-source project maintained by Daily; Memcode maintains this community
|
|
309
|
+
package and its Memcode-specific behavior.
|
|
@@ -0,0 +1,281 @@
|
|
|
1
|
+
# Pipecat Memcode
|
|
2
|
+
|
|
3
|
+
`pipecat-memcode` gives a Pipecat voice agent durable, personal long-term
|
|
4
|
+
memory backed by [Memcode](https://memcode.in). It retrieves relevant memories
|
|
5
|
+
before inference and stores only finalized user/assistant turns after assistant
|
|
6
|
+
aggregation.
|
|
7
|
+
|
|
8
|
+
This is a community-maintained Pipecat integration, maintained by Memcode. It
|
|
9
|
+
does not modify or ship as part of Pipecat core.
|
|
10
|
+
|
|
11
|
+
## Why there are two processors
|
|
12
|
+
|
|
13
|
+
One `MemcodeMemoryService` exposes two coordinated processors:
|
|
14
|
+
|
|
15
|
+
```text
|
|
16
|
+
transport.input() -> STT -> user_aggregator
|
|
17
|
+
-> memory.recall_processor()
|
|
18
|
+
-> LLM -> TTS -> transport.output()
|
|
19
|
+
-> assistant_aggregator
|
|
20
|
+
-> memory.capture_processor()
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
- Recall belongs **after the user aggregator and before the LLM**. It sees a
|
|
24
|
+
finalized user message and can enrich that inference.
|
|
25
|
+
- Capture belongs **after the assistant aggregator**. It receives Pipecat's
|
|
26
|
+
finalized `LLMContextAssistantTurnFrame`, pairs it with the finalized user
|
|
27
|
+
turn, and queues exactly that delta for ingestion.
|
|
28
|
+
|
|
29
|
+
Interim transcripts, speculative contexts, raw TTS text frames, old history,
|
|
30
|
+
and Memcode's injected context are never ingested.
|
|
31
|
+
|
|
32
|
+
## Installation
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
uv add pipecat-memcode
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Install the optional dependencies used by the runnable WebRTC example with:
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
uv add "pipecat-memcode[example]"
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
The first release targets Python 3.11-3.14, `pipecat-ai>=1.10,<1.11`, and
|
|
45
|
+
`memcode-sdk` 2.4.x. Pipecat releases outside the 1.10 line are not yet claimed
|
|
46
|
+
compatible. Source, issues, and release history live in the
|
|
47
|
+
[`pipecat-memcode` repository](https://github.com/vivekguptaxmemcode/pipecat-memcode).
|
|
48
|
+
|
|
49
|
+
## OAuth 2.1 connection
|
|
50
|
+
|
|
51
|
+
Production applications should connect each participant to Memcode using
|
|
52
|
+
Authorization Code with S256 PKCE and dynamic client registration:
|
|
53
|
+
|
|
54
|
+
1. Discover Memcode's authorization-server and protected-resource metadata.
|
|
55
|
+
2. Register the Pipecat application's exact callback URI once per deployment.
|
|
56
|
+
3. Generate a new `state`, PKCE verifier, and S256 challenge for each account
|
|
57
|
+
connection.
|
|
58
|
+
4. Send the user to Memcode's authorization and consent page, requesting the
|
|
59
|
+
Memory API resource and `memory:read memory:write` scopes.
|
|
60
|
+
5. Validate `state`, exchange the code with the verifier, and store the access
|
|
61
|
+
and rotating refresh tokens encrypted under the application's user record.
|
|
62
|
+
6. Give this package that user's `AsyncAccessTokenProvider`. The SDK resolves a
|
|
63
|
+
token for every request and performs one coordinated refresh/retry after a
|
|
64
|
+
401.
|
|
65
|
+
|
|
66
|
+
Dynamic registration is deployment setup, not per-call or per-conversation
|
|
67
|
+
work. A `MemcodeMemoryService` instance is per authenticated participant. The
|
|
68
|
+
OAuth token subject selects the personal memory scope, so this integration
|
|
69
|
+
never accepts or transmits a `user_id`.
|
|
70
|
+
|
|
71
|
+
`AsyncMemcodeOAuthClient` from `memcode-sdk>=2.4.0` implements discovery,
|
|
72
|
+
dynamic registration, PKCE, token exchange, rotation, and the access-token
|
|
73
|
+
provider interface consumed here.
|
|
74
|
+
|
|
75
|
+
Never put a refresh token, authorization code, or PKCE verifier in frontend
|
|
76
|
+
storage, logs, frame metadata, or LLM context.
|
|
77
|
+
|
|
78
|
+
## Run the foundational example
|
|
79
|
+
|
|
80
|
+
The [single-file example](examples/foundational/memcode_memory.py) is a complete
|
|
81
|
+
Small WebRTC voice bot using Deepgram STT, OpenAI, Cartesia TTS, and Memcode. Its
|
|
82
|
+
account-connection commands are separate from the real-time bot command, and it
|
|
83
|
+
never opens a browser automatically.
|
|
84
|
+
|
|
85
|
+
For a source checkout, install the package, development tools, and example
|
|
86
|
+
dependencies, then create the local environment file:
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
uv sync --group dev --extra example
|
|
90
|
+
cp .env.example .env
|
|
91
|
+
uv run python -c 'from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())'
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Put the generated value in `MEMCODE_TOKEN_ENCRYPTION_KEY` in `.env`, then fill
|
|
95
|
+
the three voice-provider keys. These are the example variables:
|
|
96
|
+
|
|
97
|
+
| Variable | Required | Purpose |
|
|
98
|
+
|---|---|---|
|
|
99
|
+
| `DEEPGRAM_API_KEY` | bot | Speech-to-text |
|
|
100
|
+
| `OPENAI_API_KEY` | bot | Language model |
|
|
101
|
+
| `OPENAI_MODEL` | no | Defaults to `gpt-4.1-mini` |
|
|
102
|
+
| `CARTESIA_API_KEY` | bot | Text-to-speech |
|
|
103
|
+
| `CARTESIA_VOICE_ID` | no | Defaults to the voice in `.env.example` |
|
|
104
|
+
| `MEMCODE_CLIENT_ID` | connect and bot | Public client ID created during registration |
|
|
105
|
+
| `MEMCODE_REDIRECT_URI` | no | Defaults to `http://127.0.0.1:8765/callback` |
|
|
106
|
+
| `MEMCODE_TOKEN_KEY` | no | Stable local grant lookup key |
|
|
107
|
+
| `MEMCODE_TOKEN_ENCRYPTION_KEY` | connect and bot | Fernet key protecting the local token file |
|
|
108
|
+
| `MEMCODE_TOKEN_PATH` | no | Defaults to `.memcode-oauth.enc` |
|
|
109
|
+
|
|
110
|
+
Register the local public client once:
|
|
111
|
+
|
|
112
|
+
```bash
|
|
113
|
+
uv run python examples/foundational/memcode_memory.py --register
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Copy the printed, non-secret client ID into `MEMCODE_CLIENT_ID` in `.env`. Then
|
|
117
|
+
start an explicit account connection:
|
|
118
|
+
|
|
119
|
+
```bash
|
|
120
|
+
uv run python examples/foundational/memcode_memory.py --connect
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
Open the printed authorization URL yourself, approve access, and paste the full
|
|
124
|
+
redirected callback URL into the hidden terminal prompt. A browser may show an
|
|
125
|
+
unreachable loopback page; the address bar still contains the callback URL.
|
|
126
|
+
The example validates OAuth state and writes access and rotating refresh tokens
|
|
127
|
+
only to the encrypted, gitignored token file.
|
|
128
|
+
|
|
129
|
+
After connection, run the bot:
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
uv run python examples/foundational/memcode_memory.py -t webrtc
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
Open the Pipecat runner URL printed in the terminal and connect your microphone.
|
|
136
|
+
The local encrypted store and its process-local refresh lock are intentionally
|
|
137
|
+
limited to this one-process example. Production deployments must use an
|
|
138
|
+
encrypted server-side `AsyncOAuthTokenStore` with an atomic save and a
|
|
139
|
+
distributed refresh lease covering every worker.
|
|
140
|
+
|
|
141
|
+
## Pipeline usage
|
|
142
|
+
|
|
143
|
+
```python
|
|
144
|
+
from pipecat.pipeline.pipeline import Pipeline
|
|
145
|
+
from pipecat.processors.aggregators.llm_context import LLMContext
|
|
146
|
+
from pipecat.processors.aggregators.llm_response_universal import (
|
|
147
|
+
LLMContextAggregatorPair,
|
|
148
|
+
)
|
|
149
|
+
from pipecat_memcode import MemcodeMemoryConfig, MemcodeMemoryService
|
|
150
|
+
|
|
151
|
+
# `token_provider` is scoped to the signed-in participant and implements
|
|
152
|
+
# memcode_sdk.AsyncAccessTokenProvider.
|
|
153
|
+
memory = MemcodeMemoryService(
|
|
154
|
+
access_token_provider=token_provider,
|
|
155
|
+
api_url="https://memory.memcode.in",
|
|
156
|
+
session_id=call_id, # use a stable room/call ID for retry idempotency
|
|
157
|
+
config=MemcodeMemoryConfig(
|
|
158
|
+
search_top_k=5,
|
|
159
|
+
search_timeout_seconds=1.5,
|
|
160
|
+
),
|
|
161
|
+
)
|
|
162
|
+
|
|
163
|
+
context = LLMContext([{"role": "developer", "content": "You are a concise, helpful assistant."}])
|
|
164
|
+
user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context)
|
|
165
|
+
|
|
166
|
+
pipeline = Pipeline(
|
|
167
|
+
[
|
|
168
|
+
transport.input(),
|
|
169
|
+
stt,
|
|
170
|
+
user_aggregator,
|
|
171
|
+
memory.recall_processor(),
|
|
172
|
+
llm,
|
|
173
|
+
tts,
|
|
174
|
+
transport.output(),
|
|
175
|
+
assistant_aggregator,
|
|
176
|
+
memory.capture_processor(),
|
|
177
|
+
]
|
|
178
|
+
)
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
Applications that already own a per-user SDK client can inject it instead:
|
|
182
|
+
|
|
183
|
+
```python
|
|
184
|
+
from memcode_sdk import AsyncMemcodeClient
|
|
185
|
+
from pipecat_memcode import MemcodeMemoryService
|
|
186
|
+
|
|
187
|
+
client = AsyncMemcodeClient(
|
|
188
|
+
api_url="https://memory.memcode.in",
|
|
189
|
+
access_token_provider=token_provider,
|
|
190
|
+
)
|
|
191
|
+
memory = MemcodeMemoryService(
|
|
192
|
+
client=client,
|
|
193
|
+
session_id=call_id,
|
|
194
|
+
close_client=False, # the application retains lifecycle ownership
|
|
195
|
+
)
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
When the service constructs the SDK client, it closes that client during
|
|
199
|
+
processor cleanup. The application still owns the injected token provider and
|
|
200
|
+
must close an `AsyncMemcodeOAuthClient` or `DelegatingMemoryTokenProvider` from
|
|
201
|
+
its own connection/session lifecycle. When an existing client is injected, the
|
|
202
|
+
application owns it unless `close_client=True` is explicitly requested.
|
|
203
|
+
|
|
204
|
+
See [`examples/foundational/memcode_memory.py`](examples/foundational/memcode_memory.py)
|
|
205
|
+
for the complete runnable integration.
|
|
206
|
+
|
|
207
|
+
## Runtime contract
|
|
208
|
+
|
|
209
|
+
Recall uses `AsyncMemcodeClient.search_v2`, not `retrieve_v2`. It always asks
|
|
210
|
+
for extracted memories only (`mode="memories"`,
|
|
211
|
+
`include_original_chunks=False`), bounds the resulting block, marks it as
|
|
212
|
+
reference-only data, and fails open if Memcode is slow or unavailable.
|
|
213
|
+
|
|
214
|
+
Capture uses `AsyncMemcodeClient.ingest_v2` in a Pipecat-managed background
|
|
215
|
+
task. Assistant-turn frames are staged because Pipecat can emit one at both a
|
|
216
|
+
tool preamble and the post-tool answer. All segments remain attached to the
|
|
217
|
+
same user turn and are written once when the next finalized user turn arrives,
|
|
218
|
+
or when graceful `EndFrame` or cleanup finalizes the session. An
|
|
219
|
+
`InterruptionFrame` discards the interrupted partial assistant turn so it
|
|
220
|
+
cannot be captured by the next user turn. Urgent `CancelFrame` discards the
|
|
221
|
+
active turn, signals any owned writes to stop, and propagates immediately
|
|
222
|
+
without waiting on Memcode. Each write carries a deterministic SHA-256
|
|
223
|
+
idempotency key derived from the stable session ID and combined finalized turn.
|
|
224
|
+
Graceful shutdown work is bounded by `shutdown_timeout_seconds`; cleanup and
|
|
225
|
+
client closing are cancellation-safe and idempotent.
|
|
226
|
+
|
|
227
|
+
The ingestion call returns a durable receipt. Memory extraction continues in
|
|
228
|
+
Memcode asynchronously; this package intentionally does not hold up the voice
|
|
229
|
+
pipeline by polling that job.
|
|
230
|
+
|
|
231
|
+
## Configuration
|
|
232
|
+
|
|
233
|
+
| Field | Default | Meaning |
|
|
234
|
+
|---|---:|---|
|
|
235
|
+
| `search_top_k` | `5` | Maximum memories requested per user turn |
|
|
236
|
+
| `search_minimum_score` | `0.0` | Minimum relevance score |
|
|
237
|
+
| `search_mode` | `"default"` | Memcode routing mode (`default` or `global`) |
|
|
238
|
+
| `search_timeout_seconds` | `1.5` | Recall latency budget before fail-open |
|
|
239
|
+
| `ingest_timeout_seconds` | `5.0` | Budget for a durable ingest receipt |
|
|
240
|
+
| `shutdown_timeout_seconds` | `2.0` | Graceful EndFrame and cleanup budget |
|
|
241
|
+
| `max_context_characters` | `4000` | Maximum complete injected context block |
|
|
242
|
+
| `context_role` | `"developer"` | Injected universal-context role |
|
|
243
|
+
| `context_header` | reference-only warning | Boundary between data and instructions |
|
|
244
|
+
| `effort_level` | `"low"` | Memcode ingest effort (`low` or `high`) |
|
|
245
|
+
|
|
246
|
+
Use a stable, non-secret `session_id` from the Pipecat call or room. If omitted,
|
|
247
|
+
the service creates a random ID, which preserves in-process retry safety but
|
|
248
|
+
cannot deduplicate the same turn after a process restart.
|
|
249
|
+
|
|
250
|
+
## Failure behavior
|
|
251
|
+
|
|
252
|
+
- Search timeout or error: the unchanged context continues to the LLM.
|
|
253
|
+
- Partial search response: available results are used; failed domains do not
|
|
254
|
+
erase valid hits.
|
|
255
|
+
- Ingest timeout or error: the voice response is never blocked or failed.
|
|
256
|
+
- Interruption or cancellation: partial active turns are discarded; urgent
|
|
257
|
+
cancellation never waits for Memcode.
|
|
258
|
+
- Duplicate context frames: recall is cached per finalized conversational
|
|
259
|
+
prefix and only one pending capture turn is created.
|
|
260
|
+
- Duplicate write attempt: the same finalized turn receives the same
|
|
261
|
+
idempotency key.
|
|
262
|
+
|
|
263
|
+
## Development
|
|
264
|
+
|
|
265
|
+
```bash
|
|
266
|
+
uv sync --group dev --extra example
|
|
267
|
+
uv run ruff check .
|
|
268
|
+
uv run ruff format --check .
|
|
269
|
+
uv run pytest
|
|
270
|
+
uv build
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
No browser is required for the unit suite. A release candidate should also be
|
|
274
|
+
verified against a real OAuth account in an explicitly authorized staging run,
|
|
275
|
+
including a session long enough to rotate an access token.
|
|
276
|
+
|
|
277
|
+
## License and attribution
|
|
278
|
+
|
|
279
|
+
This integration is released under the BSD 2-Clause License. Pipecat is an
|
|
280
|
+
open-source project maintained by Daily; Memcode maintains this community
|
|
281
|
+
package and its Memcode-specific behavior.
|