pipecat-memorysync 1.0.1__tar.gz → 1.1.1__tar.gz

pipecat_memorysync-1.1.1/CHANGELOG.md

@@ -0,0 +1,51 @@
+# Changelog
+
+All notable changes to `pipecat-memorysync` are documented here.
+
+## 1.1.1 — 2026-08-29
+
+- **Immediate, loss-proof capture.** 1.1.0's turn-complete merging
+  deferred the current utterance until the turn completed — on a browser
+  disconnect that deferral raced the brief cancel salvage window and
+  could lose the newest (usually most important) turn over slow
+  networks. Capture is immediate again: every new user/assistant message
+  is in flight the moment its frame passes, BEFORE the LLM replies.
+  Junk filtering and fact extraction now happen server-side (the
+  platform's low-value gate + conversational ingestion), so client-side
+  merging is unnecessary. Suite: 17 checks on Pipecat v1.8.1, including
+  the exact disconnect sequence that previously lost data.
+
+## 1.1.0 — 2026-08-29
+
+- **Turn-complete capture.** Voice aggregators split one utterance across
+  several context messages at speech pauses; those fragments previously
+  stored as separate rows ("and", "dinner", …). Consecutive new user
+  fragments now merge into ONE verbatim turn, stored when the assistant
+  reply completes the turn; an in-progress utterance is flushed as one
+  merged turn at end of call. Assistant turns still store immediately,
+  and a failed store releases its fragments for retry on the next frame.
+- Delta-only and idempotency guarantees unchanged; suite grows to 17
+  checks on Pipecat v1.8.1 (fragment merging, end-of-call tail flush,
+  failed-store retry).
+
+## 1.0.1 — 2026-08-29
+
+- Metadata only: the source repository moved to
+  `https://github.com/memorysyncio/memorysync-pipecat`; project URLs updated.
+  No code changes.
+
+## 1.0.0 — 2026-08-29
+
+Initial release.
+
+- `MemorySyncMemoryService`, a `FrameProcessor` for the position between the
+  user context aggregator and the LLM service.
+- Budgeted memory recall (default 1.2 s hard timeout) injected into every
+  `LLMContextFrame` as a guarded system message — a slow or unreachable
+  backend degrades to an unenriched frame, never a stalled reply.
+- Delta-only capture with deterministic idempotency seeds: only turns not
+  seen before are persisted, with role fidelity for user and assistant.
+- Injection exclusion: the injected memory block is never re-captured.
+- Graceful `EndFrame` flush (bounded 3 s window) and `CancelFrame` salvage.
+- Tested with Pipecat v1.8.1 through `pipecat.tests.utils.run_test`
+  (14-test suite).

{pipecat_memorysync-1.0.1 → pipecat_memorysync-1.1.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: pipecat-memorysync
-Version: 1.0.1
+Version: 1.1.1
 Summary: MemorySync for Pipecat: budgeted memory recall that never stalls a voice reply, delta-only conversation persistence with idempotency seeds, and a pipeline that cannot be broken by a memory outage.
 Project-URL: Homepage, https://docs.memorysync.io/guides/pipecat
 Project-URL: Documentation, https://docs.memorysync.io/guides/pipecat
@@ -81,6 +81,12 @@ enriched or not, on time.
 - **Budgeted recall.** Enrichment runs under a hard timeout (default
   **1.2 s**). A slow or dead memory backend means an unenriched frame, never a
   stalled voice reply.
+- **Immediate, loss-proof capture.** Every new message is sent the moment
+  its frame passes — the current utterance is in flight BEFORE the LLM
+  replies, so a disconnect can never lose it. Junk filtering and fact
+  extraction happen server-side (the platform's low-value gate +
+  conversational ingestion): the memory store receives curated facts,
+  not transcript clutter.
 - **Delta-only capture.** Only messages *not seen before* are stored, tracked
   by deterministic idempotency seeds. Growing a 50-message context does not
   re-store 50 messages per turn.

{pipecat_memorysync-1.0.1 → pipecat_memorysync-1.1.1}/README.md

@@ -57,6 +57,12 @@ enriched or not, on time.
 - **Budgeted recall.** Enrichment runs under a hard timeout (default
   **1.2 s**). A slow or dead memory backend means an unenriched frame, never a
   stalled voice reply.
+- **Immediate, loss-proof capture.** Every new message is sent the moment
+  its frame passes — the current utterance is in flight BEFORE the LLM
+  replies, so a disconnect can never lose it. Junk filtering and fact
+  extraction happen server-side (the platform's low-value gate +
+  conversational ingestion): the memory store receives curated facts,
+  not transcript clutter.
 - **Delta-only capture.** Only messages *not seen before* are stored, tracked
   by deterministic idempotency seeds. Growing a 50-message context does not
   re-store 50 messages per turn.

{pipecat_memorysync-1.0.1 → pipecat_memorysync-1.1.1}/src/pipecat_memorysync/_version.py

@@ -1,3 +1,3 @@
 """Version for pipecat-memorysync. Single source of truth."""
 
-__version__ = "1.0.1"
+__version__ = "1.1.1"

{pipecat_memorysync-1.0.1 → pipecat_memorysync-1.1.1}/src/pipecat_memorysync/service.py

@@ -7,9 +7,13 @@ predecessors don't:
 1. **Recall is budgeted.** Enrichment waits at most ``recall_timeout``
    seconds (default 1.2). On timeout or failure the context frame passes
    through unenriched — a voice reply is never stalled by a slow network.
-2. **Capture is delta-only.** Each turn stores only the messages that are
-   NEW since the last frame, verbatim, with cross-adapter fnv1a64
-   idempotency seeds — not the whole conversation re-sent every turn.
+2. **Capture is immediate and delta-only.** Every NEW user/assistant
+   message is sent to the platform the moment it appears in a context
+   frame — the current utterance is already in flight BEFORE the LLM
+   replies, so a disconnect can never lose it. Junk filtering and fact
+   extraction are the platform's job (the low-value gate + conversational
+   ingestion), not the client's. Only new messages are sent each frame,
+   with cross-adapter fnv1a64 idempotency seeds.
 3. **Nothing raises, nothing is dropped.** Every failure path logs and
    pushes the ORIGINAL frame through. The pipeline cannot stall and the
    LLM always gets its context.
@@ -137,6 +141,9 @@ class MemorySyncMemoryService(FrameProcessor):
         self.params = params
 
         self._stored_seeds: Set[str] = set()
+        # Message-level ledger: which context messages have been sent.
+        # A failed store releases its message so the next frame retries.
+        self._seen_messages: Set[str] = set()
         self._store_tasks: Set[asyncio.Task] = set()
         self._last_query: Optional[str] = None
 
@@ -237,10 +244,20 @@ class MemorySyncMemoryService(FrameProcessor):
         body = "\n".join(lines)
         return f"{self.params.system_prompt}\n{body}\n\n{CONTEXT_GUARD}"
 
-    # ── capture: delta-only, background ────────────────────────────────
+    # ── capture: immediate, delta-only, background ─────────────────────
 
     def _capture_delta(self, context: Any) -> None:
-        """Queue storage for messages NOT seen before. O(new), not O(all)."""
+        """Queue storage for NEW messages the moment they appear.
+
+        The platform owns junk filtering (the low-value gate) and fact
+        extraction (conversational ingestion), so the adapter's single
+        duty is delivering every turn reliably and EARLY. The current
+        user utterance is in the frame BEFORE the LLM replies — sending
+        it immediately means a mid-call disconnect can never lose it.
+        (1.1.0 deferred the current utterance until the turn completed;
+        that deferral raced the disconnect salvage window and could drop
+        the newest — usually most important — turn. Never again.)
+        """
         header = self.params.system_prompt
         for message in context.get_messages():
             role = message.get("role")
@@ -251,19 +268,33 @@ class MemorySyncMemoryService(FrameProcessor):
                 continue  # our own injection (user-role mode) never re-enters
             speaker_role = "human" if role == "user" else "ai"
             trimmed = text if len(text) <= MAX_TURN_CHARS else text[:MAX_TURN_CHARS] + "…"
-            seed = f"{speaker_role}:{trimmed}"
-            if seed in self._stored_seeds:
+            key = f"{speaker_role}:{trimmed}"
+            if key in self._seen_messages:
                 continue
-            self._stored_seeds.add(seed)
-            if len(self._stored_seeds) > 4096:
-                self._stored_seeds.clear()
-            # Plain asyncio tasks, tracked locally: pipeline teardown must
-            # not cancel a persist mid-flight — _flush owns their fate.
-            task = asyncio.create_task(self._store_turn(speaker_role, trimmed, seed))
-            self._store_tasks.add(task)
-            task.add_done_callback(self._store_tasks.discard)
-
-    async def _store_turn(self, speaker_role: str, text: str, seed: str) -> None:
+            self._seen_messages.add(key)
+            self._bound_seen()
+            self._queue_store(speaker_role, trimmed, [key])
+
+    def _bound_seen(self) -> None:
+        if len(self._seen_messages) > 4096:
+            self._seen_messages.clear()
+
+    def _queue_store(self, speaker_role: str, text: str, msg_keys: List[str]) -> None:
+        seed = f"{speaker_role}:{text}"
+        if seed in self._stored_seeds:
+            return
+        self._stored_seeds.add(seed)
+        if len(self._stored_seeds) > 4096:
+            self._stored_seeds.clear()
+        # Plain asyncio tasks, tracked locally: pipeline teardown must
+        # not cancel a persist mid-flight — _flush owns their fate.
+        task = asyncio.create_task(self._store_turn(speaker_role, text, seed, msg_keys))
+        self._store_tasks.add(task)
+        task.add_done_callback(self._store_tasks.discard)
+
+    async def _store_turn(
+        self, speaker_role: str, text: str, seed: str, msg_keys: List[str]
+    ) -> None:
         try:
             tenant = await self._api.resolve_tenant_id()
             await self._api.add_turn(
@@ -274,7 +305,11 @@ class MemorySyncMemoryService(FrameProcessor):
                 metadata={"session_id": self.scope},
             )
         except Exception as exc:  # noqa: BLE001
-            self._stored_seeds.discard(seed)  # the write never landed; retry later
+            # The write never landed: release both ledgers so the next
+            # frame re-captures this message and retries the store.
+            self._stored_seeds.discard(seed)
+            for key in msg_keys:
+                self._seen_messages.discard(key)
             logger.debug(f"memorysync: store failed: {exc}")
 
     # ── conveniences (outside the pipeline) ───────────────────────────

{pipecat_memorysync-1.0.1 → pipecat_memorysync-1.1.1}/tests/test_service.py

@@ -250,3 +250,61 @@ async def test_conveniences_answer(mock, make_service):
 async def test_missing_user_id_is_loud_at_construction(mock):
     with pytest.raises(ValueError, match="user_id"):
         MemorySyncMemoryService(api_key="ms_x", user_id="", transport=mock.transport())
+
+
+# ── capture: immediate and loss-proof ─────────────────────────────────
+
+
+async def test_current_utterance_stores_before_any_reply(mock, make_service):
+    """The user's words must be in flight the moment the frame passes —
+    BEFORE the LLM replies — so a disconnect can never lose them."""
+    service = make_service()
+    ctx = context_of([
+        {"role": "user", "content": "My age is twenty two and I completed my bachelor's."},
+    ])
+    await drive(service, ctx)
+    await drain(mock, 1)
+    assert [r["text"] for r in mock.rows] == [
+        "human: My age is twenty two and I completed my bachelor's."
+    ]
+
+
+async def test_demo_disconnect_sequence_loses_nothing(mock, make_service):
+    """The exact sequence that lost data in 1.1.0: greeting frame, then a
+    frame carrying the reply + the important utterance, then immediate
+    CancelFrame (browser disconnect). Every message must already be
+    stored — nothing may depend on a post-cancel flush window."""
+    service = make_service()
+    f1 = context_of([{"role": "user", "content": "Hello there, anyone home?"}])
+    f2 = context_of([
+        {"role": "user", "content": "Hello there, anyone home?"},
+        {"role": "assistant", "content": "Hi! How can I help you today?"},
+        {"role": "user", "content": "My age is twenty two and I completed my bachelor's."},
+    ])
+    await drive(service, f1, f2)  # run_test tears the pipeline down right after
+    await drain(mock, 3)
+    texts = sorted(r["text"] for r in mock.rows)
+    assert texts == [
+        "ai: Hi! How can I help you today?",
+        "human: Hello there, anyone home?",
+        "human: My age is twenty two and I completed my bachelor's.",
+    ]
+
+
+async def test_failed_store_releases_message_for_retry(mock, make_service):
+    """A store that never landed must not consume its message — the next
+    frame re-captures and retries, converging on one row."""
+    service = make_service()
+    mock.fail_next = 503
+    turn = [
+        {"role": "user", "content": "remember that I fly out of Hyderabad"},
+        {"role": "assistant", "content": "Noted!"},
+    ]
+    await drive(service, context_of(turn))
+    await asyncio.sleep(0.3)  # the 503 lands on exactly one of the two stores
+
+    await drive(service, context_of(turn))  # same context re-seen → retry
+    await drain(mock, 2)
+
+    texts = sorted(r["text"] for r in mock.rows)
+    assert texts == ["ai: Noted!", "human: remember that I fly out of Hyderabad"]

pipecat_memorysync-1.0.1/CHANGELOG.md

@@ -1,25 +0,0 @@
-# Changelog
-
-All notable changes to `pipecat-memorysync` are documented here.
-
-## 1.0.1 — 2026-08-29
-
-- Metadata only: the source repository moved to
-  `https://github.com/memorysyncio/memorysync-pipecat`; project URLs updated.
-  No code changes.
-
-## 1.0.0 — 2026-08-29
-
-Initial release.
-
-- `MemorySyncMemoryService`, a `FrameProcessor` for the position between the
-  user context aggregator and the LLM service.
-- Budgeted memory recall (default 1.2 s hard timeout) injected into every
-  `LLMContextFrame` as a guarded system message — a slow or unreachable
-  backend degrades to an unenriched frame, never a stalled reply.
-- Delta-only capture with deterministic idempotency seeds: only turns not
-  seen before are persisted, with role fidelity for user and assistant.
-- Injection exclusion: the injected memory block is never re-captured.
-- Graceful `EndFrame` flush (bounded 3 s window) and `CancelFrame` salvage.
-- Tested with Pipecat v1.8.1 through `pipecat.tests.utils.run_test`
-  (14-test suite).