langstate 0.2.2__tar.gz → 0.2.4__tar.gz

{langstate-0.2.2 → langstate-0.2.4}/BENCHMARK-PROVENANCE.md

@@ -16,10 +16,13 @@ Why they are excluded from current claims:
 
 - The JSON does not record a run time, source commit, full model identifier, or
   reproducible decoding/runtime configuration.
-- `bench_adapters.py` imports the legacy repository-root `adapters.py` and
-  `compress.py`, not the shipping modules under `src/langstate/`; both pairs
-  differ at current `main` commit
-  `b9a8b036ab65d0a74d27a580602cd12afd438235`.
+- When these rows were recorded, `bench_adapters.py` imported legacy
+  repository-root copies of `adapters.py` and `compress.py` (default models
+  qwen3:14b and claude-opus-4-7), not the shipping modules under
+  `src/langstate/`; both pairs differed at `main` commit
+  `b9a8b036ab65d0a74d27a580602cd12afd438235`. The root copies have since been
+  removed and the script now imports `langstate.*`, so a fresh run uses the
+  current shipping defaults rather than the configuration behind these bytes.
 - `bench_results.json` names missing-fact keys such as `corpus_size`,
   `arxiv_papers`, and `benchmarks` that are not in the current runner's
   `GROUND_TRUTH` list. The current script therefore cannot reproduce the stored

{langstate-0.2.2 → langstate-0.2.4}/MANIFEST.in

@@ -4,6 +4,5 @@
 include BENCHMARK-PROVENANCE.md
 prune tests
 prune bench
-prune _workspace
 global-exclude te_check.py
 global-exclude *.pyc

{langstate-0.2.2/src/langstate.egg-info → langstate-0.2.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: langstate
-Version: 0.2.2
+Version: 0.2.4
 Summary: Inspectable context compression for LLM conversations, with visible scaffold state and lexical fact receipts
 Author-email: Rolando Bosch <roli@hermes-labs.ai>
 Maintainer-email: Hermes Labs <info@hermes-labs.ai>
@@ -28,6 +28,8 @@ Dynamic: license-file
 
 # langstate
 
+[![Tests](https://github.com/hermes-labs-ai/langstate/actions/workflows/test.yml/badge.svg)](https://github.com/hermes-labs-ai/langstate/actions/workflows/test.yml)
+
 **Compress long LLM conversations into visible working state — then check that
 the literal facts you care about survived.**
 
@@ -49,12 +51,19 @@ semantic fidelity.
 ## Try it in two minutes
 
 ```bash
-python -m pip install langstate==0.2.2
+python -m pip install langstate==0.2.4
+langstate --version
 langstate demo
 ```
 
+Or install the CLI from the [Hermes Labs Homebrew tap](https://github.com/hermes-labs-ai/homebrew-tap):
+
+```bash
+brew install hermes-labs-ai/tap/langstate
+```
+
 It prints the compressed messages and a machine-readable receipt showing that
-both named facts survived. The proof needs no model, API key, network call, or
+both named facts survived and that the history got smaller. The proof needs no model, API key, network call, or
 Ollama installation.
 
 Python 3.10+; no runtime dependencies beyond the standard library. To use the
@@ -147,7 +156,7 @@ contract; automatic fact extraction is a convenience heuristic.
 
 ## How it works
 
-For conversations of at least six user/assistant turns, `compress`:
+For conversations of at least six user/assistant turn pairs, `compress`:
 
 1. retains all `system` messages and the requested recent suffix verbatim;
 2. sends the older non-system messages to the selected summarizer;
@@ -209,5 +218,10 @@ compress(
 validate(before, after, facts=None)
 ```
 
+`preserve_recent` counts recent user-initiated turns. Each retained turn includes
+its assistant messages, tool calls, and tool results verbatim, so the retained
+tail does not begin in the middle of a tool exchange. Older history remains
+a lossy summary; retain the original messages when exact replay is required.
+
 See `Receipt.as_dict()` for a JSON-friendly receipt. The package is
 Apache-2.0 licensed.

{langstate-0.2.2 → langstate-0.2.4}/README.md

@@ -1,5 +1,7 @@
 # langstate
 
+[![Tests](https://github.com/hermes-labs-ai/langstate/actions/workflows/test.yml/badge.svg)](https://github.com/hermes-labs-ai/langstate/actions/workflows/test.yml)
+
 **Compress long LLM conversations into visible working state — then check that
 the literal facts you care about survived.**
 
@@ -21,12 +23,19 @@ semantic fidelity.
 ## Try it in two minutes
 
 ```bash
-python -m pip install langstate==0.2.2
+python -m pip install langstate==0.2.4
+langstate --version
 langstate demo
 ```
 
+Or install the CLI from the [Hermes Labs Homebrew tap](https://github.com/hermes-labs-ai/homebrew-tap):
+
+```bash
+brew install hermes-labs-ai/tap/langstate
+```
+
 It prints the compressed messages and a machine-readable receipt showing that
-both named facts survived. The proof needs no model, API key, network call, or
+both named facts survived and that the history got smaller. The proof needs no model, API key, network call, or
 Ollama installation.
 
 Python 3.10+; no runtime dependencies beyond the standard library. To use the
@@ -119,7 +128,7 @@ contract; automatic fact extraction is a convenience heuristic.
 
 ## How it works
 
-For conversations of at least six user/assistant turns, `compress`:
+For conversations of at least six user/assistant turn pairs, `compress`:
 
 1. retains all `system` messages and the requested recent suffix verbatim;
 2. sends the older non-system messages to the selected summarizer;
@@ -181,5 +190,10 @@ compress(
 validate(before, after, facts=None)
 ```
 
+`preserve_recent` counts recent user-initiated turns. Each retained turn includes
+its assistant messages, tool calls, and tool results verbatim, so the retained
+tail does not begin in the middle of a tool exchange. Older history remains
+a lossy summary; retain the original messages when exact replay is required.
+
 See `Receipt.as_dict()` for a JSON-friendly receipt. The package is
 Apache-2.0 licensed.

{langstate-0.2.2 → langstate-0.2.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "langstate"
-version = "0.2.2"
+version = "0.2.4"
 description = "Inspectable context compression for LLM conversations, with visible scaffold state and lexical fact receipts"
 readme = "README.md"
 license = "Apache-2.0"

{langstate-0.2.2 → langstate-0.2.4}/src/langstate/__init__.py

@@ -3,5 +3,5 @@
 from langstate.compress import compress
 from langstate.validate import Receipt, extract_facts, validate
 
-__version__ = "0.2.2"
-__all__ = ["compress", "validate", "Receipt", "extract_facts", "__version__"]
+__version__ = "0.2.4"
+__all__ = ["Receipt", "__version__", "compress", "extract_facts", "validate"]

{langstate-0.2.2 → langstate-0.2.4}/src/langstate/adapters.py

@@ -17,12 +17,13 @@ Production targets (one cheap default per provider, per the Hermes spec):
 
 from __future__ import annotations
 
+import http.client
 import json
 import os
 import time
 import urllib.error
 import urllib.request
-from typing import Callable, Optional
+from collections.abc import Callable
 
 Summarizer = Callable[[str], str]
 
@@ -82,7 +83,7 @@ DEFAULT_OPENAI_MODEL = "gpt-4o-mini"
 
 def openai(
     model: str = DEFAULT_OPENAI_MODEL,
-    api_key: Optional[str] = None,
+    api_key: str | None = None,
     url: str = OPENAI_URL,
     max_tokens: int = 4000,
     temperature: float = 0.2,
@@ -138,7 +139,7 @@ ANTHROPIC_VERSION = "2023-06-01"
 
 def anthropic(
     model: str = DEFAULT_ANTHROPIC_MODEL,
-    api_key: Optional[str] = None,
+    api_key: str | None = None,
     url: str = ANTHROPIC_URL,
     max_tokens: int = 4000,
     temperature: float = 0.2,
@@ -210,7 +211,15 @@ def probe(name: str) -> dict:
         out = fn("Say OK.")
     except AdapterUnavailable as e:
         return {"name": name, "available": False, "reason": str(e), "latency_ms": None}
-    except Exception as e:  # noqa: BLE001
+    except (
+        AttributeError,
+        http.client.HTTPException,
+        IndexError,
+        KeyError,
+        RuntimeError,
+        TypeError,
+        ValueError,
+    ) as e:
         return {"name": name, "available": False, "reason": f"{type(e).__name__}: {e}", "latency_ms": None}
     return {
         "name": name,

{langstate-0.2.2 → langstate-0.2.4}/src/langstate/cli.py

@@ -10,16 +10,42 @@ from . import __version__, compress, validate
 
 
 def _demo_summary(_: str) -> str:
-    return "- Project: Atlas\\n- Budget: $4,000"
+    return "- Project: Atlas (Q4 build-cluster migration)\n- Budget: $4,000 cap, no overrun"
 
 
 def _demo_payload() -> dict[str, object]:
     before = [
         {"role": "system", "content": "Preserve named project facts."},
-        {"role": "user", "content": "The project is Atlas."},
-        {"role": "assistant", "content": "Recorded Project: Atlas."},
-        {"role": "user", "content": "The budget is $4,000."},
-        {"role": "assistant", "content": "Recorded Budget: $4,000."},
+        {
+            "role": "user",
+            "content": (
+                "We are kicking off the Q4 infrastructure work. For tracking, "
+                "the project is called Atlas, and it covers moving every build "
+                "job from the old cluster to the new one before the freeze."
+            ),
+        },
+        {
+            "role": "assistant",
+            "content": (
+                "Recorded Project: Atlas, the Q4 migration of all build jobs "
+                "from the old cluster to the new one ahead of the freeze."
+            ),
+        },
+        {
+            "role": "user",
+            "content": (
+                "Finance signed off yesterday. The budget is $4,000 for the "
+                "whole migration, contractor hours included, and we cannot go "
+                "above that without a fresh approval."
+            ),
+        },
+        {
+            "role": "assistant",
+            "content": (
+                "Recorded Budget: $4,000, all-inclusive, with no overrun "
+                "allowed without a fresh approval."
+            ),
+        },
         {"role": "user", "content": "What remains?"},
         {"role": "assistant", "content": "Review the release packet."},
     ]

{langstate-0.2.2 → langstate-0.2.4}/src/langstate/compress.py

@@ -11,13 +11,13 @@ Usage:
     response = client.chat.completions.create(messages=compressed, model="gpt-4o")
 """
 
-from typing import Callable, Optional
+from collections.abc import Callable
 
 from langstate import adapters as _adapters
 
 OLLAMA_URL = "http://localhost:11434/api/chat"
 DEFAULT_MODEL = "qwen3:4b"
-PRESERVE_RECENT = 4  # keep last N user/assistant turns verbatim
+PRESERVE_RECENT = 4  # keep last N user/assistant turn pairs verbatim
 
 Summarizer = Callable[[str], str]
 
@@ -58,28 +58,43 @@ def compress(
     model: str = DEFAULT_MODEL,
     ollama_url: str = OLLAMA_URL,
     min_turns_to_compress: int = 6,
-    summarizer: Optional[Summarizer] = None,
+    summarizer: Summarizer | None = None,
 ) -> list[dict]:
     """Summarize older messages while keeping system and recent turns verbatim.
 
     Args:
         messages: List of dicts with "role" and "content" keys.
-        preserve_recent: Number of recent user/assistant turn pairs to keep verbatim.
+        preserve_recent: Number of recent user-initiated turns to keep verbatim,
+            including their assistant messages and tool exchanges.
         model: Ollama model to use when no summarizer is given (back-compat).
         ollama_url: Ollama API endpoint (back-compat).
-        min_turns_to_compress: Don't compress if fewer than this many complete
-            user/assistant turn pairs are present.
+        min_turns_to_compress: Don't compress if fewer than this many
+            user-initiated turns are present.
         summarizer: Optional ``summarize(prompt) -> str`` callable. If None, falls back
             to a local Ollama adapter (qwen3:4b) for back-compat with the MVP.
 
     Returns:
         Compressed messages list in the same OpenAI format.
+
+    Raises:
+        ValueError: If ``preserve_recent`` is not a non-negative integer.
+        TypeError: If the summarizer returns a non-string.
+        RuntimeError: If the summarizer returns empty or whitespace-only text.
     """
+    if (
+        isinstance(preserve_recent, bool)
+        or not isinstance(preserve_recent, int)
+        or preserve_recent < 0
+    ):
+        raise ValueError(
+            f"preserve_recent must be a non-negative integer, got {preserve_recent!r}"
+        )
+
     if not messages:
         return []
 
-    # Count actual user/assistant turns, not total messages
-    n_turns = sum(1 for m in messages if m.get("role") in ("user", "assistant")) // 2
+    # Tool-call assistant messages do not start additional user turns.
+    n_turns = sum(m.get("role") == "user" for m in messages)
     if n_turns < min_turns_to_compress:
         return list(messages)
 
@@ -90,14 +105,21 @@ def compress(
     if not conv_msgs:
         return list(messages)
 
-    # Split: history to compress vs recent turns to preserve
-    # preserve_recent refers to turn pairs, so preserve_recent*2 messages
-    preserve_count = preserve_recent * 2
-    if len(conv_msgs) <= preserve_count:
-        return list(messages)
-
-    history_msgs = conv_msgs[:-preserve_count]
-    recent_msgs = conv_msgs[-preserve_count:]
+    # Keep entire user-initiated turns, including assistant tool calls and
+    # their results. Counting raw messages can split a tool exchange.
+    if preserve_recent:
+        user_indexes = [
+            index for index, message in enumerate(conv_msgs)
+            if message.get("role") == "user"
+        ]
+        if len(user_indexes) <= preserve_recent:
+            return list(messages)
+        recent_start = user_indexes[-preserve_recent]
+        history_msgs = conv_msgs[:recent_start]
+        recent_msgs = conv_msgs[recent_start:]
+    else:
+        history_msgs = conv_msgs
+        recent_msgs = []
 
     # Format history for summarization
     history_text = "\n".join(
@@ -109,6 +131,18 @@ def compress(
         summarizer = _adapters.ollama(model=model, url=ollama_url)
     summary = summarizer(_build_compression_prompt(history_text))
 
+    # Refuse to replace history with nothing: an empty scaffold would silently
+    # drop the compressed messages instead of summarizing them.
+    if not isinstance(summary, str):
+        raise TypeError(
+            f"summarizer must return str, got {type(summary).__name__}"
+        )
+    if not summary.strip():
+        raise RuntimeError(
+            f"summarizer returned empty output; refusing to drop "
+            f"{len(history_msgs)} messages without a summary"
+        )
+
     # Build compressed messages
     compressed = []
 

{langstate-0.2.2 → langstate-0.2.4}/src/langstate/validate.py

@@ -57,6 +57,17 @@ def _norm(s: str) -> str:
     return _WS.sub(" ", s).strip().lower()
 
 
+def _contains_fact(haystack: str, key: str) -> bool:
+    """True iff normalized ``key`` occurs in ``haystack`` at lexical boundaries.
+
+    Plain substring membership would credit ``42`` against ``142`` or ``API``
+    against ``rapid``. Requiring a non-word (or string-edge) character on both
+    sides of the match blocks those, while leaving punctuation-led facts like
+    ``$4,000`` / ``90.1%`` and multiword facts untouched.
+    """
+    return re.search(r"(?<!\w)" + re.escape(key) + r"(?!\w)", haystack) is not None
+
+
 def _messages_text(messages: list[dict]) -> str:
     return "\n".join(m.get("content", "") or "" for m in messages)
 
@@ -164,9 +175,10 @@ def validate(
             (heuristic — pass an explicit list for a precise receipt).
 
     Returns:
-        A :class:`Receipt`. A fact "survived" if its normalized text is a
-        substring of the concatenated, normalized ``after`` content — a lexical,
-        conservative check that under-counts rather than over-claims.
+        A :class:`Receipt`. A fact "survived" if its normalized text occurs
+        at word boundaries in the concatenated, normalized ``after`` content —
+        a lexical, conservative check that under-counts rather than over-claims
+        (``42`` does not survive inside ``142``, nor ``API`` inside ``rapid``).
     """
     if facts is None:
         facts = extract_facts(_messages_text(before))
@@ -176,7 +188,7 @@ def validate(
     dropped: list[str] = []
     for fact in facts:
         key = _norm(fact)
-        if key and key in after_text:
+        if key and _contains_fact(after_text, key):
             survived.append(fact)
         else:
             dropped.append(fact)

{langstate-0.2.2 → langstate-0.2.4/src/langstate.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: langstate
-Version: 0.2.2
+Version: 0.2.4
 Summary: Inspectable context compression for LLM conversations, with visible scaffold state and lexical fact receipts
 Author-email: Rolando Bosch <roli@hermes-labs.ai>
 Maintainer-email: Hermes Labs <info@hermes-labs.ai>
@@ -28,6 +28,8 @@ Dynamic: license-file
 
 # langstate
 
+[![Tests](https://github.com/hermes-labs-ai/langstate/actions/workflows/test.yml/badge.svg)](https://github.com/hermes-labs-ai/langstate/actions/workflows/test.yml)
+
 **Compress long LLM conversations into visible working state — then check that
 the literal facts you care about survived.**
 
@@ -49,12 +51,19 @@ semantic fidelity.
 ## Try it in two minutes
 
 ```bash
-python -m pip install langstate==0.2.2
+python -m pip install langstate==0.2.4
+langstate --version
 langstate demo
 ```
 
+Or install the CLI from the [Hermes Labs Homebrew tap](https://github.com/hermes-labs-ai/homebrew-tap):
+
+```bash
+brew install hermes-labs-ai/tap/langstate
+```
+
 It prints the compressed messages and a machine-readable receipt showing that
-both named facts survived. The proof needs no model, API key, network call, or
+both named facts survived and that the history got smaller. The proof needs no model, API key, network call, or
 Ollama installation.
 
 Python 3.10+; no runtime dependencies beyond the standard library. To use the
@@ -147,7 +156,7 @@ contract; automatic fact extraction is a convenience heuristic.
 
 ## How it works
 
-For conversations of at least six user/assistant turns, `compress`:
+For conversations of at least six user/assistant turn pairs, `compress`:
 
 1. retains all `system` messages and the requested recent suffix verbatim;
 2. sends the older non-system messages to the selected summarizer;
@@ -209,5 +218,10 @@ compress(
 validate(before, after, facts=None)
 ```
 
+`preserve_recent` counts recent user-initiated turns. Each retained turn includes
+its assistant messages, tool calls, and tool results verbatim, so the retained
+tail does not begin in the middle of a tool exchange. Older history remains
+a lossy summary; retain the original messages when exact replay is required.
+
 See `Receipt.as_dict()` for a JSON-friendly receipt. The package is
 Apache-2.0 licensed.