ctx-compact 0.1.0__py3-none-any.whl

ctx_compact-0.1.0.dist-info/METADATA

@@ -0,0 +1,141 @@
+Metadata-Version: 2.5
+Name: ctx-compact
+Version: 0.1.0
+Summary: Trim a plain OpenAI-shaped message array down to a token budget without ever orphaning a tool result.
+Project-URL: Homepage, https://github.com/pjdurden/ctx-compact
+Project-URL: Source, https://github.com/pjdurden/ctx-compact
+Author: Prajjwal Chittori
+License: MIT
+Keywords: compaction,context-window,conversation,llm,openai,token-budget,tool-calls
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Typing :: Typed
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+
+# ctx-compact
+
+Trim a plain OpenAI-shaped message array down to a token budget without ever orphaning a tool result. This is the Python port of the [ctx-compact](https://www.npmjs.com/package/ctx-compact) npm package.
+
+## The problem
+
+Every agent framework ships its own conversation compaction (LangGraph, Inspect, MS Agent Framework, the Claude SDK all have one), and each is welded to that framework's message type. If you are working with a plain list of `{role, content, ...}` messages, people tend to hand-roll a "drop the oldest N messages" loop. That works until an assistant message with `tool_calls` gets dropped but its matching `tool` result messages do not (or the reverse). Most providers reject that shape outright, so the trim silently turns into an API error on the next call. `ctx-compact` is a small, framework-neutral compactor that keeps tool-call and tool-result messages paired and dropped or kept as a unit.
+
+## Install
+
+```
+pip install ctx-compact
+```
+
+## Usage
+
+```python
+from ctx_compact import compact, compact_with_summary
+
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": "What is the weather in Denver?"},
+    {
+        "role": "assistant",
+        "content": None,
+        "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": '{"city":"Denver"}'}}],
+    },
+    {"role": "tool", "tool_call_id": "call_1", "content": '{"tempF":72}'},
+    {"role": "assistant", "content": "It is 72F in Denver."},
+    {"role": "user", "content": "What about Austin?"},
+    {
+        "role": "assistant",
+        "content": None,
+        "tool_calls": [{"id": "call_2", "type": "function", "function": {"name": "get_weather", "arguments": '{"city":"Austin"}'}}],
+    },
+    {"role": "tool", "tool_call_id": "call_2", "content": '{"tempF":88}'},
+    {"role": "assistant", "content": "It is 88F in Austin."},
+    {"role": "user", "content": "And tomorrow in Denver?"},
+]
+
+result = compact(messages, max_tokens=150, keep_head=1, keep_tail=2)
+# result.tokens_before -> 195
+# result.tokens_after  -> 124
+# result.fits          -> True (124 <= 150)
+# result.dropped       -> the 3 oldest droppable messages: the first "What is
+#                          the weather in Denver?" turn and its whole
+#                          assistant/tool_calls + tool group
+
+# Synchronous variant: summarize whatever got dropped and splice a note back in.
+with_summary = compact_with_summary(
+    messages,
+    max_tokens=150,
+    keep_head=1,
+    keep_tail=2,
+    summarize=lambda dropped: f"Earlier in this conversation: {len(dropped)} messages were removed.",
+)
+# with_summary.summary      -> 'Earlier in this conversation: 3 messages were removed.'
+# with_summary.tokens_after -> 145 (the 124 kept after dropping, plus the inserted summary message)
+# with_summary.fits         -> True (145 <= 150)
+```
+
+The example above is exact output from running this code against this package.
+
+## API
+
+### `compact(messages, *, max_tokens, count_tokens=None, keep_head=1, keep_tail=4) -> CompactResult`
+
+Drops messages from the middle of `messages` until the estimated token count fits the budget.
+
+- `messages`: list of dicts shaped like `{role, content, tool_calls?, tool_call_id?, name?}`. `role` is one of `'system' | 'user' | 'assistant' | 'tool'`.
+- `max_tokens` (required, keyword-only, `float`) - the budget. Raises `TypeError` if missing or not a positive number.
+- `count_tokens` - `(message) -> int`. Default: `math.ceil(len(json.dumps(message, separators=(",", ":"), ensure_ascii=False)) / 4)`, with the length measured in UTF-16 code units (matching JavaScript's `String.length`), not Python codepoints.
+- `keep_head` - number of leading messages always kept. Default `1`.
+- `keep_tail` - number of trailing messages always kept. Default `4`.
+
+Returns a frozen dataclass:
+
+```python
+@dataclass(frozen=True)
+class CompactResult:
+    messages: list       # the compacted list
+    dropped: list        # what was removed, in original order
+    tokens_before: int   # summed estimated tokens of the input list
+    tokens_after: int    # summed estimated tokens of the output list
+    fits: bool           # tokens_after <= max_tokens
+```
+
+If the input already fits, it is returned unchanged with `dropped: []`.
+
+### `compact_with_summary(messages, *, max_tokens, count_tokens=None, keep_head=1, keep_tail=4, summarize=None, summary_role="user") -> CompactResultWithSummary`
+
+Same as `compact`, then, if anything was dropped and `summarize` is provided, calls `summarize(dropped)` and inserts the returned string as a message `{"role": summary_role, "content": <summary>}` immediately after the head-kept messages.
+
+- `summarize` - `(dropped: list) -> str`. If omitted, behaves exactly like `compact` and returns `summary: None`.
+- `summary_role` - default `'user'`. Some providers reject a second `system` message, which is why the default is `'user'` rather than `'system'`.
+
+The inserted summary message counts toward the budget: after insertion the result is re-checked, and if it no longer fits, additional whole groups are dropped (oldest first) to make room. `fits` is reported `False` if it is still over budget after that.
+
+Returns a frozen dataclass with the same fields as `CompactResult` plus `summary: Optional[str]`.
+
+**Difference from the JavaScript version:** the JS `compactWithSummary` is `async` and does `await options.summarize(dropped)`, since JS summarizers are typically an async LLM call. This Python port is **synchronous**: `summarize` is a plain callable, `dropped -> str`, called directly with no `await`. If your summarizer needs to be async in Python, run it yourself (e.g. via `asyncio.run` or your event loop) before calling `compact_with_summary`, and pass a synchronous wrapper.
+
+### `estimate_tokens(message, count_tokens=None) -> int`
+
+Runs `count_tokens` (or the default heuristic) against a single message. Exported so callers can reuse the same estimator `compact`/`compact_with_summary` use, e.g. to pre-check a message before appending it.
+
+## How it works
+
+1. **Group first.** Before anything is dropped, the whole list is split into groups: an assistant message carrying `tool_calls` plus every immediately-following `tool` message whose `tool_call_id` matches one of that assistant's `tool_calls[].id` forms one group. Every other message (including a `tool` message with no matching assistant) is its own group. Groups are always dropped or kept whole, so a tool result is never left without its assistant call, or vice versa.
+2. **Snap keep_head/keep_tail to group boundaries.** `keep_head` and `keep_tail` are counted in messages, but if the boundary would land inside a group, it expands outward to keep that whole group.
+3. **Drop oldest-first.** Whatever is left in the middle is droppable. Groups are dropped oldest first until the running token total fits `max_tokens` or nothing droppable is left.
+4. Token counts are an estimate (`~length / 4` by default, over the compact JSON serialization of the message, or your own `count_tokens`), not a real tokenizer. There is no LLM call, no tokenizer library, and no streaming. If your `count_tokens` is inaccurate, `fits` will be inaccurate too. Pass a `count_tokens` backed by your provider's real tokenizer if you need exact numbers.
+5. `compact_with_summary` never re-summarizes after dropping additional groups to make room for the summary itself; it just drops more of the already-dropped-eligible messages. If you need every dropped message reflected in the summary text, make sure `max_tokens` leaves enough headroom for the summary you expect `summarize` to produce.
+6. The default estimator uses `json.dumps(message, separators=(",", ":"), ensure_ascii=False)`, matching the length JavaScript's `JSON.stringify` plus `.length` produces. Two of Python's `json.dumps` defaults disagree with `JSON.stringify` and both are overridden here:
+   - `separators` defaults to `", "` and `": "` (with spaces) in Python; `JSON.stringify` never inserts spaces. Passing plain `json.dumps(message)` would inflate every count.
+   - `ensure_ascii` defaults to `True` in Python, which `\uXXXX`-escapes every non-ASCII codepoint (accents, CJK, emoji); `JSON.stringify` never escapes non-ASCII text. Leaving `ensure_ascii` at its default would silently inflate the token count, and therefore over-compact, for any non-English or emoji-bearing content.
+
+   A third, more subtle difference is corrected for internally rather than in the `json.dumps` call: JavaScript's `String.length` counts UTF-16 code units, so a character outside the Basic Multilingual Plane (most emoji, e.g. U+1F389) counts as a 2-unit surrogate pair there, while Python's `len()` counts it as a single codepoint. The estimator measures the serialized JSON's length in UTF-16 code units (not `len()` codepoints) so astral-plane characters do not silently disagree with the JS package's token numbers for the same message.
+
+See the JavaScript version at the repo root (`../index.js`, `../README.md`) for the original implementation this port matches.
+
+## License
+
+MIT

ctx_compact-0.1.0.dist-info/RECORD

@@ -0,0 +1,5 @@
+ctx_compact.py,sha256=duTx7vns837MY9vBxr3q9JbyWD-pfOXEXTE13J0r-J0,13663
+py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+ctx_compact-0.1.0.dist-info/METADATA,sha256=L8W8XvEQEIlUnPooOy4I52lUCh7IJ6cFopsr9h1yBTA,9818
+ctx_compact-0.1.0.dist-info/WHEEL,sha256=zOwg4jB6zX2kU910N-cMawjivD6tO8NEWvE12je1bVk,87
+ctx_compact-0.1.0.dist-info/RECORD,,

ctx_compact-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.32.0
+Root-Is-Purelib: true
+Tag: py3-none-any

ctx_compact.py

@@ -0,0 +1,380 @@
+"""
+ctx-compact - framework-neutral conversation compaction for plain
+OpenAI-shaped message arrays. See README.md for the full picture.
+
+This is a Python port of the ctx-compact npm package. The JavaScript
+source (../index.js in the same repo) is the specification; this module
+matches its behavior exactly, with one intentional API difference:
+`compact_with_summary` is synchronous here, while the JS `compactWithSummary`
+is async. See README.md for details.
+"""
+
+import json
+import math
+from dataclasses import dataclass
+from typing import Any, Callable, Dict, List, Optional
+
+Message = Dict[str, Any]
+CountTokensFn = Callable[[Message], int]
+
+
+def _utf16_length(s: str) -> int:
+    """Number of UTF-16 code units in s, matching JavaScript's
+    String.prototype.length. JS strings are UTF-16 internally, so any
+    codepoint outside the Basic Multilingual Plane (astral-plane
+    characters, which includes most emoji, e.g. U+1F389) counts as a
+    2-unit surrogate pair there, even though it is a single Python
+    string character (Python's len() counts Unicode codepoints, not
+    UTF-16 units). Encoding to UTF-16LE and counting 2-byte units
+    reproduces JS's notion of length exactly, including that surrogate
+    pair doubling.
+    """
+    return len(s.encode("utf-16-le")) // 2
+
+
+def _default_count_tokens(message: Message) -> int:
+    """Default token estimator: ~4 chars per token on the JSON-serialized
+    message. Good enough for budgeting, not a real tokenizer.
+
+    Three ways Python's json.dumps/str differ from JavaScript's
+    JSON.stringify/String.length must be overridden or corrected for so
+    the lengths (and therefore token counts) match:
+
+    - separators: json.dumps defaults to ", " and ": " (with spaces);
+      JSON.stringify never inserts spaces. We pass separators=(",", ":").
+    - ensure_ascii: json.dumps defaults to True, which \\uXXXX-escapes
+      every non-ASCII codepoint. JSON.stringify never escapes non-ASCII
+      text, it emits it as literal characters. We pass
+      ensure_ascii=False so accented text, CJK, and emoji are not
+      inflated by escaping.
+    - string length semantics: Python's len() on a str counts Unicode
+      codepoints; JavaScript's .length (which is what JSON.stringify's
+      result length uses) counts UTF-16 code units, so an astral-plane
+      character such as an emoji counts as 2 in JS but 1 in plain Python
+      len(). We use _utf16_length() instead of len() so that difference
+      does not creep back in for emoji-bearing content.
+    """
+    serialized = json.dumps(message, separators=(",", ":"), ensure_ascii=False)
+    return math.ceil(_utf16_length(serialized) / 4)
+
+
+def estimate_tokens(message: Message, count_tokens: Optional[CountTokensFn] = None) -> int:
+    """Estimate the token cost of a single message.
+
+    Args:
+        message: A message object.
+        count_tokens: Custom estimator. Defaults to a ~4-chars-per-token
+            heuristic over the JSON-serialized message.
+
+    Returns:
+        Estimated token count.
+    """
+    fn = count_tokens if count_tokens is not None else _default_count_tokens
+    return fn(message)
+
+
+def _validate_max_tokens(max_tokens: Any) -> float:
+    if (
+        isinstance(max_tokens, bool)
+        or not isinstance(max_tokens, (int, float))
+        or not math.isfinite(max_tokens)
+        or max_tokens <= 0
+    ):
+        raise TypeError("max_tokens must be a positive number")
+    return max_tokens
+
+
+def _build_groups(messages: List[Message]) -> List[List[Message]]:
+    """Split messages into groups. An assistant message carrying
+    `tool_calls` forms one group with every immediately-following `tool`
+    message whose `tool_call_id` matches one of that assistant's
+    `tool_calls[].id`. Every other message (including an orphan `tool`
+    message) is its own group. Groups are dropped or kept whole.
+    """
+    groups: List[List[Message]] = []
+    i = 0
+    n = len(messages)
+    while i < n:
+        msg = messages[i]
+        tool_calls = msg.get("tool_calls") if isinstance(msg, dict) else None
+        if msg.get("role") == "assistant" and isinstance(tool_calls, list) and len(tool_calls) > 0:
+            ids = {tc.get("id") for tc in tool_calls}
+            group_messages = [msg]
+            j = i + 1
+            while (
+                j < n
+                and messages[j].get("role") == "tool"
+                and messages[j].get("tool_call_id") in ids
+            ):
+                group_messages.append(messages[j])
+                j += 1
+            groups.append(group_messages)
+            i = j
+        else:
+            groups.append([msg])
+            i += 1
+    return groups
+
+
+def _count_groups_covering_front(groups: List[List[Message]], keep_count: int) -> int:
+    """Walk groups from the front, accumulating message counts, until at
+    least `keep_count` messages are covered. Returns the number of whole
+    groups needed to do that (0 if keep_count <= 0).
+    """
+    if keep_count <= 0:
+        return 0
+    covered = 0
+    groups_used = 0
+    for g in groups:
+        if covered >= keep_count:
+            break
+        covered += len(g)
+        groups_used += 1
+    return groups_used
+
+
+def _count_groups_covering_back(groups: List[List[Message]], keep_count: int) -> int:
+    """Same as _count_groups_covering_front but walking from the back."""
+    if keep_count <= 0:
+        return 0
+    covered = 0
+    groups_used = 0
+    for g in reversed(groups):
+        if covered >= keep_count:
+            break
+        covered += len(g)
+        groups_used += 1
+    return groups_used
+
+
+@dataclass
+class _CoreResult:
+    """Internal result shared by compact() and compact_with_summary()."""
+
+    messages: List[Message]
+    dropped: List[Message]
+    tokens_before: int
+    tokens_after: int
+    fits: bool
+    head_kept_count: int
+    remaining_groups: List[List[Message]]
+    count_tokens_fn: CountTokensFn
+
+
+def _compact_core(
+    messages: List[Message],
+    max_tokens: Any,
+    count_tokens: Optional[CountTokensFn],
+    keep_head: int,
+    keep_tail: int,
+) -> _CoreResult:
+    max_tokens = _validate_max_tokens(max_tokens)
+    count_tokens_fn = count_tokens if count_tokens is not None else _default_count_tokens
+
+    tokens_before = sum(estimate_tokens(m, count_tokens_fn) for m in messages)
+
+    if tokens_before <= max_tokens:
+        return _CoreResult(
+            messages=list(messages),
+            dropped=[],
+            tokens_before=tokens_before,
+            tokens_after=tokens_before,
+            fits=True,
+            head_kept_count=0,
+            remaining_groups=[],
+            count_tokens_fn=count_tokens_fn,
+        )
+
+    groups = _build_groups(messages)
+    head_group_count = _count_groups_covering_front(groups, keep_head)
+    tail_group_count = _count_groups_covering_back(groups, keep_tail)
+
+    middle_start = head_group_count
+    middle_end = len(groups) - tail_group_count
+
+    if middle_start >= middle_end:
+        # keep_head + keep_tail (snapped to group boundaries) cover the
+        # whole array. Nothing is droppable.
+        return _CoreResult(
+            messages=list(messages),
+            dropped=[],
+            tokens_before=tokens_before,
+            tokens_after=tokens_before,
+            fits=False,
+            head_kept_count=0,
+            remaining_groups=[],
+            count_tokens_fn=count_tokens_fn,
+        )
+
+    head_groups = groups[:head_group_count]
+    middle_groups = groups[middle_start:middle_end]
+    tail_groups = groups[middle_end:]
+
+    current_tokens = tokens_before
+    dropped_groups: List[List[Message]] = []
+    kept_middle_groups: List[List[Message]] = []
+    for g in middle_groups:
+        if current_tokens > max_tokens:
+            group_tokens = sum(estimate_tokens(m, count_tokens_fn) for m in g)
+            current_tokens -= group_tokens
+            dropped_groups.append(g)
+        else:
+            kept_middle_groups.append(g)
+
+    kept_messages = (
+        [m for g in head_groups for m in g]
+        + [m for g in kept_middle_groups for m in g]
+        + [m for g in tail_groups for m in g]
+    )
+    dropped = [m for g in dropped_groups for m in g]
+    head_kept_count = sum(len(g) for g in head_groups)
+
+    return _CoreResult(
+        messages=kept_messages,
+        dropped=dropped,
+        tokens_before=tokens_before,
+        tokens_after=current_tokens,
+        fits=current_tokens <= max_tokens,
+        head_kept_count=head_kept_count,
+        remaining_groups=kept_middle_groups,
+        count_tokens_fn=count_tokens_fn,
+    )
+
+
+@dataclass(frozen=True)
+class CompactResult:
+    """Result of compact()."""
+
+    messages: List[Message]
+    dropped: List[Message]
+    tokens_before: int
+    tokens_after: int
+    fits: bool
+
+
+@dataclass(frozen=True)
+class CompactResultWithSummary:
+    """Result of compact_with_summary()."""
+
+    messages: List[Message]
+    dropped: List[Message]
+    tokens_before: int
+    tokens_after: int
+    fits: bool
+    summary: Optional[str]
+
+
+def compact(
+    messages: List[Message],
+    *,
+    max_tokens: Any,
+    count_tokens: Optional[CountTokensFn] = None,
+    keep_head: int = 1,
+    keep_tail: int = 4,
+) -> CompactResult:
+    """Drop messages from the middle of a conversation until the
+    estimated token count fits the budget, never splitting an
+    assistant/tool_calls group from its matching tool results.
+
+    Args:
+        messages: Plain OpenAI-shaped messages: dicts with
+            `role`, `content`, and optionally `tool_calls`, `tool_call_id`,
+            `name`.
+        max_tokens: Required token budget. Must be a positive number.
+            Raises TypeError if missing or not a positive number.
+        count_tokens: Custom token estimator. Defaults to
+            `math.ceil(utf16_length(json.dumps(message, separators=(",", ":"), ensure_ascii=False)) / 4)`,
+            where `utf16_length` counts UTF-16 code units (matching
+            JavaScript's `String.length`), not Python codepoints.
+        keep_head: Leading messages always kept (snapped outward to group
+            boundaries). Default 1.
+        keep_tail: Trailing messages always kept (snapped outward to group
+            boundaries). Default 4.
+
+    Returns:
+        CompactResult(messages, dropped, tokens_before, tokens_after, fits)
+    """
+    result = _compact_core(messages, max_tokens, count_tokens, keep_head, keep_tail)
+    return CompactResult(
+        messages=result.messages,
+        dropped=result.dropped,
+        tokens_before=result.tokens_before,
+        tokens_after=result.tokens_after,
+        fits=result.fits,
+    )
+
+
+def compact_with_summary(
+    messages: List[Message],
+    *,
+    max_tokens: Any,
+    count_tokens: Optional[CountTokensFn] = None,
+    keep_head: int = 1,
+    keep_tail: int = 4,
+    summarize: Optional[Callable[[List[Message]], str]] = None,
+    summary_role: str = "user",
+) -> CompactResultWithSummary:
+    """Same as compact(), then, if anything was dropped and `summarize` is
+    provided, calls `summarize(dropped)` and inserts the resulting string
+    as a message right after the head-kept messages. The inserted summary
+    counts toward the budget: if it pushes the result back over
+    max_tokens, additional groups are dropped (oldest first) to make room.
+
+    Note: unlike the JavaScript compactWithSummary (which is async and
+    awaits `summarize`), this Python port is synchronous. `summarize` is a
+    plain callable taking the dropped list and returning a string.
+
+    Args:
+        messages: Plain OpenAI-shaped messages.
+        max_tokens: Required token budget. Must be a positive number.
+        count_tokens: Custom token estimator.
+        keep_head: Leading messages always kept. Default 1.
+        keep_tail: Trailing messages always kept. Default 4.
+        summarize: Produces a summary of the dropped messages. If
+            omitted, behaves exactly like compact().
+        summary_role: Role for the inserted summary message. Defaults to
+            'user' since some providers reject a second 'system' message.
+
+    Returns:
+        CompactResultWithSummary(messages, dropped, tokens_before,
+        tokens_after, fits, summary)
+    """
+    base = _compact_core(messages, max_tokens, count_tokens, keep_head, keep_tail)
+
+    if len(base.dropped) == 0 or not callable(summarize):
+        return CompactResultWithSummary(
+            messages=base.messages,
+            dropped=base.dropped,
+            tokens_before=base.tokens_before,
+            tokens_after=base.tokens_after,
+            fits=base.fits,
+            summary=None,
+        )
+
+    max_tokens = _validate_max_tokens(max_tokens)
+    count_tokens_fn = base.count_tokens_fn
+    summary_text = summarize(base.dropped)
+    summary_message: Message = {"role": summary_role, "content": summary_text}
+
+    kept_messages = list(base.messages)
+    kept_messages.insert(base.head_kept_count, summary_message)
+    current_tokens = base.tokens_after + estimate_tokens(summary_message, count_tokens_fn)
+
+    extra_dropped: List[Message] = []
+    remaining = list(base.remaining_groups)
+    while current_tokens > max_tokens and remaining:
+        g = remaining.pop(0)
+        group_tokens = sum(estimate_tokens(m, count_tokens_fn) for m in g)
+        current_tokens -= group_tokens
+        extra_dropped.extend(g)
+        to_remove_ids = {id(m) for m in g}
+        kept_messages = [m for m in kept_messages if id(m) not in to_remove_ids]
+
+    return CompactResultWithSummary(
+        messages=kept_messages,
+        dropped=list(base.dropped) + extra_dropped,
+        tokens_before=base.tokens_before,
+        tokens_after=current_tokens,
+        fits=current_tokens <= max_tokens,
+        summary=summary_text,
+    )