contextpress 0.5.3__tar.gz → 0.5.5__tar.gz

contextpress-0.5.5/AGENTS.md

@@ -0,0 +1,45 @@
+# Notes for agents and contributors
+
+## Branches
+
+- **`dev`** — default working branch; all feature work and fixes land here first.
+- **`main`** — release branch; merge from `dev` when cutting a PyPI release.
+
+Always `git checkout dev` before making changes in this repo.
+
+## Where things live
+
+| Area | Path |
+|------|------|
+| Public API | `contextpress/core.py` → `ContextManager` |
+| I/O formats | `contextpress/normalizer.py` |
+| Stage presets | `contextpress/compression.py` (`STAGE_ORDER`, `VALID_STAGES`) |
+| Tier 1 stages | `contextpress/strategies/*.py` |
+| Tier 2 | `contextpress/llm/` (`LLMBackend`, `adapters.py`) |
+| Custom stages | `contextpress/registry.py` + `ContextManager.register_stage()` |
+| Orchestration | `contextpress/pipeline.py` (behavior contract at top of file) |
+| Profiles | `contextpress/profiles.py` |
+
+`contextpress/__init__.py` lazy-loads `ContextManager` so `from contextpress.models import Turn` avoids pulling optional Tier-2 / sumy import chains when possible.
+
+## Commands
+
+```bash
+pip install -e ".[dev]"
+pytest tests -q
+```
+
+## Invariants (non-negotiable)
+
+See the numbered block at the top of `contextpress/pipeline.py`. Tests assume Tier 1 is deterministic (no live LLM calls in `tests/`).
+
+## Stabilization backlog
+
+See [`AUDIT.md`](AUDIT.md) for pre-0.6.0 findings (contract gaps, refactors, fixture gaps). Prefer low-risk items from that backlog before adding stages or new dependencies.
+
+## Adding a stage
+
+1. Subclass `BaseStrategy`, implement `process`.
+2. Register it in `Pipeline._build_strategy`.
+3. Add the name to `compression.STAGE_ORDER` and presets if it should be user-selectable.
+4. Extend `profiles.Profile` / `PROFILES` if it needs per-type defaults.

contextpress-0.5.5/AUDIT.md

@@ -0,0 +1,89 @@
+# Pre-0.6.0 stabilization audit (0.5.5)
+
+Recorded after the 0.5.x ergonomics arc. This is the Phase 1 deliverable:
+findings + backlog for Phases 2–4. Behavior changes are deferred unless marked
+**fix now**.
+
+## Architecture (summary)
+
+```
+ContextManager (core.py)
+  → normalize_messages (normalizer.py)
+  → apply_stage_selection (compression.py)
+  → Pipeline.run (pipeline.py)
+      → filler → repetition → resolution → recency → budget
+      → optional Tier 2 LLM
+  → denormalize_output
+```
+
+Profiles: `chat`, `rag_doc`, `agent`. Tier 1 deps: nltk, scikit-learn, sumy, tiktoken.
+
+## Findings
+
+### Contract / correctness
+
+| ID | Severity | Finding | Action |
+|----|----------|---------|--------|
+| C1 | Medium | Invariant 1 says system turns are untouched; `BudgetStrategy` truncates system as last resort (`budget.py`) | Document in contract (Phase 2) or change behavior later |
+| C2 | Medium | LLM dedup failures in `Pipeline._run_llm_tier` fall back silently (no warning); summarize path warns | Add warning (Phase 2) |
+| C3 | Medium | Filler drops ack phrases like `"sounds good"` before resolution can use them (`filler.py` / `resolution.py`) | Document + fixture test (Phases 3–4) |
+| C4 | Low | `Turn.importance` / `Turn.resolved` set by resolution but unused by pipeline | Leave; optional cleanup later |
+
+### Duplication / maintainability (Phase 2)
+
+| ID | Finding | Action |
+|----|---------|--------|
+| R1 | TF-IDF + cosine similarity copied in `repetition.py`, `recency.py`, `resolution.py` | Extract shared helper |
+| R2 | Token counting duplicated in `stats.py` and `budget.py` | Budget uses `stats` helpers |
+| R3 | `_bootstrap.py` downloads unused `stopwords`; warning mentions recency (doesn't use NLTK) | Trim packages; fix message |
+| R4 | Strategies use ad hoc `deepcopy` vs `clone_turn` | Prefer `clone_turn` where safe |
+
+### Test / fixture gaps (Phases 3–4)
+
+| ID | Gap |
+|----|-----|
+| T1 | No assert that input `messages` are immutable after `compress()` |
+| T2 | No end-to-end `rag_doc` / richer `agent` pipeline tests |
+| T3 | Normalizer edge cases: `None`, bad types, tuple skip, LangChain turn-count mismatch |
+| T4 | No offline real-chat fixtures under `tests/fixtures/` |
+| T5 | Cross-stage filler → resolution interaction untested |
+
+### Normalizer fragility (document; fix only if fixtures break)
+
+| ID | Area |
+|----|------|
+| N1 | LangChain detection heuristic on first list element |
+| N2 | Dict/tuple skips change turn count vs input |
+| N3 | New turns (RESOLVED / LLM summary) lose `_original_dict` keys |
+| N4 | `apply_text_to_turn` only replaces the first text block |
+
+### Deferred (not in 0.5.x stabilization)
+
+- Resolution / normalizer rewrites
+- New pipeline stages
+- New LLM backends
+- New hard dependencies
+- Changing stage order without a proven bug
+
+## Phase plan
+
+| Phase | Version | Scope |
+|-------|---------|--------|
+| 1 Analysis | **0.5.5** | This document + backlog |
+| 2 Refactor | **0.5.6** | R1–R3, C1 doc, C2 warning (no semantic stage changes) |
+| 3 Fixtures | **0.5.7** | Offline chat samples + smoke/invariant tests |
+| 4 Fixes | **0.5.8** | Bugs proven by fixtures + T1–T5 regression tests |
+
+## Invariants checklist (from `pipeline.py`)
+
+1. System turns pass through stages untouched — **except** budget last-resort truncation (C1).
+2. Input never mutated — enforced by clones; **needs T1**.
+3. Output format mirrors input — denormalize; weaker after turn collapse.
+4. Last 3 non-system not compressed by recency — unit tested.
+5. Last 2 non-system not removed by budget — unit tested.
+6. Resolution (chat) needs both sides — unit tested; C3 may weaken signals.
+7. Repetition: more recent wins — unit tested.
+8. Tier 1 deterministic — assumed; sumy/NLTK have fallbacks.
+9. LLM failure → Tier 1 + warning — summarize yes; dedup **C2**.
+10. `token_budget=None` skips budget enforcement — OK.
+11–12. Presets + Tier 2 modes — covered by existing tests.

{contextpress-0.5.3 → contextpress-0.5.5}/CHANGELOG.md

@@ -4,6 +4,16 @@ All notable changes to `contextpress` are recorded here.
 The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.5.5] - 2026-07-26
+
+- **Stabilization Phase 1** — codebase audit and prioritized backlog in [`AUDIT.md`](AUDIT.md).
+- No runtime API changes; prep for low-risk refactors (0.5.6) and fixture-driven fixes (0.5.7–0.5.8).
+
+## [0.5.4] - 2026-07-25
+
+- **`compress_many()`** — batch ``compress()`` over a list of conversations.
+- Example: `examples/agent_pipeline.py` for ``type="agent"`` tool/task threads.
+
 ## [0.5.3] - 2026-07-24
 
 - **`recommend_preset()`** — pick the mildest preset (low → medium → high) that fits a token budget.

{contextpress-0.5.3 → contextpress-0.5.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: contextpress
-Version: 0.5.3
+Version: 0.5.5
 Summary: Deterministic context compression for LLM chat, RAG, and agent pipelines
 Project-URL: Homepage, https://github.com/Taha-azizi/contextpress
 Project-URL: Documentation, https://github.com/Taha-azizi/contextpress#readme
@@ -323,6 +323,12 @@ preset = cm.recommend_preset(messages, token_budget=500)
 out = cm.compress(messages, token_budget=500, compression=preset)
 ```
 
+**Batch compress** (0.5.4+):
+
+```python
+results = cm.compress_many(list_of_conversations, token_budget=2000, return_stats=True)
+```
+
 **Async** (0.5+):
 
 ```python
@@ -391,6 +397,8 @@ ContextManager(type="rag_doc")
 ContextManager(type="agent")
 ```
 
+Runnable agent example: [`examples/agent_pipeline.py`](examples/agent_pipeline.py).
+
 ## Pipeline stages
 
 1. **Filler** — Removes low-semantic filler words and (in chat/agent) drops acknowledgement-only assistant turns.
@@ -553,6 +561,7 @@ For academic use, cite this package in your paper’s software or methods sectio
 
 ## Extension and growth
 
+- **Stabilization audit** — See [`AUDIT.md`](AUDIT.md) for known contract gaps and the 0.5.5–0.5.8 backlog (refactors, fixtures, fixes).
 - **Custom stages** — Subclass `contextpress.strategies.base.BaseStrategy` and plug in via a custom `Pipeline` subclass or future registry hooks.
 - **Tier 2** — Implement `LLMBackend` (`summarize`, `deduplicate`) for provider-specific semantic compression; failures fall back to Tier 1.
 - **Presets API** — `from contextpress.compression import VALID_STAGES, STAGE_ORDER` for tooling and experiments.

{contextpress-0.5.3 → contextpress-0.5.5}/README.md

@@ -82,6 +82,12 @@ preset = cm.recommend_preset(messages, token_budget=500)
 out = cm.compress(messages, token_budget=500, compression=preset)
 ```
 
+**Batch compress** (0.5.4+):
+
+```python
+results = cm.compress_many(list_of_conversations, token_budget=2000, return_stats=True)
+```
+
 **Async** (0.5+):
 
 ```python
@@ -150,6 +156,8 @@ ContextManager(type="rag_doc")
 ContextManager(type="agent")
 ```
 
+Runnable agent example: [`examples/agent_pipeline.py`](examples/agent_pipeline.py).
+
 ## Pipeline stages
 
 1. **Filler** — Removes low-semantic filler words and (in chat/agent) drops acknowledgement-only assistant turns.
@@ -312,6 +320,7 @@ For academic use, cite this package in your paper’s software or methods sectio
 
 ## Extension and growth
 
+- **Stabilization audit** — See [`AUDIT.md`](AUDIT.md) for known contract gaps and the 0.5.5–0.5.8 backlog (refactors, fixtures, fixes).
 - **Custom stages** — Subclass `contextpress.strategies.base.BaseStrategy` and plug in via a custom `Pipeline` subclass or future registry hooks.
 - **Tier 2** — Implement `LLMBackend` (`summarize`, `deduplicate`) for provider-specific semantic compression; failures fall back to Tier 1.
 - **Presets API** — `from contextpress.compression import VALID_STAGES, STAGE_ORDER` for tooling and experiments.

{contextpress-0.5.3 → contextpress-0.5.5}/contextpress/__init__.py

@@ -16,7 +16,7 @@ __all__ = [
     "CompressionResult",
     "CompressionStats",
 ]
-__version__ = "0.5.3"
+__version__ = "0.5.5"
 
 
 def __getattr__(name: str) -> Any:

{contextpress-0.5.3 → contextpress-0.5.5}/contextpress/core.py

@@ -210,6 +210,33 @@ class ContextManager:
             return CompressionResult(messages=messages_out, stats=stats)
         return messages_out
 
+    def compress_many(
+        self,
+        conversations: list[Any],
+        token_budget: int | None = None,
+        *,
+        compression: str | None = None,
+        stages: list[str] | None = None,
+        disable: list[str] | None = None,
+        return_stats: bool = False,
+        dry_run: bool = False,
+    ) -> list[Any] | list[CompressionResult]:
+        """Run ``compress()`` on each conversation in ``conversations``."""
+        if not isinstance(conversations, list):
+            raise TypeError("conversations must be a list")
+        return [
+            self.compress(
+                messages,
+                token_budget=token_budget,
+                compression=compression,
+                stages=stages,
+                disable=disable,
+                return_stats=return_stats,
+                dry_run=dry_run,
+            )
+            for messages in conversations
+        ]
+
     async def compress_async(
         self,
         messages: Any,

contextpress-0.5.5/examples/agent_pipeline.py

@@ -0,0 +1,29 @@
+"""Agent thread compression — ``type=\"agent\"`` for tool-using pipelines."""
+
+from __future__ import annotations
+
+from contextpress import ContextManager
+
+messages = [
+    {"role": "system", "content": "You are a task agent with tools."},
+    {"role": "user", "content": "Check whether api-v2 is deployed to staging."},
+    {
+        "role": "assistant",
+        "content": "Checking deploy status <tool_call> get_deploy_status(api-v2)",
+    },
+    {"role": "user", "content": "We've decided on using the new pipeline for deploy."},
+    {"role": "assistant", "content": "Confirmed — staging deploy is scheduled."},
+    {"role": "user", "content": "What's the current status?"},
+    {"role": "assistant", "content": "Basically, honestly, " + "still rolling out " * 30},
+]
+
+cm = ContextManager(type="agent", compression="high")
+result = cm.compress(messages, token_budget=300, return_stats=True)
+
+print("context type:", result.stats.context_type)
+print("turns:", result.stats.turns_before, "->", result.stats.turns_after)
+print("tokens:", result.stats.tokens_before, "->", result.stats.tokens_after)
+print("stages:", result.stats.stages_run)
+for i, m in enumerate(result.messages):
+    preview = str(m.get("content", ""))[:120]
+    print(i, m["role"] + ":", preview)

{contextpress-0.5.3 → contextpress-0.5.5}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "contextpress"
-version = "0.5.3"
+version = "0.5.5"
 description = "Deterministic context compression for LLM chat, RAG, and agent pipelines"
 readme = "README.md"
 license = { file = "LICENSE" }
@@ -59,7 +59,7 @@ packages = ["contextpress"]
 "contextpress/py.typed" = "contextpress/py.typed"
 
 [tool.hatch.build.targets.sdist]
-include = ["/contextpress", "/tests", "/examples", "/README.md", "/LICENSE", "/NOTICE", "/CHANGELOG.md", "/CITATION.cff", "/CONTRIBUTING.md"]
+include = ["/contextpress", "/tests", "/examples", "/README.md", "/LICENSE", "/NOTICE", "/CHANGELOG.md", "/CITATION.cff", "/CONTRIBUTING.md", "/AUDIT.md", "/AGENTS.md"]
 
 [tool.black]
 line-length = 100

contextpress-0.5.5/tests/test_v054.py

@@ -0,0 +1,53 @@
+import pytest
+
+from contextpress import CompressionResult, ContextManager
+
+
+def test_compress_many_returns_list():
+    cm = ContextManager(type="chat")
+    batches = [
+        [{"role": "user", "content": "hello basically"}],
+        [{"role": "user", "content": "thanks basically"}],
+    ]
+    out = cm.compress_many(batches, token_budget=200)
+    assert isinstance(out, list)
+    assert len(out) == 2
+    assert all(isinstance(item, list) for item in out)
+
+
+def test_compress_many_with_stats():
+    cm = ContextManager(type="chat")
+    batches = [
+        [{"role": "user", "content": "hello basically there"}],
+        [{"role": "assistant", "content": "sounds good"}],
+    ]
+    results = cm.compress_many(batches, token_budget=500, return_stats=True)
+    assert len(results) == 2
+    assert all(isinstance(r, CompressionResult) for r in results)
+    assert results[0].stats.turns_before == 1
+    assert results[1].stats.turns_before == 1
+
+
+def test_compress_many_empty():
+    cm = ContextManager(type="chat")
+    assert cm.compress_many([], token_budget=100) == []
+
+
+def test_compress_many_requires_list():
+    cm = ContextManager(type="chat")
+    with pytest.raises(TypeError, match="conversations must be a list"):
+        cm.compress_many("not a list")  # type: ignore[arg-type]
+
+
+def test_agent_pipeline_compresses():
+    cm = ContextManager(type="agent", compression="high")
+    messages = [
+        {"role": "system", "content": "Agent."},
+        {"role": "user", "content": "Deploy api-v2?"},
+        {"role": "assistant", "content": "Using pipeline <tool_call> deploy"},
+        {"role": "user", "content": "We've decided on using the new pipeline for deploy."},
+        {"role": "assistant", "content": "Confirmed."},
+    ]
+    result = cm.compress(messages, token_budget=200, return_stats=True)
+    assert result.stats.context_type == "agent"
+    assert result.stats.tokens_after <= result.stats.tokens_before