caveman-middleware 0.1.0a1__tar.gz

caveman_middleware-0.1.0a1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Julius Brussee
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

caveman_middleware-0.1.0a1/PKG-INFO

@@ -0,0 +1,143 @@
+Metadata-Version: 2.4
+Name: caveman-middleware
+Version: 0.1.0a1
+Summary: Native framework adapters for the Caveman compression runtime.
+License-Expression: MIT
+Project-URL: Repository, https://github.com/JuliusBrussee/caveman
+Project-URL: Issues, https://github.com/JuliusBrussee/caveman/issues
+Classifier: Development Status :: 3 - Alpha
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Typing :: Typed
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: caveman-sdk<2,>=1.1
+Provides-Extra: openai
+Requires-Dist: openai<4,>=2.20; extra == "openai"
+Provides-Extra: anthropic
+Requires-Dist: anthropic<2,>=1.4; extra == "anthropic"
+Provides-Extra: google
+Requires-Dist: google-genai<3,>=2.22; extra == "google"
+Provides-Extra: langchain
+Requires-Dist: langchain<2,>=1.4; extra == "langchain"
+Requires-Dist: langchain-core<2,>=1.6; extra == "langchain"
+Requires-Dist: langgraph<2,>=1.2; extra == "langchain"
+Provides-Extra: litellm
+Requires-Dist: litellm<2,>=1.100; extra == "litellm"
+Provides-Extra: strands
+Requires-Dist: strands-agents<2,>=1.55; extra == "strands"
+Provides-Extra: agno
+Requires-Dist: agno<4,>=3.0; extra == "agno"
+Provides-Extra: asgi
+Requires-Dist: fastapi<1,>=0.141; extra == "asgi"
+Requires-Dist: starlette<2,>=1.6; extra == "asgi"
+Provides-Extra: mcp
+Requires-Dist: mcp<3,>=2.2; extra == "mcp"
+Provides-Extra: crewai
+Requires-Dist: crewai<2,>=1.15; extra == "crewai"
+Provides-Extra: pydantic-ai
+Requires-Dist: pydantic-ai<3,>=2.42; extra == "pydantic-ai"
+Provides-Extra: autogen
+Requires-Dist: autogen-agentchat<0.8,>=0.7; extra == "autogen"
+Requires-Dist: autogen-core<0.8,>=0.7; extra == "autogen"
+Requires-Dist: autogen-ext<0.8,>=0.7; extra == "autogen"
+Provides-Extra: llama-index
+Requires-Dist: llama-index-core<0.15,>=0.14; extra == "llama-index"
+Dynamic: license-file
+
+# Caveman Python middleware
+
+Native framework adapters over `caveman_cloud.middleware`. Your framework keeps
+its models, tool loop, retries, streams and stored conversation. Caveman only
+changes the text of selected tool results in the outbound model view, and only
+when the local runtime answers in time.
+
+Alpha. Install the extra for the framework you use:
+
+```sh
+pip install "caveman-middleware[langchain]"
+```
+
+```python
+from caveman_cloud.middleware import MiddlewareRuntime, Scope
+from caveman_middleware.langchain import with_caveman_agent
+from langchain.agents import create_agent
+
+runtime = MiddlewareRuntime(endpoint="http://127.0.0.1:8787")
+runtime.ready()
+scope = Scope("my-app", session_id="conversation-1", branch_id="main", cache_epoch="0")
+
+agent = create_agent(**with_caveman_agent(existing_agent_options, runtime=runtime, scope=scope))
+```
+
+Call `runtime.close()` at shutdown. Nothing in your stored message history is
+rewritten; only the request handed to the provider is.
+
+## Supported frameworks
+
+One extra per adapter. The range is the band the adapter's version gate accepts;
+the left-hand value is the version its wire handling was written against.
+
+| Extra | Module | Tested range |
+|---|---|---|
+| `langchain` | `caveman_middleware.langchain` | `langchain>=1.4,<2`, `langchain-core>=1.6,<2`, `langgraph>=1.2,<2` |
+| `litellm` | `caveman_middleware.litellm` | `litellm>=1.100,<2` |
+| `openai` | `caveman_middleware.openai` | `openai>=3.10,<4` |
+| `anthropic` | `caveman_middleware.anthropic` | `anthropic>=1.4,<2` |
+| `google` | `caveman_middleware.google` | `google-genai>=2.22,<3` |
+| `strands` | `caveman_middleware.strands` | `strands-agents>=1.55,<2` |
+| `agno` | `caveman_middleware.agno` | `agno>=3.0,<4` |
+| `crewai` | `caveman_middleware.crewai` | `crewai>=1.15,<2` |
+| `autogen` | `caveman_middleware.autogen` | `autogen-core`, `autogen-agentchat`, `autogen-ext` all `>=0.7,<0.8` |
+| `pydantic-ai` | `caveman_middleware.pydantic_ai` | `pydantic-ai-slim>=2.42,<3` |
+| `llama-index` | `caveman_middleware.llama_index` | `llama-index-core>=0.14,<0.15` |
+| `asgi` | `caveman_middleware.asgi` | `fastapi>=0.141,<1`, `starlette>=1.6,<2` |
+| `mcp` | `caveman_middleware.mcp` | `mcp>=2.2,<3` |
+
+Outside its range an adapter keeps the caller's native input unchanged and
+reports `unsupported_version`. It never raises.
+
+Extras carry no provider pins, so most combine: `pip install
+"caveman-middleware[langchain,agno,litellm]"` resolves. Three pairs cannot, and
+the cause is upstream, not this package: `strands-agents` and `crewai` both
+require `mcp<2.2` while the MCP adapter needs `>=2.2`, and `agno` and `crewai`
+disagree over `litellm`. Install those families in separate environments.
+
+## What you lose
+
+Read this before turning on `compress`.
+
+- **Markers are scoped.** A replacement marker is bound to
+  `namespace` + `session_id` + `branch_id` + `cache_epoch` and to the
+  authenticated principal that created it. Replaying it under any other scope
+  returns nothing.
+- **Markers expire.** A scope lives 24 hours, renewed each time it is used.
+  After that the stored original is released.
+- **Persisted compressed history is a trap.** If you save the transformed
+  messages and replay them later — under a different scope, or after the scope
+  expired — that text is gone for good. Caveman does not rewrite your stored
+  history for this reason: keep the originals, let Caveman transform the
+  outbound copy on every call.
+- **Recovery needs a registered executor.** In `compress` mode a lossy
+  transform is only used when your framework really holds the
+  `caveman_retrieve` tool. Without it, only recovery-free transforms apply.
+- **Eligible content is narrow.** Successful tool-result text and explicitly
+  passed document bodies. Never system prompts, user messages, assistant
+  reasoning, errored tool results, images or other non-text parts, anything
+  marked protected, and never a payload the adapter cannot parse exactly.
+- **Failure is silent and safe.** Runtime down, over deadline, out of capacity,
+  expired scope: the original request goes to the provider and the decision is
+  reported as skipped. Set `strict=True` only if you want a pre-inference error
+  instead.
+
+## Reports
+
+`on_report=` receives one immutable record per native call: status, reason,
+transform ids, replacement and reuse counts. No original content, no provider
+credentials. `runtime.last_report` holds the most recent one. A callback that
+raises cannot affect inference.
+
+Token counts in reports are estimates from the runtime's tokenizer over the
+segments it saw. They are not measured billing savings, and this package does
+not claim a reduction figure.

caveman_middleware-0.1.0a1/README.md

@@ -0,0 +1,95 @@
+# Caveman Python middleware
+
+Native framework adapters over `caveman_cloud.middleware`. Your framework keeps
+its models, tool loop, retries, streams and stored conversation. Caveman only
+changes the text of selected tool results in the outbound model view, and only
+when the local runtime answers in time.
+
+Alpha. Install the extra for the framework you use:
+
+```sh
+pip install "caveman-middleware[langchain]"
+```
+
+```python
+from caveman_cloud.middleware import MiddlewareRuntime, Scope
+from caveman_middleware.langchain import with_caveman_agent
+from langchain.agents import create_agent
+
+runtime = MiddlewareRuntime(endpoint="http://127.0.0.1:8787")
+runtime.ready()
+scope = Scope("my-app", session_id="conversation-1", branch_id="main", cache_epoch="0")
+
+agent = create_agent(**with_caveman_agent(existing_agent_options, runtime=runtime, scope=scope))
+```
+
+Call `runtime.close()` at shutdown. Nothing in your stored message history is
+rewritten; only the request handed to the provider is.
+
+## Supported frameworks
+
+One extra per adapter. The range is the band the adapter's version gate accepts;
+the left-hand value is the version its wire handling was written against.
+
+| Extra | Module | Tested range |
+|---|---|---|
+| `langchain` | `caveman_middleware.langchain` | `langchain>=1.4,<2`, `langchain-core>=1.6,<2`, `langgraph>=1.2,<2` |
+| `litellm` | `caveman_middleware.litellm` | `litellm>=1.100,<2` |
+| `openai` | `caveman_middleware.openai` | `openai>=3.10,<4` |
+| `anthropic` | `caveman_middleware.anthropic` | `anthropic>=1.4,<2` |
+| `google` | `caveman_middleware.google` | `google-genai>=2.22,<3` |
+| `strands` | `caveman_middleware.strands` | `strands-agents>=1.55,<2` |
+| `agno` | `caveman_middleware.agno` | `agno>=3.0,<4` |
+| `crewai` | `caveman_middleware.crewai` | `crewai>=1.15,<2` |
+| `autogen` | `caveman_middleware.autogen` | `autogen-core`, `autogen-agentchat`, `autogen-ext` all `>=0.7,<0.8` |
+| `pydantic-ai` | `caveman_middleware.pydantic_ai` | `pydantic-ai-slim>=2.42,<3` |
+| `llama-index` | `caveman_middleware.llama_index` | `llama-index-core>=0.14,<0.15` |
+| `asgi` | `caveman_middleware.asgi` | `fastapi>=0.141,<1`, `starlette>=1.6,<2` |
+| `mcp` | `caveman_middleware.mcp` | `mcp>=2.2,<3` |
+
+Outside its range an adapter keeps the caller's native input unchanged and
+reports `unsupported_version`. It never raises.
+
+Extras carry no provider pins, so most combine: `pip install
+"caveman-middleware[langchain,agno,litellm]"` resolves. Three pairs cannot, and
+the cause is upstream, not this package: `strands-agents` and `crewai` both
+require `mcp<2.2` while the MCP adapter needs `>=2.2`, and `agno` and `crewai`
+disagree over `litellm`. Install those families in separate environments.
+
+## What you lose
+
+Read this before turning on `compress`.
+
+- **Markers are scoped.** A replacement marker is bound to
+  `namespace` + `session_id` + `branch_id` + `cache_epoch` and to the
+  authenticated principal that created it. Replaying it under any other scope
+  returns nothing.
+- **Markers expire.** A scope lives 24 hours, renewed each time it is used.
+  After that the stored original is released.
+- **Persisted compressed history is a trap.** If you save the transformed
+  messages and replay them later — under a different scope, or after the scope
+  expired — that text is gone for good. Caveman does not rewrite your stored
+  history for this reason: keep the originals, let Caveman transform the
+  outbound copy on every call.
+- **Recovery needs a registered executor.** In `compress` mode a lossy
+  transform is only used when your framework really holds the
+  `caveman_retrieve` tool. Without it, only recovery-free transforms apply.
+- **Eligible content is narrow.** Successful tool-result text and explicitly
+  passed document bodies. Never system prompts, user messages, assistant
+  reasoning, errored tool results, images or other non-text parts, anything
+  marked protected, and never a payload the adapter cannot parse exactly.
+- **Failure is silent and safe.** Runtime down, over deadline, out of capacity,
+  expired scope: the original request goes to the provider and the decision is
+  reported as skipped. Set `strict=True` only if you want a pre-inference error
+  instead.
+
+## Reports
+
+`on_report=` receives one immutable record per native call: status, reason,
+transform ids, replacement and reuse counts. No original content, no provider
+credentials. `runtime.last_report` holds the most recent one. A callback that
+raises cannot affect inference.
+
+Token counts in reports are estimates from the runtime's tokenizer over the
+segments it saw. They are not measured billing savings, and this package does
+not claim a reduction figure.

caveman_middleware-0.1.0a1/caveman_middleware/__init__.py

@@ -0,0 +1,2 @@
+"""Optional native adapters. Import only the framework module your app uses."""
+__version__ = "0.1.0a1"

caveman_middleware-0.1.0a1/caveman_middleware/_google_wire.py

@@ -0,0 +1,84 @@
+"""Bounded Google JSON string offsets; untouched serialized text stays exact."""
+import json
+
+
+def parse(text):
+    if len(text) > 2 << 20:
+        return None
+    decoder, strings, index, nodes = json.JSONDecoder(), {}, 0, 0
+
+    def white():
+        nonlocal index
+        while index < len(text) and text[index] in " \t\r\n":
+            index += 1
+
+    def walk(path, depth=0):
+        nonlocal index, nodes
+        nodes += 1
+        if depth > 64 or nodes > 65536:
+            raise ValueError("bounded")
+        white()
+        start = index
+        if text[index] == '"':
+            value, index = decoder.raw_decode(text, index)
+            strings[path] = (start, index, value)
+            return value
+        if text[index] == "{":
+            index += 1
+            result = {}
+            white()
+            if text[index] == "}":
+                index += 1
+                return result
+            while True:
+                white()
+                key, index = decoder.raw_decode(text, index)
+                if type(key) is not str or key in result:
+                    raise ValueError("duplicate")
+                white()
+                if text[index] != ":":
+                    raise ValueError("colon")
+                index += 1
+                result[key] = walk((*path, key), depth + 1)
+                white()
+                end = text[index]
+                index += 1
+                if end == "}":
+                    return result
+                if end != ",":
+                    raise ValueError("comma")
+        if text[index] == "[":
+            index += 1
+            result = []
+            white()
+            if text[index] == "]":
+                index += 1
+                return result
+            while True:
+                result.append(walk((*path, len(result)), depth + 1))
+                white()
+                end = text[index]
+                index += 1
+                if end == "]":
+                    return result
+                if end != ",":
+                    raise ValueError("comma")
+        value, index = decoder.raw_decode(text, index)
+        return value
+
+    try:
+        value = walk(())
+        white()
+        return (value, strings) if index == len(text) else None
+    except (ValueError, IndexError, RecursionError):
+        return None
+
+
+def patch(text, replacements):
+    end = len(text)
+    for (start, stop, original), replacement in sorted(replacements, reverse=True):
+        if start < 0 or stop > end or start >= stop or json.loads(text[start:stop]) != original:
+            return None
+        text = text[:start] + json.dumps(replacement, ensure_ascii=False) + text[stop:]
+        end = start
+    return text

caveman_middleware-0.1.0a1/caveman_middleware/_httpx2.py

@@ -0,0 +1,190 @@
+"""Pull-driven response accounting for the pinned native provider SDK transport."""
+from __future__ import annotations
+
+import httpx2
+import asyncio
+import hashlib
+import uuid
+
+from ._native import Attempt, owner
+
+
+from ._usage import UsageReader
+
+
+class OpenAICall:
+    """One prepared native operation and its observed physical HTTP attempts."""
+    def __init__(self, attempt, transport):
+        self.attempt, self.transport, self.attempt_count = attempt, transport, 0
+
+    def next_attempt(self, request):
+        original = self.attempt
+        attempt = original if self.attempt_count == 0 else Attempt(original.runtime, original.scope,
+            original.logical_call_id, str(uuid.uuid4()), optimization=original.optimization, plan_id=original.plan_id)
+        self.attempt_count += 1
+        try:
+            attempt.wire_sha256 = hashlib.sha256(request.content).hexdigest()
+        except httpx2.RequestNotRead:
+            pass
+        return attempt
+
+
+class CavemanOpenAITransport(httpx2.BaseTransport):
+    """Wrap an application-owned HTTPX2 transport before client construction.
+
+    Pass this same object as ``transport`` to ``with_caveman_openai``. Requests
+    outside that native wrapper pass through. The supplied transport retains its
+    connection pool, TLS/proxy settings, and close behavior; this adds no retries.
+    """
+    def __init__(self, transport):
+        if not isinstance(transport, httpx2.BaseTransport):
+            raise TypeError("Expected an HTTPX2 BaseTransport")
+        self.transport = transport
+
+    def handle_request(self, request):
+        call = owner.get()
+        if not isinstance(call, OpenAICall) or call.transport is not self:
+            return self.transport.handle_request(request)
+        attempt = call.next_attempt(request)
+        attempt.observe("dispatch_intent")
+        token = owner.set(attempt)
+        try:
+            response = self.transport.handle_request(request)
+            observe_response(response, attempt)
+            return response
+        except BaseException:
+            attempt.observe("failed")
+            raise
+        finally:
+            owner.reset(token)
+
+    def close(self):
+        self.transport.close()
+
+
+class CavemanAsyncOpenAITransport(httpx2.AsyncBaseTransport):
+    """Async counterpart of CavemanOpenAITransport, using the native pool."""
+    def __init__(self, transport):
+        if not isinstance(transport, httpx2.AsyncBaseTransport):
+            raise TypeError("Expected an HTTPX2 AsyncBaseTransport")
+        self.transport = transport
+
+    async def handle_async_request(self, request):
+        call = owner.get()
+        if not isinstance(call, OpenAICall) or call.transport is not self:
+            return await self.transport.handle_async_request(request)
+        attempt = call.next_attempt(request)
+        attempt.observe("dispatch_intent")
+        token = owner.set(attempt)
+        try:
+            response = await self.transport.handle_async_request(request)
+            observe_response(response, attempt)
+            return response
+        except (asyncio.CancelledError, GeneratorExit):
+            attempt.observe("cancelled")
+            raise
+        except BaseException:
+            attempt.observe("failed")
+            raise
+        finally:
+            owner.reset(token)
+
+    async def aclose(self):
+        await self.transport.aclose()
+
+
+class SyncObservedStream(httpx2.SyncByteStream):
+    def __init__(self, original, attempt, parser):
+        self.original, self.attempt, self.parser, self.ended = original, attempt, parser, False
+
+    def finish(self, event):
+        if not self.ended:
+            self.ended = True
+            self.attempt.observe(event, self.parser.finish() if event == "completed" else None)
+
+    def __iter__(self):
+        iterator = iter(self.original)
+        try:
+            while True:
+                token = owner.set(self.attempt)
+                try:
+                    chunk = next(iterator)
+                except StopIteration:
+                    self.finish("completed")
+                    return
+                finally:
+                    owner.reset(token)
+                self.parser.feed(chunk)
+                yield chunk
+        except GeneratorExit:
+            self.finish("completed" if self.parser.terminal else "cancelled")
+            raise
+        except BaseException:
+            self.finish("failed")
+            raise
+
+    def close(self):
+        try:
+            self.original.close()
+        finally:
+            self.finish("completed" if self.parser.terminal else "cancelled")
+
+
+class AsyncObservedStream(httpx2.AsyncByteStream):
+    def __init__(self, original, attempt, parser):
+        self.original, self.attempt, self.parser, self.ended = original, attempt, parser, False
+
+    def finish(self, event):
+        if not self.ended:
+            self.ended = True
+            self.attempt.observe(event, self.parser.finish() if event == "completed" else None)
+
+    async def __aiter__(self):
+        iterator = self.original.__aiter__()
+        try:
+            while True:
+                token = owner.set(self.attempt)
+                try:
+                    chunk = await anext(iterator)
+                except StopAsyncIteration:
+                    self.finish("completed")
+                    return
+                finally:
+                    owner.reset(token)
+                self.parser.feed(chunk)
+                yield chunk
+        except GeneratorExit:
+            self.finish("completed" if self.parser.terminal else "cancelled")
+            raise
+        except asyncio.CancelledError:
+            self.finish("cancelled")
+            raise
+        except BaseException:
+            self.finish("failed")
+            raise
+
+    async def aclose(self):
+        try:
+            await self.original.aclose()
+        finally:
+            self.finish("completed" if self.parser.terminal else "cancelled")
+
+
+def observe_response(response, attempt):
+    if not isinstance(response, httpx2.Response):
+        return
+    if response.status_code >= 400:
+        attempt.observe("failed")
+        return
+    content_type = response.headers.get("content-type", "")
+    if "json" not in content_type and "text/event-stream" not in content_type:
+        attempt.observe("completed")
+        return
+    parser = UsageReader("text/event-stream" in content_type)
+    if response.is_stream_consumed:
+        parser.feed(response.content)
+        attempt.observe("completed", parser.finish())
+    elif isinstance(response.stream, httpx2.SyncByteStream):
+        response.stream = SyncObservedStream(response.stream, attempt, parser)
+    elif isinstance(response.stream, httpx2.AsyncByteStream):
+        response.stream = AsyncObservedStream(response.stream, attempt, parser)