axio-context-sqlite 0.6.1__tar.gz → 0.7.0__tar.gz

{axio_context_sqlite-0.6.1 → axio_context_sqlite-0.7.0}/.gitignore

@@ -10,3 +10,5 @@ dist/
 build/
 _build/
 .DS_Store
+*.sqlite
+*.db

{axio_context_sqlite-0.6.1 → axio_context_sqlite-0.7.0}/PKG-INFO

@@ -1,9 +1,9 @@
 Metadata-Version: 2.4
 Name: axio-context-sqlite
-Version: 0.6.1
+Version: 0.7.0
 Summary: SQLite-backed context store for Axio
-Project-URL: Homepage, https://github.com/axio-agent/monorepo
-Project-URL: Repository, https://github.com/axio-agent/monorepo
+Project-URL: Homepage, https://github.com/mosquito/axio-agent
+Project-URL: Repository, https://github.com/mosquito/axio-agent
 License: MIT
 Keywords: agent,ai,context,llm,sqlite,storage
 Requires-Python: >=3.12
@@ -17,7 +17,7 @@ Description-Content-Type: text/markdown
 [![Python](https://img.shields.io/pypi/pyversions/axio-context-sqlite)](https://pypi.org/project/axio-context-sqlite/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
 
-SQLite-backed persistent context store for [axio](https://github.com/axio-agent/monorepo).
+SQLite-backed persistent context store for [axio](https://github.com/mosquito/axio-agent).
 
 ## Installation
 
@@ -27,6 +27,32 @@ pip install axio-context-sqlite
 
 ## Usage
 
+### `connect(db_path)`
+
+Open (or create) a SQLite database at `db_path` and initialise the schema.
+Returns an `aiosqlite.Connection`. The caller is responsible for closing it.
+
+```python
+async def main():
+    conn = await connect("~/.axio/chat.db")
+    # ... use the connection ...
+    await conn.close()
+```
+
+The helper enables WAL journal mode and a 5-second busy timeout so concurrent
+readers and a single writer can safely share the same database file.
+
+### `SQLiteContextStore(conn, session_id, project=None, db_name="axio_context")`
+
+Create a context store bound to one session.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `conn` | `aiosqlite.Connection` | — | Open database connection (from `connect()`) |
+| `session_id` | `str` | — | Unique identifier for this conversation session |
+| `project` | `str \| None` | `str(Path.cwd().resolve())` | Logical project scope used to group and list sessions. Defaults to the current working directory. |
+| `db_name` | `str` | `"axio_context"` | Prefix used for the database table names (`axio_context_messages`, `axio_context_tokens`). |
+
 Open a connection with `connect()`, then create a `SQLiteContextStore` bound to a
 session. The caller owns the connection and is responsible for closing it.
 
@@ -56,6 +82,22 @@ asyncio.run(main())
 conversation history across process restarts. Multiple sessions can coexist in
 the same database file, isolated by `session_id` and `project`.
 
+#### Storage and compression
+
+Message content is stored as serialized JSON. Payloads larger than 512 bytes
+are automatically compressed with gzip (compresslevel 6) and stored
+base64-encoded with a `gzip:` prefix. Smaller payloads are stored as-is with a
+`plain:` prefix. Decompression happens transparently on read — callers never
+see the encoded form.
+
+#### SQLite performance settings
+
+Every connection opened by `connect()` is configured with:
+
+- `PRAGMA journal_mode=WAL` — enables concurrent readers alongside one writer
+- `PRAGMA busy_timeout=5000` — waits up to 5 seconds before raising a lock error
+- `PRAGMA synchronous=NORMAL` — balances durability and write throughput
+
 ### Agent integration
 
 <!--
@@ -112,6 +154,16 @@ async def main() -> None:
 asyncio.run(main())
 ```
 
+### Token accounting
+
+`add_context_tokens(input_tokens, output_tokens)` atomically increments the
+stored token counts for the current session and project using a SQL UPSERT
+(`INSERT ... ON CONFLICT DO UPDATE SET ... = ... + excluded....`). This is safe
+to call concurrently from multiple coroutines without an application-level lock.
+
+`set_context_tokens()` replaces the counts unconditionally, and
+`get_context_tokens()` returns a `(input_tokens, output_tokens)` tuple.
+
 ### Forking
 
 `fork()` copies the current session's messages into a new session — useful for

{axio_context_sqlite-0.6.1 → axio_context_sqlite-0.7.0}/README.md

@@ -4,7 +4,7 @@
 [![Python](https://img.shields.io/pypi/pyversions/axio-context-sqlite)](https://pypi.org/project/axio-context-sqlite/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
 
-SQLite-backed persistent context store for [axio](https://github.com/axio-agent/monorepo).
+SQLite-backed persistent context store for [axio](https://github.com/mosquito/axio-agent).
 
 ## Installation
 
@@ -14,6 +14,32 @@ pip install axio-context-sqlite
 
 ## Usage
 
+### `connect(db_path)`
+
+Open (or create) a SQLite database at `db_path` and initialise the schema.
+Returns an `aiosqlite.Connection`. The caller is responsible for closing it.
+
+```python
+async def main():
+    conn = await connect("~/.axio/chat.db")
+    # ... use the connection ...
+    await conn.close()
+```
+
+The helper enables WAL journal mode and a 5-second busy timeout so concurrent
+readers and a single writer can safely share the same database file.
+
+### `SQLiteContextStore(conn, session_id, project=None, db_name="axio_context")`
+
+Create a context store bound to one session.
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `conn` | `aiosqlite.Connection` | — | Open database connection (from `connect()`) |
+| `session_id` | `str` | — | Unique identifier for this conversation session |
+| `project` | `str \| None` | `str(Path.cwd().resolve())` | Logical project scope used to group and list sessions. Defaults to the current working directory. |
+| `db_name` | `str` | `"axio_context"` | Prefix used for the database table names (`axio_context_messages`, `axio_context_tokens`). |
+
 Open a connection with `connect()`, then create a `SQLiteContextStore` bound to a
 session. The caller owns the connection and is responsible for closing it.
 
@@ -43,6 +69,22 @@ asyncio.run(main())
 conversation history across process restarts. Multiple sessions can coexist in
 the same database file, isolated by `session_id` and `project`.
 
+#### Storage and compression
+
+Message content is stored as serialized JSON. Payloads larger than 512 bytes
+are automatically compressed with gzip (compresslevel 6) and stored
+base64-encoded with a `gzip:` prefix. Smaller payloads are stored as-is with a
+`plain:` prefix. Decompression happens transparently on read — callers never
+see the encoded form.
+
+#### SQLite performance settings
+
+Every connection opened by `connect()` is configured with:
+
+- `PRAGMA journal_mode=WAL` — enables concurrent readers alongside one writer
+- `PRAGMA busy_timeout=5000` — waits up to 5 seconds before raising a lock error
+- `PRAGMA synchronous=NORMAL` — balances durability and write throughput
+
 ### Agent integration
 
 <!--
@@ -99,6 +141,16 @@ async def main() -> None:
 asyncio.run(main())
 ```
 
+### Token accounting
+
+`add_context_tokens(input_tokens, output_tokens)` atomically increments the
+stored token counts for the current session and project using a SQL UPSERT
+(`INSERT ... ON CONFLICT DO UPDATE SET ... = ... + excluded....`). This is safe
+to call concurrently from multiple coroutines without an application-level lock.
+
+`set_context_tokens()` replaces the counts unconditionally, and
+`get_context_tokens()` returns a `(input_tokens, output_tokens)` tuple.
+
 ### Forking
 
 `fork()` copies the current session's messages into a new session — useful for

{axio_context_sqlite-0.6.1 → axio_context_sqlite-0.7.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "axio-context-sqlite"
-version = "0.6.1"
+version = "0.7.0"
 description = "SQLite-backed context store for Axio"
 readme = "README.md"
 requires-python = ">=3.12"
@@ -9,8 +9,8 @@ keywords = ["llm", "agent", "ai", "sqlite", "context", "storage"]
 dependencies = ["axio", "aiosqlite>=0.20"]
 
 [project.urls]
-Homepage = "https://github.com/axio-agent/monorepo"
-Repository = "https://github.com/axio-agent/monorepo"
+Homepage = "https://github.com/mosquito/axio-agent"
+Repository = "https://github.com/mosquito/axio-agent"
 
 [build-system]
 requires = ["hatchling"]

axio_context_sqlite-0.7.0/tests/test_autocompact_sqlite.py

@@ -0,0 +1,158 @@
+"""Tests for AutoCompactStore wrapping SQLiteContextStore."""
+
+from __future__ import annotations
+
+from collections.abc import AsyncGenerator, AsyncIterator
+from pathlib import Path
+from typing import Any
+from uuid import uuid4
+
+import aiosqlite
+import pytest
+from axio.blocks import TextBlock
+from axio.compaction import AutoCompactStore
+from axio.events import StreamEvent
+from axio.messages import Message
+from axio.testing import StubTransport, make_text_response
+from axio.tool import Tool
+
+from axio_context_sqlite import SQLiteContextStore, connect
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _msg(role: str = "user", text: str = "x") -> Message:
+    return Message(role=role, content=[TextBlock(text=text)])  # type: ignore[arg-type]
+
+
+class FailTransport:
+    def stream(self, messages: list[Message], tools: list[Tool[Any]], system: str) -> AsyncIterator[StreamEvent]:
+        raise RuntimeError("boom")
+
+
+# ---------------------------------------------------------------------------
+# Fixtures
+# ---------------------------------------------------------------------------
+
+
+@pytest.fixture
+def db_path(tmp_path: Path) -> Path:
+    return tmp_path / "autocompact_test.db"
+
+
+@pytest.fixture
+async def conn(db_path: Path) -> AsyncGenerator[aiosqlite.Connection, None]:
+    c = await connect(db_path)
+    yield c
+    await c.close()
+
+
+@pytest.fixture
+async def sqlite_store(conn: aiosqlite.Connection) -> SQLiteContextStore:
+    return SQLiteContextStore(conn, uuid4().hex, "test-project")
+
+
+# ---------------------------------------------------------------------------
+# Tests
+# ---------------------------------------------------------------------------
+
+
+class TestAutoCompactWithSQLite:
+    async def test_history_written_back_to_sqlite(self, sqlite_store: SQLiteContextStore) -> None:
+        """After compaction, messages in the DB are the compacted ones."""
+        for i in range(14):
+            role = "user" if i % 2 == 0 else "assistant"
+            await sqlite_store.append(_msg(role, f"msg-{i}"))
+
+        transport = StubTransport([make_text_response("Summary of earlier work")])
+        store = AutoCompactStore(sqlite_store, transport, max_tokens=1, keep_recent=4)
+
+        await store.add_context_tokens(2, 0)
+
+        history = await sqlite_store.get_history()
+        # 2 (summary pair) + 4 recent
+        assert len(history) == 6
+        assert history[0].role == "user"
+        block0 = history[0].content[0]
+        assert isinstance(block0, TextBlock)
+        assert "Summary of earlier work" in block0.text
+        assert history[1].role == "assistant"
+
+    async def test_cumulative_tokens_persisted_in_sqlite(self, sqlite_store: SQLiteContextStore) -> None:
+        """Cumulative token counts survive compaction in the SQLite table."""
+        for i in range(14):
+            role = "user" if i % 2 == 0 else "assistant"
+            await sqlite_store.append(_msg(role, f"msg-{i}"))
+        await sqlite_store.set_context_tokens(400_000, 15_000)
+
+        transport = StubTransport([make_text_response("summary")])
+        store = AutoCompactStore(sqlite_store, transport, max_tokens=1, keep_recent=4)
+
+        await store.add_context_tokens(2, 0)
+
+        in_tok, out_tok = await sqlite_store.get_context_tokens()
+        assert in_tok == 400_002
+        assert out_tok == 15_000
+
+    async def test_sqlite_session_id_preserved(self, sqlite_store: SQLiteContextStore) -> None:
+        """session_id of wrapper matches the inner SQLiteContextStore."""
+        store = AutoCompactStore(sqlite_store, StubTransport([]), max_tokens=999_999)
+        assert store.session_id == sqlite_store.session_id
+
+    async def test_compact_does_not_corrupt_on_failure(self, sqlite_store: SQLiteContextStore) -> None:
+        """If compaction fails, the original history in the DB is untouched."""
+        msgs = [_msg("user" if i % 2 == 0 else "assistant", f"msg-{i}") for i in range(14)]
+        for m in msgs:
+            await sqlite_store.append(m)
+
+        store = AutoCompactStore(sqlite_store, FailTransport(), max_tokens=1, keep_recent=4)
+        await store.add_context_tokens(2, 0)
+
+        history = await sqlite_store.get_history()
+        assert len(history) == 14
+
+    async def test_fork_creates_independent_sqlite_session(self, sqlite_store: SQLiteContextStore) -> None:
+        """fork() returns an AutoCompactStore with a new session_id; mutations are independent."""
+        for i in range(6):
+            role = "user" if i % 2 == 0 else "assistant"
+            await sqlite_store.append(_msg(role, f"msg-{i}"))
+
+        store = AutoCompactStore(sqlite_store, StubTransport([]), max_tokens=999_999)
+        forked = await store.fork()
+
+        assert forked.session_id != store.session_id
+
+        # append to fork — parent unaffected
+        await forked.append(_msg(text="extra"))
+        assert len(await store.get_history()) == 6
+        assert len(await forked.get_history()) == 7
+
+    async def test_compact_uses_fork_as_snapshot(self, sqlite_store: SQLiteContextStore) -> None:
+        """compact_context receives a fork of the store, not the live store itself."""
+        for i in range(14):
+            role = "user" if i % 2 == 0 else "assistant"
+            await sqlite_store.append(_msg(role, f"msg-{i}"))
+
+        seen_session_ids: list[str] = []
+
+        import axio.compaction as _mod
+
+        original = _mod.compact_context
+
+        async def _capturing(ctx, transport, **kw):  # type: ignore[no-untyped-def]
+            seen_session_ids.append(ctx.session_id)
+            return await original(ctx, StubTransport([make_text_response("summary")]), **kw)
+
+        _mod.compact_context = _capturing  # type: ignore[assignment]
+        try:
+            store = AutoCompactStore(
+                sqlite_store, StubTransport([make_text_response("summary")]), max_tokens=1, keep_recent=4
+            )
+            await store.add_context_tokens(2, 0)
+        finally:
+            _mod.compact_context = original
+
+        assert len(seen_session_ids) == 1
+        assert seen_session_ids[0] != sqlite_store.session_id