memkernel-mcp 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Varad Ganjoo
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1 @@
1
+ prune tests
@@ -0,0 +1,265 @@
1
+ Metadata-Version: 2.4
2
+ Name: memkernel-mcp
3
+ Version: 0.1.0
4
+ Summary: Event-sourced, branchable memory kernel and MCP server for multi-agent coding systems
5
+ Author: Varad Ganjoo
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/varadganjoo/memkernel
8
+ Project-URL: Repository, https://github.com/varadganjoo/memkernel
9
+ Project-URL: Issues, https://github.com/varadganjoo/memkernel/issues
10
+ Project-URL: Documentation, https://github.com/varadganjoo/memkernel/blob/main/docs/integrations.md
11
+ Project-URL: Whitepaper, https://github.com/varadganjoo/memkernel/blob/main/docs/whitepaper.md
12
+ Keywords: mcp,agents,memory,event-sourcing,sqlite,llm
13
+ Classifier: Development Status :: 4 - Beta
14
+ Classifier: Framework :: AsyncIO
15
+ Classifier: Intended Audience :: Developers
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Programming Language :: Python :: 3.13
20
+ Classifier: Programming Language :: Python :: 3.14
21
+ Classifier: Operating System :: OS Independent
22
+ Classifier: Programming Language :: Python :: 3.13
23
+ Classifier: Topic :: Software Development :: Libraries
24
+ Requires-Python: >=3.11
25
+ Description-Content-Type: text/markdown
26
+ License-File: LICENSE
27
+ Requires-Dist: mcp>=1.30
28
+ Requires-Dist: aiosqlite>=0.22
29
+ Requires-Dist: pydantic>=2.13
30
+ Requires-Dist: tiktoken>=0.14
31
+ Provides-Extra: dev
32
+ Requires-Dist: pytest>=9.1; extra == "dev"
33
+ Requires-Dist: pytest-asyncio>=1.4; extra == "dev"
34
+ Dynamic: license-file
35
+
36
+ # MemKernel
37
+
38
+ **An event-sourced, branchable memory kernel for teams of coding agents, served over the Model Context Protocol.**
39
+
40
+ Cursor, Claude Code, Codex and your own harnesses each keep their own chat
41
+ history. None of them knows what the others decided, which files they are
42
+ editing, or which "fact" was already disproven three turns ago. MemKernel gives
43
+ all of them one shared memory with the following properties:
44
+
45
+ - **Deterministic.** State is a pure fold over an append-only, hash-chained event log. Replaying the log reproduces the state exactly.
46
+ - **Branchable.** Agents fork a timeline to investigate, diff it against `main`, and roll it back when an experiment or a hallucination fails. Forks copy no rows.
47
+ - **Coordinated.** Leases with fencing tokens keep two agents from editing the same file, and a stale writer is rejected at the storage layer.
48
+ - **Budgeted.** An autonomous GC compacts noisy tool output into semantic snapshots. In the benchmarks it keeps each agent's context under a token budget while preserving every decision.
49
+
50
+ It is a plain Python package: SQLite (WAL) for storage, `pydantic` v2 for
51
+ schemas, and the official `mcp` SDK (`FastMCP` on 1.x, `MCPServer` on 2.x) for
52
+ the stdio server.
53
+
54
+ **Runs on Linux, macOS and Windows** with Python 3.11 through 3.14 (3.14 recommended). The
55
+ test suite passes on Windows 11 and on Linux (Debian 13) under Python 3.14.7, and CI covers
56
+ all three operating systems. It works with Claude Code, Codex CLI, Antigravity and Cursor.
57
+
58
+ ```text
59
+ Cursor Claude Code Codex CLI Antigravity your harness
60
+ | | | | |
61
+ | stdio/MCP | stdio/MCP | stdio/MCP | stdio/MCP | in-process
62
+ v v v v v
63
+ memkernel-server (one process per agent, own agent_id) --> MemKernel facade
64
+ |
65
+ +-------------------------------+-----------------------+
66
+ v v v
67
+ AsyncEventStore LockManager MemoryGCEngine
68
+ append-only, hash-chained log leases + snapshots within
69
+ + copy-on-write branches fencing tokens a token budget
70
+ \_______________ one SQLite file (WAL) per project ______/
71
+ ```
72
+
73
+ Each agent runs its own server process. All of them share one database file,
74
+ and SQLite's `BEGIN IMMEDIATE` transactions serialize writers across processes.
75
+
76
+ ## Quickstart: three commands
77
+
78
+ **1. Install** (Python 3.14 recommended; 3.11+ supported). `uv tool install` gives you a global `memkernel` command in its own isolated environment:
79
+
80
+ ```bash
81
+ uv tool install memkernel-mcp
82
+ ```
83
+
84
+ `pipx install memkernel-mcp` does the same. For an existing isolated environment, use `pip install memkernel-mcp`.
85
+
86
+ The distribution is named **`memkernel-mcp`**; the Python import and commands remain `memkernel`. The `memkernel` project on PyPI belongs to a different author. Do not use `pip install memkernel` for this repository, or install both projects into the same Python environment. To install from source instead, use `uv tool install git+https://github.com/varadganjoo/MemKernel.git`.
87
+
88
+ **2. Attach** it to your agents, from your project's root directory:
89
+
90
+ ```bash
91
+ memkernel attach claude-code cursor codex
92
+ ```
93
+
94
+ Use `all` to include Antigravity too. `attach` merges a `memkernel` entry into each agent's own MCP config and leaves the other servers in those files alone. It creates `.memkernel/` for the project's shared database and adds that directory to `.gitignore`. Global configs are backed up first. Re-running it is safe, and `--dry-run` shows the changes without writing them.
95
+
96
+ | Agent | File written | Scope |
97
+ |---|---|---|
98
+ | Claude Code | `./.mcp.json` | project |
99
+ | Cursor | `./.cursor/mcp.json` | project |
100
+ | Codex CLI | `~/.codex/config.toml` | global: each project gets its own memory via discovery |
101
+ | Antigravity | `~/.gemini/.../mcp_config.json` | global: each project gets its own memory via discovery |
102
+
103
+ **3. Verify:**
104
+
105
+ ```bash
106
+ memkernel doctor
107
+ ```
108
+
109
+ `doctor` opens the database, checks its hash chain, boots the MCP server over stdio exactly as an agent would, and checks every agent configuration. Then restart your agents. Each one now has `memkernel_*` tools, and they share one memory per project. To see what the agents currently remember:
110
+
111
+ ```bash
112
+ memkernel status
113
+ ```
114
+
115
+ > **Slow machine?** Cold start is dominated by importing the MCP SDK. Claude Code gives MCP servers 30 s to start; raise it with `MCP_TIMEOUT=120000`. The Codex config written by `attach` already allows 120 s.
116
+
117
+ Full per-agent details, manual configuration and troubleshooting are in [docs/integrations.md](https://github.com/varadganjoo/memkernel/blob/main/docs/integrations.md).
118
+
119
+ ### Try it without an agent
120
+
121
+ From a checkout:
122
+
123
+ ```bash
124
+ pip install -e ".[dev]"
125
+ ```
126
+
127
+ ```bash
128
+ pytest
129
+ ```
130
+
131
+ Two simulations assert their invariants and exit non-zero on violation:
132
+
133
+ ```bash
134
+ python examples/multi_agent_collaboration.py
135
+ ```
136
+
137
+ ```bash
138
+ python examples/rollback_on_hallucination.py
139
+ ```
140
+
141
+ ## MCP tools
142
+
143
+ | Tool | Purpose |
144
+ |---|---|
145
+ | `memkernel_query_state(branch_id="main", include_ephemeral=False)` | Consolidated memory as JSON: tasks, decisions, workspace facts, deprecated assumptions, blockers, claims, leases, recent events. |
146
+ | `memkernel_append_decision(decision_key, decision_value, rationale)` | Durable architectural decision. It survives compaction, and a new value supersedes the old one. |
147
+ | `memkernel_acquire_lease(file_path, timeout_seconds=60)` | Exclusive lease with a TTL. Returns a fencing token, or the current holder. |
148
+ | `memkernel_release_lease(file_path, fencing_token)` | Release. Returns `false` for a stale token. |
149
+ | `memkernel_heartbeat_lease(file_path, fencing_token)` | Extend a held lease. |
150
+ | `memkernel_fork_investigation(fork_branch_name)` | Fork the active branch at its head and switch to it. |
151
+ | `memkernel_checkout(branch_id)` | Switch the session's active branch. |
152
+ | `memkernel_rollback(target_event_id)` | Revert the active branch's live timeline to an event. History stays auditable. |
153
+ | `memkernel_history(limit=30)` | Recent live events, useful for picking rollback targets. |
154
+ | `memkernel_diff_branches(branch_a, branch_b)` | Merge base, divergent events, conflicting decisions and facts. |
155
+ | `memkernel_trigger_gc(target_branch="main")` | Compact now and report the token savings. |
156
+ | `memkernel_init_task(directive, constraints)` | Record the agent's task and constraints. |
157
+ | `memkernel_log_execution(command, exit_code, output, facts)` | Log tool output. A failure opens a blocker and a later success clears it. `facts` updates workspace state. |
158
+ | `memkernel_note(note)` | Ephemeral scratchpad turn. |
159
+ | `memkernel_list_branches()` | All branches, plus the session's active branch. |
160
+
161
+ ## Concepts in one page
162
+
163
+ **Events.** An event is an immutable `EventEnvelope` with fields `event_id`
164
+ (globally monotonic), `sequence` (position in the branch lineage), `timestamp`,
165
+ `branch_id`, `agent_id`, `parent_event_id` (lineage predecessor), `event_type`,
166
+ `payload`, `metadata` and `hash = SHA-256(prev_hash ‖ canonical(event))`. The
167
+ event types are `AGENT_TASK_INIT`, `ARCHITECTURAL_DECISION`,
168
+ `FILE_CLAIM_ACQUIRED/RELEASED`, `TOOL_EXECUTION_LOG`, `AGENT_SCRATCHPAD`,
169
+ `SEMANTIC_SNAPSHOT`, `BRANCH_FORKED` and `ROLLBACK_TRIGGERED`. SQLite triggers
170
+ reject every `UPDATE` and `DELETE` on the event table.
171
+
172
+ **Branches.** A branch is a pointer (parent, fork point, head). Its lineage is
173
+ its parent's lineage up to the fork point, followed by its own events. Forking
174
+ copies no rows.
175
+
176
+ **Rollback.** A rollback is a compensating `ROLLBACK_TRIGGERED` event. The
177
+ *live* timeline is a left fold in which a rollback truncates back to its
178
+ target. Nothing is deleted, so the reverted events stay auditable and forkable.
179
+
180
+ **State.** `WorkspaceState = fold(live events)`. It holds tasks, decisions (with
181
+ rationale and provenance), active workspace facts, deprecated or overruled
182
+ assumptions, execution blockers, file claims, an execution digest, and insights.
183
+
184
+ **GC.** When `[latest snapshot] + [delta]` exceeds the token budget, older delta
185
+ events fold into a new `SEMANTIC_SNAPSHOT`, an exact serialization of the fold.
186
+ The invariant `fold(snapshot, delta) == fold(full history)` is tested. The
187
+ retained raw tail is bounded by both an event count and a token low-water mark,
188
+ and every automatic compaction must free at least a quarter of the budget, so
189
+ GC does not thrash.
190
+
191
+ **Leases.** Leases are time-bounded, and every grant carries a fencing token
192
+ that strictly increases per resource. `fenced_append` checks the token and
193
+ appends in one transaction, so a writer whose lease expired cannot land a
194
+ write.
195
+
196
+ ## Results
197
+
198
+ These numbers come from `python benchmarks/run_benchmarks.py`. The full
199
+ methodology is in [the whitepaper](https://github.com/varadganjoo/memkernel/blob/main/docs/whitepaper.md#5-empirical-evaluation--benchmarks).
200
+
201
+ | Workflow | Events | Raw history tokens | MemKernel context tokens | Reduction | Snapshots | State equivalent |
202
+ |---|---:|---:|---:|---:|---:|:---:|
203
+ | single-agent-test-fix | 73 | 31,346 | 3,558 | 88.6% | 13 | yes |
204
+ | multi-agent-refactor | 99 | 33,598 | 2,360 | 93.0% | 13 | yes |
205
+ | lint-heavy-exploration | 73 | 42,445 | 3,459 | 91.9% | 19 | yes |
206
+
207
+ On a 50,000-event log, a warm (fast-path) context read takes 1.7 ms on Windows 11 and 1.1 ms on Linux, a fork at the head takes under 2 ms, and a cold open plus context read takes 21 ms on Windows 11 and 4.5 ms on Linux. Measured with Python 3.14.7 on Windows 11 (build 26200, SQLite 3.53.1) and in a Debian 13 container (SQLite 3.46.1) on the same workstation. The token results are the same on both platforms.
208
+
209
+ ### Live agents, with evidence
210
+
211
+ `benchmarks/live_agents.py` drives the real Claude Code, Codex CLI and Antigravity CLIs on a
212
+ small repository with two failing tests. They coordinate only through MemKernel: leases,
213
+ decisions, a fork with a rollback, and recall from shared memory. The script saves their
214
+ transcripts, the event database, the code diff and the test results. The recorded run is
215
+ in [`evidence/live-agents/20260928T161716Z/REPORT.md`](https://github.com/varadganjoo/memkernel/blob/main/evidence/live-agents/20260928T161716Z/REPORT.md).
216
+ In that run, the concurrently started agents contended for the same file, one was refused
217
+ and took other work, both bugs were fixed, and the hash chain verified intact in an
218
+ independent Linux container.
219
+
220
+ ## Repository layout
221
+
222
+ ```text
223
+ memkernel/
224
+ ├── core/events.py event taxonomy, envelope schema, hashing
225
+ ├── core/store.py AsyncEventStore: WAL, lineage, fork, rollback, diff, verify
226
+ ├── core/branch.py pure lineage algebra: live fold, rollback planning, diff, chain verification
227
+ ├── core/locks.py LockManager: leases, heartbeats, fencing tokens
228
+ ├── memory/state_view.py WorkspaceState fold, context split, rendering
229
+ ├── memory/gc.py MemoryGCEngine, heuristic and LLM summarizers, daemon
230
+ ├── memory/token_counter.py tiktoken with a conservative regex fallback
231
+ ├── kernel.py MemKernel facade (atomic lease+claim, fenced appends)
232
+ ├── server/mcp_server.py MCP server over stdio
233
+ ├── clients/adapter.py MemKernelClient (in-process) and McpStdioClient
234
+ └── cli.py `memkernel` command: attach, doctor, status, serve
235
+ examples/ multi-agent and rollback simulations
236
+ benchmarks/ reproducible measurements and the live-agent harness
237
+ evidence/ recorded live-agent run (transcripts, database, report)
238
+ docs/ whitepaper and integration guide
239
+ tests/ store, branching, locks, GC, server and stdio e2e
240
+ ```
241
+
242
+ ## Development
243
+
244
+ ```bash
245
+ pytest
246
+ ```
247
+
248
+ The suite runs offline and is deterministic. One opt-in test exercises
249
+ `LLMSummarizer` against a real OpenAI-compatible model:
250
+
251
+ ```bash
252
+ MEMKERNEL_LIVE_LLM_TESTS=1 OPENAI_API_KEY=... OPENAI_LLM=<model> pytest -k live
253
+ ```
254
+
255
+ ## Limitations
256
+
257
+ - **Single host.** Coordination relies on SQLite file locking on one machine; network filesystems are not supported. A replicated log is future work (see the whitepaper).
258
+ - **Self-asserted identity.** `agent_id` comes from the server's launch configuration. MemKernel records provenance but does not authenticate agents; see the threat model.
259
+ - **O(lineage) operations.** Rollback, arbitrary-point forks and full replay are linear in lineage length. Context reads and forks at the head are not.
260
+ - **No automatic merge.** `diff` reports conflicts. Reconciling them is an agent's (or a human's) decision, recorded as a new `ARCHITECTURAL_DECISION`.
261
+
262
+ ## License
263
+
264
+ MIT. See [LICENSE](https://github.com/varadganjoo/memkernel/blob/main/LICENSE).
265
+
@@ -0,0 +1,230 @@
1
+ # MemKernel
2
+
3
+ **An event-sourced, branchable memory kernel for teams of coding agents, served over the Model Context Protocol.**
4
+
5
+ Cursor, Claude Code, Codex and your own harnesses each keep their own chat
6
+ history. None of them knows what the others decided, which files they are
7
+ editing, or which "fact" was already disproven three turns ago. MemKernel gives
8
+ all of them one shared memory with the following properties:
9
+
10
+ - **Deterministic.** State is a pure fold over an append-only, hash-chained event log. Replaying the log reproduces the state exactly.
11
+ - **Branchable.** Agents fork a timeline to investigate, diff it against `main`, and roll it back when an experiment or a hallucination fails. Forks copy no rows.
12
+ - **Coordinated.** Leases with fencing tokens keep two agents from editing the same file, and a stale writer is rejected at the storage layer.
13
+ - **Budgeted.** An autonomous GC compacts noisy tool output into semantic snapshots. In the benchmarks it keeps each agent's context under a token budget while preserving every decision.
14
+
15
+ It is a plain Python package: SQLite (WAL) for storage, `pydantic` v2 for
16
+ schemas, and the official `mcp` SDK (`FastMCP` on 1.x, `MCPServer` on 2.x) for
17
+ the stdio server.
18
+
19
+ **Runs on Linux, macOS and Windows** with Python 3.11 through 3.14 (3.14 recommended). The
20
+ test suite passes on Windows 11 and on Linux (Debian 13) under Python 3.14.7, and CI covers
21
+ all three operating systems. It works with Claude Code, Codex CLI, Antigravity and Cursor.
22
+
23
+ ```text
24
+ Cursor Claude Code Codex CLI Antigravity your harness
25
+ | | | | |
26
+ | stdio/MCP | stdio/MCP | stdio/MCP | stdio/MCP | in-process
27
+ v v v v v
28
+ memkernel-server (one process per agent, own agent_id) --> MemKernel facade
29
+ |
30
+ +-------------------------------+-----------------------+
31
+ v v v
32
+ AsyncEventStore LockManager MemoryGCEngine
33
+ append-only, hash-chained log leases + snapshots within
34
+ + copy-on-write branches fencing tokens a token budget
35
+ \_______________ one SQLite file (WAL) per project ______/
36
+ ```
37
+
38
+ Each agent runs its own server process. All of them share one database file,
39
+ and SQLite's `BEGIN IMMEDIATE` transactions serialize writers across processes.
40
+
41
+ ## Quickstart: three commands
42
+
43
+ **1. Install** (Python 3.14 recommended; 3.11+ supported). `uv tool install` gives you a global `memkernel` command in its own isolated environment:
44
+
45
+ ```bash
46
+ uv tool install memkernel-mcp
47
+ ```
48
+
49
+ `pipx install memkernel-mcp` does the same. For an existing isolated environment, use `pip install memkernel-mcp`.
50
+
51
+ The distribution is named **`memkernel-mcp`**; the Python import and commands remain `memkernel`. The `memkernel` project on PyPI belongs to a different author. Do not use `pip install memkernel` for this repository, or install both projects into the same Python environment. To install from source instead, use `uv tool install git+https://github.com/varadganjoo/MemKernel.git`.
52
+
53
+ **2. Attach** it to your agents, from your project's root directory:
54
+
55
+ ```bash
56
+ memkernel attach claude-code cursor codex
57
+ ```
58
+
59
+ Use `all` to include Antigravity too. `attach` merges a `memkernel` entry into each agent's own MCP config and leaves the other servers in those files alone. It creates `.memkernel/` for the project's shared database and adds that directory to `.gitignore`. Global configs are backed up first. Re-running it is safe, and `--dry-run` shows the changes without writing them.
60
+
61
+ | Agent | File written | Scope |
62
+ |---|---|---|
63
+ | Claude Code | `./.mcp.json` | project |
64
+ | Cursor | `./.cursor/mcp.json` | project |
65
+ | Codex CLI | `~/.codex/config.toml` | global: each project gets its own memory via discovery |
66
+ | Antigravity | `~/.gemini/.../mcp_config.json` | global: each project gets its own memory via discovery |
67
+
68
+ **3. Verify:**
69
+
70
+ ```bash
71
+ memkernel doctor
72
+ ```
73
+
74
+ `doctor` opens the database, checks its hash chain, boots the MCP server over stdio exactly as an agent would, and checks every agent configuration. Then restart your agents. Each one now has `memkernel_*` tools, and they share one memory per project. To see what the agents currently remember:
75
+
76
+ ```bash
77
+ memkernel status
78
+ ```
79
+
80
+ > **Slow machine?** Cold start is dominated by importing the MCP SDK. Claude Code gives MCP servers 30 s to start; raise it with `MCP_TIMEOUT=120000`. The Codex config written by `attach` already allows 120 s.
81
+
82
+ Full per-agent details, manual configuration and troubleshooting are in [docs/integrations.md](https://github.com/varadganjoo/memkernel/blob/main/docs/integrations.md).
83
+
84
+ ### Try it without an agent
85
+
86
+ From a checkout:
87
+
88
+ ```bash
89
+ pip install -e ".[dev]"
90
+ ```
91
+
92
+ ```bash
93
+ pytest
94
+ ```
95
+
96
+ Two simulations assert their invariants and exit non-zero on violation:
97
+
98
+ ```bash
99
+ python examples/multi_agent_collaboration.py
100
+ ```
101
+
102
+ ```bash
103
+ python examples/rollback_on_hallucination.py
104
+ ```
105
+
106
+ ## MCP tools
107
+
108
+ | Tool | Purpose |
109
+ |---|---|
110
+ | `memkernel_query_state(branch_id="main", include_ephemeral=False)` | Consolidated memory as JSON: tasks, decisions, workspace facts, deprecated assumptions, blockers, claims, leases, recent events. |
111
+ | `memkernel_append_decision(decision_key, decision_value, rationale)` | Durable architectural decision. It survives compaction, and a new value supersedes the old one. |
112
+ | `memkernel_acquire_lease(file_path, timeout_seconds=60)` | Exclusive lease with a TTL. Returns a fencing token, or the current holder. |
113
+ | `memkernel_release_lease(file_path, fencing_token)` | Release. Returns `false` for a stale token. |
114
+ | `memkernel_heartbeat_lease(file_path, fencing_token)` | Extend a held lease. |
115
+ | `memkernel_fork_investigation(fork_branch_name)` | Fork the active branch at its head and switch to it. |
116
+ | `memkernel_checkout(branch_id)` | Switch the session's active branch. |
117
+ | `memkernel_rollback(target_event_id)` | Revert the active branch's live timeline to an event. History stays auditable. |
118
+ | `memkernel_history(limit=30)` | Recent live events, useful for picking rollback targets. |
119
+ | `memkernel_diff_branches(branch_a, branch_b)` | Merge base, divergent events, conflicting decisions and facts. |
120
+ | `memkernel_trigger_gc(target_branch="main")` | Compact now and report the token savings. |
121
+ | `memkernel_init_task(directive, constraints)` | Record the agent's task and constraints. |
122
+ | `memkernel_log_execution(command, exit_code, output, facts)` | Log tool output. A failure opens a blocker and a later success clears it. `facts` updates workspace state. |
123
+ | `memkernel_note(note)` | Ephemeral scratchpad turn. |
124
+ | `memkernel_list_branches()` | All branches, plus the session's active branch. |
125
+
126
+ ## Concepts in one page
127
+
128
+ **Events.** An event is an immutable `EventEnvelope` with fields `event_id`
129
+ (globally monotonic), `sequence` (position in the branch lineage), `timestamp`,
130
+ `branch_id`, `agent_id`, `parent_event_id` (lineage predecessor), `event_type`,
131
+ `payload`, `metadata` and `hash = SHA-256(prev_hash ‖ canonical(event))`. The
132
+ event types are `AGENT_TASK_INIT`, `ARCHITECTURAL_DECISION`,
133
+ `FILE_CLAIM_ACQUIRED/RELEASED`, `TOOL_EXECUTION_LOG`, `AGENT_SCRATCHPAD`,
134
+ `SEMANTIC_SNAPSHOT`, `BRANCH_FORKED` and `ROLLBACK_TRIGGERED`. SQLite triggers
135
+ reject every `UPDATE` and `DELETE` on the event table.
136
+
137
+ **Branches.** A branch is a pointer (parent, fork point, head). Its lineage is
138
+ its parent's lineage up to the fork point, followed by its own events. Forking
139
+ copies no rows.
140
+
141
+ **Rollback.** A rollback is a compensating `ROLLBACK_TRIGGERED` event. The
142
+ *live* timeline is a left fold in which a rollback truncates back to its
143
+ target. Nothing is deleted, so the reverted events stay auditable and forkable.
144
+
145
+ **State.** `WorkspaceState = fold(live events)`. It holds tasks, decisions (with
146
+ rationale and provenance), active workspace facts, deprecated or overruled
147
+ assumptions, execution blockers, file claims, an execution digest, and insights.
148
+
149
+ **GC.** When `[latest snapshot] + [delta]` exceeds the token budget, older delta
150
+ events fold into a new `SEMANTIC_SNAPSHOT`, an exact serialization of the fold.
151
+ The invariant `fold(snapshot, delta) == fold(full history)` is tested. The
152
+ retained raw tail is bounded by both an event count and a token low-water mark,
153
+ and every automatic compaction must free at least a quarter of the budget, so
154
+ GC does not thrash.
155
+
156
+ **Leases.** Leases are time-bounded, and every grant carries a fencing token
157
+ that strictly increases per resource. `fenced_append` checks the token and
158
+ appends in one transaction, so a writer whose lease expired cannot land a
159
+ write.
160
+
161
+ ## Results
162
+
163
+ These numbers come from `python benchmarks/run_benchmarks.py`. The full
164
+ methodology is in [the whitepaper](https://github.com/varadganjoo/memkernel/blob/main/docs/whitepaper.md#5-empirical-evaluation--benchmarks).
165
+
166
+ | Workflow | Events | Raw history tokens | MemKernel context tokens | Reduction | Snapshots | State equivalent |
167
+ |---|---:|---:|---:|---:|---:|:---:|
168
+ | single-agent-test-fix | 73 | 31,346 | 3,558 | 88.6% | 13 | yes |
169
+ | multi-agent-refactor | 99 | 33,598 | 2,360 | 93.0% | 13 | yes |
170
+ | lint-heavy-exploration | 73 | 42,445 | 3,459 | 91.9% | 19 | yes |
171
+
172
+ On a 50,000-event log, a warm (fast-path) context read takes 1.7 ms on Windows 11 and 1.1 ms on Linux, a fork at the head takes under 2 ms, and a cold open plus context read takes 21 ms on Windows 11 and 4.5 ms on Linux. Measured with Python 3.14.7 on Windows 11 (build 26200, SQLite 3.53.1) and in a Debian 13 container (SQLite 3.46.1) on the same workstation. The token results are the same on both platforms.
173
+
174
+ ### Live agents, with evidence
175
+
176
+ `benchmarks/live_agents.py` drives the real Claude Code, Codex CLI and Antigravity CLIs on a
177
+ small repository with two failing tests. They coordinate only through MemKernel: leases,
178
+ decisions, a fork with a rollback, and recall from shared memory. The script saves their
179
+ transcripts, the event database, the code diff and the test results. The recorded run is
180
+ in [`evidence/live-agents/20260928T161716Z/REPORT.md`](https://github.com/varadganjoo/memkernel/blob/main/evidence/live-agents/20260928T161716Z/REPORT.md).
181
+ In that run, the concurrently started agents contended for the same file, one was refused
182
+ and took other work, both bugs were fixed, and the hash chain verified intact in an
183
+ independent Linux container.
184
+
185
+ ## Repository layout
186
+
187
+ ```text
188
+ memkernel/
189
+ ├── core/events.py event taxonomy, envelope schema, hashing
190
+ ├── core/store.py AsyncEventStore: WAL, lineage, fork, rollback, diff, verify
191
+ ├── core/branch.py pure lineage algebra: live fold, rollback planning, diff, chain verification
192
+ ├── core/locks.py LockManager: leases, heartbeats, fencing tokens
193
+ ├── memory/state_view.py WorkspaceState fold, context split, rendering
194
+ ├── memory/gc.py MemoryGCEngine, heuristic and LLM summarizers, daemon
195
+ ├── memory/token_counter.py tiktoken with a conservative regex fallback
196
+ ├── kernel.py MemKernel facade (atomic lease+claim, fenced appends)
197
+ ├── server/mcp_server.py MCP server over stdio
198
+ ├── clients/adapter.py MemKernelClient (in-process) and McpStdioClient
199
+ └── cli.py `memkernel` command: attach, doctor, status, serve
200
+ examples/ multi-agent and rollback simulations
201
+ benchmarks/ reproducible measurements and the live-agent harness
202
+ evidence/ recorded live-agent run (transcripts, database, report)
203
+ docs/ whitepaper and integration guide
204
+ tests/ store, branching, locks, GC, server and stdio e2e
205
+ ```
206
+
207
+ ## Development
208
+
209
+ ```bash
210
+ pytest
211
+ ```
212
+
213
+ The suite runs offline and is deterministic. One opt-in test exercises
214
+ `LLMSummarizer` against a real OpenAI-compatible model:
215
+
216
+ ```bash
217
+ MEMKERNEL_LIVE_LLM_TESTS=1 OPENAI_API_KEY=... OPENAI_LLM=<model> pytest -k live
218
+ ```
219
+
220
+ ## Limitations
221
+
222
+ - **Single host.** Coordination relies on SQLite file locking on one machine; network filesystems are not supported. A replicated log is future work (see the whitepaper).
223
+ - **Self-asserted identity.** `agent_id` comes from the server's launch configuration. MemKernel records provenance but does not authenticate agents; see the threat model.
224
+ - **O(lineage) operations.** Rollback, arbitrary-point forks and full replay are linear in lineage length. Context reads and forks at the head are not.
225
+ - **No automatic merge.** `diff` reports conflicts. Reconciling them is an agent's (or a human's) decision, recorded as a new `ARCHITECTURAL_DECISION`.
226
+
227
+ ## License
228
+
229
+ MIT. See [LICENSE](https://github.com/varadganjoo/memkernel/blob/main/LICENSE).
230
+
@@ -0,0 +1,33 @@
1
+ """MemKernel: an event-sourced memory kernel for multi-agent coding systems."""
2
+
3
+ from memkernel.core.events import EventEnvelope, EventType
4
+ from memkernel.core.locks import LockManager
5
+ from memkernel.core.store import AsyncEventStore, BranchExistsError, BranchNotFoundError, StaleHeadError
6
+ from memkernel.kernel import DEFAULT_BRANCH, FencingTokenError, LeaseGrant, MemKernel
7
+ from memkernel.memory.gc import GCConfig, GCResult, HeuristicSummarizer, LLMSummarizer, MemoryGCEngine
8
+ from memkernel.memory.state_view import WorkspaceState
9
+ from memkernel.memory.token_counter import TokenCounter
10
+
11
+ __version__ = "0.1.0"
12
+
13
+ __all__ = [
14
+ "AsyncEventStore",
15
+ "BranchExistsError",
16
+ "BranchNotFoundError",
17
+ "DEFAULT_BRANCH",
18
+ "EventEnvelope",
19
+ "EventType",
20
+ "FencingTokenError",
21
+ "GCConfig",
22
+ "GCResult",
23
+ "HeuristicSummarizer",
24
+ "LLMSummarizer",
25
+ "LeaseGrant",
26
+ "LockManager",
27
+ "MemKernel",
28
+ "MemoryGCEngine",
29
+ "StaleHeadError",
30
+ "TokenCounter",
31
+ "WorkspaceState",
32
+ "__version__",
33
+ ]