ctxttools 0.2.1__tar.gz → 0.3.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. {ctxttools-0.2.1/ctxttools.egg-info → ctxttools-0.3.1}/PKG-INFO +119 -45
  2. {ctxttools-0.2.1 → ctxttools-0.3.1}/README.md +117 -42
  3. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/agents.py +424 -21
  4. ctxttools-0.3.1/ctools/backup.py +457 -0
  5. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/ccat.py +76 -47
  6. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/cconnect.py +55 -32
  7. ctxttools-0.3.1/ctools/ccopy.py +654 -0
  8. ctxttools-0.3.1/ctools/cdir.py +789 -0
  9. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/cdu.py +80 -29
  10. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/cextract.py +185 -46
  11. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/cgrep.py +178 -57
  12. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/cli.py +34 -22
  13. ctxttools-0.3.1/ctools/crecover.py +234 -0
  14. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/crm.py +60 -38
  15. ctxttools-0.3.1/ctools/ctools-web.py +30 -0
  16. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/ctools_mcp.py +162 -0
  17. ctxttools-0.3.1/ctools/filterlib.py +463 -0
  18. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/lib.py +25 -0
  19. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/strategy.py +19 -2
  20. ctxttools-0.3.1/ctools/testing.py +61 -0
  21. ctxttools-0.3.1/ctools/webui.py +2907 -0
  22. {ctxttools-0.2.1 → ctxttools-0.3.1/ctxttools.egg-info}/PKG-INFO +119 -45
  23. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctxttools.egg-info/SOURCES.txt +5 -0
  24. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctxttools.egg-info/entry_points.txt +2 -0
  25. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctxttools.egg-info/requires.txt +1 -2
  26. {ctxttools-0.2.1 → ctxttools-0.3.1}/pyproject.toml +4 -3
  27. ctxttools-0.2.1/ctools/ccopy.py +0 -478
  28. ctxttools-0.2.1/ctools/cdir.py +0 -505
  29. ctxttools-0.2.1/ctools/filterlib.py +0 -116
  30. {ctxttools-0.2.1 → ctxttools-0.3.1}/LICENSE.MIT +0 -0
  31. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/__init__.py +0 -0
  32. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/log.py +0 -0
  33. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctxttools.egg-info/dependency_links.txt +0 -0
  34. {ctxttools-0.2.1 → ctxttools-0.3.1}/ctxttools.egg-info/top_level.txt +0 -0
  35. {ctxttools-0.2.1 → ctxttools-0.3.1}/setup.cfg +0 -0
@@ -1,16 +1,15 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: ctxttools
3
- Version: 0.2.1
3
+ Version: 0.3.1
4
4
  Summary: CLI tools for LLM conversations
5
5
  Requires-Python: >=3.10
6
6
  Description-Content-Type: text/markdown
7
7
  License-File: LICENSE.MIT
8
- Requires-Dist: typer>=0.9.0
9
8
  Requires-Dist: rich>=13.0.0
10
9
  Requires-Dist: tiktoken>=0.5.0
11
10
  Requires-Dist: structlog>=24.0
12
11
  Provides-Extra: mcp
13
- Requires-Dist: mcp>=1.2.0; extra == "mcp"
12
+ Requires-Dist: mcp<2.0,>=1.2.0; extra == "mcp"
14
13
  Provides-Extra: dev
15
14
  Requires-Dist: pytest>=7.0.0; extra == "dev"
16
15
  Dynamic: license-file
@@ -25,9 +24,9 @@ Dynamic: license-file
25
24
  </p>
26
25
 
27
26
  ------
28
- **ctools** is a suite of simple tools for navigating through the sessions in various agents. It allows you to search your history, transfer conversations, filter them, and even share them among hosts and users through ssh.
27
+ **ctools** is a suite of simple tools for navigating through the sessions of the most popular LLM agents and coding tools. It allows you to search your history, transfer conversations, filter them, and even share them among machines and users.
29
28
 
30
- It supports [Claude Desktop](https://claude.com/download), [Claude Code](https://github.com/anthropics/claude-code), [Codex](https://github.com/openai/codex), [Pi](https://github.com/earendil-works/pi), [Hermes](https://github.com/NousResearch/hermes-agent), [Goose](https://github.com/aaif-goose/goose), [Kilo](https://github.com/Kilo-Org/kilocode), [Hermes](https://github.com/NousResearch/hermes-agent), [FreeBuff](https://github.com/CodebuffAI/freebuff), [Cline](https://github.com/cline/cline), and [omp](https://github.com/can1357/oh-my-pi).
29
+ It supports [Claude Desktop](https://claude.com/download), [Claude Code](https://github.com/anthropics/claude-code), [Codex](https://github.com/openai/codex), [Pi](https://github.com/earendil-works/pi), [Hermes](https://github.com/NousResearch/hermes-agent), [Goose](https://github.com/aaif-goose/goose), [Kilo](https://github.com/Kilo-Org/kilocode), [FreeBuff](https://github.com/CodebuffAI/freebuff), [Cline](https://github.com/cline/cline), [omp](https://github.com/can1357/oh-my-pi), and [OpenClaw](https://openclaw.ai/).
31
30
 
32
31
  Here's an example:
33
32
 
@@ -58,38 +57,6 @@ Resume it with:
58
57
 
59
58
  As you can see, these are unix-friendly tools using simple paradigms that you are already familiar with.
60
59
 
61
- ----
62
-
63
- ### 1. `cgrep` — find it in your past conversations
64
-
65
- You already know the answer is in a session you had months ago. You just don't remember which one. `cgrep` searches **every session you ever had, across every agent you use**, with a plain regex:
66
-
67
- ```shell
68
- $ cgrep -i "ssl"
69
- claude-code/a351eedf:142:assistant: … the TLS handshake fails with SSL wrong version number …
70
- opencode/ses_000d460f:16:user: ok graflex has an erorr in the check. i see this: … [[SSL: WRONG_VERSION_NUMBER] …
71
- opencode/ses_000d460f:34:assistant: Found it. Root cause: `_check_host` (graflex/__init__.py:128) tries `http`, and …
72
- ```
73
-
74
- No agent argument? It searches **every installed agent at once** — Claude Code, Opencode, Kilo, Codex, Pi, Goose, Hermes, Cline, omp, and Freebuff, all in one pass. That last line is the root-cause analysis you wrote six months ago and would never have found again.
75
-
76
- ### 2. `ccopy` — move your project from one tool to another
77
-
78
- Switched from opencode to Claude Code mid-project? You don't have to start from zero. Copy the whole conversation — context, decisions, the lot — into a fresh session in the other tool:
79
-
80
- ```shell
81
- $ ccopy opencode/ses_abc123 claude-code
82
- Copied 128 message(s) from opencode/ses_abc123 to a new claude-code session: 1a2b3c
83
- Resume it with:
84
- claude-code --resume 1a2b3c
85
- ```
86
-
87
- Your work travels with you. The source is never touched, so it's a true copy. And when the destination is on another machine, it's just `ccopy opencode/ses_abc123 ssh://chris@remote/codex` — it pulls the agent's storage over your normal ssh (the remote only needs `sshd` and `tar`, not ctools).
88
-
89
- ctools is a set of GNU-style tools for the history your coding agents leave behind. These two are the headline; the rest (`cdir` to list sessions, `ccat` to read them, `cextract` to move concepts, `cconnect` to pipe them live, `cdu` to count tokens, `crm` to prune) are in service of them.
90
-
91
- **Works with the agents you already use:** Claude Desktop, Claude Code, Opencode, Kilo, Codex, Pi, Goose, Hermes, Cline, omp (oh-my-pi), and Freebuff (Codebuff).
92
-
93
60
  ## Tools
94
61
 
95
62
  | Tool | What it does | Docs |
@@ -98,13 +65,21 @@ ctools is a set of GNU-style tools for the history your coding agents leave behi
98
65
  | `cdir` | List sessions (ls for your history) | [docs/cdir.md](docs/cdir.md) |
99
66
  | `ccat` | Print one or more conversations (cat for your history) | [docs/ccat.md](docs/ccat.md) |
100
67
  | `ccopy` | Copy a whole conversation from one agent to a new session in another | [docs/ccopy.md](docs/ccopy.md) |
68
+ | `crecover` | List, verify, and restore the storage backups ccopy takes before each write | [docs/crecover.md](docs/crecover.md) |
101
69
  | `cextract` | Extract concepts from a session into files, and inject them back | [docs/cextract.md](docs/cextract.md) |
102
70
  | `cconnect` | Live concept pipelines between sessions | [docs/cconnect.md](docs/cconnect.md) |
103
71
  | `cdu` | Token usage per session (du for context windows) | [docs/cdu.md](docs/cdu.md) |
104
72
  | `crm` | Remove concepts from a session (mdel) | [docs/crm.md](docs/crm.md) |
73
+ | `ctools-web` | Browser dashboard for all of the above | [Web dashboard](#web-dashboard) |
105
74
 
106
75
  Every tool takes `--version`. The session-mover tools (`ccopy`, `cextract`, `cconnect`, `crm`) also take `--verbose` for structured logging. Each tool's full flag reference, output formats, and examples live in its `docs/` page — the table above is just the 30-second version.
107
76
 
77
+ ## The problem
78
+
79
+ You talk to LLMs all day. Over weeks, you build up a set of constraints, preferences, and goals. These live in your conversations as system messages. They are valuable. They are also trapped.
80
+
81
+ Say you have been working with opencode for a month. You have refined your coding style through dozens of sessions. Now you start a new Claude Code project and you want those same preferences. You could copy them by hand. Or you could `cextract` them out and `cextract` them back in, or copy the whole conversation across with `ccopy`. Either way, your memory travels with you.
82
+
108
83
  ## How it works
109
84
 
110
85
  ctools is a substrate for moving memory between context windows. Cross-platform, agent-agnostic, designed like a bus.
@@ -114,6 +89,8 @@ Two stages: **extraction** and **filtering**.
114
89
  - A **strategy** extracts concepts from a conversation. It defines what counts as a concept and how to find it: regex patterns, LLM extraction, whatever. Different strategies produce different ontologies from the same conversation, because context is contestable.
115
90
  - A **filter** selects which extracted concepts reach a destination. It's a binary classifier: pass through or filter out. One concept list can fan out to many destinations, each with its own filter. A coding agent gets coding preferences, a security agent gets security constraints, a PM agent gets goals, all from the same source.
116
91
 
92
+ A filter can be as simple as a type or substring match, or as smart as a **decision model** — a small local classifier that scores each concept and lets it through only when confident. Point a filter at a model and you get *semantic* filtering with no free text to parse: ask "is this concept about security?" and gate on the answer. The model is asked either through Ollama's typed `/v1/systemone` endpoint (calibrated probabilities, the home of decision-dedicated models) or through any OpenAI-compatible `/v1/chat/completions` server (portable, lettered decision-task prompt) — one `endpoint` field decides. See [cextract](docs/cextract.md) for the decision-model and JSON-RPC filter configs.
93
+
117
94
  ```mermaid
118
95
  graph LR
119
96
  SRC["source session"] --> STRAT["strategy<br/>(extraction)"]
@@ -126,13 +103,33 @@ graph LR
126
103
  FC --> DEST_C["destination C"]
127
104
  ```
128
105
 
129
- Context windows are endpoints: opencode, Claude Code, Codex, Pi. They all speak different protocols, but they all consume the same packets. See [cextract](docs/cextract.md) for the concept packet format, strategies, and the filter (JSON-RPC) protocol.
106
+ Context windows are endpoints: opencode, Claude Code, Codex, Pi. They all speak different protocols, but they all consume the same packets. See [cextract](docs/cextract.md) for the concept packet format, strategies, and the filter configs (decision model and JSON-RPC).
130
107
 
131
- ## The problem
108
+ ## Write safety
132
109
 
133
- You talk to LLMs all day. Over weeks, you build up a set of constraints, preferences, and goals. These live in your conversations as system messages. They are valuable. They are also trapped.
110
+ The session-mover tools write into your agents' **live** storage — for the
111
+ SQLite-backed agents (opencode, goose, kilo, hermes) that's the database that
112
+ holds every session of that agent. A write that fails or is interrupted
113
+ part-way can corrupt it, and losing an agent's whole history would be
114
+ unforgivable.
134
115
 
135
- Say you have been working with opencode for a month. You have refined your coding style through dozens of sessions. Now you start a new Claude Code project and you want those same preferences. You could copy them by hand. Or you could `cextract` them out and `cextract` them back in, or copy the whole conversation across with `ccopy`. Either way, your memory travels with you.
116
+ So every local write is guarded:
117
+
118
+ 1. ctools **snapshots the agent's storage first** (SQLite via `VACUUM INTO` —
119
+ a consistent copy even mid-write; file agents via a tree copy), kept under
120
+ `~/.local/share/ctools/backups/<agent>/` (20 newest per agent).
121
+ 2. After the write it **verifies** the storage: the DB opens,
122
+ `PRAGMA integrity_check` passes, the session table still exists, and the
123
+ session count did not decrease. A database that is structurally valid but
124
+ emptied of data is caught too.
125
+ 3. If anything looks wrong, the snapshot is **restored automatically** — you
126
+ get a clean error, not a broken database.
127
+
128
+ And because a restore is itself a write, it snapshots what it overwrites, so
129
+ **every restore is undoable**. For the out-of-band case (you notice days later
130
+ that an agent's history vanished), `crecover list` / `verify` / `restore` are
131
+ there. Details: [ccopy — safety](docs/ccopy.md#safety-automatic-backups) and
132
+ [crecover](docs/crecover.md).
136
133
 
137
134
  ## Supported endpoints
138
135
 
@@ -149,6 +146,12 @@ Say you have been working with opencode for a month. You have refined your codin
149
146
  | cline | JSON (`~/.cline/data/tasks/`) |
150
147
  | omp | JSONL (`~/.omp/agent/sessions/`, tree journal) |
151
148
  | freebuff | JSON (`~/.config/<codebuff>/projects/`) |
149
+ | openclaw | SQLite (per-agent, `~/.openclaw/agents/`, `OPENCLAW_STATE_DIR`) |
150
+
151
+ OpenClaw is a **personal assistant** rather than a terminal coding agent, so it
152
+ runs in a local Gateway that owns its store; ctools reads and exports its
153
+ sessions but does not seed new ones (the store has canonical-index and integrity
154
+ checks that a raw insert cannot satisfy).
152
155
 
153
156
  Run `cdir` to see which endpoints are found on your system and where they store data.
154
157
 
@@ -174,23 +177,87 @@ Anything the storage cannot do — pi's session trees cannot be rewritten in pla
174
177
 
175
178
  ## MCP server
176
179
 
177
- There is an MCP server for use from Claude, opencode, Cursor, or anything else that speaks MCP.
180
+ There is an MCP server for use from pi, Claude, opencode, Cursor, or anything else that speaks MCP.
178
181
 
179
182
  Tools: `list_agents`, `list_sessions`, `search_sessions`, `export_session`, `extract_concepts`, `copy_concepts`, `get_session_concepts`.
180
183
 
181
- Add to your MCP config:
184
+ ### Quick install (pi)
185
+
186
+ One command does the whole job — install into an isolated venv (if needed) and register with pi:
187
+
188
+ ```sh
189
+ pipx run 'ctxttools[mcp]' ctools-mcp install
190
+ # or, with uv:
191
+ uv tool run 'ctxttools[mcp]' ctools-mcp install
192
+ ```
193
+
194
+ It detects an existing `uv tool` or `pipx` install of `ctxttools` and uses it instead. Verify with:
195
+
196
+ ```sh
197
+ pi mcp list # expect: ctools: connected, 7 tools
198
+ ```
199
+
200
+ Then use it inside pi — run `/mcp` to inspect, or just ask the agent (the tools are also reachable through codemode / tool search).
201
+
202
+ Uninstall with `ctools-mcp uninstall` (removes the pi entry; pass `--all` to also remove Claude Code and opencode entries).
203
+
204
+ ### Other clients
205
+
206
+ Run `ctools-mcp print-config` to get the JSON snippet for any MCP host, e.g. for opencode (`opencode.json` under `"mcp"`) or Claude (`.mcp.json`):
182
207
 
183
208
  ```json
184
209
  {
185
210
  "mcpServers": {
186
211
  "ctools": {
187
212
  "command": "python",
188
- "args": ["/ABSOLUTE/PATH/TO/ctools/ctools_mcp.py"]
213
+ "args": ["-m", "ctools.ctools_mcp"]
189
214
  }
190
215
  }
191
216
  }
192
217
  ```
193
218
 
219
+ (`command` should point at the interpreter of a venv that has `ctxttools[mcp]` installed — `ctools-mcp install` prints the exact one for your machine.)
220
+
221
+ ## Web dashboard
222
+
223
+ Prefer a mouse to a man page? ctools ships a browser dashboard — a small local
224
+ web server (Python stdlib only, no web dependencies) with the same library
225
+ underneath. Same on macOS, Windows, and Linux:
226
+
227
+ ```sh
228
+ $ ctools-web # installed command
229
+ # or, straight from a checkout, no install needed:
230
+ $ python3 ctools/ctools-web.py # or: python -m ctools.webui
231
+ ctools 0.2.1 dashboard: http://127.0.0.1:8765
232
+ ```
233
+
234
+ Open the URL. The navbar has **Overview**, **Sessions**, and **Concepts**, plus
235
+ a search box (press <b>/</b>) and a power button to stop the server. Clicking a
236
+ session anywhere opens its **Conversation** view:
237
+
238
+ - **Overview** — every supported agent, installed or not, with session counts
239
+ and total tokens, and a jump into each agent's sessions.
240
+ - **Search** — `cgrep` in the browser: one box, searches session ids, names,
241
+ paths <b>and</b> the content of every conversation at once; click a match to
242
+ open that conversation.
243
+ - **Sessions** — `cdir` for one agent (or all of them): newest, largest, or by
244
+ token usage, filterable.
245
+ - **Conversation** — `ccat` with the session's messages, token usage, the
246
+ resume command, concept extraction (`cextract`), a markdown export, and
247
+ **Copy** — `ccopy` to any other installed agent, which creates a fresh
248
+ session there (source untouched) and tells you how to resume it.
249
+
250
+ The server binds to `127.0.0.1` by default, so nothing is exposed to your
251
+ network unless you pass `--host 0.0.0.0`. There is one write operation:
252
+ Copy. Everything else is read-only against the agents' storage.
253
+
254
+ Stop it with **Ctrl-C**, or with the **power button** (⏻) in the top-right of
255
+ the page — it asks for confirmation, then shuts the server down cleanly.
256
+
257
+ ```sh
258
+ ctools-web --port 9000
259
+ ```
260
+
194
261
  ## Installation
195
262
 
196
263
  ```sh
@@ -213,7 +280,7 @@ from ctools.lib import get_formatter
213
280
  from ctools.cgrep import grep_session
214
281
  from ctools.cextract import extract_concepts_from_messages
215
282
  from ctools.cdu import count_tokens, get_session_tokens
216
- from ctools.filterlib import JSONRPCFilter, load_filter
283
+ from ctools.filterlib import JSONRPCFilter, SystemOneFilter, load_filter
217
284
  from ctools.log import configure_logging, get_logger
218
285
  ```
219
286
 
@@ -238,6 +305,13 @@ from ctools.agents import OpencodeAgent
238
305
  agent = OpencodeAgent('/backup/opencode')
239
306
  ```
240
307
 
308
+ ## Where to go next
309
+
310
+ - **Per-tool reference** — the [Tools](#tools) table above links to a `docs/` page for each command: every flag, output format, and a worked example.
311
+ - **The interchange format** — how a conversation becomes a portable, lossless envelope and back, and why 2N agents compose into N² transfers. See [interchange](docs/interchange.md).
312
+ - **Filtering with decision models** — the full decision-model and JSON-RPC filter configs, with live examples. See [cextract](docs/cextract.md#filters) and [cconnect](docs/cconnect.md#filter-configuration).
313
+ - **Adding an endpoint** — one `Agent` subclass plus a line in `AGENT_CLASSES`. See [Supported endpoints](#supported-endpoints).
314
+
241
315
  ## Credits
242
316
 
243
317
  ctools was extracted from [Gab n' Go](https://github.com/day50-dev/gabngo). It's named after [GNU mtools](https://www.gnu.org/software/mtools/), which does the same thing for DOS floppies — because your context window is about the size of a DOS-floppy. The flag habits are GNU's too: `--version`, `-1`, `-S`, `-u`, and the rest behave the way you'd expect.
@@ -8,9 +8,9 @@
8
8
  </p>
9
9
 
10
10
  ------
11
- **ctools** is a suite of simple tools for navigating through the sessions in various agents. It allows you to search your history, transfer conversations, filter them, and even share them among hosts and users through ssh.
11
+ **ctools** is a suite of simple tools for navigating through the sessions of the most popular LLM agents and coding tools. It allows you to search your history, transfer conversations, filter them, and even share them among machines and users.
12
12
 
13
- It supports [Claude Desktop](https://claude.com/download), [Claude Code](https://github.com/anthropics/claude-code), [Codex](https://github.com/openai/codex), [Pi](https://github.com/earendil-works/pi), [Hermes](https://github.com/NousResearch/hermes-agent), [Goose](https://github.com/aaif-goose/goose), [Kilo](https://github.com/Kilo-Org/kilocode), [Hermes](https://github.com/NousResearch/hermes-agent), [FreeBuff](https://github.com/CodebuffAI/freebuff), [Cline](https://github.com/cline/cline), and [omp](https://github.com/can1357/oh-my-pi).
13
+ It supports [Claude Desktop](https://claude.com/download), [Claude Code](https://github.com/anthropics/claude-code), [Codex](https://github.com/openai/codex), [Pi](https://github.com/earendil-works/pi), [Hermes](https://github.com/NousResearch/hermes-agent), [Goose](https://github.com/aaif-goose/goose), [Kilo](https://github.com/Kilo-Org/kilocode), [FreeBuff](https://github.com/CodebuffAI/freebuff), [Cline](https://github.com/cline/cline), [omp](https://github.com/can1357/oh-my-pi), and [OpenClaw](https://openclaw.ai/).
14
14
 
15
15
  Here's an example:
16
16
 
@@ -41,38 +41,6 @@ Resume it with:
41
41
 
42
42
  As you can see, these are unix-friendly tools using simple paradigms that you are already familiar with.
43
43
 
44
- ----
45
-
46
- ### 1. `cgrep` — find it in your past conversations
47
-
48
- You already know the answer is in a session you had months ago. You just don't remember which one. `cgrep` searches **every session you ever had, across every agent you use**, with a plain regex:
49
-
50
- ```shell
51
- $ cgrep -i "ssl"
52
- claude-code/a351eedf:142:assistant: … the TLS handshake fails with SSL wrong version number …
53
- opencode/ses_000d460f:16:user: ok graflex has an erorr in the check. i see this: … [[SSL: WRONG_VERSION_NUMBER] …
54
- opencode/ses_000d460f:34:assistant: Found it. Root cause: `_check_host` (graflex/__init__.py:128) tries `http`, and …
55
- ```
56
-
57
- No agent argument? It searches **every installed agent at once** — Claude Code, Opencode, Kilo, Codex, Pi, Goose, Hermes, Cline, omp, and Freebuff, all in one pass. That last line is the root-cause analysis you wrote six months ago and would never have found again.
58
-
59
- ### 2. `ccopy` — move your project from one tool to another
60
-
61
- Switched from opencode to Claude Code mid-project? You don't have to start from zero. Copy the whole conversation — context, decisions, the lot — into a fresh session in the other tool:
62
-
63
- ```shell
64
- $ ccopy opencode/ses_abc123 claude-code
65
- Copied 128 message(s) from opencode/ses_abc123 to a new claude-code session: 1a2b3c
66
- Resume it with:
67
- claude-code --resume 1a2b3c
68
- ```
69
-
70
- Your work travels with you. The source is never touched, so it's a true copy. And when the destination is on another machine, it's just `ccopy opencode/ses_abc123 ssh://chris@remote/codex` — it pulls the agent's storage over your normal ssh (the remote only needs `sshd` and `tar`, not ctools).
71
-
72
- ctools is a set of GNU-style tools for the history your coding agents leave behind. These two are the headline; the rest (`cdir` to list sessions, `ccat` to read them, `cextract` to move concepts, `cconnect` to pipe them live, `cdu` to count tokens, `crm` to prune) are in service of them.
73
-
74
- **Works with the agents you already use:** Claude Desktop, Claude Code, Opencode, Kilo, Codex, Pi, Goose, Hermes, Cline, omp (oh-my-pi), and Freebuff (Codebuff).
75
-
76
44
  ## Tools
77
45
 
78
46
  | Tool | What it does | Docs |
@@ -81,13 +49,21 @@ ctools is a set of GNU-style tools for the history your coding agents leave behi
81
49
  | `cdir` | List sessions (ls for your history) | [docs/cdir.md](docs/cdir.md) |
82
50
  | `ccat` | Print one or more conversations (cat for your history) | [docs/ccat.md](docs/ccat.md) |
83
51
  | `ccopy` | Copy a whole conversation from one agent to a new session in another | [docs/ccopy.md](docs/ccopy.md) |
52
+ | `crecover` | List, verify, and restore the storage backups ccopy takes before each write | [docs/crecover.md](docs/crecover.md) |
84
53
  | `cextract` | Extract concepts from a session into files, and inject them back | [docs/cextract.md](docs/cextract.md) |
85
54
  | `cconnect` | Live concept pipelines between sessions | [docs/cconnect.md](docs/cconnect.md) |
86
55
  | `cdu` | Token usage per session (du for context windows) | [docs/cdu.md](docs/cdu.md) |
87
56
  | `crm` | Remove concepts from a session (mdel) | [docs/crm.md](docs/crm.md) |
57
+ | `ctools-web` | Browser dashboard for all of the above | [Web dashboard](#web-dashboard) |
88
58
 
89
59
  Every tool takes `--version`. The session-mover tools (`ccopy`, `cextract`, `cconnect`, `crm`) also take `--verbose` for structured logging. Each tool's full flag reference, output formats, and examples live in its `docs/` page — the table above is just the 30-second version.
90
60
 
61
+ ## The problem
62
+
63
+ You talk to LLMs all day. Over weeks, you build up a set of constraints, preferences, and goals. These live in your conversations as system messages. They are valuable. They are also trapped.
64
+
65
+ Say you have been working with opencode for a month. You have refined your coding style through dozens of sessions. Now you start a new Claude Code project and you want those same preferences. You could copy them by hand. Or you could `cextract` them out and `cextract` them back in, or copy the whole conversation across with `ccopy`. Either way, your memory travels with you.
66
+
91
67
  ## How it works
92
68
 
93
69
  ctools is a substrate for moving memory between context windows. Cross-platform, agent-agnostic, designed like a bus.
@@ -97,6 +73,8 @@ Two stages: **extraction** and **filtering**.
97
73
  - A **strategy** extracts concepts from a conversation. It defines what counts as a concept and how to find it: regex patterns, LLM extraction, whatever. Different strategies produce different ontologies from the same conversation, because context is contestable.
98
74
  - A **filter** selects which extracted concepts reach a destination. It's a binary classifier: pass through or filter out. One concept list can fan out to many destinations, each with its own filter. A coding agent gets coding preferences, a security agent gets security constraints, a PM agent gets goals, all from the same source.
99
75
 
76
+ A filter can be as simple as a type or substring match, or as smart as a **decision model** — a small local classifier that scores each concept and lets it through only when confident. Point a filter at a model and you get *semantic* filtering with no free text to parse: ask "is this concept about security?" and gate on the answer. The model is asked either through Ollama's typed `/v1/systemone` endpoint (calibrated probabilities, the home of decision-dedicated models) or through any OpenAI-compatible `/v1/chat/completions` server (portable, lettered decision-task prompt) — one `endpoint` field decides. See [cextract](docs/cextract.md) for the decision-model and JSON-RPC filter configs.
77
+
100
78
  ```mermaid
101
79
  graph LR
102
80
  SRC["source session"] --> STRAT["strategy<br/>(extraction)"]
@@ -109,13 +87,33 @@ graph LR
109
87
  FC --> DEST_C["destination C"]
110
88
  ```
111
89
 
112
- Context windows are endpoints: opencode, Claude Code, Codex, Pi. They all speak different protocols, but they all consume the same packets. See [cextract](docs/cextract.md) for the concept packet format, strategies, and the filter (JSON-RPC) protocol.
90
+ Context windows are endpoints: opencode, Claude Code, Codex, Pi. They all speak different protocols, but they all consume the same packets. See [cextract](docs/cextract.md) for the concept packet format, strategies, and the filter configs (decision model and JSON-RPC).
113
91
 
114
- ## The problem
92
+ ## Write safety
115
93
 
116
- You talk to LLMs all day. Over weeks, you build up a set of constraints, preferences, and goals. These live in your conversations as system messages. They are valuable. They are also trapped.
94
+ The session-mover tools write into your agents' **live** storage — for the
95
+ SQLite-backed agents (opencode, goose, kilo, hermes) that's the database that
96
+ holds every session of that agent. A write that fails or is interrupted
97
+ part-way can corrupt it, and losing an agent's whole history would be
98
+ unforgivable.
117
99
 
118
- Say you have been working with opencode for a month. You have refined your coding style through dozens of sessions. Now you start a new Claude Code project and you want those same preferences. You could copy them by hand. Or you could `cextract` them out and `cextract` them back in, or copy the whole conversation across with `ccopy`. Either way, your memory travels with you.
100
+ So every local write is guarded:
101
+
102
+ 1. ctools **snapshots the agent's storage first** (SQLite via `VACUUM INTO` —
103
+ a consistent copy even mid-write; file agents via a tree copy), kept under
104
+ `~/.local/share/ctools/backups/<agent>/` (20 newest per agent).
105
+ 2. After the write it **verifies** the storage: the DB opens,
106
+ `PRAGMA integrity_check` passes, the session table still exists, and the
107
+ session count did not decrease. A database that is structurally valid but
108
+ emptied of data is caught too.
109
+ 3. If anything looks wrong, the snapshot is **restored automatically** — you
110
+ get a clean error, not a broken database.
111
+
112
+ And because a restore is itself a write, it snapshots what it overwrites, so
113
+ **every restore is undoable**. For the out-of-band case (you notice days later
114
+ that an agent's history vanished), `crecover list` / `verify` / `restore` are
115
+ there. Details: [ccopy — safety](docs/ccopy.md#safety-automatic-backups) and
116
+ [crecover](docs/crecover.md).
119
117
 
120
118
  ## Supported endpoints
121
119
 
@@ -132,6 +130,12 @@ Say you have been working with opencode for a month. You have refined your codin
132
130
  | cline | JSON (`~/.cline/data/tasks/`) |
133
131
  | omp | JSONL (`~/.omp/agent/sessions/`, tree journal) |
134
132
  | freebuff | JSON (`~/.config/<codebuff>/projects/`) |
133
+ | openclaw | SQLite (per-agent, `~/.openclaw/agents/`, `OPENCLAW_STATE_DIR`) |
134
+
135
+ OpenClaw is a **personal assistant** rather than a terminal coding agent, so it
136
+ runs in a local Gateway that owns its store; ctools reads and exports its
137
+ sessions but does not seed new ones (the store has canonical-index and integrity
138
+ checks that a raw insert cannot satisfy).
135
139
 
136
140
  Run `cdir` to see which endpoints are found on your system and where they store data.
137
141
 
@@ -157,23 +161,87 @@ Anything the storage cannot do — pi's session trees cannot be rewritten in pla
157
161
 
158
162
  ## MCP server
159
163
 
160
- There is an MCP server for use from Claude, opencode, Cursor, or anything else that speaks MCP.
164
+ There is an MCP server for use from pi, Claude, opencode, Cursor, or anything else that speaks MCP.
161
165
 
162
166
  Tools: `list_agents`, `list_sessions`, `search_sessions`, `export_session`, `extract_concepts`, `copy_concepts`, `get_session_concepts`.
163
167
 
164
- Add to your MCP config:
168
+ ### Quick install (pi)
169
+
170
+ One command does the whole job — install into an isolated venv (if needed) and register with pi:
171
+
172
+ ```sh
173
+ pipx run 'ctxttools[mcp]' ctools-mcp install
174
+ # or, with uv:
175
+ uv tool run 'ctxttools[mcp]' ctools-mcp install
176
+ ```
177
+
178
+ It detects an existing `uv tool` or `pipx` install of `ctxttools` and uses it instead. Verify with:
179
+
180
+ ```sh
181
+ pi mcp list # expect: ctools: connected, 7 tools
182
+ ```
183
+
184
+ Then use it inside pi — run `/mcp` to inspect, or just ask the agent (the tools are also reachable through codemode / tool search).
185
+
186
+ Uninstall with `ctools-mcp uninstall` (removes the pi entry; pass `--all` to also remove Claude Code and opencode entries).
187
+
188
+ ### Other clients
189
+
190
+ Run `ctools-mcp print-config` to get the JSON snippet for any MCP host, e.g. for opencode (`opencode.json` under `"mcp"`) or Claude (`.mcp.json`):
165
191
 
166
192
  ```json
167
193
  {
168
194
  "mcpServers": {
169
195
  "ctools": {
170
196
  "command": "python",
171
- "args": ["/ABSOLUTE/PATH/TO/ctools/ctools_mcp.py"]
197
+ "args": ["-m", "ctools.ctools_mcp"]
172
198
  }
173
199
  }
174
200
  }
175
201
  ```
176
202
 
203
+ (`command` should point at the interpreter of a venv that has `ctxttools[mcp]` installed — `ctools-mcp install` prints the exact one for your machine.)
204
+
205
+ ## Web dashboard
206
+
207
+ Prefer a mouse to a man page? ctools ships a browser dashboard — a small local
208
+ web server (Python stdlib only, no web dependencies) with the same library
209
+ underneath. Same on macOS, Windows, and Linux:
210
+
211
+ ```sh
212
+ $ ctools-web # installed command
213
+ # or, straight from a checkout, no install needed:
214
+ $ python3 ctools/ctools-web.py # or: python -m ctools.webui
215
+ ctools 0.2.1 dashboard: http://127.0.0.1:8765
216
+ ```
217
+
218
+ Open the URL. The navbar has **Overview**, **Sessions**, and **Concepts**, plus
219
+ a search box (press <b>/</b>) and a power button to stop the server. Clicking a
220
+ session anywhere opens its **Conversation** view:
221
+
222
+ - **Overview** — every supported agent, installed or not, with session counts
223
+ and total tokens, and a jump into each agent's sessions.
224
+ - **Search** — `cgrep` in the browser: one box, searches session ids, names,
225
+ paths <b>and</b> the content of every conversation at once; click a match to
226
+ open that conversation.
227
+ - **Sessions** — `cdir` for one agent (or all of them): newest, largest, or by
228
+ token usage, filterable.
229
+ - **Conversation** — `ccat` with the session's messages, token usage, the
230
+ resume command, concept extraction (`cextract`), a markdown export, and
231
+ **Copy** — `ccopy` to any other installed agent, which creates a fresh
232
+ session there (source untouched) and tells you how to resume it.
233
+
234
+ The server binds to `127.0.0.1` by default, so nothing is exposed to your
235
+ network unless you pass `--host 0.0.0.0`. There is one write operation:
236
+ Copy. Everything else is read-only against the agents' storage.
237
+
238
+ Stop it with **Ctrl-C**, or with the **power button** (⏻) in the top-right of
239
+ the page — it asks for confirmation, then shuts the server down cleanly.
240
+
241
+ ```sh
242
+ ctools-web --port 9000
243
+ ```
244
+
177
245
  ## Installation
178
246
 
179
247
  ```sh
@@ -196,7 +264,7 @@ from ctools.lib import get_formatter
196
264
  from ctools.cgrep import grep_session
197
265
  from ctools.cextract import extract_concepts_from_messages
198
266
  from ctools.cdu import count_tokens, get_session_tokens
199
- from ctools.filterlib import JSONRPCFilter, load_filter
267
+ from ctools.filterlib import JSONRPCFilter, SystemOneFilter, load_filter
200
268
  from ctools.log import configure_logging, get_logger
201
269
  ```
202
270
 
@@ -221,6 +289,13 @@ from ctools.agents import OpencodeAgent
221
289
  agent = OpencodeAgent('/backup/opencode')
222
290
  ```
223
291
 
292
+ ## Where to go next
293
+
294
+ - **Per-tool reference** — the [Tools](#tools) table above links to a `docs/` page for each command: every flag, output format, and a worked example.
295
+ - **The interchange format** — how a conversation becomes a portable, lossless envelope and back, and why 2N agents compose into N² transfers. See [interchange](docs/interchange.md).
296
+ - **Filtering with decision models** — the full decision-model and JSON-RPC filter configs, with live examples. See [cextract](docs/cextract.md#filters) and [cconnect](docs/cconnect.md#filter-configuration).
297
+ - **Adding an endpoint** — one `Agent` subclass plus a line in `AGENT_CLASSES`. See [Supported endpoints](#supported-endpoints).
298
+
224
299
  ## Credits
225
300
 
226
301
  ctools was extracted from [Gab n' Go](https://github.com/day50-dev/gabngo). It's named after [GNU mtools](https://www.gnu.org/software/mtools/), which does the same thing for DOS floppies — because your context window is about the size of a DOS-floppy. The flag habits are GNU's too: `--version`, `-1`, `-S`, `-u`, and the rest behave the way you'd expect.