ctxttools 0.2.1__tar.gz → 0.3.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {ctxttools-0.2.1/ctxttools.egg-info → ctxttools-0.3.1}/PKG-INFO +119 -45
- {ctxttools-0.2.1 → ctxttools-0.3.1}/README.md +117 -42
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/agents.py +424 -21
- ctxttools-0.3.1/ctools/backup.py +457 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/ccat.py +76 -47
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/cconnect.py +55 -32
- ctxttools-0.3.1/ctools/ccopy.py +654 -0
- ctxttools-0.3.1/ctools/cdir.py +789 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/cdu.py +80 -29
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/cextract.py +185 -46
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/cgrep.py +178 -57
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/cli.py +34 -22
- ctxttools-0.3.1/ctools/crecover.py +234 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/crm.py +60 -38
- ctxttools-0.3.1/ctools/ctools-web.py +30 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/ctools_mcp.py +162 -0
- ctxttools-0.3.1/ctools/filterlib.py +463 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/lib.py +25 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/strategy.py +19 -2
- ctxttools-0.3.1/ctools/testing.py +61 -0
- ctxttools-0.3.1/ctools/webui.py +2907 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1/ctxttools.egg-info}/PKG-INFO +119 -45
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctxttools.egg-info/SOURCES.txt +5 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctxttools.egg-info/entry_points.txt +2 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctxttools.egg-info/requires.txt +1 -2
- {ctxttools-0.2.1 → ctxttools-0.3.1}/pyproject.toml +4 -3
- ctxttools-0.2.1/ctools/ccopy.py +0 -478
- ctxttools-0.2.1/ctools/cdir.py +0 -505
- ctxttools-0.2.1/ctools/filterlib.py +0 -116
- {ctxttools-0.2.1 → ctxttools-0.3.1}/LICENSE.MIT +0 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/__init__.py +0 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctools/log.py +0 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctxttools.egg-info/dependency_links.txt +0 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/ctxttools.egg-info/top_level.txt +0 -0
- {ctxttools-0.2.1 → ctxttools-0.3.1}/setup.cfg +0 -0
|
@@ -1,16 +1,15 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: ctxttools
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.3.1
|
|
4
4
|
Summary: CLI tools for LLM conversations
|
|
5
5
|
Requires-Python: >=3.10
|
|
6
6
|
Description-Content-Type: text/markdown
|
|
7
7
|
License-File: LICENSE.MIT
|
|
8
|
-
Requires-Dist: typer>=0.9.0
|
|
9
8
|
Requires-Dist: rich>=13.0.0
|
|
10
9
|
Requires-Dist: tiktoken>=0.5.0
|
|
11
10
|
Requires-Dist: structlog>=24.0
|
|
12
11
|
Provides-Extra: mcp
|
|
13
|
-
Requires-Dist: mcp
|
|
12
|
+
Requires-Dist: mcp<2.0,>=1.2.0; extra == "mcp"
|
|
14
13
|
Provides-Extra: dev
|
|
15
14
|
Requires-Dist: pytest>=7.0.0; extra == "dev"
|
|
16
15
|
Dynamic: license-file
|
|
@@ -25,9 +24,9 @@ Dynamic: license-file
|
|
|
25
24
|
</p>
|
|
26
25
|
|
|
27
26
|
------
|
|
28
|
-
**ctools** is a suite of simple tools for navigating through the sessions
|
|
27
|
+
**ctools** is a suite of simple tools for navigating through the sessions of the most popular LLM agents and coding tools. It allows you to search your history, transfer conversations, filter them, and even share them among machines and users.
|
|
29
28
|
|
|
30
|
-
It supports [Claude Desktop](https://claude.com/download), [Claude Code](https://github.com/anthropics/claude-code), [Codex](https://github.com/openai/codex), [Pi](https://github.com/earendil-works/pi), [Hermes](https://github.com/NousResearch/hermes-agent), [Goose](https://github.com/aaif-goose/goose), [Kilo](https://github.com/Kilo-Org/kilocode), [
|
|
29
|
+
It supports [Claude Desktop](https://claude.com/download), [Claude Code](https://github.com/anthropics/claude-code), [Codex](https://github.com/openai/codex), [Pi](https://github.com/earendil-works/pi), [Hermes](https://github.com/NousResearch/hermes-agent), [Goose](https://github.com/aaif-goose/goose), [Kilo](https://github.com/Kilo-Org/kilocode), [FreeBuff](https://github.com/CodebuffAI/freebuff), [Cline](https://github.com/cline/cline), [omp](https://github.com/can1357/oh-my-pi), and [OpenClaw](https://openclaw.ai/).
|
|
31
30
|
|
|
32
31
|
Here's an example:
|
|
33
32
|
|
|
@@ -58,38 +57,6 @@ Resume it with:
|
|
|
58
57
|
|
|
59
58
|
As you can see, these are unix-friendly tools using simple paradigms that you are already familiar with.
|
|
60
59
|
|
|
61
|
-
----
|
|
62
|
-
|
|
63
|
-
### 1. `cgrep` — find it in your past conversations
|
|
64
|
-
|
|
65
|
-
You already know the answer is in a session you had months ago. You just don't remember which one. `cgrep` searches **every session you ever had, across every agent you use**, with a plain regex:
|
|
66
|
-
|
|
67
|
-
```shell
|
|
68
|
-
$ cgrep -i "ssl"
|
|
69
|
-
claude-code/a351eedf:142:assistant: … the TLS handshake fails with SSL wrong version number …
|
|
70
|
-
opencode/ses_000d460f:16:user: ok graflex has an erorr in the check. i see this: … [[SSL: WRONG_VERSION_NUMBER] …
|
|
71
|
-
opencode/ses_000d460f:34:assistant: Found it. Root cause: `_check_host` (graflex/__init__.py:128) tries `http`, and …
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
No agent argument? It searches **every installed agent at once** — Claude Code, Opencode, Kilo, Codex, Pi, Goose, Hermes, Cline, omp, and Freebuff, all in one pass. That last line is the root-cause analysis you wrote six months ago and would never have found again.
|
|
75
|
-
|
|
76
|
-
### 2. `ccopy` — move your project from one tool to another
|
|
77
|
-
|
|
78
|
-
Switched from opencode to Claude Code mid-project? You don't have to start from zero. Copy the whole conversation — context, decisions, the lot — into a fresh session in the other tool:
|
|
79
|
-
|
|
80
|
-
```shell
|
|
81
|
-
$ ccopy opencode/ses_abc123 claude-code
|
|
82
|
-
Copied 128 message(s) from opencode/ses_abc123 to a new claude-code session: 1a2b3c
|
|
83
|
-
Resume it with:
|
|
84
|
-
claude-code --resume 1a2b3c
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
Your work travels with you. The source is never touched, so it's a true copy. And when the destination is on another machine, it's just `ccopy opencode/ses_abc123 ssh://chris@remote/codex` — it pulls the agent's storage over your normal ssh (the remote only needs `sshd` and `tar`, not ctools).
|
|
88
|
-
|
|
89
|
-
ctools is a set of GNU-style tools for the history your coding agents leave behind. These two are the headline; the rest (`cdir` to list sessions, `ccat` to read them, `cextract` to move concepts, `cconnect` to pipe them live, `cdu` to count tokens, `crm` to prune) are in service of them.
|
|
90
|
-
|
|
91
|
-
**Works with the agents you already use:** Claude Desktop, Claude Code, Opencode, Kilo, Codex, Pi, Goose, Hermes, Cline, omp (oh-my-pi), and Freebuff (Codebuff).
|
|
92
|
-
|
|
93
60
|
## Tools
|
|
94
61
|
|
|
95
62
|
| Tool | What it does | Docs |
|
|
@@ -98,13 +65,21 @@ ctools is a set of GNU-style tools for the history your coding agents leave behi
|
|
|
98
65
|
| `cdir` | List sessions (ls for your history) | [docs/cdir.md](docs/cdir.md) |
|
|
99
66
|
| `ccat` | Print one or more conversations (cat for your history) | [docs/ccat.md](docs/ccat.md) |
|
|
100
67
|
| `ccopy` | Copy a whole conversation from one agent to a new session in another | [docs/ccopy.md](docs/ccopy.md) |
|
|
68
|
+
| `crecover` | List, verify, and restore the storage backups ccopy takes before each write | [docs/crecover.md](docs/crecover.md) |
|
|
101
69
|
| `cextract` | Extract concepts from a session into files, and inject them back | [docs/cextract.md](docs/cextract.md) |
|
|
102
70
|
| `cconnect` | Live concept pipelines between sessions | [docs/cconnect.md](docs/cconnect.md) |
|
|
103
71
|
| `cdu` | Token usage per session (du for context windows) | [docs/cdu.md](docs/cdu.md) |
|
|
104
72
|
| `crm` | Remove concepts from a session (mdel) | [docs/crm.md](docs/crm.md) |
|
|
73
|
+
| `ctools-web` | Browser dashboard for all of the above | [Web dashboard](#web-dashboard) |
|
|
105
74
|
|
|
106
75
|
Every tool takes `--version`. The session-mover tools (`ccopy`, `cextract`, `cconnect`, `crm`) also take `--verbose` for structured logging. Each tool's full flag reference, output formats, and examples live in its `docs/` page — the table above is just the 30-second version.
|
|
107
76
|
|
|
77
|
+
## The problem
|
|
78
|
+
|
|
79
|
+
You talk to LLMs all day. Over weeks, you build up a set of constraints, preferences, and goals. These live in your conversations as system messages. They are valuable. They are also trapped.
|
|
80
|
+
|
|
81
|
+
Say you have been working with opencode for a month. You have refined your coding style through dozens of sessions. Now you start a new Claude Code project and you want those same preferences. You could copy them by hand. Or you could `cextract` them out and `cextract` them back in, or copy the whole conversation across with `ccopy`. Either way, your memory travels with you.
|
|
82
|
+
|
|
108
83
|
## How it works
|
|
109
84
|
|
|
110
85
|
ctools is a substrate for moving memory between context windows. Cross-platform, agent-agnostic, designed like a bus.
|
|
@@ -114,6 +89,8 @@ Two stages: **extraction** and **filtering**.
|
|
|
114
89
|
- A **strategy** extracts concepts from a conversation. It defines what counts as a concept and how to find it: regex patterns, LLM extraction, whatever. Different strategies produce different ontologies from the same conversation, because context is contestable.
|
|
115
90
|
- A **filter** selects which extracted concepts reach a destination. It's a binary classifier: pass through or filter out. One concept list can fan out to many destinations, each with its own filter. A coding agent gets coding preferences, a security agent gets security constraints, a PM agent gets goals, all from the same source.
|
|
116
91
|
|
|
92
|
+
A filter can be as simple as a type or substring match, or as smart as a **decision model** — a small local classifier that scores each concept and lets it through only when confident. Point a filter at a model and you get *semantic* filtering with no free text to parse: ask "is this concept about security?" and gate on the answer. The model is asked either through Ollama's typed `/v1/systemone` endpoint (calibrated probabilities, the home of decision-dedicated models) or through any OpenAI-compatible `/v1/chat/completions` server (portable, lettered decision-task prompt) — one `endpoint` field decides. See [cextract](docs/cextract.md) for the decision-model and JSON-RPC filter configs.
|
|
93
|
+
|
|
117
94
|
```mermaid
|
|
118
95
|
graph LR
|
|
119
96
|
SRC["source session"] --> STRAT["strategy<br/>(extraction)"]
|
|
@@ -126,13 +103,33 @@ graph LR
|
|
|
126
103
|
FC --> DEST_C["destination C"]
|
|
127
104
|
```
|
|
128
105
|
|
|
129
|
-
Context windows are endpoints: opencode, Claude Code, Codex, Pi. They all speak different protocols, but they all consume the same packets. See [cextract](docs/cextract.md) for the concept packet format, strategies, and the filter (JSON-RPC)
|
|
106
|
+
Context windows are endpoints: opencode, Claude Code, Codex, Pi. They all speak different protocols, but they all consume the same packets. See [cextract](docs/cextract.md) for the concept packet format, strategies, and the filter configs (decision model and JSON-RPC).
|
|
130
107
|
|
|
131
|
-
##
|
|
108
|
+
## Write safety
|
|
132
109
|
|
|
133
|
-
|
|
110
|
+
The session-mover tools write into your agents' **live** storage — for the
|
|
111
|
+
SQLite-backed agents (opencode, goose, kilo, hermes) that's the database that
|
|
112
|
+
holds every session of that agent. A write that fails or is interrupted
|
|
113
|
+
part-way can corrupt it, and losing an agent's whole history would be
|
|
114
|
+
unforgivable.
|
|
134
115
|
|
|
135
|
-
|
|
116
|
+
So every local write is guarded:
|
|
117
|
+
|
|
118
|
+
1. ctools **snapshots the agent's storage first** (SQLite via `VACUUM INTO` —
|
|
119
|
+
a consistent copy even mid-write; file agents via a tree copy), kept under
|
|
120
|
+
`~/.local/share/ctools/backups/<agent>/` (20 newest per agent).
|
|
121
|
+
2. After the write it **verifies** the storage: the DB opens,
|
|
122
|
+
`PRAGMA integrity_check` passes, the session table still exists, and the
|
|
123
|
+
session count did not decrease. A database that is structurally valid but
|
|
124
|
+
emptied of data is caught too.
|
|
125
|
+
3. If anything looks wrong, the snapshot is **restored automatically** — you
|
|
126
|
+
get a clean error, not a broken database.
|
|
127
|
+
|
|
128
|
+
And because a restore is itself a write, it snapshots what it overwrites, so
|
|
129
|
+
**every restore is undoable**. For the out-of-band case (you notice days later
|
|
130
|
+
that an agent's history vanished), `crecover list` / `verify` / `restore` are
|
|
131
|
+
there. Details: [ccopy — safety](docs/ccopy.md#safety-automatic-backups) and
|
|
132
|
+
[crecover](docs/crecover.md).
|
|
136
133
|
|
|
137
134
|
## Supported endpoints
|
|
138
135
|
|
|
@@ -149,6 +146,12 @@ Say you have been working with opencode for a month. You have refined your codin
|
|
|
149
146
|
| cline | JSON (`~/.cline/data/tasks/`) |
|
|
150
147
|
| omp | JSONL (`~/.omp/agent/sessions/`, tree journal) |
|
|
151
148
|
| freebuff | JSON (`~/.config/<codebuff>/projects/`) |
|
|
149
|
+
| openclaw | SQLite (per-agent, `~/.openclaw/agents/`, `OPENCLAW_STATE_DIR`) |
|
|
150
|
+
|
|
151
|
+
OpenClaw is a **personal assistant** rather than a terminal coding agent, so it
|
|
152
|
+
runs in a local Gateway that owns its store; ctools reads and exports its
|
|
153
|
+
sessions but does not seed new ones (the store has canonical-index and integrity
|
|
154
|
+
checks that a raw insert cannot satisfy).
|
|
152
155
|
|
|
153
156
|
Run `cdir` to see which endpoints are found on your system and where they store data.
|
|
154
157
|
|
|
@@ -174,23 +177,87 @@ Anything the storage cannot do — pi's session trees cannot be rewritten in pla
|
|
|
174
177
|
|
|
175
178
|
## MCP server
|
|
176
179
|
|
|
177
|
-
There is an MCP server for use from Claude, opencode, Cursor, or anything else that speaks MCP.
|
|
180
|
+
There is an MCP server for use from pi, Claude, opencode, Cursor, or anything else that speaks MCP.
|
|
178
181
|
|
|
179
182
|
Tools: `list_agents`, `list_sessions`, `search_sessions`, `export_session`, `extract_concepts`, `copy_concepts`, `get_session_concepts`.
|
|
180
183
|
|
|
181
|
-
|
|
184
|
+
### Quick install (pi)
|
|
185
|
+
|
|
186
|
+
One command does the whole job — install into an isolated venv (if needed) and register with pi:
|
|
187
|
+
|
|
188
|
+
```sh
|
|
189
|
+
pipx run 'ctxttools[mcp]' ctools-mcp install
|
|
190
|
+
# or, with uv:
|
|
191
|
+
uv tool run 'ctxttools[mcp]' ctools-mcp install
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
It detects an existing `uv tool` or `pipx` install of `ctxttools` and uses it instead. Verify with:
|
|
195
|
+
|
|
196
|
+
```sh
|
|
197
|
+
pi mcp list # expect: ctools: connected, 7 tools
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
Then use it inside pi — run `/mcp` to inspect, or just ask the agent (the tools are also reachable through codemode / tool search).
|
|
201
|
+
|
|
202
|
+
Uninstall with `ctools-mcp uninstall` (removes the pi entry; pass `--all` to also remove Claude Code and opencode entries).
|
|
203
|
+
|
|
204
|
+
### Other clients
|
|
205
|
+
|
|
206
|
+
Run `ctools-mcp print-config` to get the JSON snippet for any MCP host, e.g. for opencode (`opencode.json` under `"mcp"`) or Claude (`.mcp.json`):
|
|
182
207
|
|
|
183
208
|
```json
|
|
184
209
|
{
|
|
185
210
|
"mcpServers": {
|
|
186
211
|
"ctools": {
|
|
187
212
|
"command": "python",
|
|
188
|
-
"args": ["
|
|
213
|
+
"args": ["-m", "ctools.ctools_mcp"]
|
|
189
214
|
}
|
|
190
215
|
}
|
|
191
216
|
}
|
|
192
217
|
```
|
|
193
218
|
|
|
219
|
+
(`command` should point at the interpreter of a venv that has `ctxttools[mcp]` installed — `ctools-mcp install` prints the exact one for your machine.)
|
|
220
|
+
|
|
221
|
+
## Web dashboard
|
|
222
|
+
|
|
223
|
+
Prefer a mouse to a man page? ctools ships a browser dashboard — a small local
|
|
224
|
+
web server (Python stdlib only, no web dependencies) with the same library
|
|
225
|
+
underneath. Same on macOS, Windows, and Linux:
|
|
226
|
+
|
|
227
|
+
```sh
|
|
228
|
+
$ ctools-web # installed command
|
|
229
|
+
# or, straight from a checkout, no install needed:
|
|
230
|
+
$ python3 ctools/ctools-web.py # or: python -m ctools.webui
|
|
231
|
+
ctools 0.2.1 dashboard: http://127.0.0.1:8765
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
Open the URL. The navbar has **Overview**, **Sessions**, and **Concepts**, plus
|
|
235
|
+
a search box (press <b>/</b>) and a power button to stop the server. Clicking a
|
|
236
|
+
session anywhere opens its **Conversation** view:
|
|
237
|
+
|
|
238
|
+
- **Overview** — every supported agent, installed or not, with session counts
|
|
239
|
+
and total tokens, and a jump into each agent's sessions.
|
|
240
|
+
- **Search** — `cgrep` in the browser: one box, searches session ids, names,
|
|
241
|
+
paths <b>and</b> the content of every conversation at once; click a match to
|
|
242
|
+
open that conversation.
|
|
243
|
+
- **Sessions** — `cdir` for one agent (or all of them): newest, largest, or by
|
|
244
|
+
token usage, filterable.
|
|
245
|
+
- **Conversation** — `ccat` with the session's messages, token usage, the
|
|
246
|
+
resume command, concept extraction (`cextract`), a markdown export, and
|
|
247
|
+
**Copy** — `ccopy` to any other installed agent, which creates a fresh
|
|
248
|
+
session there (source untouched) and tells you how to resume it.
|
|
249
|
+
|
|
250
|
+
The server binds to `127.0.0.1` by default, so nothing is exposed to your
|
|
251
|
+
network unless you pass `--host 0.0.0.0`. There is one write operation:
|
|
252
|
+
Copy. Everything else is read-only against the agents' storage.
|
|
253
|
+
|
|
254
|
+
Stop it with **Ctrl-C**, or with the **power button** (⏻) in the top-right of
|
|
255
|
+
the page — it asks for confirmation, then shuts the server down cleanly.
|
|
256
|
+
|
|
257
|
+
```sh
|
|
258
|
+
ctools-web --port 9000
|
|
259
|
+
```
|
|
260
|
+
|
|
194
261
|
## Installation
|
|
195
262
|
|
|
196
263
|
```sh
|
|
@@ -213,7 +280,7 @@ from ctools.lib import get_formatter
|
|
|
213
280
|
from ctools.cgrep import grep_session
|
|
214
281
|
from ctools.cextract import extract_concepts_from_messages
|
|
215
282
|
from ctools.cdu import count_tokens, get_session_tokens
|
|
216
|
-
from ctools.filterlib import JSONRPCFilter, load_filter
|
|
283
|
+
from ctools.filterlib import JSONRPCFilter, SystemOneFilter, load_filter
|
|
217
284
|
from ctools.log import configure_logging, get_logger
|
|
218
285
|
```
|
|
219
286
|
|
|
@@ -238,6 +305,13 @@ from ctools.agents import OpencodeAgent
|
|
|
238
305
|
agent = OpencodeAgent('/backup/opencode')
|
|
239
306
|
```
|
|
240
307
|
|
|
308
|
+
## Where to go next
|
|
309
|
+
|
|
310
|
+
- **Per-tool reference** — the [Tools](#tools) table above links to a `docs/` page for each command: every flag, output format, and a worked example.
|
|
311
|
+
- **The interchange format** — how a conversation becomes a portable, lossless envelope and back, and why 2N agents compose into N² transfers. See [interchange](docs/interchange.md).
|
|
312
|
+
- **Filtering with decision models** — the full decision-model and JSON-RPC filter configs, with live examples. See [cextract](docs/cextract.md#filters) and [cconnect](docs/cconnect.md#filter-configuration).
|
|
313
|
+
- **Adding an endpoint** — one `Agent` subclass plus a line in `AGENT_CLASSES`. See [Supported endpoints](#supported-endpoints).
|
|
314
|
+
|
|
241
315
|
## Credits
|
|
242
316
|
|
|
243
317
|
ctools was extracted from [Gab n' Go](https://github.com/day50-dev/gabngo). It's named after [GNU mtools](https://www.gnu.org/software/mtools/), which does the same thing for DOS floppies — because your context window is about the size of a DOS-floppy. The flag habits are GNU's too: `--version`, `-1`, `-S`, `-u`, and the rest behave the way you'd expect.
|
|
@@ -8,9 +8,9 @@
|
|
|
8
8
|
</p>
|
|
9
9
|
|
|
10
10
|
------
|
|
11
|
-
**ctools** is a suite of simple tools for navigating through the sessions
|
|
11
|
+
**ctools** is a suite of simple tools for navigating through the sessions of the most popular LLM agents and coding tools. It allows you to search your history, transfer conversations, filter them, and even share them among machines and users.
|
|
12
12
|
|
|
13
|
-
It supports [Claude Desktop](https://claude.com/download), [Claude Code](https://github.com/anthropics/claude-code), [Codex](https://github.com/openai/codex), [Pi](https://github.com/earendil-works/pi), [Hermes](https://github.com/NousResearch/hermes-agent), [Goose](https://github.com/aaif-goose/goose), [Kilo](https://github.com/Kilo-Org/kilocode), [
|
|
13
|
+
It supports [Claude Desktop](https://claude.com/download), [Claude Code](https://github.com/anthropics/claude-code), [Codex](https://github.com/openai/codex), [Pi](https://github.com/earendil-works/pi), [Hermes](https://github.com/NousResearch/hermes-agent), [Goose](https://github.com/aaif-goose/goose), [Kilo](https://github.com/Kilo-Org/kilocode), [FreeBuff](https://github.com/CodebuffAI/freebuff), [Cline](https://github.com/cline/cline), [omp](https://github.com/can1357/oh-my-pi), and [OpenClaw](https://openclaw.ai/).
|
|
14
14
|
|
|
15
15
|
Here's an example:
|
|
16
16
|
|
|
@@ -41,38 +41,6 @@ Resume it with:
|
|
|
41
41
|
|
|
42
42
|
As you can see, these are unix-friendly tools using simple paradigms that you are already familiar with.
|
|
43
43
|
|
|
44
|
-
----
|
|
45
|
-
|
|
46
|
-
### 1. `cgrep` — find it in your past conversations
|
|
47
|
-
|
|
48
|
-
You already know the answer is in a session you had months ago. You just don't remember which one. `cgrep` searches **every session you ever had, across every agent you use**, with a plain regex:
|
|
49
|
-
|
|
50
|
-
```shell
|
|
51
|
-
$ cgrep -i "ssl"
|
|
52
|
-
claude-code/a351eedf:142:assistant: … the TLS handshake fails with SSL wrong version number …
|
|
53
|
-
opencode/ses_000d460f:16:user: ok graflex has an erorr in the check. i see this: … [[SSL: WRONG_VERSION_NUMBER] …
|
|
54
|
-
opencode/ses_000d460f:34:assistant: Found it. Root cause: `_check_host` (graflex/__init__.py:128) tries `http`, and …
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
No agent argument? It searches **every installed agent at once** — Claude Code, Opencode, Kilo, Codex, Pi, Goose, Hermes, Cline, omp, and Freebuff, all in one pass. That last line is the root-cause analysis you wrote six months ago and would never have found again.
|
|
58
|
-
|
|
59
|
-
### 2. `ccopy` — move your project from one tool to another
|
|
60
|
-
|
|
61
|
-
Switched from opencode to Claude Code mid-project? You don't have to start from zero. Copy the whole conversation — context, decisions, the lot — into a fresh session in the other tool:
|
|
62
|
-
|
|
63
|
-
```shell
|
|
64
|
-
$ ccopy opencode/ses_abc123 claude-code
|
|
65
|
-
Copied 128 message(s) from opencode/ses_abc123 to a new claude-code session: 1a2b3c
|
|
66
|
-
Resume it with:
|
|
67
|
-
claude-code --resume 1a2b3c
|
|
68
|
-
```
|
|
69
|
-
|
|
70
|
-
Your work travels with you. The source is never touched, so it's a true copy. And when the destination is on another machine, it's just `ccopy opencode/ses_abc123 ssh://chris@remote/codex` — it pulls the agent's storage over your normal ssh (the remote only needs `sshd` and `tar`, not ctools).
|
|
71
|
-
|
|
72
|
-
ctools is a set of GNU-style tools for the history your coding agents leave behind. These two are the headline; the rest (`cdir` to list sessions, `ccat` to read them, `cextract` to move concepts, `cconnect` to pipe them live, `cdu` to count tokens, `crm` to prune) are in service of them.
|
|
73
|
-
|
|
74
|
-
**Works with the agents you already use:** Claude Desktop, Claude Code, Opencode, Kilo, Codex, Pi, Goose, Hermes, Cline, omp (oh-my-pi), and Freebuff (Codebuff).
|
|
75
|
-
|
|
76
44
|
## Tools
|
|
77
45
|
|
|
78
46
|
| Tool | What it does | Docs |
|
|
@@ -81,13 +49,21 @@ ctools is a set of GNU-style tools for the history your coding agents leave behi
|
|
|
81
49
|
| `cdir` | List sessions (ls for your history) | [docs/cdir.md](docs/cdir.md) |
|
|
82
50
|
| `ccat` | Print one or more conversations (cat for your history) | [docs/ccat.md](docs/ccat.md) |
|
|
83
51
|
| `ccopy` | Copy a whole conversation from one agent to a new session in another | [docs/ccopy.md](docs/ccopy.md) |
|
|
52
|
+
| `crecover` | List, verify, and restore the storage backups ccopy takes before each write | [docs/crecover.md](docs/crecover.md) |
|
|
84
53
|
| `cextract` | Extract concepts from a session into files, and inject them back | [docs/cextract.md](docs/cextract.md) |
|
|
85
54
|
| `cconnect` | Live concept pipelines between sessions | [docs/cconnect.md](docs/cconnect.md) |
|
|
86
55
|
| `cdu` | Token usage per session (du for context windows) | [docs/cdu.md](docs/cdu.md) |
|
|
87
56
|
| `crm` | Remove concepts from a session (mdel) | [docs/crm.md](docs/crm.md) |
|
|
57
|
+
| `ctools-web` | Browser dashboard for all of the above | [Web dashboard](#web-dashboard) |
|
|
88
58
|
|
|
89
59
|
Every tool takes `--version`. The session-mover tools (`ccopy`, `cextract`, `cconnect`, `crm`) also take `--verbose` for structured logging. Each tool's full flag reference, output formats, and examples live in its `docs/` page — the table above is just the 30-second version.
|
|
90
60
|
|
|
61
|
+
## The problem
|
|
62
|
+
|
|
63
|
+
You talk to LLMs all day. Over weeks, you build up a set of constraints, preferences, and goals. These live in your conversations as system messages. They are valuable. They are also trapped.
|
|
64
|
+
|
|
65
|
+
Say you have been working with opencode for a month. You have refined your coding style through dozens of sessions. Now you start a new Claude Code project and you want those same preferences. You could copy them by hand. Or you could `cextract` them out and `cextract` them back in, or copy the whole conversation across with `ccopy`. Either way, your memory travels with you.
|
|
66
|
+
|
|
91
67
|
## How it works
|
|
92
68
|
|
|
93
69
|
ctools is a substrate for moving memory between context windows. Cross-platform, agent-agnostic, designed like a bus.
|
|
@@ -97,6 +73,8 @@ Two stages: **extraction** and **filtering**.
|
|
|
97
73
|
- A **strategy** extracts concepts from a conversation. It defines what counts as a concept and how to find it: regex patterns, LLM extraction, whatever. Different strategies produce different ontologies from the same conversation, because context is contestable.
|
|
98
74
|
- A **filter** selects which extracted concepts reach a destination. It's a binary classifier: pass through or filter out. One concept list can fan out to many destinations, each with its own filter. A coding agent gets coding preferences, a security agent gets security constraints, a PM agent gets goals, all from the same source.
|
|
99
75
|
|
|
76
|
+
A filter can be as simple as a type or substring match, or as smart as a **decision model** — a small local classifier that scores each concept and lets it through only when confident. Point a filter at a model and you get *semantic* filtering with no free text to parse: ask "is this concept about security?" and gate on the answer. The model is asked either through Ollama's typed `/v1/systemone` endpoint (calibrated probabilities, the home of decision-dedicated models) or through any OpenAI-compatible `/v1/chat/completions` server (portable, lettered decision-task prompt) — one `endpoint` field decides. See [cextract](docs/cextract.md) for the decision-model and JSON-RPC filter configs.
|
|
77
|
+
|
|
100
78
|
```mermaid
|
|
101
79
|
graph LR
|
|
102
80
|
SRC["source session"] --> STRAT["strategy<br/>(extraction)"]
|
|
@@ -109,13 +87,33 @@ graph LR
|
|
|
109
87
|
FC --> DEST_C["destination C"]
|
|
110
88
|
```
|
|
111
89
|
|
|
112
|
-
Context windows are endpoints: opencode, Claude Code, Codex, Pi. They all speak different protocols, but they all consume the same packets. See [cextract](docs/cextract.md) for the concept packet format, strategies, and the filter (JSON-RPC)
|
|
90
|
+
Context windows are endpoints: opencode, Claude Code, Codex, Pi. They all speak different protocols, but they all consume the same packets. See [cextract](docs/cextract.md) for the concept packet format, strategies, and the filter configs (decision model and JSON-RPC).
|
|
113
91
|
|
|
114
|
-
##
|
|
92
|
+
## Write safety
|
|
115
93
|
|
|
116
|
-
|
|
94
|
+
The session-mover tools write into your agents' **live** storage — for the
|
|
95
|
+
SQLite-backed agents (opencode, goose, kilo, hermes) that's the database that
|
|
96
|
+
holds every session of that agent. A write that fails or is interrupted
|
|
97
|
+
part-way can corrupt it, and losing an agent's whole history would be
|
|
98
|
+
unforgivable.
|
|
117
99
|
|
|
118
|
-
|
|
100
|
+
So every local write is guarded:
|
|
101
|
+
|
|
102
|
+
1. ctools **snapshots the agent's storage first** (SQLite via `VACUUM INTO` —
|
|
103
|
+
a consistent copy even mid-write; file agents via a tree copy), kept under
|
|
104
|
+
`~/.local/share/ctools/backups/<agent>/` (20 newest per agent).
|
|
105
|
+
2. After the write it **verifies** the storage: the DB opens,
|
|
106
|
+
`PRAGMA integrity_check` passes, the session table still exists, and the
|
|
107
|
+
session count did not decrease. A database that is structurally valid but
|
|
108
|
+
emptied of data is caught too.
|
|
109
|
+
3. If anything looks wrong, the snapshot is **restored automatically** — you
|
|
110
|
+
get a clean error, not a broken database.
|
|
111
|
+
|
|
112
|
+
And because a restore is itself a write, it snapshots what it overwrites, so
|
|
113
|
+
**every restore is undoable**. For the out-of-band case (you notice days later
|
|
114
|
+
that an agent's history vanished), `crecover list` / `verify` / `restore` are
|
|
115
|
+
there. Details: [ccopy — safety](docs/ccopy.md#safety-automatic-backups) and
|
|
116
|
+
[crecover](docs/crecover.md).
|
|
119
117
|
|
|
120
118
|
## Supported endpoints
|
|
121
119
|
|
|
@@ -132,6 +130,12 @@ Say you have been working with opencode for a month. You have refined your codin
|
|
|
132
130
|
| cline | JSON (`~/.cline/data/tasks/`) |
|
|
133
131
|
| omp | JSONL (`~/.omp/agent/sessions/`, tree journal) |
|
|
134
132
|
| freebuff | JSON (`~/.config/<codebuff>/projects/`) |
|
|
133
|
+
| openclaw | SQLite (per-agent, `~/.openclaw/agents/`, `OPENCLAW_STATE_DIR`) |
|
|
134
|
+
|
|
135
|
+
OpenClaw is a **personal assistant** rather than a terminal coding agent, so it
|
|
136
|
+
runs in a local Gateway that owns its store; ctools reads and exports its
|
|
137
|
+
sessions but does not seed new ones (the store has canonical-index and integrity
|
|
138
|
+
checks that a raw insert cannot satisfy).
|
|
135
139
|
|
|
136
140
|
Run `cdir` to see which endpoints are found on your system and where they store data.
|
|
137
141
|
|
|
@@ -157,23 +161,87 @@ Anything the storage cannot do — pi's session trees cannot be rewritten in pla
|
|
|
157
161
|
|
|
158
162
|
## MCP server
|
|
159
163
|
|
|
160
|
-
There is an MCP server for use from Claude, opencode, Cursor, or anything else that speaks MCP.
|
|
164
|
+
There is an MCP server for use from pi, Claude, opencode, Cursor, or anything else that speaks MCP.
|
|
161
165
|
|
|
162
166
|
Tools: `list_agents`, `list_sessions`, `search_sessions`, `export_session`, `extract_concepts`, `copy_concepts`, `get_session_concepts`.
|
|
163
167
|
|
|
164
|
-
|
|
168
|
+
### Quick install (pi)
|
|
169
|
+
|
|
170
|
+
One command does the whole job — install into an isolated venv (if needed) and register with pi:
|
|
171
|
+
|
|
172
|
+
```sh
|
|
173
|
+
pipx run 'ctxttools[mcp]' ctools-mcp install
|
|
174
|
+
# or, with uv:
|
|
175
|
+
uv tool run 'ctxttools[mcp]' ctools-mcp install
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
It detects an existing `uv tool` or `pipx` install of `ctxttools` and uses it instead. Verify with:
|
|
179
|
+
|
|
180
|
+
```sh
|
|
181
|
+
pi mcp list # expect: ctools: connected, 7 tools
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
Then use it inside pi — run `/mcp` to inspect, or just ask the agent (the tools are also reachable through codemode / tool search).
|
|
185
|
+
|
|
186
|
+
Uninstall with `ctools-mcp uninstall` (removes the pi entry; pass `--all` to also remove Claude Code and opencode entries).
|
|
187
|
+
|
|
188
|
+
### Other clients
|
|
189
|
+
|
|
190
|
+
Run `ctools-mcp print-config` to get the JSON snippet for any MCP host, e.g. for opencode (`opencode.json` under `"mcp"`) or Claude (`.mcp.json`):
|
|
165
191
|
|
|
166
192
|
```json
|
|
167
193
|
{
|
|
168
194
|
"mcpServers": {
|
|
169
195
|
"ctools": {
|
|
170
196
|
"command": "python",
|
|
171
|
-
"args": ["
|
|
197
|
+
"args": ["-m", "ctools.ctools_mcp"]
|
|
172
198
|
}
|
|
173
199
|
}
|
|
174
200
|
}
|
|
175
201
|
```
|
|
176
202
|
|
|
203
|
+
(`command` should point at the interpreter of a venv that has `ctxttools[mcp]` installed — `ctools-mcp install` prints the exact one for your machine.)
|
|
204
|
+
|
|
205
|
+
## Web dashboard
|
|
206
|
+
|
|
207
|
+
Prefer a mouse to a man page? ctools ships a browser dashboard — a small local
|
|
208
|
+
web server (Python stdlib only, no web dependencies) with the same library
|
|
209
|
+
underneath. Same on macOS, Windows, and Linux:
|
|
210
|
+
|
|
211
|
+
```sh
|
|
212
|
+
$ ctools-web # installed command
|
|
213
|
+
# or, straight from a checkout, no install needed:
|
|
214
|
+
$ python3 ctools/ctools-web.py # or: python -m ctools.webui
|
|
215
|
+
ctools 0.2.1 dashboard: http://127.0.0.1:8765
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
Open the URL. The navbar has **Overview**, **Sessions**, and **Concepts**, plus
|
|
219
|
+
a search box (press <b>/</b>) and a power button to stop the server. Clicking a
|
|
220
|
+
session anywhere opens its **Conversation** view:
|
|
221
|
+
|
|
222
|
+
- **Overview** — every supported agent, installed or not, with session counts
|
|
223
|
+
and total tokens, and a jump into each agent's sessions.
|
|
224
|
+
- **Search** — `cgrep` in the browser: one box, searches session ids, names,
|
|
225
|
+
paths <b>and</b> the content of every conversation at once; click a match to
|
|
226
|
+
open that conversation.
|
|
227
|
+
- **Sessions** — `cdir` for one agent (or all of them): newest, largest, or by
|
|
228
|
+
token usage, filterable.
|
|
229
|
+
- **Conversation** — `ccat` with the session's messages, token usage, the
|
|
230
|
+
resume command, concept extraction (`cextract`), a markdown export, and
|
|
231
|
+
**Copy** — `ccopy` to any other installed agent, which creates a fresh
|
|
232
|
+
session there (source untouched) and tells you how to resume it.
|
|
233
|
+
|
|
234
|
+
The server binds to `127.0.0.1` by default, so nothing is exposed to your
|
|
235
|
+
network unless you pass `--host 0.0.0.0`. There is one write operation:
|
|
236
|
+
Copy. Everything else is read-only against the agents' storage.
|
|
237
|
+
|
|
238
|
+
Stop it with **Ctrl-C**, or with the **power button** (⏻) in the top-right of
|
|
239
|
+
the page — it asks for confirmation, then shuts the server down cleanly.
|
|
240
|
+
|
|
241
|
+
```sh
|
|
242
|
+
ctools-web --port 9000
|
|
243
|
+
```
|
|
244
|
+
|
|
177
245
|
## Installation
|
|
178
246
|
|
|
179
247
|
```sh
|
|
@@ -196,7 +264,7 @@ from ctools.lib import get_formatter
|
|
|
196
264
|
from ctools.cgrep import grep_session
|
|
197
265
|
from ctools.cextract import extract_concepts_from_messages
|
|
198
266
|
from ctools.cdu import count_tokens, get_session_tokens
|
|
199
|
-
from ctools.filterlib import JSONRPCFilter, load_filter
|
|
267
|
+
from ctools.filterlib import JSONRPCFilter, SystemOneFilter, load_filter
|
|
200
268
|
from ctools.log import configure_logging, get_logger
|
|
201
269
|
```
|
|
202
270
|
|
|
@@ -221,6 +289,13 @@ from ctools.agents import OpencodeAgent
|
|
|
221
289
|
agent = OpencodeAgent('/backup/opencode')
|
|
222
290
|
```
|
|
223
291
|
|
|
292
|
+
## Where to go next
|
|
293
|
+
|
|
294
|
+
- **Per-tool reference** — the [Tools](#tools) table above links to a `docs/` page for each command: every flag, output format, and a worked example.
|
|
295
|
+
- **The interchange format** — how a conversation becomes a portable, lossless envelope and back, and why 2N agents compose into N² transfers. See [interchange](docs/interchange.md).
|
|
296
|
+
- **Filtering with decision models** — the full decision-model and JSON-RPC filter configs, with live examples. See [cextract](docs/cextract.md#filters) and [cconnect](docs/cconnect.md#filter-configuration).
|
|
297
|
+
- **Adding an endpoint** — one `Agent` subclass plus a line in `AGENT_CLASSES`. See [Supported endpoints](#supported-endpoints).
|
|
298
|
+
|
|
224
299
|
## Credits
|
|
225
300
|
|
|
226
301
|
ctools was extracted from [Gab n' Go](https://github.com/day50-dev/gabngo). It's named after [GNU mtools](https://www.gnu.org/software/mtools/), which does the same thing for DOS floppies — because your context window is about the size of a DOS-floppy. The flag habits are GNU's too: `--version`, `-1`, `-S`, `-u`, and the rest behave the way you'd expect.
|