opencode-jev-compaction 0.1.1 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. package/NOTICE +26 -19
  2. package/README.md +101 -88
  3. package/package.json +6 -4
  4. package/scripts/laya-server.py +113 -0
  5. package/scripts/report.mjs +332 -0
  6. package/src/server.ts +365 -428
package/NOTICE CHANGED
@@ -1,25 +1,32 @@
1
1
  # Attribution
2
2
 
3
- The compaction strategy implemented here is adapted from
4
- [fast-jev-compaction](https://github.com/tamaratran/fast-jev-compaction),
5
- which is MIT licensed, Copyright (c) 2025.
3
+ The original strategy in this project (v0.1) was adapted from
4
+ [fast-jev-compaction](https://github.com/tamaratran/fast-jev-compaction), which is MIT
5
+ licensed, Copyright (c) 2025. Specifically derived from it:
6
6
 
7
- Specifically derived from that project:
8
-
9
- - the graded two-question decision protocol (`keepCall` / `keepResult` as
10
- `noul` probabilities) and the keep / truncate-result / drop-call actions
11
- - the staged state-fitting ladder and its constants (tool inputs truncated to
12
- 1000 / 200 / 60 characters, `TEXT_HEAD` 400, `TEXT_TAIL` 150)
13
- - the calibrated token estimator (a word per six letters, half a token per
14
- digit, 0.9 per other symbol)
7
+ - the idea of replacing a lossy compaction summary with per-tool-call keep / truncate /
8
+ drop decisions, so everything kept stays verbatim
9
+ - the calibrated token estimator (a word per six letters, half a token per digit, 0.9 per
10
+ other symbol)
15
11
  - the pinning rule (first message plus the newest N)
16
- - the state context wording and the batching-under-a-token-budget approach
12
+ - the Jev `system_one` request/response shape (`answers[name].noul`)
13
+ - the batching-under-a-token-budget approach, in the early versions
14
+
15
+ ## Divergence in v0.3
16
+
17
+ v0.3 keeps the estimator, the pinning rule and the protocol, and replaces the decision core
18
+ entirely:
19
+
20
+ - decisions are made from facts computed exactly (supersession, exact-string reference,
21
+ error resolution) rather than from model judgements
22
+ - the model is an optional local refinement (Laya, via MLX), used only for the residual
23
+ question, and it can only ever justify a truncation
24
+ - deletion requires deterministic evidence; nothing is deleted on a probabilistic answer
25
+ - the payload per question is a few hundred tokens rather than a 25k-token wholesale state
17
26
 
18
- This project is an independent reimplementation for
19
- [opencode](https://opencode.ai), not a fork. It targets opencode's part model,
20
- where a single `tool` part carries both the call and its result, so the
21
- original's orphaned-result invariant is unnecessary. It also runs before every
22
- model request rather than only at a compaction boundary, never throws (any
23
- failure leaves the messages untouched), and adds a daily request ceiling.
27
+ The finding that drove this is recorded in the README: with a `noul` primitive, factual
28
+ questions are reliable and judgement questions are not.
24
29
 
25
- All credit for the underlying idea and the decision protocol belongs upstream.
30
+ For the open-source model this now defaults to: Laya by Convai Innovations
31
+ (https://huggingface.co/convaiinnovations/laya), run through the MLX runtime
32
+ (https://github.com/mizorewww/laya-mlx). Both are third-party projects, unmodified.
package/README.md CHANGED
@@ -1,26 +1,44 @@
1
1
  # opencode-jev-compaction
2
2
 
3
- Two [opencode](https://opencode.ai) plugins that replace lossy compaction with
4
- decisions: ask a fast model which tool calls and results are still needed, drop
5
- or truncate the ones that aren't, and leave every user and assistant message
6
- verbatim.
3
+ Two [opencode](https://opencode.ai) plugins that shrink context by **deleting what is
4
+ provably stale and truncating what is probably done with** — never by summarizing.
7
5
 
8
- - **`./server`** — the pruner. Runs before every model request, and adds a note
9
- to the compaction prompt so shortened results aren't mistaken for failures.
10
- - **`./tui`** — a sidebar widget showing how much context the pruner has removed.
6
+ - **`./server`** — the pruner, a server plugin that runs before every model request.
7
+ - **`./tui`** — a sidebar widget showing how much context it removed.
11
8
 
12
- Strategy adapted from [fast-jev-compaction](https://github.com/tamaratran/fast-jev-compaction) (MIT). See [NOTICE](./NOTICE).
9
+ Runs entirely locally by default. No API key, no per-request cost.
13
10
 
14
- ## Why
11
+ ## Why this looks the way it does
15
12
 
16
- When a context window fills, the usual answer is to summarize old turns. A
17
- summary is lossy: a file path, an exact error, or a constraint can vanish even
18
- when it matters later. This never rewrites anything. It only removes what a
19
- model says is no longer needed, and everything kept stays byte-for-byte.
13
+ v0.1 asked a hosted model a **judgement** per tool call ("should this still be in the
14
+ history?"). That failed in a specific, instructive way: it deleted a short file of hard
15
+ constraints, and the scores were mushy. Two independent measurements agreed on the cause —
16
+ with a `noul`-style primitive (calibrated P(true)), **factual questions are reliable and
17
+ judgement questions are not** (0.996 on an explicit fact versus 0.003–0.28 on judgements).
20
18
 
21
- opencode already has a pruner, but its decision is purely recency and size — it
22
- keeps a fixed window of recent tool output and erases the rest. This one decides
23
- by relevance.
19
+ It was also expensive. A 25k-token state resent on every request, roughly 1,000 times a
20
+ day, cost about **$1/day** — against a saving measured at $0.0001 on a model whose cached
21
+ input is $0.003/M. The economics were upside down.
22
+
23
+ v0.3 asks **only facts**, computes the ones it can exactly, and treats the model as a
24
+ narrow refinement rather than the decision-maker.
25
+
26
+ ## How it decides
27
+
28
+ | reason | how | action |
29
+ | --- | --- | --- |
30
+ | `referenced` | the target string (path, command) appears in later **prose** — exact search | keep |
31
+ | `superseded` | a later call with the same tool and target — exact | **drop** |
32
+ | `error-resolved` | this call errored, a later call to the same target succeeded — exact | **drop** |
33
+ | `small-result` | under `SMALL_RESULT_CHARS`, not worth touching | keep |
34
+ | `model-unreferenced` | large, unmentioned, not superseded; local model says nothing quotes it | truncate |
35
+ | `model-referenced` | as above, but the model says something does | keep |
36
+ | `inconclusive` | the question could not be answered (no backend, timeout, low confidence) | truncate |
37
+
38
+ **Deletion requires deterministic evidence.** A model answer can only ever cause a
39
+ *truncation*, which keeps a bounded head plus a `[laya-compaction truncated …; re-run the
40
+ tool if needed]` note, so the model can recover by re-running. Nothing is ever deleted on a
41
+ probabilistic answer.
24
42
 
25
43
  ## Install
26
44
 
@@ -28,91 +46,86 @@ by relevance.
28
46
  opencode plugin opencode-jev-compaction --global
29
47
  ```
30
48
 
31
- That detects both the `./server` and `./tui` entrypoints and writes each to the
32
- right config (`opencode.json` for the server plugin, `tui.json` for the widget).
33
- Restart opencode afterwards.
34
-
35
- From a checkout instead:
49
+ Then run the local backend:
36
50
 
37
51
  ```sh
38
- opencode plugin github:JLegends/opencode-jev-compaction --global
52
+ # once
53
+ uv venv -p 3.12 ~/laya-server/.venv
54
+ uv pip install -p ~/laya-server/.venv laya-mlx
55
+
56
+ # run (first start downloads a few hundred MB of weights)
57
+ ~/laya-server/.venv/bin/python ~/opencode-jev-compaction/scripts/laya-server.py
39
58
  ```
40
59
 
60
+ `laya-server.py` is a thin transport: Laya already returns the Jev response shape, so it
61
+ exists only so the plugin can speak HTTP to a local process. It binds `127.0.0.1:8000` and
62
+ serializes inference (MLX is not reliably reentrant).
63
+
64
+ To keep it running across logins, wrap that command in a launchd agent or run it under
65
+ `tmux`. Startup takes a few seconds plus the one-time download.
66
+
41
67
  ## Configure
42
68
 
43
- The key comes from the environment, or from the macOS Keychain if you point it at
44
- one:
69
+ | Variable | Default | Purpose |
70
+ | --- | --- | --- |
71
+ | `LAYA_BASE_URL` | `http://127.0.0.1:8000/v1/systemone` | Backend endpoint. A hosted Jev endpoint works too. |
72
+ | `LAYA_COMPACTION` | on | `0` disables everything. |
73
+ | `LAYA_COMPACTION_THRESHOLD` | `60000` | Estimated context tokens before it engages. |
74
+ | `LAYA_PRESERVE_RECENT` | `6` | Newest messages never touched, minimum 1. |
75
+ | `LAYA_SMALL_RESULT_CHARS` | `600` | Results this size or smaller are left alone. |
76
+ | `LAYA_TRUNCATE_HEAD` | `300` | Characters kept when a result is truncated. |
77
+ | `LAYA_EXCERPT_CHARS` / `LAYA_AFTER_CHARS` | `400` / `1000` | What the model sees. Keep these small: Laya's sequence budget is 512 tokens. |
78
+ | `LAYA_REFERENCED_HIGH` | `0.7` | Probability of "quotes" needed to keep. |
79
+ | `LAYA_TIMEOUT_MS` / `LAYA_CONCURRENCY` | `8000` / `4` | Per-question timeout, parallel questions. |
80
+ | `LAYA_MAX_QUESTIONS` | `40` | Cap on model questions per prune. |
81
+ | `LAYA_DAILY_REQUEST_CAP` | `400` | Requests per day, per process. |
82
+ | `LAYA_DEBUG` | off | `1` appends a trace to `~/.local/share/opencode/laya-compaction.log`. |
83
+
84
+ ## Degraded mode
85
+
86
+ If the backend is unreachable, times out, or answers below `LAYA_REFERENCED_HIGH`, the
87
+ `model-*` rows simply do not apply: the deterministic reasons still fire and every residual
88
+ becomes `inconclusive` → truncate. The plugin is fully functional without any model, and
89
+ `LAYA_COMPACTION=0` turns it off entirely.
90
+
91
+ ## Metrics and reporting
92
+
93
+ `~/.local/share/opencode/laya-compaction.json` (totals), `-ledger.jsonl` (one line per run
94
+ that changed something, with the reason breakdown and re-run counts), `-usage.json`.
45
95
 
46
96
  ```sh
47
- export TYPESAFE_API_KEY=... # or:
48
- export JEV_KEYCHAIN_SERVICE=... # keychain service name
49
- export JEV_KEYCHAIN_ACCOUNT=... # keychain account name
97
+ npm run report
50
98
  ```
51
99
 
52
- | Variable | Default | Purpose |
53
- | --- | --- | --- |
54
- | `TYPESAFE_API_KEY` | — | API key. Required unless the keychain is configured. |
55
- | `JEV_KEYCHAIN_SERVICE` / `JEV_KEYCHAIN_ACCOUNT` | — | Read the key from the macOS Keychain instead of the environment. |
56
- | `JEV_COMPACTION` | on | `0` disables everything. |
57
- | `JEV_COMPACTION_THRESHOLD` | `60000` | Estimated tokens before it engages. Below this it does nothing and costs nothing. |
58
- | `JEV_KEEP_THRESHOLD` | `0.35` | Minimum probability for a call or result to be kept. Lower keeps more; a call below it is deleted outright, which is irreversible, so this is deliberately conservative. |
59
- | `JEV_PRESERVE_RECENT` | `6` | Newest messages never touched. Values below `1` are clamped to `1`; setting it to `0` drops the results the model is actively using and causes re-run loops. |
60
- | `JEV_MAX_STATE_TOKENS` | `25000` | Ceiling for the state sent to Jev. |
61
- | `JEV_MAX_REQUEST_TOKENS` | `30000` | Ceiling for state plus one batch of questions. |
62
- | `JEV_TRUNCATE_HEAD` | `300` | Characters of a dropped result kept before its note. |
63
- | `JEV_SMALL_RESULT_CHARS` | `600` | Results at or below this size are shown to Jev in full instead of as a note. |
64
- | `JEV_TIMEOUT_MS` | `20000` | Per-request timeout. Failures are skipped silently. |
65
- | `JEV_DAILY_REQUEST_CAP` | `200` | Hard ceiling on Jev requests per day. |
66
- | `JEV_MODEL` | `jev-latest` | Model name. |
67
- | `JEV_BASE_URL` | System One endpoint | Override the endpoint. |
68
- | `JEV_DEBUG` | off | `1` appends a trace to `~/.local/share/opencode/jev-compaction.log`. |
69
-
70
- ## Cost
71
-
72
- Jev is priced per input token with free output. At the default 25k state ceiling
73
- and the 200-request daily cap, worst-case spend is about **$0.21/day**, and it
74
- cannot exceed that. It also removes input tokens from every subsequent request,
75
- which is the point.
76
-
77
- Set `JEV_DAILY_REQUEST_CAP` lower if you want a tighter bound.
78
-
79
- ## How it works
80
-
81
- 1. Every finished `tool` part is a candidate, except those in the first message
82
- or the newest `JEV_PRESERVE_RECENT` messages, which are pinned.
83
- 2. The whole conversation is sent as state, oldest first, with tool outputs
84
- replaced by a short note (`ok, 4213 chars (omitted)`). Tool inputs and all
85
- text are included. The state is shrunk in stages until it fits
86
- `JEV_MAX_STATE_TOKENS`: inputs truncated to 1000, then 200, then 60
87
- characters; long texts abridged head and tail; old messages collapsed;
88
- old calls reduced to one line each. If it still doesn't fit, the run is
89
- skipped.
90
- 3. Jev answers two graded questions per call: should the **call** stay, and
91
- should the **result** stay verbatim. Questions are split into as many
92
- requests as needed so state plus questions fits `JEV_MAX_REQUEST_TOKENS`, and
93
- those requests run concurrently.
94
- 4. `keepResult >= threshold` keeps both. Otherwise `keepCall >= threshold` keeps
95
- the call and truncates the result to its first `JEV_TRUNCATE_HEAD`
96
- characters. Otherwise the call and its result go.
97
- 5. Decisions are cached per call for the life of the process and are monotonic:
98
- once dropped, always dropped.
99
-
100
- Nothing here throws. A missing key, a timeout, a malformed answer, or a history
101
- too large to fit leaves the messages exactly as they were, so a Jev outage can
102
- slow nothing down and break nothing.
103
-
104
- ## Requirements
105
-
106
- - opencode `>= 1.18.31`
107
- - A TypeSafe API key with access to Jev
100
+ Reports engagement, the reason breakdown, whether decisions are good (re-run rate),
101
+ pruned vs unpruned sessions, before/after the install boundary, and the subagent cost share
102
+ measured directly. It refuses to print a quality verdict when the installed version does
103
+ not record re-runs, so a `0` cannot be misread as "nothing was undone".
104
+
105
+ ## Measured caveats
106
+
107
+ Recorded here because they are the reason the design is conservative:
108
+
109
+ - **`noul` cannot answer a statement about text.** Against this same local model, a
110
+ statement and its own negation both scored ~0.95. As a two-option `choice` with explicit
111
+ criteria, the same cases separate cleanly (0.75–0.99 on a real quote, 0.80–0.91 on
112
+ unrelated text). That is why the code uses `choice` and says not to change it back.
113
+ - **The model sees only about 1000 characters of what came after** — Laya's sequence budget
114
+ is 512 tokens. Remote references are found by the exact string search, not by the model.
115
+ - **Accuracy is model- and task-specific.** Independent comparison found the hosted Jev
116
+ model ahead of open-weight Laya on ambiguous inputs (78% vs 57% on 40 tickets), and
117
+ confidently wrong on a multi-intent case. That is a small sample: treat it as directional.
118
+ - **Correlation, not causation, in the cohort tables.** A session is only pruned once it is
119
+ large, so the pruned cohort is longer by construction.
108
120
 
109
121
  ## Not affiliated
110
122
 
111
- Not built by, endorsed by, or affiliated with the opencode team or TypeSafe.
112
- "opencode", "Jev", and "TypeSafe" are used only to describe what this plugs into.
123
+ Not built by, endorsed by, or affiliated with the opencode team, TypeSafe, or Convai
124
+ Innovations. Names are used only to describe what this plugs into.
113
125
 
114
126
  ## License
115
127
 
116
- MIT. The compaction strategy is adapted from
117
- [fast-jev-compaction](https://github.com/tamaratran/fast-jev-compaction) (MIT) —
118
- see [NOTICE](./NOTICE).
128
+ MIT. The original strategy was adapted from
129
+ [fast-jev-compaction](https://github.com/tamaratran/fast-jev-compaction) (MIT) — see
130
+ [NOTICE](./NOTICE). v0.3 diverges from it substantially: the decision core is now
131
+ deterministic, and the model is an optional local refinement.
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "opencode-jev-compaction",
3
- "version": "0.1.1",
4
- "description": "opencode plugins that replace lossy compaction with Jev decisions: score every tool call and result, drop or truncate the stale ones, keep everything else verbatim.",
3
+ "version": "0.3.0",
4
+ "description": "opencode plugins that shrink context by deleting provably-stale tool calls and truncating the rest, deterministically first, with an optional local Laya backend.",
5
5
  "type": "module",
6
6
  "license": "MIT",
7
7
  "author": "JLegends",
@@ -25,7 +25,8 @@
25
25
  "src",
26
26
  "README.md",
27
27
  "LICENSE",
28
- "NOTICE"
28
+ "NOTICE",
29
+ "scripts"
29
30
  ],
30
31
  "keywords": [
31
32
  "opencode",
@@ -53,6 +54,7 @@
53
54
  }
54
55
  },
55
56
  "scripts": {
56
- "check": "node --check src/tui.js && bun build src/server.ts --target node --outfile .check.js && rm -f .check.js"
57
+ "check": "node --check src/tui.js && bun build src/server.ts --target node --outfile .check.js && rm -f .check.js",
58
+ "report": "node scripts/report.mjs"
57
59
  }
58
60
  }
package/scripts/laya-server.py ADDED
@@ -0,0 +1,113 @@
1
+ #!/usr/bin/env python3
2
+ """A TypeSafe-Jev-compatible HTTP server backed by local Laya (MLX, Apple Silicon).
3
+
4
+ Laya is the open-weight System-1 decision model, and its response shape is already the
5
+ Jev one (`answers[name].noul`), so this is a thin transport: it exists only so the
6
+ opencode plugin can speak HTTP to a local process instead of a paid API.
7
+
8
+ python laya-server.py # 127.0.0.1:8000, multilingual checkpoint
9
+ LAYA_PORT=8010 python laya-server.py
10
+ LAYA_SUBFOLDER=typed-decisions python laya-server.py
11
+
12
+ Endpoints:
13
+ POST /v1/systemone {model, state, questions} -> {model, answers, usage}
14
+ GET /v1/models [{id, object}]
15
+
16
+ First start downloads the checkpoint (a few hundred MB) into the Hugging Face cache.
17
+
18
+ Requirements: Python 3.11+, Apple Silicon, `pip install laya-mlx`.
19
+ """
20
+
21
+ import json
22
+ import os
23
+ import sys
24
+ import threading
25
+ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
26
+
27
+ HOST = os.environ.get("LAYA_HOST", "127.0.0.1")
28
+ PORT = int(os.environ.get("LAYA_PORT", "8000"))
29
+ REPO = os.environ.get("LAYA_REPO", "convaiinnovations/laya")
30
+ SUBFOLDER = os.environ.get("LAYA_SUBFOLDER", "multilingual")
31
+
32
+ _lock = threading.Lock()
33
+ _agent = None
34
+
35
+
36
+ def load():
37
+ global _agent
38
+ import laya_mlx as laya
39
+
40
+ print(f"[laya] loading {REPO} (subfolder={SUBFOLDER}) ...", flush=True)
41
+ # load() is called without a subfolder when it would be redundant, so a locally
42
+ # exported checkpoint can be pointed at with LAYA_REPO=/path/to/model.
43
+ try:
44
+ _agent = laya.load(REPO, subfolder=SUBFOLDER)
45
+ except TypeError:
46
+ _agent = laya.load(REPO)
47
+ print("[laya] ready", flush=True)
48
+
49
+
50
+ class Handler(BaseHTTPRequestHandler):
51
+ protocol_version = "HTTP/1.1"
52
+
53
+ def log_message(self, format, *args): # keep the console readable
54
+ return
55
+
56
+ def _send(self, status, payload):
57
+ body = json.dumps(payload).encode()
58
+ self.send_response(status)
59
+ self.send_header("content-type", "application/json")
60
+ self.send_header("content-length", str(len(body)))
61
+ self.end_headers()
62
+ self.wfile.write(body)
63
+
64
+ def do_GET(self):
65
+ if self.path.rstrip("/") in ("/v1/models", "/models"):
66
+ return self._send(200, {"object": "list", "data": [{"id": "laya", "object": "model"}]})
67
+ return self._send(404, {"error": "not found"})
68
+
69
+ def do_POST(self):
70
+ if self.path.rstrip("/") not in ("/v1/systemone", "/systemone"):
71
+ return self._send(404, {"error": "not found"})
72
+ try:
73
+ length = int(self.headers.get("content-length") or 0)
74
+ body = json.loads(self.rfile.read(length) or b"{}")
75
+ except Exception as error:
76
+ return self._send(400, {"error": f"bad request: {error}"})
77
+
78
+ state = body.get("state")
79
+ questions = body.get("questions")
80
+ if not isinstance(questions, dict) or not questions:
81
+ return self._send(400, {"error": "questions must be a non-empty object"})
82
+
83
+ try:
84
+ # One model instance, one inference at a time: MLX is not reliably reentrant.
85
+ with _lock:
86
+ result = _agent.system_one(state, questions)
87
+ except Exception as error:
88
+ return self._send(500, {"error": f"{type(error).__name__}: {error}"})
89
+
90
+ return self._send(200, result)
91
+
92
+
93
+ def main():
94
+ try:
95
+ load()
96
+ except ImportError:
97
+ print("[laya] laya-mlx is not installed. pip install laya-mlx", file=sys.stderr)
98
+ sys.exit(2)
99
+ except Exception as error:
100
+ print(f"[laya] failed to load: {type(error).__name__}: {error}", file=sys.stderr)
101
+ sys.exit(2)
102
+
103
+ server = ThreadingHTTPServer((HOST, PORT), Handler)
104
+ print(f"[laya] listening on http://{HOST}:{PORT}/v1/systemone", flush=True)
105
+ try:
106
+ server.serve_forever()
107
+ except KeyboardInterrupt:
108
+ print("\n[laya] stopping", flush=True)
109
+ server.server_close()
110
+
111
+
112
+ if __name__ == "__main__":
113
+ main()