opencode-jev-compaction 0.2.0 → 0.3.0

package/NOTICE

@@ -1,25 +1,32 @@
 # Attribution
 
-The compaction strategy implemented here is adapted from
-[fast-jev-compaction](https://github.com/tamaratran/fast-jev-compaction),
-which is MIT licensed, Copyright (c) 2025.
+The original strategy in this project (v0.1) was adapted from
+[fast-jev-compaction](https://github.com/tamaratran/fast-jev-compaction), which is MIT
+licensed, Copyright (c) 2025. Specifically derived from it:
 
-Specifically derived from that project:
-
-- the graded two-question decision protocol (`keepCall` / `keepResult` as
-  `noul` probabilities) and the keep / truncate-result / drop-call actions
-- the staged state-fitting ladder and its constants (tool inputs truncated to
-  1000 / 200 / 60 characters, `TEXT_HEAD` 400, `TEXT_TAIL` 150)
-- the calibrated token estimator (a word per six letters, half a token per
-  digit, 0.9 per other symbol)
+- the idea of replacing a lossy compaction summary with per-tool-call keep / truncate /
+  drop decisions, so everything kept stays verbatim
+- the calibrated token estimator (a word per six letters, half a token per digit, 0.9 per
+  other symbol)
 - the pinning rule (first message plus the newest N)
-- the state context wording and the batching-under-a-token-budget approach
+- the Jev `system_one` request/response shape (`answers[name].noul`)
+- the batching-under-a-token-budget approach, in the early versions
+
+## Divergence in v0.3
+
+v0.3 keeps the estimator, the pinning rule and the protocol, and replaces the decision core
+entirely:
+
+- decisions are made from facts computed exactly (supersession, exact-string reference,
+  error resolution) rather than from model judgements
+- the model is an optional local refinement (Laya, via MLX), used only for the residual
+  question, and it can only ever justify a truncation
+- deletion requires deterministic evidence; nothing is deleted on a probabilistic answer
+- the payload per question is a few hundred tokens rather than a 25k-token wholesale state
 
-This project is an independent reimplementation for
-[opencode](https://opencode.ai), not a fork. It targets opencode's part model,
-where a single `tool` part carries both the call and its result, so the
-original's orphaned-result invariant is unnecessary. It also runs before every
-model request rather than only at a compaction boundary, never throws (any
-failure leaves the messages untouched), and adds a daily request ceiling.
+The finding that drove this is recorded in the README: with a `noul` primitive, factual
+questions are reliable and judgement questions are not.
 
-All credit for the underlying idea and the decision protocol belongs upstream.
+For the open-source model this now defaults to: Laya by Convai Innovations
+(https://huggingface.co/convaiinnovations/laya), run through the MLX runtime
+(https://github.com/mizorewww/laya-mlx). Both are third-party projects, unmodified.

package/README.md

@@ -1,26 +1,44 @@
 # opencode-jev-compaction
 
-Two [opencode](https://opencode.ai) plugins that replace lossy compaction with
-decisions: ask a fast model which tool calls and results are still needed, drop
-or truncate the ones that aren't, and leave every user and assistant message
-verbatim.
+Two [opencode](https://opencode.ai) plugins that shrink context by **deleting what is
+provably stale and truncating what is probably done with** — never by summarizing.
 
-- **`./server`** — the pruner. Runs before every model request, and adds a note
-  to the compaction prompt so shortened results aren't mistaken for failures.
-- **`./tui`** — a sidebar widget showing how much context the pruner has removed.
+- **`./server`** — the pruner, a server plugin that runs before every model request.
+- **`./tui`** — a sidebar widget showing how much context it removed.
 
-Strategy adapted from [fast-jev-compaction](https://github.com/tamaratran/fast-jev-compaction) (MIT). See [NOTICE](./NOTICE).
+Runs entirely locally by default. No API key, no per-request cost.
 
-## Why
+## Why this looks the way it does
 
-When a context window fills, the usual answer is to summarize old turns. A
-summary is lossy: a file path, an exact error, or a constraint can vanish even
-when it matters later. This never rewrites anything. It only removes what a
-model says is no longer needed, and everything kept stays byte-for-byte.
+v0.1 asked a hosted model a **judgement** per tool call ("should this still be in the
+history?"). That failed in a specific, instructive way: it deleted a short file of hard
+constraints, and the scores were mushy. Two independent measurements agreed on the cause —
+with a `noul`-style primitive (calibrated P(true)), **factual questions are reliable and
+judgement questions are not** (0.996 on an explicit fact versus 0.003–0.28 on judgements).
 
-opencode already has a pruner, but its decision is purely recency and size — it
-keeps a fixed window of recent tool output and erases the rest. This one decides
-by relevance.
+It was also expensive. A 25k-token state resent on every request, roughly 1,000 times a
+day, cost about **$1/day** — against a saving measured at $0.0001 on a model whose cached
+input is $0.003/M. The economics were upside down.
+
+v0.3 asks **only facts**, computes the ones it can exactly, and treats the model as a
+narrow refinement rather than the decision-maker.
+
+## How it decides
+
+| reason | how | action |
+| --- | --- | --- |
+| `referenced` | the target string (path, command) appears in later **prose** — exact search | keep |
+| `superseded` | a later call with the same tool and target — exact | **drop** |
+| `error-resolved` | this call errored, a later call to the same target succeeded — exact | **drop** |
+| `small-result` | under `SMALL_RESULT_CHARS`, not worth touching | keep |
+| `model-unreferenced` | large, unmentioned, not superseded; local model says nothing quotes it | truncate |
+| `model-referenced` | as above, but the model says something does | keep |
+| `inconclusive` | the question could not be answered (no backend, timeout, low confidence) | truncate |
+
+**Deletion requires deterministic evidence.** A model answer can only ever cause a
+*truncation*, which keeps a bounded head plus a `[laya-compaction truncated …; re-run the
+tool if needed]` note, so the model can recover by re-running. Nothing is ever deleted on a
+probabilistic answer.
 
 ## Install
 
@@ -28,123 +46,86 @@ by relevance.
 opencode plugin opencode-jev-compaction --global
 ```
 
-That detects both the `./server` and `./tui` entrypoints and writes each to the
-right config (`opencode.json` for the server plugin, `tui.json` for the widget).
-Restart opencode afterwards.
-
-From a checkout instead:
+Then run the local backend:
 
 ```sh
-opencode plugin github:JLegends/opencode-jev-compaction --global
+# once
+uv venv -p 3.12 ~/laya-server/.venv
+uv pip install -p ~/laya-server/.venv laya-mlx
+
+# run (first start downloads a few hundred MB of weights)
+~/laya-server/.venv/bin/python ~/opencode-jev-compaction/scripts/laya-server.py
 ```
 
+`laya-server.py` is a thin transport: Laya already returns the Jev response shape, so it
+exists only so the plugin can speak HTTP to a local process. It binds `127.0.0.1:8000` and
+serializes inference (MLX is not reliably reentrant).
+
+To keep it running across logins, wrap that command in a launchd agent or run it under
+`tmux`. Startup takes a few seconds plus the one-time download.
+
 ## Configure
 
-The key comes from the environment, or from the macOS Keychain if you point it at
-one:
+| Variable | Default | Purpose |
+| --- | --- | --- |
+| `LAYA_BASE_URL` | `http://127.0.0.1:8000/v1/systemone` | Backend endpoint. A hosted Jev endpoint works too. |
+| `LAYA_COMPACTION` | on | `0` disables everything. |
+| `LAYA_COMPACTION_THRESHOLD` | `60000` | Estimated context tokens before it engages. |
+| `LAYA_PRESERVE_RECENT` | `6` | Newest messages never touched, minimum 1. |
+| `LAYA_SMALL_RESULT_CHARS` | `600` | Results this size or smaller are left alone. |
+| `LAYA_TRUNCATE_HEAD` | `300` | Characters kept when a result is truncated. |
+| `LAYA_EXCERPT_CHARS` / `LAYA_AFTER_CHARS` | `400` / `1000` | What the model sees. Keep these small: Laya's sequence budget is 512 tokens. |
+| `LAYA_REFERENCED_HIGH` | `0.7` | Probability of "quotes" needed to keep. |
+| `LAYA_TIMEOUT_MS` / `LAYA_CONCURRENCY` | `8000` / `4` | Per-question timeout, parallel questions. |
+| `LAYA_MAX_QUESTIONS` | `40` | Cap on model questions per prune. |
+| `LAYA_DAILY_REQUEST_CAP` | `400` | Requests per day, per process. |
+| `LAYA_DEBUG` | off | `1` appends a trace to `~/.local/share/opencode/laya-compaction.log`. |
+
+## Degraded mode
+
+If the backend is unreachable, times out, or answers below `LAYA_REFERENCED_HIGH`, the
+`model-*` rows simply do not apply: the deterministic reasons still fire and every residual
+becomes `inconclusive` → truncate. The plugin is fully functional without any model, and
+`LAYA_COMPACTION=0` turns it off entirely.
+
+## Metrics and reporting
+
+`~/.local/share/opencode/laya-compaction.json` (totals), `-ledger.jsonl` (one line per run
+that changed something, with the reason breakdown and re-run counts), `-usage.json`.
 
 ```sh
-export TYPESAFE_API_KEY=...            # or:
-export JEV_KEYCHAIN_SERVICE=...        # keychain service name
-export JEV_KEYCHAIN_ACCOUNT=...        # keychain account name
+npm run report
 ```
 
-| Variable | Default | Purpose |
-| --- | --- | --- |
-| `TYPESAFE_API_KEY` | — | API key. Required unless the keychain is configured. |
-| `JEV_KEYCHAIN_SERVICE` / `JEV_KEYCHAIN_ACCOUNT` | — | Read the key from the macOS Keychain instead of the environment. |
-| `JEV_COMPACTION` | on | `0` disables everything. |
-| `JEV_COMPACTION_THRESHOLD` | `60000` | Estimated tokens before it engages. Below this it does nothing and costs nothing. |
-| `JEV_KEEP_THRESHOLD` | `0.35` | Minimum probability for a call or result to be kept. Lower keeps more; a call below it is deleted outright, which is irreversible, so this is deliberately conservative. |
-| `JEV_PRESERVE_RECENT` | `6` | Newest messages never touched. Values below `1` are clamped to `1`; setting it to `0` drops the results the model is actively using and causes re-run loops. |
-| `JEV_MAX_STATE_TOKENS` | `25000` | Ceiling for the state sent to Jev. |
-| `JEV_MAX_REQUEST_TOKENS` | `30000` | Ceiling for state plus one batch of questions. |
-| `JEV_TRUNCATE_HEAD` | `300` | Characters of a dropped result kept before its note. |
-| `JEV_SMALL_RESULT_CHARS` | `600` | Results at or below this size are shown to Jev in full instead of as a note. |
-| `JEV_TIMEOUT_MS` | `20000` | Per-request timeout. Failures are skipped silently. |
-| `JEV_DAILY_REQUEST_CAP` | `200` | Hard ceiling on Jev requests per day. |
-| `JEV_MODEL` | `jev-latest` | Model name. |
-| `JEV_BASE_URL` | System One endpoint | Override the endpoint. |
-| `JEV_DEBUG` | off | `1` appends a trace to `~/.local/share/opencode/jev-compaction.log`. |
-
-## Cost
-
-Jev is priced per input token with free output. At the default 25k state ceiling
-and the 200-request daily cap, worst-case spend is about **$0.21/day**, and it
-cannot exceed that. It also removes input tokens from every subsequent request,
-which is the point.
-
-Set `JEV_DAILY_REQUEST_CAP` lower if you want a tighter bound.
-
-## Metrics
-
-Savings alone do not tell you whether the decisions are good, so the plugin records
-the cost of being wrong too. All of it lives in `~/.local/share/opencode/`:
-
-| File | What it is |
-| --- | --- |
-| `jev-compaction.json` | Running totals. `tokensSaved` uses the same calibrated estimator as the threshold, not a characters-per-token guess. |
-| `jev-compaction-ledger.jsonl` | One line per run that changed something, for analysis over time. |
-| `jev-compaction-usage.json` | Requests made today, against the daily ceiling. |
-| `jev-compaction.log` | Per-decision trace, only when `JEV_DEBUG=1`. |
-
-The totals include:
-
-- `runs`, `tokensSaved`, `callsSeen`, `dropped`, `truncated`
-- **`rerunAfterDrop`** and **`rerunAfterTruncate`** — the metrics that matter. A
-  re-run is detected when the model issues the same tool call, with the same input,
-  under a new id, after we removed or shortened the original. That is a decision the
-  model had to pay to undo.
-- `transformCalls`, `engaged`, `belowThreshold`, `capReached`, `overflow`, `noKey` —
-  so you can tell "working well" apart from "never ran".
-
-How to read it: if `rerunAfterDrop` climbs alongside `dropped`, the keep threshold is
-too high or the questions are being asked about content Jev cannot see. If `dropped`
-stays near zero and `belowThreshold` dominates, the trigger is higher than your
-sessions ever reach and the plugin is dormant — raise nothing, lower
-`JEV_COMPACTION_THRESHOLD` if you want it to actually act.
-
-Each ledger line carries `tokensBefore`, `tokensAfter`, `tokensSaved`, `dropped`,
-`truncated`, `requests`, `stage`, the re-run counts, and the `session`, so savings and
-mistakes can be attributed rather than averaged over everything.
-
-## How it works
-
-1. Every finished `tool` part is a candidate, except those in the first message
-   or the newest `JEV_PRESERVE_RECENT` messages, which are pinned.
-2. The whole conversation is sent as state, oldest first, with tool outputs
-   replaced by a short note (`ok, 4213 chars (omitted)`). Tool inputs and all
-   text are included. The state is shrunk in stages until it fits
-   `JEV_MAX_STATE_TOKENS`: inputs truncated to 1000, then 200, then 60
-   characters; long texts abridged head and tail; old messages collapsed;
-   old calls reduced to one line each. If it still doesn't fit, the run is
-   skipped.
-3. Jev answers two graded questions per call: should the **call** stay, and
-   should the **result** stay verbatim. Questions are split into as many
-   requests as needed so state plus questions fits `JEV_MAX_REQUEST_TOKENS`, and
-   those requests run concurrently.
-4. `keepResult >= threshold` keeps both. Otherwise `keepCall >= threshold` keeps
-   the call and truncates the result to its first `JEV_TRUNCATE_HEAD`
-   characters. Otherwise the call and its result go.
-5. Decisions are cached per call for the life of the process and are monotonic:
-   once dropped, always dropped.
-
-Nothing here throws. A missing key, a timeout, a malformed answer, or a history
-too large to fit leaves the messages exactly as they were, so a Jev outage can
-slow nothing down and break nothing.
-
-## Requirements
-
-- opencode `>= 1.18.31`
-- A TypeSafe API key with access to Jev
+Reports engagement, the reason breakdown, whether decisions are good (re-run rate),
+pruned vs unpruned sessions, before/after the install boundary, and the subagent cost share
+measured directly. It refuses to print a quality verdict when the installed version does
+not record re-runs, so a `0` cannot be misread as "nothing was undone".
+
+## Measured caveats
+
+Recorded here because they are the reason the design is conservative:
+
+- **`noul` cannot answer a statement about text.** Against this same local model, a
+  statement and its own negation both scored ~0.95. As a two-option `choice` with explicit
+  criteria, the same cases separate cleanly (0.75–0.99 on a real quote, 0.80–0.91 on
+  unrelated text). That is why the code uses `choice` and says not to change it back.
+- **The model sees only about 1000 characters of what came after** — Laya's sequence budget
+  is 512 tokens. Remote references are found by the exact string search, not by the model.
+- **Accuracy is model- and task-specific.** Independent comparison found the hosted Jev
+  model ahead of open-weight Laya on ambiguous inputs (78% vs 57% on 40 tickets), and
+  confidently wrong on a multi-intent case. That is a small sample: treat it as directional.
+- **Correlation, not causation, in the cohort tables.** A session is only pruned once it is
+  large, so the pruned cohort is longer by construction.
 
 ## Not affiliated
 
-Not built by, endorsed by, or affiliated with the opencode team or TypeSafe.
-"opencode", "Jev", and "TypeSafe" are used only to describe what this plugs into.
+Not built by, endorsed by, or affiliated with the opencode team, TypeSafe, or Convai
+Innovations. Names are used only to describe what this plugs into.
 
 ## License
 
-MIT. The compaction strategy is adapted from
-[fast-jev-compaction](https://github.com/tamaratran/fast-jev-compaction) (MIT) —
-see [NOTICE](./NOTICE).
+MIT. The original strategy was adapted from
+[fast-jev-compaction](https://github.com/tamaratran/fast-jev-compaction) (MIT) — see
+[NOTICE](./NOTICE). v0.3 diverges from it substantially: the decision core is now
+deterministic, and the model is an optional local refinement.

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "opencode-jev-compaction",
-  "version": "0.2.0",
-  "description": "opencode plugins that replace lossy compaction with Jev decisions: score every tool call and result, drop or truncate the stale ones, keep everything else verbatim.",
+  "version": "0.3.0",
+  "description": "opencode plugins that shrink context by deleting provably-stale tool calls and truncating the rest, deterministically first, with an optional local Laya backend.",
   "type": "module",
   "license": "MIT",
   "author": "JLegends",
@@ -25,7 +25,8 @@
     "src",
     "README.md",
     "LICENSE",
-    "NOTICE"
+    "NOTICE",
+    "scripts"
   ],
   "keywords": [
     "opencode",
@@ -53,6 +54,7 @@
     }
   },
   "scripts": {
-    "check": "node --check src/tui.js && bun build src/server.ts --target node --outfile .check.js && rm -f .check.js"
+    "check": "node --check src/tui.js && bun build src/server.ts --target node --outfile .check.js && rm -f .check.js",
+    "report": "node scripts/report.mjs"
   }
 }

package/scripts/laya-server.py

@@ -0,0 +1,113 @@
+#!/usr/bin/env python3
+"""A TypeSafe-Jev-compatible HTTP server backed by local Laya (MLX, Apple Silicon).
+
+Laya is the open-weight System-1 decision model, and its response shape is already the
+Jev one (`answers[name].noul`), so this is a thin transport: it exists only so the
+opencode plugin can speak HTTP to a local process instead of a paid API.
+
+    python laya-server.py                     # 127.0.0.1:8000, multilingual checkpoint
+    LAYA_PORT=8010 python laya-server.py
+    LAYA_SUBFOLDER=typed-decisions python laya-server.py
+
+Endpoints:
+    POST /v1/systemone   {model, state, questions} -> {model, answers, usage}
+    GET  /v1/models      [{id, object}]
+
+First start downloads the checkpoint (a few hundred MB) into the Hugging Face cache.
+
+Requirements: Python 3.11+, Apple Silicon, `pip install laya-mlx`.
+"""
+
+import json
+import os
+import sys
+import threading
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+HOST = os.environ.get("LAYA_HOST", "127.0.0.1")
+PORT = int(os.environ.get("LAYA_PORT", "8000"))
+REPO = os.environ.get("LAYA_REPO", "convaiinnovations/laya")
+SUBFOLDER = os.environ.get("LAYA_SUBFOLDER", "multilingual")
+
+_lock = threading.Lock()
+_agent = None
+
+
+def load():
+    global _agent
+    import laya_mlx as laya
+
+    print(f"[laya] loading {REPO} (subfolder={SUBFOLDER}) ...", flush=True)
+    # load() is called without a subfolder when it would be redundant, so a locally
+    # exported checkpoint can be pointed at with LAYA_REPO=/path/to/model.
+    try:
+        _agent = laya.load(REPO, subfolder=SUBFOLDER)
+    except TypeError:
+        _agent = laya.load(REPO)
+    print("[laya] ready", flush=True)
+
+
+class Handler(BaseHTTPRequestHandler):
+    protocol_version = "HTTP/1.1"
+
+    def log_message(self, format, *args):  # keep the console readable
+        return
+
+    def _send(self, status, payload):
+        body = json.dumps(payload).encode()
+        self.send_response(status)
+        self.send_header("content-type", "application/json")
+        self.send_header("content-length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def do_GET(self):
+        if self.path.rstrip("/") in ("/v1/models", "/models"):
+            return self._send(200, {"object": "list", "data": [{"id": "laya", "object": "model"}]})
+        return self._send(404, {"error": "not found"})
+
+    def do_POST(self):
+        if self.path.rstrip("/") not in ("/v1/systemone", "/systemone"):
+            return self._send(404, {"error": "not found"})
+        try:
+            length = int(self.headers.get("content-length") or 0)
+            body = json.loads(self.rfile.read(length) or b"{}")
+        except Exception as error:
+            return self._send(400, {"error": f"bad request: {error}"})
+
+        state = body.get("state")
+        questions = body.get("questions")
+        if not isinstance(questions, dict) or not questions:
+            return self._send(400, {"error": "questions must be a non-empty object"})
+
+        try:
+            # One model instance, one inference at a time: MLX is not reliably reentrant.
+            with _lock:
+                result = _agent.system_one(state, questions)
+        except Exception as error:
+            return self._send(500, {"error": f"{type(error).__name__}: {error}"})
+
+        return self._send(200, result)
+
+
+def main():
+    try:
+        load()
+    except ImportError:
+        print("[laya] laya-mlx is not installed. pip install laya-mlx", file=sys.stderr)
+        sys.exit(2)
+    except Exception as error:
+        print(f"[laya] failed to load: {type(error).__name__}: {error}", file=sys.stderr)
+        sys.exit(2)
+
+    server = ThreadingHTTPServer((HOST, PORT), Handler)
+    print(f"[laya] listening on http://{HOST}:{PORT}/v1/systemone", flush=True)
+    try:
+        server.serve_forever()
+    except KeyboardInterrupt:
+        print("\n[laya] stopping", flush=True)
+        server.server_close()
+
+
+if __name__ == "__main__":
+    main()