opencode-lossless 0.1.0 → 0.1.2

package/CHANGELOG.md

@@ -1 +1,16 @@
 # opencode-lossless
+
+## 0.1.2
+
+### Patch Changes
+
+- de419eb: Ship a versioned Intent skill with each package (`skills/<name>/SKILL.md` in the published tarball): install, command and recall vocabulary, config essentials, and common mistakes, stamped with the package version on release. Adds `skills` to each package's published files and validates skills on publish and in CI.
+
+## 0.1.1
+
+### Patch Changes
+
+- 6748df1: Refresh every external dependency to its newest published version: Vite+ 1.4.0 and Vitest 5.0.5, TypeScript 7.0.2, `@types/node` 26.6.5, `@changesets/cli` 3.0.3, Pi 1.4.0, OpenCode 2.1.4, Typebox 1.4.3, Blume 2.2.5, and Bun 1.4.4. Pi host packages now use the `*` peer range its package guide documents for packages the host supplies.
+- 6748df1: Derive the heal source window from the summary target instead of clipping every source message at a fixed 3,000 tokens. The window is clamped to the summarizer request budget and has an explicit `healSourceTokens` override in Pi and OpenCode. A leaf that exceeds one window is mapped through up to eight exact windows and reduced once, so a large tool result can become clean without one enormous prompt. A source beyond the bounded plan is blocked without another model call, while a mechanical node gets one improvement. Heal reports now separate clean repairs from improvements and record source-blocked nodes.
+- 6748df1: Add per-node summary timing, session-correlated engine rows, and adapter-routed metrics. OpenCode now writes engine and summarizer rows to its sidecar log, exposes the summarizer call and pass budgets, marks character-derived usage as estimated, and reports summary latency, cap outcomes, and abandoned rows. Add persistent summary quality: clean, clipped, mechanical, and derived. Existing stores migrate to generation 2, and a failed parent stays visible as derived. Add a bounded heal command in both adapters. It finds tainted summaries and their ancestors, re-derives them from stored source bottom-up, rejects another mechanical result, and supports a dry run. A heal owns its store connection so a plugin reload cannot finalize it, and one session allows one heal at a time. Mechanical work runs before a clipped retry, so a bounded run keeps making progress.
+- 6748df1: Add the OpenCode v2 adapter. It mirrors prompts, assistant text, and tool traffic verbatim into a per-session SQLite sidecar; registers `lcm_grep`, `lcm_describe`, `lcm_expand`, and the `/lossless-status`, `/lossless-report`, `/lossless-pass`, `/lossless-backup`, `/lossless-export`, `/lossless-import`, `/lossless-diagnose`, `/lossless-migrate`, `/lossless-gc`, and `/lossless-heal` commands; and records the host's compaction boundaries and summaries beside the rows they replace. `assembly` defaults to true: it builds the leaf and condensed DAG, projects the aged frontier into `session.context`, and hands the same frontier to the host as its compaction checkpoint, marked so the next request re-applies it. `assembly: false` leaves compaction to the host and omits `/lossless-pass` and `lcm_expand_query`. `lcm_expand_query` runs a child session narrowed to the three recall tools, reads the parent's sidecar, returns findings only, and removes the child with its transcript sidecar. Every seam writes a diagnostic row to `metrics.jsonl` (projection, checkpoint, pass, recall, reader), and `/lossless-report` summarizes this session's rows. `lcm_grep` gains `scope` and `lcm_expand` gains `session`, which read other sessions' sidecars read-only; OpenCode keeps one global sidecar directory, so `sessions` and `all_sessions` are equivalent and the reader remains session-scoped. `/lossless-diagnose` builds a redacted bundle and can repair the full-text index. `/lossless-gc` is age-based because the plugin API cannot list host sessions; it exports each transcript before deleting and never touches the current or open sidecar. There is no cockpit in V2 scope: the host TUI extension point exists, so a separate TUI plugin could reuse the same command core later. The adapter has no runtime dependencies, and sidecar files and their WAL are owner-only (0600) with a 0700 directory.

package/README.md

@@ -1,274 +1,88 @@
 # opencode-lossless
 
-Lossless Context Management for OpenCode v2, built on
-[`lossless-core`](https://npmx.dev/package/lossless-core). The adapter mirrors
-every prompt, assistant reply, and tool exchange into a per-session SQLite
-sidecar. The model reads that history back through `lcm_grep`, `lcm_describe`,
-and `lcm_expand`; with `assembly` true (the default) it also gets
-`lcm_expand_query`, which answers through a child session without pulling the
-originals into the parent.
+**Lossless Context Management (LCM) for OpenCode v2. Long sessions keep working past the point where the model would forget, and every verbatim message stays searchable and readable.**
 
-With assembly on, the adapter also owns the context boundary: once a request
-crosses the swap point, the DAG projection replaces the aged history, and when
-the host compacts, the adapter hands it the frontier as the checkpoint instead
-of letting it summarize. With `assembly: false`, the host compacts natively.
-Either way the sidecar keeps every address the reader can name, including the
-compaction summaries.
+Install it, and the adapter mirrors every prompt, assistant reply, and tool exchange into a per-session SQLite sidecar. The model reads that history back through `lcm_grep`, `lcm_describe`, and `lcm_expand`; with `assembly` true (the default) it also gets `lcm_expand_query`, which answers through a child session without pulling the originals into the parent. With assembly on, the adapter also owns the context boundary: past the swap point the DAG projection replaces the aged history, and when the host compacts, the adapter hands it the frontier as the checkpoint instead of letting it summarize.
 
-Verified against OpenCode v2.1.3 in live sessions: the plugin loads from its
-directory; prompt text, assistant text, tool input and result, and a manual
-compaction mark with its summary text all mirror verbatim; `lcm_grep` answers
-from the sidecar; and auto-compaction ran after a project config narrowed the
-window to 6k tokens. Compaction marks and summaries keep distinct ids across a
-plugin reload. In the live run, the host stored the DAG frontier as its
-checkpoint with the adapter's marker on the message, and the next request
-re-applied it.
+This package is the OpenCode v2 adapter over
+[`lossless-core`](https://npmx.dev/package/lossless-core), the engine that
+ingests the history and builds the summaries. Verified against OpenCode v2 in
+live sessions: the plugin loads from its directory, all mirrored text matches
+verbatim, and host compactions store the DAG frontier as their checkpoint.
 
 ## Install
 
-Install it with the OpenCode CLI, which adds the `plugins` entry to the global
-config:
-
 ```sh
 opencode plugin add opencode-lossless
 ```
 
-The equivalent entry is:
+The equivalent entry in `opencode.json`:
 
 ```json
 {
-  "plugins": [
-    {
-      "package": "opencode-lossless",
-      "options": { "sidecarDir": "./.lossless" }
-    }
-  ]
+  "plugins": [{ "package": "opencode-lossless", "options": { "sidecarDir": "./.lossless" } }]
 }
 ```
 
-## Live loop
+## Quick start
 
-From the checkout:
+Zero config required: passive mirroring starts immediately, and swaps begin
+once context crosses the swap point. Ten commands, all `lossless-*`:
 
 ```sh
-bun run pack           # build the bundle the host loads
-bun run opencode:check # read-only: host version, config, bundle hash, sidecar counts
-opencode2 plugin list  # confirms the host loaded this checkout's dist/index.mjs
-bun run opencode:smoke # one real model call through the running server
-bun run opencode:compact --mode manual # compaction gate, makes model calls
+/lossless-status    # message and compaction summary counts per live sidecar
+/lossless-report    # this session's metrics: projection, checkpoint, pass, recall, reader
+/lossless-pass      # run one DAG pass over the aged span now, without waiting for the swap
+/lossless-heal      # dry-run repair of tainted summaries; --apply spends model calls
+/lossless-backup    # copy this session's sidecar beside it
+/lossless-export    # write the verbatim transcript and summaries as JSONL (mode 0600)
+/lossless-import    # restore rows from an export
+/lossless-diagnose  # redacted, pasteable diagnostics block; --save writes it at mode 0600
+/lossless-migrate   # upgrade older-schema sidecars in place (dry-run by default)
+/lossless-gc        # list old sidecars, or export and delete them (never the current session)
 ```
 
-`bun run opencode:check` makes no model calls. It prints the host version, the
-config path, the plugin entry resolved from the config document's directory,
-the bundle's sha256 and mtime, the resolved sidecar directory, and one line per
-store with message counts by role, the compaction summary count, and the newest
-entry. It takes `--json` and `--limit N`. The live scripts require an absolute
-`sidecarDir` in the plugin options, because the host resolves a relative one
-from its own working directory.
-
-The server watches the configured plugin directory and reloads it after
-`bun run pack`. If a change does not take effect, run `opencode2 reload`; a
-service restart is the fallback.
-
-`bun run opencode:smoke` sends `Reply with exactly: pong.` through
-`opencode2 run` in a temporary directory, then asserts that the session sidecar
-holds a user row and an assistant row with the reply, that the sidecar
-directory is 0700, and that the database files are 0600. It deletes the sidecar
-afterwards unless `--keep` is passed. `--model provider/model[#variant]`
-overrides the model, which defaults to the config's `model`. The npm free tier
-rejects CLI runs, so the smoke does not default to a free model.
-
-`bun run opencode:compact` is the compaction gate, and it makes real model
-calls. `--mode manual` asks for a compaction through the API, waits for the
-step boundary to store a mark and a summary, reloads the plugin, then compacts
-again and asserts that both runs left distinct ids and that the first rows
-survived. `--mode auto` (the default is manual) writes a temporary project
-config with a 6k token window and auto-compaction, then sends a long prompt
-until the host compacts on its own. Both modes assert the 0700 directory and the
-0600 database files, delete the sidecar unless `--keep` is passed, and take
-`--model` and `--timeout-ms`. Auto mode uses `--model`, or the script's fallback
-model when none is passed; manual mode uses `--model`, then the config's
-`model`, then the script fallback.
+The model gets four recall tools: `lcm_grep` (search this session, `scope`
+reaches past sessions), `lcm_describe` (inspect one summary without expanding
+it), `lcm_expand` (read the originals a summary covers), and
+`lcm_expand_query` (a bounded child session reads and answers, so the
+originals never enter the parent's context).
 
 ## Config
 
 ```json
 {
   "enabled": true,
-  "sidecarDir": "./.lossless"
+  "sidecarDir": "./.lossless",
+  "assembly": true
 }
 ```
 
-`enabled` is a master flag and defaults to true. Listing the plugin is intent
-enough, while `false` registers nothing. `sidecarDir` holds one SQLite file
-per session and is created when missing. It defaults to `./.lossless`,
-resolved from the server working directory. Delete a sidecar to forget that
-session. The mirror never writes to the session record. With assembly on, the
-host stores the checkpoint summary and its marker on the compaction message.
-
-`assembly` (default true) wires the context boundary. It registers the
-`session.context` projection, the blocking `session.compaction` checkpoint,
-`lcm_expand_query`, and the `/lossless-pass` command. The projection swaps the
-aged history for the DAG frontier at `swapAtRatio` (default 0.7) and re-cuts at
-`recutAtRatio` (default 0.85); `keepRecentTokens` (default 20000) is the
-verbatim tail, `summaryTokens` (default 1500) the render budget, `contextWindow`
-(default 128000) the window used when the catalogue cannot name one, `smartZone`
-and `zones` override the thresholds, and `summarizer` names the model chain,
-where `auto` is the session model. This host exposes no synchronous model
-lookup, so a concrete model id in the chain falls back to the session model.
-`summarizerTokensPerCall` (default 60000) caps one request, estimate plus output
-cap, before it is refused rather than paid for; `summarizerTokensPerPass`
-(default 250000) caps what one pass may spend across its calls.
-`summarizerTokensPerHeal` (default 120000) caps what one
-`/lossless-heal --apply` run may spend across its calls; a dry run spends
-nothing. `healSourceTokens` (default 8x `summaryTokens`, clamped 3000 through
-24000 and to `summarizerTokensPerCall`) is the source window one heal map call
-may read. A leaf that fits one window is sent once; a larger leaf maps up to
-eight windows and reduces them once. A source beyond eight windows is blocked
-without a model call, and a mechanical node gets one improvement.
-`readerTimeoutMs` (default 120000) is how long one expand-query child may run
-before it is interrupted. A malformed value keeps its default rather than
-failing the plugin load. Set `assembly: false` to leave compaction to the host.
-
-## What the model gets
-
-Three tools always, and a fourth by default; `assembly: false` leaves three.
-`lcm_grep`, `lcm_describe`, and `lcm_expand` use the same recall vocabulary as
-the Pi adapter; the OpenCode descriptions add scope and session text.
-`lcm_expand_query` matches Pi's tool: it runs a bounded
-reader and returns findings, so the raw originals never enter the parent's
-context.
-
-- `lcm_grep`: full-text search over the calling session's stored history.
-  Each hit names the summary that covers it. `scope: "sessions"` or
-  `"all_sessions"` searches the other sidecars in the directory; OpenCode keeps
-  one global sidecar directory, so the two scopes are equivalent today and no
-  project index is kept. A past-session hit is labelled `[lcm:session <id>]`
-  and capped at 20 store files, 40 hits, and 1.5 s.
-- `lcm_describe`: inspect one summary without expanding it. This reads the
-  calling session only.
-- `lcm_expand`: read the original messages a summary covers, or one named
-  entry. Pass `session` with an `[lcm:session <id>]` hit to read that past
-  session's sidecar read-only; the text comes back labelled as a past
-  session's history.
-- `lcm_expand_query` (assembly default): recover compacted history and answer a
-  question about it. Pass `summary_id` or `entry_id` to read that source, or
-  neither to search first. `max_tokens` (default 10000, clamped to 500..40000)
-  sets the budget stated in the brief, and `readerTimeoutMs` bounds the child's
-  run. The reader is session-scoped in this version: it reads the calling
-  session's history, not a past session's, and the child's recall tools refuse
-  `scope: "sessions"` and a past `session` argument.
-
-The reader is an ordinary child session created against the parent. It runs
-the host's `explore` agent with a session ruleset that denies everything and
-then allows only `lcm_grep`, `lcm_describe`, and `lcm_expand`; the
-`session.context` hook replaces its system prompt with the reader instructions
-and skips the projection and checkpoint. Its recall tools resolve to the
-parent's sidecar. The parent's `lcm_expand_query` waits for the child, reads
-the last assistant text, redacts it, removes the child, and deletes the child's
-transcript sidecar. A named summary or entry the store does not hold is refused
-before a child exists. The reader is session-scoped, so there is no past-session
-reader, and there is no per-child token or step cap: the child's model decides
-how much it reads within the brief.
-
-Ten commands by default; `/lossless-pass` is absent with `assembly: false`:
+`sidecarDir` holds one SQLite file per session (0700 dir, 0600 files; delete
+one to forget that session). `assembly: false` leaves compaction to the host
+and drops `/lossless-pass` and `lcm_expand_query`. Every other key (thresholds,
+summarizer chain and budgets, heal window, reader timeout) lives on the
+[config reference](https://stainless-code.com/lossless/reference/opencode/config)
+with its default.
 
-- `/lossless-status`: report the message and compaction summary counts per live
-  sidecar. A command cannot return text directly, so the status arrives as a
-  prompt the loop answers from.
-- `/lossless-report`: summarize this session's rows in `metrics.jsonl`:
-  projection decisions with their top bail and kick reasons, checkpoints with
-  the last summary ids and text size, pass outcomes with leaf and condensed
-  totals, summary node counts with the level-3 fallback share and node latency,
-  summarizer usage, errors, caps, and estimated counts, abandoned pass counts,
-  heal runs, recall calls per tool, reader runs with their average duration, and
-  the last decision's action and occupancy.
-- `/lossless-pass`: run one DAG pass over the aged span now, without waiting for
-  the swap threshold.
-- `/lossless-heal [--apply] [--max-nodes <n>]`: repair tainted summaries from
-  their stored sources. Dry-run by default and makes no model call; `--apply`
-  spends model calls, and `--max-nodes` overrides the engine's 32-node cap.
-- `/lossless-backup [path]`: copy this session's sidecar beside it, or to the
-  path you name.
-- `/lossless-export [path]`: write the verbatim transcript and summaries as
-  JSONL, mode 0600. The default lands beside the sidecars.
-- `/lossless-import <path>`: restore rows from an export. Refused while a pass
-  is running, or into a store that already holds foreign messages.
-- `/lossless-diagnose [--save <path>] [--repair]`: build a redacted, pasteable
-  diagnostics block; `--save` writes it at mode 0600, and `--repair` rebuilds
-  the full-text index first.
-- `/lossless-migrate [--apply]`: upgrade older-schema sidecars in place.
-  Dry-run by default.
-- `/lossless-gc [--older-than <days>] [--session <id>] [--apply]`: list old
-  sidecars, or export and delete them. Never touches the current session.
+## Verify
 
-`gc` is age-based because the plugin API cannot list host sessions: it defaults
-to 30 days, never touches the current session, and exports each transcript
-before deleting it.
+From a checkout of this repo:
 
-There is no cockpit in V2 scope: the host exposes a TUI extension point with slot mounts (including `session.panel`), so a separate TUI plugin could reuse the same command core later.
-
-## How it works
-
-The plugin mirrors four boundaries into the sidecar, each under a stable
-entry id, so a redelivered event inserts nothing:
-
-| Boundary                                                  | Entry                                                                       |
-| --------------------------------------------------------- | --------------------------------------------------------------------------- |
-| `session.prompt` hook                                     | the user text plus attached file names                                      |
-| `session.text.ended` bus event                            | each assistant text block at its full-value boundary                        |
-| `tool.execute.after` hook                                 | the tool name, the input that caused it, and the result or error text       |
-| `session.compaction` hook plus `session.compaction.ended` | a checkpoint mark naming what left the window, then the summary text itself |
-
-Only text is stored. Media, reasoning, and streaming deltas stay out, which
-matches what the model saw as text. Native checkpoints that publish an empty
-summary are skipped because there is nothing to store.
-
-With assembly enabled (the default), the same compaction hook is the blocking boundary. It reads
-the stored history, cuts the aged prefix the way the projection does, and hands
-the host the frontier render as `event.result.summary` plus a marker
-(`lcm`, `summaryIds`, `cutCount`, `checkpoint`, `keepID`). The host stores the summary and
-the marker on the compaction message, so the next request's projection
-recognizes its own checkpoint by the marker, not by the text, and re-applies the
-host's rendering of it. A checkpoint the adapter did not write leaves the
-request alone. If the frontier does not cover the aged span the hook runs one
-pass and waits; a failed pass leaves the result unset so the host summarizes
-natively, and records a `compaction-checkpoint` metric row.
-
-Every seam writes a row to `metrics.jsonl` beside the sidecars: one
-`projection` decision per request, one `compaction-checkpoint` per boundary,
-one `pass` per summarizer run, one `recall` per recall-tool call, and one
-`reader` per expand-query child with its outcome, duration, answer size, and
-cited ids. The engine and summarizer rows a pass causes (`summary-stored`,
-`summarizer-usage`, `compaction-complete`, summarizer errors and caps, and
-abandoned counts) land in that same OpenCode log through the pass's sink, not
-in Pi's metrics file. A heal writes `heal-start`, `heal-node`, and
-`heal-complete` rows to the same log. The log is created 0600 in a 0700 directory, rotates to
-`metrics.jsonl.<timestamp>` at 5 MB, keeps three generations, and a read skips
-a missing file or a malformed line. `/lossless-report` renders it per session.
-
-The published bundle names no host SDK. `src/host.ts` is the whole part of
-the plugin API this package uses, written as local structural types, and
-`test/host-compat.test.ts` assigns the real host types to each one. A host
-upgrade that changes the API fails typecheck instead of packing a silent
-mismatch.
+```sh
+bun run opencode:check   # read-only: host version, config, bundle hash, sidecar counts
+bun run opencode:smoke   # one real model call, asserts the verbatim mirror
+bun run opencode:compact # compaction gate, makes real model calls
+```
 
-## Security notes
+## Documentation
 
-- Sidecars are plaintext SQLite files under `sidecarDir`. The directory is
-  created 0700, and the database, WAL, and shared-memory files are 0600. A
-  sidecar holds the session's secrets verbatim, so it deserves the same care
-  as the session itself.
-- Redaction is best-effort pattern matching, not a hard boundary. It runs on
-  the way out, at every tool that hands stored text to a model, and it never
-  touches a stored row.
-- Recall tools read the calling session's sidecar, except inside an
-  expand-query child, which reads its parent's. A `lcm_grep` scope of
-  `sessions` or `all_sessions` opens other sidecars through a `queryOnly`
-  handle, so a scan never rewrites a session it is only reading, and
-  `lcm_expand` with `session` reads the same way. A session with no history
-  gets an explanation, not an empty result, and a lookup never creates a store. The
-  child's own sidecar is a transcript and is deleted with the child.
+|                                                                                 |                                                           |
+| ------------------------------------------------------------------------------- | --------------------------------------------------------- |
+| [The docs site](https://stainless-code.com/lossless/)                           | How LCM behaves in a session, the config keys, the limits |
+| [Commands](https://stainless-code.com/lossless/reference/opencode/command)      | What each `/lossless-*` command does                      |
+| [Comparison](https://stainless-code.com/lossless/reference/opencode/comparison) | The competing engines on one set of axes, verified cells  |
+| [CHANGELOG.md](./CHANGELOG.md)                                                  | Version history (changesets)                              |
 
 ## Credits
 

package/dist/index.d.mts

@@ -1,19 +1,4 @@
-/** One model's thresholds. A token field takes precedence over its ratio
- * counterpart when both are set. */
-interface ThresholdConfig {
-  /** Swap when the request reaches this share of the model window. */
-  swapAtRatio?: number;
-  /** Swap when the request reaches this many tokens; takes precedence over
-   * `swapAtRatio`. */
-  swapAtTokens?: number;
-  /** Re-cut when the request reaches this share of the model window. */
-  recutAtRatio?: number;
-  /** Re-cut when the request reaches this many tokens; takes precedence over
-   * `recutAtRatio`. */
-  recutAtTokens?: number;
-  /** Swap at this absolute token count, deriving the recut from it. */
-  smartZone?: number;
-}
+import { LcmStore, SidecarStores, ThresholdConfig } from "lossless-core";
 /** The part of the OpenCode v2 plugin API this package uses, written
  * as local structural types. The published bundle and its declarations name
  * no host SDK: every host value arrives through these shapes, and