opencode-diane 0.0.5

package/CHANGELOG.md

@@ -0,0 +1,180 @@
+# Changelog
+
+All notable changes to this project are documented in this file.
+
+The format is loosely based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
+this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
+on the understanding that the public surface for SemVer purposes is the
+tool list (`memory_*`) and the documented `UserConfig` options.
+
+## [0.0.5] — 2026-05-22
+
+### Added
+- **Live-session activity recording (`recordSessionActivity`, default
+  `true`).** The current session's file edits and bash commands are
+  now rolled up into a single memory under `session-trace` →
+  `live:${sessionId}`, upserted in place after each event. Lets the
+  current session recall what it has already touched without scanning
+  the OpenCode SDK, and pre-seeds the trace for the moment this
+  session becomes "past" to a successor. The memory is not pinned —
+  transient state should be evictable. JSONL logs are unchanged; this
+  is a recall surface, not an audit log. New module
+  `src/ingest/live-session.ts` (~160 lines), bounded by
+  `MAX_CONTENT_BYTES` and per-buffer caps so a long session can't
+  swell the store.
+- **Bash file-change tracking (`bashFileTrackingMaxFiles`,
+  default `20`).** After every `bash` tool call the plugin runs
+  `git status --porcelain` and refreshes the code-map for each
+  touched file (modified or untracked, deletions skipped). Closes the
+  long-standing "bash-driven changes are a known gap" caveat —
+  `git checkout other-branch`, `npm run format`, `cargo fmt --all`,
+  and similar now keep the code map fresh. Capped to bound post-hook
+  latency on mass-checkout situations; raise or set to `0` to
+  disable.
+- **Auto git re-ingest on HEAD movement (`autoReingestGitOnHeadChange`,
+  default `true`).** After every `bash` call the plugin polls
+  `git rev-parse HEAD`; if HEAD moved (pull / merge / rebase /
+  checkout / reset), it queues a background re-ingest of git
+  history. Idempotent — already-known commits are skipped via
+  `insertIfMissing`. Concurrent triggers coalesce (one re-ingest at a
+  time; further detections re-arm the flag for the next poll). Closes
+  the post-merge invisibility gap surfaced by the v0.0.4 verdict.
+- **`memory_ingest_git` tool (the 10th).** Explicit on-demand
+  re-ingest of git history for cases the auto-detect can't cover
+  (a `git fetch` without merge that nonetheless brings new commits
+  via another mechanism). Returns a human-readable summary of new
+  commit / co-change / churn / recency memories added.
+- New helpers in `src/utils/shell.ts`: `currentHead(cwd)` (returns
+  the HEAD SHA or null) and `changedFilesInWorktree(cwd)` (parses
+  `git status --porcelain` into a file list, handling renames,
+  deletions, and C-quoted paths). Both are non-throwing — they
+  return safe empty values on non-git directories or git failures.
+
+### Changed
+- The tool count is now **ten** (was nine in v0.0.4). The new tool is
+  additive — existing tool semantics are unchanged.
+- Live-session memories share the `session-trace` category with past
+  sessions, distinguished by subject prefix: `task:` / `trace:` for
+  past, `live:` for current.
+
+### Notes
+- All three new behaviours default ON. Set the corresponding config
+  to `false` (or `0` for `bashFileTrackingMaxFiles`) to opt out. The
+  JSONL audit log captures the same events regardless.
+- Test count: **674 assertions across 24 test suites** (up from
+  641/23 in v0.0.4). New suite `tests/live-session.test.ts` adds
+  33 assertions; two pre-existing plugin-test assertions and one
+  smoke-script assertion were updated from "nine tools" to "ten".
+
+## [0.0.4] — 2026-05-22
+
+### Added
+- **Grammar-agnostic cross-reference ingester (`ingestCrossRefs`,
+  default `true`).** Discovers file→file edges for languages
+  tree-sitter doesn't cover (Pascal, Ruby, Perl, Elixir, Lua, R,
+  Haskell, Scala, Kotlin, Swift, Crystal, Nim, Tcl, OCaml, F#, Dart,
+  Erlang, Clojure, Elm, Zig, Ada, **plus Verilog, SystemVerilog,
+  VHDL, COBOL, Fortran (modern, .f90+), Solidity, D, Vim script,
+  Smalltalk, Racket/Scheme, Common Lisp, Modula-2**) and for
+  low-code DSLs (GitHub Actions, Docker Compose, k8s kustomize,
+  Terraform, Ansible, n8n-shaped JSON workflows, OpenAPI, Protocol
+  Buffers, Thrift, GraphQL). Uses **multi-signal corroboration** to
+  keep FP low: path-resolvable strings (filesystem-grounded; single
+  signal sufficient) and rare identifier references corroborated by
+  import-line context OR filename↔identifier coupling (lexical
+  matches alone are deliberately rejected). Three new import-line
+  patterns: Verilog backtick-include (`` `include "file.v" ``),
+  COBOL `COPY copybook.cpy.`, Vim `runtime path/file.vim`.
+  Per-extension comment-line skipping prevents Lisp `; alu` in
+  comments from spuriously coupling to `alu.v`. Each edge becomes a
+  memory under `xref:<from>→<to>`. 52 assertions in
+  `tests/cross-refs.test.ts` exercise the language + DSL matrix
+  including FP-control negatives.
+- **Generic markdown ingestion (`ingestDocs`, default `true`).** Walks
+  `<root>/docs/` recursively and a fixed set of conventional root
+  docs (CHANGELOG, CONTRIBUTING, ARCHITECTURE, ROADMAP, TODO, NOTES,
+  SECURITY, HISTORY, GOVERNANCE, MAINTAINERS, AUTHORS,
+  CODE_OF_CONDUCT). Each H1/H2/H3 heading becomes a recallable
+  section pointer: `memory_recall { query: "installation" }` now
+  returns `docs/install.md:15  ## Installation` + first paragraph —
+  the agent gets a precise pointer it can hand to `read` without
+  grepping the docs tree. Headings inside fenced code blocks are
+  correctly ignored. Bounded walk (200 files, 50 headings/file,
+  256 KB/file). Tests in `tests/docs-ingest.test.ts`.
+- **Project-notes ingestion (`ingestProjectNotes`, default `true`).**
+  Indexes root-level agent-instruction files as project facts:
+  `AGENTS.md`, `CLAUDE.md`, `GEMINI.md`, `COPILOT.md`,
+  `CONVENTIONS.md`, `.cursorrules`, `.windsurfrules`, `.clinerules`.
+  One memory per file (truncated to 6 KB) plus a directory-summary
+  memory listing what was found, so the agent learns the repo's
+  house rules in the first recall. Tests in
+  `tests/extra-ingesters.test.ts`.
+- **Table-header ingestion (`ingestTableHeaders`, default `true`).**
+  Walks for `.csv`, `.tsv`, `.xlsx`, `.xls`, `.xlsm` files and
+  indexes *only the column headers* — `users.csv (CSV, 4 columns):
+  id, email, signup_date, plan_tier`; for spreadsheets, each sheet
+  becomes its own memory (`table:budget.xlsx#users`,
+  `table:budget.xlsx#events`). Row data is never loaded; even a 10 GB
+  CSV or a workbook with 1 M rows costs the same as a 10-row file.
+  RFC-4180-ish CSV parsing handles quoted fields with embedded
+  commas; single-column files / sheets and binary files are
+  rejected. XLSX support uses SheetJS, **lazy-imported only when a
+  spreadsheet is actually encountered** — projects with no
+  spreadsheets pay no module-load cost. SheetJS is invoked with
+  macros, formulas, and styles disabled (minimum surface). Tests
+  in `tests/extra-ingesters.test.ts`.
+
+### Changed
+- **`enableCodeMap` now defaults to `true`.** The tree-sitter
+  signature map is built on first startup so the agent has the
+  structural shape of the codebase from the first session. Set to
+  `false` to disable (the ~10 MB grammars ship in the package
+  regardless).
+- **Soft-force adoption.** A new `installUsageSkill` option (default
+  `true`) writes `<skillsOutputDir>/<prefix>using-memory/SKILL.md` on
+  first startup so OpenCode surfaces a "call `memory_recall` first"
+  instruction to the agent at session start. Written only when the
+  file does not already exist — user edits and user deletions both
+  survive every subsequent startup. Set `installUsageSkill: false` to
+  never write it.
+
+## [0.0.3] — Unreleased
+
+First public preview release.
+
+### Added
+- Nine `memory_*` tools (`recall`, `remember`, `snapshot`, `status`,
+  `code_map`, `outline`, `ingest_sessions`, `mine_skills`, `skill`).
+- BM25 lexical recall with one-hop co-change boosting; opt-in
+  Personalized PageRank for multi-hop recall (`personalizedPageRank`).
+- Tree-sitter code-map across 10 languages plus JSON/CSS/HTML
+  (`enableCodeMap`).
+- Cross-lingual semantic search via a small multilingual e5 model
+  (`enableSemanticSearch`, optional `@huggingface/transformers`).
+- Skill mining: cluster-derived `SKILL.md` files from past sessions.
+- Structured JSONL logs under `$OPENCODE_DIANE_LOG_DIR` (or
+  `os.tmpdir()/opencode-diane/`) with `analyze-logs.py` reader.
+- Plugin version flows from `package.json#version` to the
+  `plugin.active` event and `memory_status` output — single source of
+  truth, no second place to update.
+- Defensive legacy-JSON → SQLite migration that never crashes plugin
+  startup even when another plugin is contending for the database.
+
+### Reliability
+- `MemoryRepository.load` accepts an `onMigrationError` callback so a
+  failed legacy migration logs cleanly and the plugin continues with
+  an empty database rather than failing to start. Regression test in
+  `tests/store.test.ts`.
+- Auto-detects coexisting plugins (`oh-my-opencode`/`oh-my-openagent`/
+  `oh-my-opencode-slim`, and any `caveman` packaging) by reading the
+  `plugin` array in `opencode.json`. When a peer is present and the
+  user hasn't overridden the relevant option, disables the
+  tool-output nudge hook (oh-my-opencode also rewrites tool output)
+  and prefixes mined-skill subdirectories with `diane-` (so skills
+  in the shared `.opencode/skills/` directory don't collide with the
+  peer's slugs). Standalone behaviour is unchanged. Tests in
+  `tests/peer-compat.test.ts`.
+
+### Coverage
+- 460+ assertions across 16 suites; 91%+ line coverage; size ceiling
+  enforced in CI.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 opencode-diane contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,206 @@
+# opencode-diane
+
+[![npm](https://img.shields.io/npm/v/opencode-diane.svg)](https://www.npmjs.com/package/opencode-diane)
+[![CI](https://github.com/two-coats-guaranteed/opencode-diane/actions/workflows/ci.yml/badge.svg)](https://github.com/two-coats-guaranteed/opencode-diane/actions/workflows/ci.yml)
+[![license](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
+
+A memory layer for [OpenCode](https://opencode.ai). It gives the
+coding agent a persistent, searchable store of structural facts about
+a repository, so it stops re-discovering the same things — with raw
+`git log`, `grep`, and file reads — every single session.
+
+Named for Diane in *Twin Peaks* — the recipient of Dale Cooper's
+recorded case notes. The plugin works the same way: it keeps the
+record of what your coding agent learns about a codebase.
+
+## TL;DR for a decision-maker
+
+- **What it is.** A hierarchical, BM25-ranked memory layer for any git
+  repository. It pre-fills itself from git history, project files,
+  docs, agent-instruction files (AGENTS.md, CLAUDE.md, .cursorrules),
+  table column headers, and grammar-agnostic cross-file edges; the
+  agent reaches it through ten `memory_*` tools.
+- **The problem it solves.** An agent re-greps and re-reads the same
+  files every session. One bounded `memory_recall` replaces many raw
+  discovery calls.
+- **Token reduction.** 80–89 % measured *when a recall covers the
+  task* — a ceiling, not a promise; lower on terse-history, mature, or
+  tiny repos. The bundled `dry-run.mjs` gives *your* repo a GOOD /
+  MODERATE / LOW verdict before you rely on it.
+- **Deterministic.** BM25 over a hand-built index — no embeddings, no
+  model, no API key, no GPU, no network. Reproducible and debuggable.
+  (One opt-in exception: semantic search — off by default — adds an
+  embedding model for cross-lingual recall. See below.)
+- **Convention-free.** It never parses commit messages for meaning;
+  every signal is a physical fact (files touched, lines ±, co-change).
+  It behaves identically on a `wip` / `.` / `更新` history and a
+  pristine one.
+- **Languages.** Code map covers 13 tree-sitter grammars. The
+  grammar-agnostic cross-reference ingester extends this to 30+
+  languages (Pascal, Ruby, Perl, Elixir, Verilog, VHDL, COBOL,
+  Fortran, Solidity, Smalltalk, Racket, Common Lisp, Vim script, and
+  more) plus low-code DSLs (GitHub Actions, k8s, Terraform, n8n).
+- **What it costs.** ~1.6 MB packed, ~17 MB unpacked (includes
+  tree-sitter grammar `.wasm` files and SheetJS for spreadsheet
+  headers). A few hundred MB RAM for a large store. Dependencies:
+  `@opencode-ai/plugin`, `web-tree-sitter`, `xlsx`.
+- **What it is not.** Not an LLM, not an unbounded archive (a
+  configurable disk budget, 50 MB by default, ages out least-used
+  facts), not a replacement for `AGENTS.md` (though we *read*
+  AGENTS.md and index its contents). Not a vector store by default —
+  lexical BM25 — though cross-lingual semantic search is available as
+  an explicit opt-in.
+- **Maturity.** 674 assertions across 24 test suites, ~90 % line
+  coverage; verified against the documented plugin contract in 30+
+  languages and against live builds with oh-my-opencode and caveman
+  as coexisting plugins. Not yet run end-to-end inside a live OpenCode
+  *server* — see the WIKI.
+
+**The full design — how the memory is structured, how retrieval works,
+what happens without git, scaling numbers, how it compares to other
+approaches, and every honest limitation — is in
+[WIKI.md](./WIKI.md).** Start there with *Straight answers for a
+decision-maker*.
+
+## The tools
+
+| Tool | What it does |
+|---|---|
+| `memory_recall` | BM25 search over the store — co-change-boosted, token-budgeted, with optional `category` / `subject` filters. The recall-first entry point. |
+| `memory_code_map` | Aider-style structural map: per-file signatures of functions/classes/types, ranked and budgeted. Needs `enableCodeMap`. |
+| `memory_remember` | Store an explicit note for future turns. |
+| `memory_snapshot` | Record this session's *understanding* — mental model, decisions, conventions — for a later or parallel session to resume from. |
+| `memory_outline` | Counts per category — token-cheap orientation. |
+| `memory_status` | Size, byte usage vs budget, last-ingest timestamps. |
+| `memory_ingest_sessions` | Pull task + tool-trace summaries from past OpenCode sessions. |
+| `memory_ingest_git` | Re-scan git history for new commits after a pull / merge / rebase. Idempotent — already-known commits are skipped. The plugin also auto-runs this in the background when it detects HEAD moved as a side effect of a `bash` call. |
+| `memory_mine_skills` | Cluster memories by subject into `SKILL.md` files. Runs in the background. |
+| `memory_skill` | List the mined skill files, or load one into the conversation — so a skill mined this session is usable now, no restart. |
+
+## Install
+
+```bash
+npm install opencode-diane
+```
+
+Then in `opencode.json`:
+
+```json
+{
+  "$schema": "https://opencode.ai/config.json",
+  "plugin": ["opencode-diane"]
+}
+```
+
+Open OpenCode in any git repository, in any language. The plugin
+loads, runs prefill in the background, registers all ten tools, and
+the agent can use them immediately. If the directory is neither a git
+repo nor has a recognised manifest, the plugin logs one idle line and
+does nothing.
+
+The optional Aider-style code map is **off by default** because its
+tree-sitter grammars add ~16 MB to the install. To enable it, use the
+`[name, options]` tuple form and restart OpenCode:
+
+```json
+{
+  "plugin": [["opencode-diane", { "enableCodeMap": true }]]
+}
+```
+
+### Install from a local clone
+
+```bash
+git clone <repo-url> opencode-diane
+cd opencode-diane
+bun install      # fetches @opencode-ai/plugin into ./node_modules
+bun run build    # compiles src/ -> dist/
+```
+
+Then point `opencode.json` at the built file:
+
+```json
+{
+  "plugin": ["file:///absolute/path/to/opencode-diane/dist/index.js"]
+}
+```
+
+`bun install` is required for the local form — OpenCode resolves
+plugin imports through the module resolver, so
+`node_modules/@opencode-ai/plugin` must sit next to `dist/`.
+
+## Configuration
+
+Every setting is optional with a sensible default. To override, list
+the plugin as a `[name, options]` tuple — OpenCode passes the options
+object straight through, and bad or unknown keys are ignored so a
+malformed config never breaks the plugin.
+
+```ts
+interface UserConfig {
+  maxMemoryDiskMB?: number       // default 50
+  autoIngestOnStartup?: boolean  // default true
+  gitHistoryDepth?: number       // default 500
+  forceActive?: boolean          // default false
+  skillsOutputDir?: string       // default ".opencode/skills"
+  skillMiningMinCluster?: number // default 3
+  ingestSessions?: boolean       // default true
+  enableCodeMap?: boolean        // default true  (see WIKI: Code map)
+  installUsageSkill?: boolean    // default true  — write a using-memory skill on first startup
+  ingestDocs?: boolean           // default true  — index docs/ headings as section pointers
+  ingestProjectNotes?: boolean   // default true  — index AGENTS.md, CLAUDE.md, .cursorrules, …
+  ingestTableHeaders?: boolean   // default true  — index CSV/TSV/XLSX column headers
+  ingestCrossRefs?: boolean      // default true  — grammar-agnostic cross-file edge discovery
+  crossRefsRarityThreshold?: number // default 3 — max files a symbol can appear in to count
+  enableNudgeHook?: boolean      // default true  (see WIKI: Compatibility)
+  adaptive?: boolean             // default true  (see WIKI: Adaptive sizing)
+  enableSemanticSearch?: boolean // default false (see WIKI: Semantic search)
+  embeddingModel?: string        // default "Xenova/multilingual-e5-small"
+  personalizedPageRank?: boolean // default false (co-change ranking; see WIKI)
+  recordSessionActivity?: boolean      // default true  — record this session's edits + bash as a rolling memory
+  bashFileTrackingMaxFiles?: number    // default 20    — refresh code-map for files a bash call touched (0 = off)
+  autoReingestGitOnHeadChange?: boolean // default true — re-ingest git when bash moves HEAD (pull/merge/rebase)
+}
+```
+
+With `adaptive` on (the default), prefill measures one cheap signal —
+commit count, or file count when there is no git — and scales the
+history depth and code-map file cap to the repo's size. An explicit
+value in your config always wins.
+
+`maxMemoryDiskMB` is the disk budget for the memory store: once it is
+exceeded, least-used memories are evicted (pinned ones never). The
+default is 50 MB — generous enough that a realistic store (even on a
+large repo, ~4–6 MB) never hits it, so eviction acts as a safety valve
+rather than a routine clip. Note it also bounds RAM: the search index
+is held in memory at roughly 70 MB of heap per 1 MB of budget *if
+filled*, so on an unusually large monorepo, lower it to cap memory or
+raise it for a deeper store. See *Performance* in the WIKI.
+
+`enableSemanticSearch` (default **off**) adds opt-in cross-lingual
+recall: with it on, the plugin also embeds memories with a small
+multilingual e5 model and fuses vector similarity with the BM25
+ranking, so a query in one language (Russian, Chinese, …) can retrieve
+code and comments written in another. It needs the optional
+`@huggingface/transformers` dependency (`bun add @huggingface/transformers`);
+the model downloads on first use. When off — the default — no model is
+downloaded, the dependency is never loaded, and retrieval is the
+unchanged lexical path. See *Semantic search* in the WIKI.
+
+## Learn more
+
+[WIKI.md](./WIKI.md) covers everything else, including:
+
+- *Straight answers for a decision-maker* — the questions above, in depth
+- *How the memory is structured* — the record shape and the hierarchy, with diagrams
+- *The pillars* — retrieval, prefill, code-health, and the rest, with diagrams
+- *How it compares* — versus embeddings, aider's repo-map, and `AGENTS.md`
+- *Without git history* — what works, what doesn't, and why
+- *Semantic search* — the opt-in cross-lingual embedding feature
+- *Token savings* — what reduction to expect, and how it is measured
+- *Performance* & *Scaling* — measured numbers, and the honest heap caveat
+- *Code map*, *Session snapshots*, *Skill mining*, *Rich logs*, *Tests & CI*
+
+## License
+
+MIT.