dsh-engramory 0.1.1 → 0.2.1

package/README.md

@@ -1,8 +1,10 @@
 # dsh-engramory
 
 Curated, file-based long-term memory for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) —
-plain markdown notes you can read and `git log`, one store shared across every host you
-use, and an index cap that is a **real refusal** rather than a request.
+plain markdown notes you can open with anything, one store shared across every host you
+use, and an index cap that is a **real refusal** rather than a request. (If the store
+lives inside a project's git repo it must be git-ignored — memories carry machine-local
+detail; see SKILL.md §1. A *dedicated private* repo for the store itself is fine.)
 
 Part of [Engramory](https://github.com/tinqiao-oss/engramory).
 
@@ -16,11 +18,17 @@ because the index loads every session and everything past the cap silently stops
 recalled. On most hosts that limit is rules plus a checker the agent has to remember to
 run. dsh exposes `ctx.tools.guard()` — a synchronous, **monotonic** refusal: once a guard
 returns a reason, no later listener can turn it back into an allow. So here the limit is
-enforced, not requested. Outside Claude Code, this is the only host where that is true.
-
-**The protocol arrives with the plugin.** `ctx.skills.register()` contributes the skill at
-runtime, so it does not depend on landing files in one of the five skill roots dsh scans —
-a path that is easy to get subtly wrong and fails silently when you do.
+enforced, not requested. Use **0.2.1 or later**: 0.2.0 shipped before profile installs
+worked upstream and carried an `inject` declaration this Cordis does not accept — it
+installed but never activated (issue #8). Outside Claude Code, dsh is the first host with
+the shim actually written (a few others expose equivalent pre-write seams — see
+PORTING.md — but have no shim yet).
+
+**The protocol arrives with the plugin.** The skill is registered at runtime through
+dsh's skill registry — via a reactive `ctx.inject(['skills'], …)` child, so a profile
+without a registry still gets the cap and one whose registry activates late still gets
+the skill — which means it does not depend on landing files in one of the five skill
+roots dsh scans, a path that is easy to get subtly wrong and fails silently when you do.
 
 ## Install
 
@@ -30,16 +38,16 @@ dsh plugin --profile <name> add dsh-engramory
 
 That is the whole install. The package ships a `dsh.bundle` manifest pointing at its own
 `cordis.patch.yml`, so the row is inserted into the profile's plugin tree for you — no
-hand-editing. To change a default, restate the row in your profile's own patch layer:
+hand-editing. To change a default, patch the EXISTING row by id in your profile's own
+patch layer — do **not** use a second `- insert:` (insert always appends, so you would
+end up with two engramory rows and the original caps still enforced):
 
 ```yaml
-- insert:
-    - id: engramory
-      name: dsh-engramory
-      config:
-        indexName: MEMORY.md
-        maxLines: 200
-        maxBytes: 25600
+- id: engramory
+  config:
+    indexName: MEMORY.md
+    maxLines: 200
+    maxBytes: 25600
 ```
 
 A patch replaces the targeted row's whole `config`, so list every key you mean to keep.
@@ -52,7 +60,7 @@ supplies the protocol; it does not create the store.
 
 | Field | Default | Meaning |
 |---|---|---|
-| `indexName` | `MEMORY.md` | Basename treated as the memory index. Nothing else is inspected. |
+| `indexName` | `MEMORY.md` | Basename treated as the memory index (case-insensitive; both `file_path` and `str_replace_editor`'s `path` are checked). An empty or non-string value falls back to the default. Nothing else is inspected. |
 | `maxLines` | `200` | Hard line cap. Non-positive or non-finite values fall back to the default. |
 | `maxBytes` | `25600` | Hard UTF-8 byte cap (25 KB). |
 | `registerSkill` | `true` | Set `false` to keep the cap but skip the runtime skill. |
@@ -62,32 +70,47 @@ supplies the protocol; it does not create the store.
 
 | Call | Decision |
 |---|---|
-| `write` to the index, within caps | allow |
-| `write` to the index, over either cap | **deny**, naming the numbers and what to compact |
-| Partial edit of an index that is already over cap | **deny** — do not grow a breach |
-| Partial edit of a healthy index | allow |
+| `write` (or `str_replace_editor` `create`) ending within caps | allow |
+| `write`/`create` that GROWS the index past a cap | **deny**, naming the numbers and what to compact |
+| `write`/`create` that SHRINKS or keeps an over-cap index | allow — incremental compaction (210 → 205 → 198) must stay possible |
+| Edit carrying `old_str`/`new_str`: result simulated | judged by the RESULT, same grow/shrink rule as a write |
+| Unsimulable partial (e.g. `insert`) on an over-cap index | **deny**, telling the agent a shrinking whole-file write passes |
+| Unsimulable partial on a healthy index | allow |
+| `read`, `view`, or any unknown tool | allow — recall must never be blocked, even over cap |
 | Any write to any other file | allow — not the guard's business |
-| Missing/unreadable index, malformed arguments | allow — a guard must never block work by accident |
+| Missing/unreadable current index on a whole-file write | treated as EMPTY (mirrors the Python guard) — a within-caps write passes, an over-cap first write is refused |
+| Missing/unreadable index on a partial edit | allow — a guard must never block work over a path it cannot read |
+| Malformed execution (no arguments, non-string path/content) | allow — not recognisably a write |
 
-The line count ignores a trailing newline, matching `hooks/engramory_index_guard.py`, so
-an index sitting exactly at the cap stays writable.
+The deny rule mirrors `hooks/engramory_index_guard.py` exactly: refuse only a result
+that is over a cap AND grew past the current file. The line count ignores a trailing
+newline, so an index sitting exactly at the cap stays writable.
 
 ## Known limits
 
-- **An edit that crosses the cap from under it is not caught.** A partial write does not
-  carry the resulting text, and a synchronous guard has no business reconstructing it.
-  The breach is caught by the next whole-file write, or by `engramory_check.py`. What is
-  guaranteed here is that an already-over index cannot be grown further.
-- **Not yet verified end to end inside dsh.** The guard's decision table is covered by
-  `node --test` (13 cases, run in Engramory's CI), and the wiring for the AGENTS.md block
-  and skill discovery was dogfooded against a live `deepseek-v4-flash` session. But
-  installing a *third-party plugin* into a profile currently fails on dsh 0.1.0-rc.6:
-  `dsh plugin` shells out to `pnpm` (not bundled), and a direct `pnpm add` then dies on
-  `ERR_PNPM_FETCH_404` for `@deepseek-ai/dsh-type-meta`, a package the published tree
-  depends on but which is not on the registry. That is a preview-packaging problem
-  upstream, not a plugin defect; this note goes away once a profile install resolves.
+- **An unsimulable partial that crosses the cap from under it is not caught.** `insert`
+  and friends do not carry the resulting text. The breach is caught by the next
+  whole-file write, or by `engramory_check.py`. What is guaranteed: an already-over
+  index cannot be grown further, and a shrinking write always passes.
+- **The tool roster follows dsh's documented tool-fs contract** (`write`/`edit` with
+  `file_path`, `str_replace_editor` with `path`) and is deliberately conservative:
+  unknown tools pass.
+- **0.2.0 never activated; 0.2.1 is the first version that runs.** While the rc.6
+  preview bug blocked all third-party profile installs upstream, the only coverage a
+  plugin could have was mocks — and 0.2.0's mock hid two activation bugs: an
+  older-Cordis `{ required, optional }` inject shape (read as waiting for services
+  literally named `required`/`optional`, pending forever) and a bare `ctx.skills` read
+  of an undeclared service (throws under Cordis' reflective context). The first field
+  install caught both the moment installs became possible (issue #8). 0.2.1 fixes them,
+  verified end to end on a live dsh 0.1.0-rc.7 web profile (0.2.0 reproduces the boot
+  failure byte for byte; 0.2.1 boots and serves) and against the vendored
+  `@deepseek-ai/cordis` resolver (activation with, without, and with a late-mounted
+  skill registry). The test mock now mirrors the reflective-context access rules, and
+  the guard's decision table stays covered by `node --test` (22 cases, run in
+  Engramory's CI).
 - dsh is a developer preview and its plugin API can change. This plugin deliberately
-  touches only `ctx.tools.guard()` and `ctx.skills.register()` so it stays cheap to fix.
+  touches only `ctx.tools.guard()` and a reactive `ctx.inject(['skills'], …)` child
+  that registers the skill, so it stays cheap to fix.
 
 ## License
 

package/index.js

@@ -7,12 +7,20 @@
  *      monotonic refusal — once a guard returns a reason, no later waterfall
  *      listener can turn it back into an allow. That makes it a stronger seam than
  *      most hosts expose, and it is why the 200-line / 25 KB limit can be enforced
- *      here rather than merely asked for. Everywhere except Claude Code, Engramory's
- *      cap degrades to "rules plus a checker the agent has to remember to run".
+ *      here rather than merely asked for. Without a shim like this one, everywhere
+ *      except Claude Code the cap degrades to "rules plus a checker the agent has to
+ *      remember to run".
  *   2. Skill delivery that does not depend on install paths. Registering at runtime
  *      sidesteps the five-root scan entirely, so the protocol is present because the
  *      plugin is loaded, not because a directory happened to be right.
  *
+ * The decision table mirrors hooks/engramory_index_guard.py: deny only a write that
+ * ENDS over a cap AND grew past the current file — a shrinking rewrite always passes,
+ * so an over-cap index can be compacted incrementally (210 → 205 → 198). Only known
+ * mutating tools are gated; `read` and unknown tools are never refused (the first cut
+ * gated everything that named the index, which blocked recall of an over-cap index at
+ * exactly the moment compaction was needed).
+ *
  * Zero dependencies, no build step: plain ESM, node: builtins only.
  */
 import { readFileSync } from 'node:fs'
@@ -20,39 +28,65 @@ import { basename } from 'node:path'
 
 export const name = 'engramory'
 
-// `tools` carries the cap and is required. `skills` is optional so the plugin still
-// loads (and still caps) on a profile that mounts no skill registry.
-export const inject = { required: ['tools'], optional: ['skills'] }
+// `tools` carries the cap and is the plugin's only declared dependency. Cordis reads
+// `inject` as an array of service names (or an object KEYED by them) and treats every
+// entry as a hard wait — the `{ required, optional }` shape is older-Cordis syntax that
+// the build dsh vendors resolves as services literally named "required"/"optional",
+// which never appear, so the plugin sat pending forever (issue #8). `skills` is
+// deliberately NOT declared: the cap must mount even on a profile with no skill
+// registry, so apply() probes it at runtime instead.
+export const inject = ['tools']
 
 /** Mirrors hooks/engramory_index_guard.py — the caps are the protocol's, not this port's. */
+const DEFAULT_INDEX_NAME = 'MEMORY.md'
 const DEFAULT_MAX_LINES = 200
 const DEFAULT_MAX_BYTES = 25600
 
-/** Tools that replace a file wholesale, so `arguments.content` IS the post-write text. */
+/** Tools that replace a file wholesale, so the post-write text is in the arguments. */
 const WHOLE_FILE_WRITES = new Set(['write'])
 
+/**
+ * Tools that mutate a file in place. Only tools named here (plus
+ * `str_replace_editor`'s write commands, handled explicitly) count as partial
+ * writes. Unknown tools pass through: this guard is a discipline rail, not a
+ * security boundary (SECURITY.md), and a false refusal of `read` costs more than a
+ * miss — the next whole-file write is measured exactly regardless.
+ */
+const PARTIAL_WRITES = new Set(['edit', 'str_replace', 'insert'])
+
 export function apply(ctx, config = {}) {
   const settings = {
-    indexName: config.indexName ?? 'MEMORY.md',
+    indexName: indexNameOf(config.indexName),
     maxLines: positive(config.maxLines, DEFAULT_MAX_LINES),
     maxBytes: positive(config.maxBytes, DEFAULT_MAX_BYTES),
   }
 
   ctx.tools.guard((exec) => refuseOversizedIndex(exec, settings))
 
-  if (ctx.skills && config.registerSkill !== false) {
-    const skill = config.skill ?? builtinSkillBody()
-    ctx.skills.register({
-      name: 'engramory',
-      description:
-        'Curated file-based long-term memory: recall through MEMORY.md at the start of ' +
-        'a task, save durable user/feedback/project/reference facts, and sync before ' +
-        'compacting or opening a fresh thread.',
-      whenToUse:
-        'Starting or resuming work, learning something durable worth a future session, ' +
-        'or approaching a compact/clear/new-thread boundary.',
-      source: 'runtime',
-      content: skill,
+  // The skill registry is optional AND may activate after this plugin — fiber order
+  // between unrelated providers is not guaranteed, so a one-shot probe here would
+  // silently skip registration on such boots. ctx.inject() is Cordis' reactive form:
+  // the callback runs once `skills` is available (however late), re-runs if the
+  // registry reloads, and never makes the cap wait. A bare `ctx.skills` read at THIS
+  // level would throw (undeclared service — issue #8's second crash); inside the
+  // callback the service is declared, so the read is legal.
+  if (config.registerSkill !== false) {
+    ctx.inject(['skills'], (inner) => {
+      const skill = config.skill ?? builtinSkillBody()
+      // effect() ties the registration to the child fiber: it is disposed when the
+      // registry unloads, so a reload cannot accumulate duplicate registrations.
+      inner.effect(() => inner.skills.register({
+        name: 'engramory',
+        description:
+          'Curated file-based long-term memory: recall through MEMORY.md at the start of ' +
+          'a task, save durable user/feedback/project/reference facts, and sync before ' +
+          'compacting or opening a fresh thread.',
+        whenToUse:
+          'Starting or resuming work, learning something durable worth a future session, ' +
+          'or approaching a compact/clear/new-thread boundary.',
+        source: 'runtime',
+        content: skill,
+      }))
     })
   }
 }
@@ -62,52 +96,127 @@ export function apply(ctx, config = {}) {
  *
  * Guards are synchronous by contract, so this stays cheap: one basename comparison
  * rejects the overwhelming majority of calls before anything is measured, and the
- * only I/O is a single read of the index on a partial write.
+ * only I/O is a single read of the index on a gated write.
  */
 function refuseOversizedIndex(exec, settings) {
   const args = exec?.arguments
   if (!args || typeof args !== 'object') return undefined
 
-  const filePath = args.file_path
-  if (typeof filePath !== 'string' || basename(filePath) !== settings.indexName) {
+  // dsh's file tools carry the target as `file_path`; `str_replace_editor` calls it
+  // `path`. Compare basenames case-INsensitively: on the case-insensitive
+  // filesystems most stores live on (Windows/macOS), `memory.md` IS the guarded
+  // index, and an exact compare let that spelling through. (Same trade-off as the
+  // Python guard: on a case-sensitive FS this can gate an unrelated lowercase twin —
+  // rename it, or point `indexName` elsewhere.)
+  const filePath = typeof args.file_path === 'string' ? args.file_path
+    : typeof args.path === 'string' ? args.path
+      : undefined
+  if (filePath === undefined) return undefined
+  if (basename(filePath).toLowerCase() !== settings.indexName.toLowerCase()) {
     return undefined
   }
 
-  if (WHOLE_FILE_WRITES.has(exec.name)) {
+  const tool = typeof exec.name === 'string' ? exec.name : ''
+
+  if (WHOLE_FILE_WRITES.has(tool)) {
     if (typeof args.content !== 'string') return undefined
-    return verdictFor(args.content, filePath, settings)
+    return verdict(args.content, readIfPossible(filePath), filePath, settings)
+  }
+  if (tool === 'str_replace_editor' && args.command === 'create') {
+    if (typeof args.file_text !== 'string') return undefined
+    return verdict(args.file_text, readIfPossible(filePath), filePath, settings)
   }
 
-  // A partial write (an edit / replace) does not carry the resulting text, and a
-  // guard must not do expensive work to reconstruct it. The honest, cheap rule: if
-  // the index is ALREADY over, refuse to grow it further. An edit that crosses the
-  // line from under it still gets through here — engramory_check.py is the backstop,
-  // and the next whole-file write is caught exactly.
-  const current = readIfPossible(filePath)
-  if (current === undefined) return undefined
+  const partial = PARTIAL_WRITES.has(tool)
+    || (tool === 'str_replace_editor'
+      && (args.command === 'str_replace' || args.command === 'insert'))
+  if (!partial) return undefined // read / view / list / unknown: never refused
+
+  const currentBuf = readIfPossible(filePath)
+  if (currentBuf === undefined) return undefined
+  const current = currentBuf.toString('utf8')
+  const simulated = simulateEdit(current, args)
+  if (simulated !== undefined) {
+    return verdict(simulated, currentBuf, filePath, settings)
+  }
+
+  // The edit's result genuinely cannot be reconstructed from its arguments. Honest,
+  // cheap rule: if the index is ALREADY over, refuse to mutate it blind — and name
+  // the open door, because "compact first" while refusing edits reads as a dead end.
   const breach = measure(current, settings)
   if (!breach) return undefined
   return (
-    `${settings.indexName} is already over the Engramory cap (${breach}). ` +
-    `Editing it further would push it deeper past the limit. Compact it first: ` +
-    `pointer-ify over-long lines, merge duplicates, and archive cold notes — the ` +
-    `index is a table of contents, and anything past the cap silently stops being ` +
-    `recalled.`
+    `${settings.indexName} is already over the Engramory cap (${breach}) and this ` +
+    `edit's result cannot be measured. Compact it with a whole-file write instead: ` +
+    `a rewrite that SHRINKS the index always passes, even while still over the cap. ` +
+    `Pointer-ify over-long lines, merge duplicates, archive cold notes — the index ` +
+    `is a table of contents, and anything past the cap silently stops being recalled.`
   )
 }
 
-function verdictFor(text, filePath, settings) {
-  const breach = measure(text, settings)
-  if (!breach) return undefined
+/**
+ * Mirrors the Python guard's deny rule: refuse only when a dimension ends OVER its
+ * cap AND grew past the current file, so a shrinking/keeping write on an over-cap
+ * index always passes and incremental compaction stays possible. A missing or
+ * unreadable current file counts as empty — a first write past the cap is refused.
+ */
+function verdict(text, currentBuf, filePath, settings) {
+  const lines = countLines(text)
+  const bytes = Buffer.byteLength(text, 'utf8')
+  // Current byte size comes from the RAW buffer, mirroring how the Python guard
+  // sizes the on-disk file: decoding first inflated a non-UTF-8 index ~3x (every
+  // bad byte becomes a 3-byte U+FFFD), which made a genuinely growing write look
+  // like a shrink and pass. Newlines survive a lossy decode, so the line count may
+  // use the decoded text.
+  const curLines = currentBuf === undefined ? 0 : countLines(currentBuf.toString('utf8'))
+  const curBytes = currentBuf === undefined ? 0 : currentBuf.length
+  const over = []
+  if (lines > settings.maxLines && lines > curLines) {
+    over.push(`${lines} lines > ${settings.maxLines}`)
+  }
+  if (bytes > settings.maxBytes && bytes > curBytes) {
+    over.push(`${bytes} bytes > ${settings.maxBytes}`)
+  }
+  if (!over.length) return undefined
   return (
     `This write would put ${basename(filePath)} over the Engramory index cap ` +
-    `(${breach}). The index is loaded every session and the host only reads so far, ` +
-    `so anything past the cap silently stops being recalled. Compact before writing: ` +
-    `move detail into the linked note files, merge duplicates, archive cold notes, ` +
-    `and keep every line to "one short hook + link".`
+    `(${over.join(', ')}). The index is loaded every session and the host only reads ` +
+    `so far, so anything past the cap silently stops being recalled. Compact before ` +
+    `writing: move detail into the linked note files, merge duplicates, archive cold ` +
+    `notes, and keep every line to "one short hook + link". A write that SHRINKS the ` +
+    `index always passes, so you can compact step by step.`
   )
 }
 
+/**
+ * Reconstruct a partial edit's post-write text when its arguments carry enough to
+ * do it (`old_str`/`new_str`, or the Claude-style `old_string`/`new_string`).
+ * Mirrors the Python guard's Edit simulation: replace the unique occurrence, or all
+ * of them under a replace-all flag; an absent or ambiguous old-string means the
+ * real tool errors and changes nothing, so the current text is the honest
+ * prediction. Returns undefined when the shape isn't recognised.
+ */
+function simulateEdit(current, args) {
+  const oldStr = firstString(args.old_str, args.old_string)
+  if (oldStr === undefined || oldStr === '') return undefined
+  const newStr = firstString(args.new_str, args.new_string) ?? ''
+  if (args.replace_all === true || args.replaceAll === true) {
+    return current.split(oldStr).join(newStr)
+  }
+  const first = current.indexOf(oldStr)
+  if (first === -1) return current // the real tool errors: nothing changes
+  const second = current.indexOf(oldStr, first + oldStr.length)
+  if (second !== -1) return current // ambiguous: the real tool errors, nothing changes
+  return current.slice(0, first) + newStr + current.slice(first + oldStr.length)
+}
+
+function firstString(...values) {
+  for (const v of values) {
+    if (typeof v === 'string') return v
+  }
+  return undefined
+}
+
 /** Returns a human-readable breach description, or `null` when the text is within caps. */
 function measure(text, { maxLines, maxBytes }) {
   const lines = countLines(text)
@@ -126,18 +235,31 @@ function countLines(text) {
 
 /**
  * A missing or unreadable index is not a breach: the guard's job is to stop an
- * oversized index, never to block work because a path could not be read.
+ * oversized index, never to block work because a path could not be read. Returns the
+ * RAW buffer — byte comparisons must use on-disk bytes (see verdict), decode at the
+ * call sites that need text.
  */
 function readIfPossible(filePath) {
   try {
-    return readFileSync(filePath, 'utf8')
+    return readFileSync(filePath)
   } catch {
     return undefined
   }
 }
 
 function positive(value, fallback) {
-  return Number.isFinite(value) && value > 0 ? Math.floor(value) : fallback
+  if (!Number.isFinite(value)) return fallback
+  const floored = Math.floor(value)
+  return floored > 0 ? floored : fallback
+}
+
+/**
+ * An empty or non-string indexName must fall back, not silently disable the cap:
+ * the numeric caps already recover from nonsense values, and this field is the one
+ * that decides whether the guard fires at all.
+ */
+function indexNameOf(value) {
+  return typeof value === 'string' && value.trim() ? value.trim() : DEFAULT_INDEX_NAME
 }
 
 /**
@@ -194,9 +316,10 @@ function builtinSkillBody() {
     '',
     '## The cap is enforced here',
     '',
-    'This host denies a write that would push the index past 200 lines / 25 KB. That is',
-    'a real refusal, not a warning: compact first — the index is loaded every session,',
-    'and anything past the cap silently stops being recalled.',
+    'This host denies a write that would GROW the index past 200 lines / 25 KB. That is',
+    'a real refusal, not a warning — but a write that shrinks the index always passes,',
+    'so compact first, step by step if needed: the index is loaded every session, and',
+    'anything past the cap silently stops being recalled.',
     '',
     'Never write credentials, keys, tokens, or cookies into memory — record only where',
     'the secret lives.',

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "dsh-engramory",
-  "version": "0.1.1",
-  "description": "Curated file-based long-term memory for DeepSeek Harness — human-readable markdown notes, one git-versioned store shared across hosts, and a deterministic index cap enforced by ctx.tools.guard()",
+  "version": "0.2.1",
+  "description": "Curated file-based long-term memory for DeepSeek Harness — human-readable markdown notes, one plain-file store shared across hosts (git-ignored when it lives inside a project repo), and a deterministic index cap enforced by ctx.tools.guard()",
   "keywords": [
     "dsh-plugin",
     "deepseek-harness",