dsh-engramory 0.1.0 → 0.2.0

package/README.md

@@ -1,8 +1,10 @@
 # dsh-engramory
 
 Curated, file-based long-term memory for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) —
-plain markdown notes you can read and `git log`, one store shared across every host you
-use, and an index cap that is a **real refusal** rather than a request.
+plain markdown notes you can open with anything, one store shared across every host you
+use, and an index cap that is a **real refusal** rather than a request. (If the store
+lives inside a project's git repo it must be git-ignored — memories carry machine-local
+detail; see SKILL.md §1. A *dedicated private* repo for the store itself is fine.)
 
 Part of [Engramory](https://github.com/tinqiao-oss/engramory).
 
@@ -16,7 +18,10 @@ because the index loads every session and everything past the cap silently stops
 recalled. On most hosts that limit is rules plus a checker the agent has to remember to
 run. dsh exposes `ctx.tools.guard()` — a synchronous, **monotonic** refusal: once a guard
 returns a reason, no later listener can turn it back into an allow. So here the limit is
-enforced, not requested. Outside Claude Code, this is the only host where that is true.
+enforced, not requested — **once the plugin is actually installed and running**; a
+profile install currently fails upstream (see "Known limits"), and until then the cap on
+dsh is still rules plus the checker. Outside Claude Code, this is the only host where a
+deterministic cap is even possible.
 
 **The protocol arrives with the plugin.** `ctx.skills.register()` contributes the skill at
 runtime, so it does not depend on landing files in one of the five skill roots dsh scans —
@@ -28,15 +33,22 @@ a path that is easy to get subtly wrong and fails silently when you do.
 dsh plugin --profile <name> add dsh-engramory
 ```
 
-Then mount it in that profile's `cordis.patch.yml`:
+That is the whole install. The package ships a `dsh.bundle` manifest pointing at its own
+`cordis.patch.yml`, so the row is inserted into the profile's plugin tree for you — no
+hand-editing. To change a default, patch the EXISTING row by id in your profile's own
+patch layer — do **not** use a second `- insert:` (insert always appends, so you would
+end up with two engramory rows and the original caps still enforced):
 
 ```yaml
 - id: engramory
-  name: dsh-engramory
   config:
-    indexName: MEMORY.md   # optional
+    indexName: MEMORY.md
+    maxLines: 200
+    maxBytes: 25600
 ```
 
+A patch replaces the targeted row's whole `config`, so list every key you mean to keep.
+
 For the store itself and the always-on block, use the installer in the Engramory repo
 (`python tools/engramory_init.py dsh --install-skill`). This plugin enforces the cap and
 supplies the protocol; it does not create the store.
@@ -45,7 +57,7 @@ supplies the protocol; it does not create the store.
 
 | Field | Default | Meaning |
 |---|---|---|
-| `indexName` | `MEMORY.md` | Basename treated as the memory index. Nothing else is inspected. |
+| `indexName` | `MEMORY.md` | Basename treated as the memory index (case-insensitive; both `file_path` and `str_replace_editor`'s `path` are checked). An empty or non-string value falls back to the default. Nothing else is inspected. |
 | `maxLines` | `200` | Hard line cap. Non-positive or non-finite values fall back to the default. |
 | `maxBytes` | `25600` | Hard UTF-8 byte cap (25 KB). |
 | `registerSkill` | `true` | Set `false` to keep the cap but skip the runtime skill. |
@@ -55,24 +67,34 @@ supplies the protocol; it does not create the store.
 
 | Call | Decision |
 |---|---|
-| `write` to the index, within caps | allow |
-| `write` to the index, over either cap | **deny**, naming the numbers and what to compact |
-| Partial edit of an index that is already over cap | **deny** — do not grow a breach |
-| Partial edit of a healthy index | allow |
+| `write` (or `str_replace_editor` `create`) ending within caps | allow |
+| `write`/`create` that GROWS the index past a cap | **deny**, naming the numbers and what to compact |
+| `write`/`create` that SHRINKS or keeps an over-cap index | allow — incremental compaction (210 → 205 → 198) must stay possible |
+| Edit carrying `old_str`/`new_str`: result simulated | judged by the RESULT, same grow/shrink rule as a write |
+| Unsimulable partial (e.g. `insert`) on an over-cap index | **deny**, telling the agent a shrinking whole-file write passes |
+| Unsimulable partial on a healthy index | allow |
+| `read`, `view`, or any unknown tool | allow — recall must never be blocked, even over cap |
 | Any write to any other file | allow — not the guard's business |
-| Missing/unreadable index, malformed arguments | allow — a guard must never block work by accident |
+| Missing/unreadable current index on a whole-file write | treated as EMPTY (mirrors the Python guard) — a within-caps write passes, an over-cap first write is refused |
+| Missing/unreadable index on a partial edit | allow — a guard must never block work over a path it cannot read |
+| Malformed execution (no arguments, non-string path/content) | allow — not recognisably a write |
 
-The line count ignores a trailing newline, matching `hooks/engramory_index_guard.py`, so
-an index sitting exactly at the cap stays writable.
+The deny rule mirrors `hooks/engramory_index_guard.py` exactly: refuse only a result
+that is over a cap AND grew past the current file. The line count ignores a trailing
+newline, so an index sitting exactly at the cap stays writable.
 
 ## Known limits
 
-- **An edit that crosses the cap from under it is not caught.** A partial write does not
-  carry the resulting text, and a synchronous guard has no business reconstructing it.
-  The breach is caught by the next whole-file write, or by `engramory_check.py`. What is
-  guaranteed here is that an already-over index cannot be grown further.
+- **An unsimulable partial that crosses the cap from under it is not caught.** `insert`
+  and friends do not carry the resulting text. The breach is caught by the next
+  whole-file write, or by `engramory_check.py`. What is guaranteed: an already-over
+  index cannot be grown further, and a shrinking write always passes.
+- **The tool roster follows dsh's documented tool-fs contract** (`write`/`edit` with
+  `file_path`, `str_replace_editor` with `path`) and is deliberately conservative:
+  unknown tools pass. Until the plugin runs inside a real profile this mapping is
+  documentation-verified only.
 - **Not yet verified end to end inside dsh.** The guard's decision table is covered by
-  `node --test` (13 cases, run in Engramory's CI), and the wiring for the AGENTS.md block
+  `node --test` (20 cases, run in Engramory's CI), and the wiring for the AGENTS.md block
   and skill discovery was dogfooded against a live `deepseek-v4-flash` session. But
   installing a *third-party plugin* into a profile currently fails on dsh 0.1.0-rc.6:
   `dsh plugin` shells out to `pnpm` (not bundled), and a direct `pnpm add` then dies on

package/cordis.patch.yml

@@ -0,0 +1,16 @@
+# The dsh-engramory bundle patch: one row, inserted over whatever layers came before
+# it. A patch replaces the targeted row's whole `config`, so a profile overriding this
+# row must restate every key it means to keep.
+#
+# The numbers below are the protocol's own caps, not this port's opinion: 200 lines /
+# 25 KB is what keeps MEMORY.md cheap enough to load at the start of every session, and
+# it is the same limit hooks/engramory_index_guard.py enforces on Claude Code. They live
+# here, beside the row, so a deployment can raise or lower them without touching code.
+
+- insert:
+    - id: engramory
+      name: dsh-engramory
+      config:
+        indexName: MEMORY.md
+        maxLines: 200
+        maxBytes: 25600

package/index.js

@@ -13,6 +13,13 @@
  *      sidesteps the five-root scan entirely, so the protocol is present because the
  *      plugin is loaded, not because a directory happened to be right.
  *
+ * The decision table mirrors hooks/engramory_index_guard.py: deny only a write that
+ * ENDS over a cap AND grew past the current file — a shrinking rewrite always passes,
+ * so an over-cap index can be compacted incrementally (210 → 205 → 198). Only known
+ * mutating tools are gated; `read` and unknown tools are never refused (the first cut
+ * gated everything that named the index, which blocked recall of an over-cap index at
+ * exactly the moment compaction was needed).
+ *
  * Zero dependencies, no build step: plain ESM, node: builtins only.
  */
 import { readFileSync } from 'node:fs'
@@ -25,15 +32,25 @@ export const name = 'engramory'
 export const inject = { required: ['tools'], optional: ['skills'] }
 
 /** Mirrors hooks/engramory_index_guard.py — the caps are the protocol's, not this port's. */
+const DEFAULT_INDEX_NAME = 'MEMORY.md'
 const DEFAULT_MAX_LINES = 200
 const DEFAULT_MAX_BYTES = 25600
 
-/** Tools that replace a file wholesale, so `arguments.content` IS the post-write text. */
+/** Tools that replace a file wholesale, so the post-write text is in the arguments. */
 const WHOLE_FILE_WRITES = new Set(['write'])
 
+/**
+ * Tools that mutate a file in place. Only tools named here (plus
+ * `str_replace_editor`'s write commands, handled explicitly) count as partial
+ * writes. Unknown tools pass through: this guard is a discipline rail, not a
+ * security boundary (SECURITY.md), and a false refusal of `read` costs more than a
+ * miss — the next whole-file write is measured exactly regardless.
+ */
+const PARTIAL_WRITES = new Set(['edit', 'str_replace', 'insert'])
+
 export function apply(ctx, config = {}) {
   const settings = {
-    indexName: config.indexName ?? 'MEMORY.md',
+    indexName: indexNameOf(config.indexName),
     maxLines: positive(config.maxLines, DEFAULT_MAX_LINES),
     maxBytes: positive(config.maxBytes, DEFAULT_MAX_BYTES),
   }
@@ -62,52 +79,127 @@ export function apply(ctx, config = {}) {
  *
  * Guards are synchronous by contract, so this stays cheap: one basename comparison
  * rejects the overwhelming majority of calls before anything is measured, and the
- * only I/O is a single read of the index on a partial write.
+ * only I/O is a single read of the index on a gated write.
  */
 function refuseOversizedIndex(exec, settings) {
   const args = exec?.arguments
   if (!args || typeof args !== 'object') return undefined
 
-  const filePath = args.file_path
-  if (typeof filePath !== 'string' || basename(filePath) !== settings.indexName) {
+  // dsh's file tools carry the target as `file_path`; `str_replace_editor` calls it
+  // `path`. Compare basenames case-INsensitively: on the case-insensitive
+  // filesystems most stores live on (Windows/macOS), `memory.md` IS the guarded
+  // index, and an exact compare let that spelling through. (Same trade-off as the
+  // Python guard: on a case-sensitive FS this can gate an unrelated lowercase twin —
+  // rename it, or point `indexName` elsewhere.)
+  const filePath = typeof args.file_path === 'string' ? args.file_path
+    : typeof args.path === 'string' ? args.path
+      : undefined
+  if (filePath === undefined) return undefined
+  if (basename(filePath).toLowerCase() !== settings.indexName.toLowerCase()) {
     return undefined
   }
 
-  if (WHOLE_FILE_WRITES.has(exec.name)) {
+  const tool = typeof exec.name === 'string' ? exec.name : ''
+
+  if (WHOLE_FILE_WRITES.has(tool)) {
     if (typeof args.content !== 'string') return undefined
-    return verdictFor(args.content, filePath, settings)
+    return verdict(args.content, readIfPossible(filePath), filePath, settings)
+  }
+  if (tool === 'str_replace_editor' && args.command === 'create') {
+    if (typeof args.file_text !== 'string') return undefined
+    return verdict(args.file_text, readIfPossible(filePath), filePath, settings)
   }
 
-  // A partial write (an edit / replace) does not carry the resulting text, and a
-  // guard must not do expensive work to reconstruct it. The honest, cheap rule: if
-  // the index is ALREADY over, refuse to grow it further. An edit that crosses the
-  // line from under it still gets through here — engramory_check.py is the backstop,
-  // and the next whole-file write is caught exactly.
-  const current = readIfPossible(filePath)
-  if (current === undefined) return undefined
+  const partial = PARTIAL_WRITES.has(tool)
+    || (tool === 'str_replace_editor'
+      && (args.command === 'str_replace' || args.command === 'insert'))
+  if (!partial) return undefined // read / view / list / unknown: never refused
+
+  const currentBuf = readIfPossible(filePath)
+  if (currentBuf === undefined) return undefined
+  const current = currentBuf.toString('utf8')
+  const simulated = simulateEdit(current, args)
+  if (simulated !== undefined) {
+    return verdict(simulated, currentBuf, filePath, settings)
+  }
+
+  // The edit's result genuinely cannot be reconstructed from its arguments. Honest,
+  // cheap rule: if the index is ALREADY over, refuse to mutate it blind — and name
+  // the open door, because "compact first" while refusing edits reads as a dead end.
   const breach = measure(current, settings)
   if (!breach) return undefined
   return (
-    `${settings.indexName} is already over the Engramory cap (${breach}). ` +
-    `Editing it further would push it deeper past the limit. Compact it first: ` +
-    `pointer-ify over-long lines, merge duplicates, and archive cold notes — the ` +
-    `index is a table of contents, and anything past the cap silently stops being ` +
-    `recalled.`
+    `${settings.indexName} is already over the Engramory cap (${breach}) and this ` +
+    `edit's result cannot be measured. Compact it with a whole-file write instead: ` +
+    `a rewrite that SHRINKS the index always passes, even while still over the cap. ` +
+    `Pointer-ify over-long lines, merge duplicates, archive cold notes — the index ` +
+    `is a table of contents, and anything past the cap silently stops being recalled.`
   )
 }
 
-function verdictFor(text, filePath, settings) {
-  const breach = measure(text, settings)
-  if (!breach) return undefined
+/**
+ * Mirrors the Python guard's deny rule: refuse only when a dimension ends OVER its
+ * cap AND grew past the current file, so a shrinking/keeping write on an over-cap
+ * index always passes and incremental compaction stays possible. A missing or
+ * unreadable current file counts as empty — a first write past the cap is refused.
+ */
+function verdict(text, currentBuf, filePath, settings) {
+  const lines = countLines(text)
+  const bytes = Buffer.byteLength(text, 'utf8')
+  // Current byte size comes from the RAW buffer, mirroring how the Python guard
+  // sizes the on-disk file: decoding first inflated a non-UTF-8 index ~3x (every
+  // bad byte becomes a 3-byte U+FFFD), which made a genuinely growing write look
+  // like a shrink and pass. Newlines survive a lossy decode, so the line count may
+  // use the decoded text.
+  const curLines = currentBuf === undefined ? 0 : countLines(currentBuf.toString('utf8'))
+  const curBytes = currentBuf === undefined ? 0 : currentBuf.length
+  const over = []
+  if (lines > settings.maxLines && lines > curLines) {
+    over.push(`${lines} lines > ${settings.maxLines}`)
+  }
+  if (bytes > settings.maxBytes && bytes > curBytes) {
+    over.push(`${bytes} bytes > ${settings.maxBytes}`)
+  }
+  if (!over.length) return undefined
   return (
     `This write would put ${basename(filePath)} over the Engramory index cap ` +
-    `(${breach}). The index is loaded every session and the host only reads so far, ` +
-    `so anything past the cap silently stops being recalled. Compact before writing: ` +
-    `move detail into the linked note files, merge duplicates, archive cold notes, ` +
-    `and keep every line to "one short hook + link".`
+    `(${over.join(', ')}). The index is loaded every session and the host only reads ` +
+    `so far, so anything past the cap silently stops being recalled. Compact before ` +
+    `writing: move detail into the linked note files, merge duplicates, archive cold ` +
+    `notes, and keep every line to "one short hook + link". A write that SHRINKS the ` +
+    `index always passes, so you can compact step by step.`
   )
 }
 
+/**
+ * Reconstruct a partial edit's post-write text when its arguments carry enough to
+ * do it (`old_str`/`new_str`, or the Claude-style `old_string`/`new_string`).
+ * Mirrors the Python guard's Edit simulation: replace the unique occurrence, or all
+ * of them under a replace-all flag; an absent or ambiguous old-string means the
+ * real tool errors and changes nothing, so the current text is the honest
+ * prediction. Returns undefined when the shape isn't recognised.
+ */
+function simulateEdit(current, args) {
+  const oldStr = firstString(args.old_str, args.old_string)
+  if (oldStr === undefined || oldStr === '') return undefined
+  const newStr = firstString(args.new_str, args.new_string) ?? ''
+  if (args.replace_all === true || args.replaceAll === true) {
+    return current.split(oldStr).join(newStr)
+  }
+  const first = current.indexOf(oldStr)
+  if (first === -1) return current // the real tool errors: nothing changes
+  const second = current.indexOf(oldStr, first + oldStr.length)
+  if (second !== -1) return current // ambiguous: the real tool errors, nothing changes
+  return current.slice(0, first) + newStr + current.slice(first + oldStr.length)
+}
+
+function firstString(...values) {
+  for (const v of values) {
+    if (typeof v === 'string') return v
+  }
+  return undefined
+}
+
 /** Returns a human-readable breach description, or `null` when the text is within caps. */
 function measure(text, { maxLines, maxBytes }) {
   const lines = countLines(text)
@@ -126,18 +218,31 @@ function countLines(text) {
 
 /**
  * A missing or unreadable index is not a breach: the guard's job is to stop an
- * oversized index, never to block work because a path could not be read.
+ * oversized index, never to block work because a path could not be read. Returns the
+ * RAW buffer — byte comparisons must use on-disk bytes (see verdict), decode at the
+ * call sites that need text.
  */
 function readIfPossible(filePath) {
   try {
-    return readFileSync(filePath, 'utf8')
+    return readFileSync(filePath)
   } catch {
     return undefined
   }
 }
 
 function positive(value, fallback) {
-  return Number.isFinite(value) && value > 0 ? Math.floor(value) : fallback
+  if (!Number.isFinite(value)) return fallback
+  const floored = Math.floor(value)
+  return floored > 0 ? floored : fallback
+}
+
+/**
+ * An empty or non-string indexName must fall back, not silently disable the cap:
+ * the numeric caps already recover from nonsense values, and this field is the one
+ * that decides whether the guard fires at all.
+ */
+function indexNameOf(value) {
+  return typeof value === 'string' && value.trim() ? value.trim() : DEFAULT_INDEX_NAME
 }
 
 /**
@@ -194,9 +299,10 @@ function builtinSkillBody() {
     '',
     '## The cap is enforced here',
     '',
-    'This host denies a write that would push the index past 200 lines / 25 KB. That is',
-    'a real refusal, not a warning: compact first — the index is loaded every session,',
-    'and anything past the cap silently stops being recalled.',
+    'This host denies a write that would GROW the index past 200 lines / 25 KB. That is',
+    'a real refusal, not a warning — but a write that shrinks the index always passes,',
+    'so compact first, step by step if needed: the index is loaded every session, and',
+    'anything past the cap silently stops being recalled.',
     '',
     'Never write credentials, keys, tokens, or cookies into memory — record only where',
     'the secret lives.',

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "dsh-engramory",
-  "version": "0.1.0",
-  "description": "Curated file-based long-term memory for DeepSeek Harness — human-readable markdown notes, one git-versioned store shared across hosts, and a deterministic index cap enforced by ctx.tools.guard()",
+  "version": "0.2.0",
+  "description": "Curated file-based long-term memory for DeepSeek Harness — human-readable markdown notes, one plain-file store shared across hosts (git-ignored when it lives inside a project repo), and a deterministic index cap enforced by ctx.tools.guard()",
   "keywords": [
     "dsh-plugin",
     "deepseek-harness",
@@ -14,12 +14,18 @@
   ],
   "type": "module",
   "main": "index.js",
+  "dsh": {
+    "bundle": {
+      "patch": "./cordis.patch.yml"
+    }
+  },
   "exports": {
     ".": "./index.js",
     "./package.json": "./package.json"
   },
   "files": [
     "index.js",
+    "cordis.patch.yml",
     "README.md",
     "LICENSE"
   ],