@try-works/dsh-memory 0.1.27 → 0.1.28

package/STATUS.md

@@ -3,8 +3,8 @@
 A tracker and planner: what exists, what is verified working, what is written but not
 loaded, and what is left. Kept as a record of MEASURED state, not intention.
 
-**Last measured: 2026-10-12** (408 tests passing; plugin at 0.1.26, published and installed
-byte-identical to source; store committed and pushed to `origin`).
+**Last measured: 2026-10-17** (408 tests passing; plugin at 0.1.27, published and installed
+byte-identical and confirmed LOADED by the running host; queue terminal-clean).
 
 ## How to read this
 
@@ -68,24 +68,28 @@ Observed in the running host. Each row names the effect that was seen, not a cla
 | Git sync works live | after the 0.1.22+ boot the log shows real `sync` events, `state/sync.json` records `ok:true remote:https://github.com/try-works/dsh-memory-store` (2026-10-11T10:34:54Z), and `git log` in the store holds `memory(store): update [dsh-memory]` commits — the first syncs this plugin has ever performed |
 | History ingest works live | `history-check ok=true` entries after the boot, including `count=1` at 09:37Z — a commit was actually ingested; the hourly `null.run` error era is over |
 
-## WRITTEN but NOT LOADED
+## Shipped and LOADED
 
-Implemented and tested on disk. The running host predates them, so none of this is in effect.
+Formerly "WRITTEN but NOT LOADED". Every row below was verified present in the **installed**
+0.1.27 package (grepped in `profiles/web/node_modules/@try-works/dsh-memory`), not merely in the
+source tree — that distinction is the whole point of this document. One row used to sit here as
+"what remains pending"; nothing does now.
 
-| change | why it matters | how it was verified on disk |
+| change | why it matters | how it was verified |
 |---|---|---|
 | `due()` priority ordering | retry → abandon → work → recover, so a bounded batch reaches a due retry | measured: the retry sat at position 101 of 111 and is now position 1 |
-| `hostExtras(host)` as one source | `index.js` and the wiring test shared a copied list that drifted | the test now derives extras from the host object |
-| `memory_recover` report omits its `entries` field rather than blanking it | the old line added `entries: undefined`, which is NOT valid JSON, so the harness rejected the whole result | the new guard round-trips every tool output; reintroducing it fails with `value.entries is undefined` |
-| Publisher stops when the harness does not stamp `ignorable` (0.1.21) | ≤0.1.19 appended `memory/map` WITHOUT the marker into ten real session logs, so every later load of those sessions failed ("unknown to this harness and not marked ignorable"). The guard checks the RETURN VALUE of `session.append` and suppresses publishing for that session at the first refusal, instead of poisoning the log | 403 tests pass; the new test drives a harness-shaped stub that returns an unstamped event and asserts the next turn appends nothing, even with a changed store and a passed throttle |
-| Git runner resolves the subprocess service lazily (0.1.22) | `makeGitRunner` probed `ctx.get('subprocess')` ONCE at wire time; cordis `get()` only returns services whose providing fiber is ACTIVE (vendor/cordis reflect.ts), so during mount it returns undefined, git stayed null forever, and every git path was dead: zero sync events ever recorded, and `history-check` threw `Cannot read properties of null (reading 'run')` hourly since 2026-10-06. The runner now re-probes at use and reports an honest reason while absent | 405 tests pass; the new test pins no-probe-at-mount, an honest refusal while the service is absent, and working ingest after it appears |
-| `classifyGitResult` reads `exitCode`, the name the runner actually returns (0.1.23-0.1.25) | the doc promised `code` and the code read `code`, but the injected runner returns `exitCode` — every successful git command was classified as a failure, so pull/push always reported failed and sync "had never once succeeded". `memory_sync` also stopped counting local commits as `synced: true` | sync tests drive the runner through the real classifier; an empty remote is asserted as the first push, not a failed pull |
-| `memory_recover` run path gets real services (0.1.26) | `recoverNow.run` called `recoverOne(store, services, ...)` where `services` was NEVER DEFINED — a ReferenceError the catch quietly turned into "1 declined or refused", so 75 recoverable candidates sat behind a tool that could only refuse them (observed live: limit:1 rebuilt 0). Services are now resolved at call time (`sessionQuery`), and the summary uses the plugin's own memory-model ladder | 408 tests pass; the new test executes the REAL run path through the wiring: plan → requeue → readSession → capture → digest file written, queue leaves ORPHANED |
-
-How to tell loaded from not: ask the host. `memory_recover` used to reply *"unavailable in this
-context"* and now replies *"returned invalid output: value is not lossless JSON"*. The change of
-ERROR is the proof the first fix is loaded and the tool now executes — a different failure, one
-layer further along. The report fix in the last row is what remains pending.
+| `hostExtras(host)` as one source | `index.js` and the wiring test shared a copied list that drifted | the test derives extras from the host object |
+| `memory_recover` report omits its `entries` field rather than blanking it | the old line added `entries: undefined`, which is NOT valid JSON, so the harness rejected the whole result | the guard round-trips every tool output; reintroducing it fails with `value.entries is undefined` |
+| Publisher stops when the harness does not stamp `ignorable` (0.1.21) | ≤0.1.19 appended `memory/map` WITHOUT the marker into ten real session logs, so every later load of those sessions failed ("unknown to this harness and not marked ignorable"). The guard checks the RETURN VALUE of `session.append` and suppresses publishing for that session at the first refusal, instead of poisoning the log | the test drives a harness-shaped stub that returns an unstamped event and asserts the next turn appends nothing, even with a changed store and a passed throttle |
+| Git runner resolves the subprocess service lazily (0.1.22) | `makeGitRunner` probed `ctx.get('subprocess')` ONCE at wire time; cordis `get()` only returns services whose providing fiber is ACTIVE, so during mount it returns undefined, git stayed null forever, and every git path was dead: zero sync events ever recorded, and `history-check` threw `Cannot read properties of null (reading 'run')` hourly from 2026-10-06. The runner now re-probes at use and reports an honest reason while absent | the test pins no-probe-at-mount, an honest refusal while the service is absent, and working ingest after it appears; live proof: real `sync` events and `state/sync.json` ok:true |
+| `classifyGitResult` reads `exitCode`, the name the runner actually returns (0.1.23-0.1.25) | the doc promised `code` and the code read `code`, but the injected runner returns `exitCode` — every successful git command was classified as a failure, so pull/push always reported failed and sync "had never once succeeded". `memory_sync` also stopped counting local commits as `synced: true` | sync tests drive the runner through the real classifier; an empty remote is asserted as the first push, not a failed pull; live proof: the store pushed to GitHub |
+| `memory_recover` run path gets real services (0.1.26) | `recoverNow.run` called `recoverOne(store, services, ...)` where `services` was NEVER DEFINED — a ReferenceError the catch quietly turned into "1 declined or refused", so 75 recoverable candidates sat behind a tool that could only refuse them | the test executes the REAL run path through the wiring: plan → requeue → readSession → capture → digest file written, queue leaves ORPHANED; live proof: all 75 recovered |
+| Consolidation never sees a header-only digest (0.1.27) | `runConsolidation` batched `withContent.concat(headerOnly)`, so a digest with no summary section went to the model as a bare header. The reply could only be "nothing durable", which never advanced the entry, so the same empty shells were re-sent every cadence — one entry reached `consolidateAttempts: 39`. Capture writes summaries; consolidation now only reads them | the test asserts the model is never called for a header-only digest |
+
+How to tell loaded from not: ask the host, and read the **installed** file rather than the source
+tree. A version string and a successful `install_bundle` are both insufficient — on 2026-10-12 a
+fix was committed but the version was not bumped, so the install was a no-op and the host kept
+running the old code while the source tree looked correct.
 ### The tasks layer (new since the last restart)
 
 A fifth layer, `tasks`, global only: `global/tasks/` holds one JSON file per DSH scheduled task
@@ -94,8 +98,9 @@ your automated tasks and is in no git repository. The host DELETES a task when i
 stops, so a task can vanish with nothing to observe; the backup runs on the sync cadence and a
 restore re-arms what is missing at the first session after boot.
 
-Verified on disk: 396 tests pass, and `@try-works/dsh-memory@0.1.16` is published and installed
-byte-identical.
+Verified on disk: 408 tests pass, and `@try-works/dsh-memory@0.1.27` is published and installed
+byte-identical. (The layer landed at 0.1.16; that figure was left in place here long after it
+stopped being the current release.)
 
 - `backupTasks` against `E:\dsh-memory`: two cron tasks captured, a second pass `written=0`.
 - The layer is at `global/tasks/`. It first shipped at `<root>/tasks`; every test used a

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@try-works/dsh-memory",
-  "version": "0.1.27",
+  "version": "0.1.28",
   "description": "Git-backed, workspace-scoped memory for the DeepSeek Harness (global layer + per-workspace layers)",
   "type": "module",
   "main": "src/index.js",
@@ -38,6 +38,7 @@
   },
   "peerDependencies": {
     "@deepseek-ai/cordis": "^4.0.1",
+    "@deepseek-ai/dsh-session": "^0.2.1-rc.2",
     "@deepseek-ai/dsh-tools": "^0.2.0-rc.2",
     "@deepseek-ai/dsh-llm": "^0.2.0-rc.2"
   },

package/scripts/orphan-for-recovery.ts

@@ -11,5 +11,14 @@ if (!id) throw new Error('usage: orphan-for-recovery.ts <root> <id>')
 
 const entry = readQueue(ROOT).get(id)
 if (!entry) throw new Error(id + ' is not in the queue')
+const from = entry.state
+// ORPHANED is only reachable from PENDING (and FAILED/ABANDONED). A candidate parked in HELD
+// must be released back to PENDING first, which is legal, then orphaned. Do the shortest legal
+// walk rather than guessing at the transition graph.
+if (from === STATES.HELD) {
+  transition(ROOT, id, STATES.PENDING)
+} else if (from !== STATES.PENDING && from !== STATES.FAILED && from !== STATES.ABANDONED) {
+  throw new Error(`${id} is ${from}, which has no legal route to orphaned`)
+}
 transition(ROOT, id, STATES.ORPHANED, { reason: REASONS.DIGEST_GONE })
-console.log(`orphaned ${id} (was ${entry.state}); now run memory_recover({ id: '${id}' })`)
+console.log(`orphaned ${id} (was ${from}); now run memory_recover({ id: '${id}' })`)

package/scripts/probe-open.ts

@@ -0,0 +1,13 @@
+import { Context } from "@deepseek-ai/cordis"
+import { SessionStore } from "@deepseek-ai/dsh-session"
+import JsonlSessionPersistence from "@deepseek-ai/dsh-session-persistence-jsonl"
+const ctx = new Context()
+await ctx.plugin(SessionStore)
+await ctx.plugin(JsonlSessionPersistence, { root: "C:/Users/erikb/.dsh/sessions", compression: "zstd" })
+const p = ctx.get("sessionPersistence")
+const id = "session-b21ecc7e-7743-4b06-b07f-453541dbcde5"
+try {
+  const h = await p.open(id, "read")
+  const snap = await h.read()
+  console.log("REAL OPEN OK — events:", snap?.events?.length ?? snap?.log?.length ?? "(shape)")
+} catch (e) { console.log("REAL OPEN FAILED:", String(e.message).slice(0, 300)) }

package/scripts/probe-rows.ts

@@ -0,0 +1,16 @@
+import { readFileSync } from "node:fs"
+import { scanZstdFrames, decompressZstdFrame, decompressZstdPrefix } from "file:///D:/deepseek-harness/packages/session/session-persistence-jsonl/src/zstd.ts"
+import { validateStoredEvents } from "file:///D:/deepseek-harness/packages/session/session-persistence/src/storage-contract.ts"
+const buf = readFileSync(process.argv[2])
+const scan = scanZstdFrames(buf)
+let text = ""
+for (const f of scan.frames) text += (await decompressZstdFrame(buf.subarray(f.start, f.end))).toString("utf8")
+if (scan.tornStart !== undefined) text += (await decompressZstdPrefix(buf.subarray(scan.tornStart))).toString("utf8")
+const rows = text.split("\n").filter(Boolean).map(l => JSON.parse(l))
+const meta = rows[0]; const events = rows.slice(1)
+const dups = new Map(); for (const e of events) dups.set(e.seq, (dups.get(e.seq) ?? 0) + 1)
+console.log("events:", events.length, "| duplicate seqs:", [...dups].filter(([,n]) => n > 1).slice(0, 5))
+const bad = events.filter(e => e.type === "memory/map" && e.ignorable !== true)
+console.log("memory/map without marker:", bad.length)
+try { validateStoredEvents(meta, events, { kind: "jsonl", path: process.argv[2] }); console.log("VALIDATOR: OK") }
+catch (e) { console.log("VALIDATOR THREW:", String(e.message).slice(0, 300)) }

package/src/client/index.js

@@ -400,7 +400,13 @@ window.__ModuleLoader__.load({
         // The host PUBLISHES the map into the log itself, so the tab fills in on its own and needs
         // nobody to call a tool first. This is the route that matters; the tool/result route below
         // is kept so a session whose snapshots predate the publisher still draws.
-        if (event.type === 'memory/map') {
+        //
+        // BOTH TYPE NAMES. Harness 0.2.1 writes out-of-repo records through appendPluginRecord,
+        // which names them under the `plugin:` grammar; older builds wrote the bare `memory/map`
+        // type, and a V3 log migrated to V4 is renamed into `plugin:` but a V4 log written by an
+        // old build keeps the bare name. Accepting only one of them makes the tab silently empty
+        // for half the session history.
+        if (event.type === 'plugin:memory/map' || event.type === 'memory/map') {
           const published = event.data
           if (published === null || typeof published !== 'object') continue
           if (typeof published.generatedAt !== 'string' || typeof published.scope !== 'string') continue

package/src/host.js

@@ -3,6 +3,10 @@
 import { existsSync, mkdirSync, rmSync, writeFileSync } from 'node:fs'
 import { join } from 'node:path'
 import { BlockAssembler, ReasoningEffortId, createUserMessage } from '@deepseek-ai/dsh-llm'
+// Namespace import on purpose: an older harness does not export appendPluginRecord, and a named
+// import of a missing ESM export fails at link time — which would break the whole plugin load
+// instead of falling back to the append-with-ignorable path.
+import * as dshSession from '@deepseek-ai/dsh-session'
 import { captureSession, pendingDigests, pruneDigests } from './capture.js'
 import { runConsolidation } from './consolidate-run.js'
 import { runRefresh } from './refresh.js'
@@ -807,14 +811,14 @@ export function wireHost(ctx, store, cfg) {
       if (!force && lastMapSignature.get(session.id) === signature) return
       lastMapSignature.set(session.id, signature)
       // Out-of-repo event types are absent from KNOWN_SESSION_EVENT_TYPES; without
-      // ignorable:true the persistence read path refuses the whole session log.
-      // The append returns the logged event: if the harness did NOT stamp
-      // ignorable onto it (a harness older than the option), the very next load
-      // of this session would fail with "unknown to this harness and not marked
-      // ignorable". Detecting that from the return value lets us stop before a
-      // log is poisoned, instead of discovering it when history fails to load.
-      const logged = session.append('memory/map', payload, { ignorable: true })
-      if (logged?.ignorable !== true) {
+      // ignorable:true the persistence read path refuses the whole session log. Harness 0.2.1
+      // moved that contract: `Session.append` no longer accepts an `ignorable` option at all
+      // (its third argument is surface metadata, valid only for surface event types), and an
+      // out-of-repo type is written through `appendPluginRecord`, which stamps `ignorable: true`
+      // by construction under the `plugin:` namespace. Both paths are handled by appendMapRecord;
+      // if the harness refuses to stamp the marker we suppress publishing rather than poison the
+      // log, so a later load cannot fail with "unknown to this harness and not marked ignorable".
+      if (!appendMapRecord(session, payload)) {
         if (!mapPublishSuppressed.has(session.id)) {
           mapPublishSuppressed.add(session.id)
           logRun(store.root, 'map-publish', { ok: false, error: 'harness did not stamp ignorable; publishing suppressed for this session' })
@@ -828,6 +832,28 @@ export function wireHost(ctx, store, cfg) {
     }
   }
 
+  /**
+   * Write the map snapshot into the session log as an ignorable record.
+   *
+   * Two harness contracts, chosen at call time rather than captured at wire time:
+   *  - 0.2.1+: `appendPluginRecord(session, 'plugin:memory/map', data)`; the harness owns the
+   *    envelope and always stamps `ignorable: true`, which is exactly what out-of-repo records
+   *    require.
+   *  - older: `session.append(type, data, { ignorable: true })`, whose RETURN VALUE must be
+   *    checked — a harness that ignores the option returns an event without the marker, and
+   *    every further append would poison the log.
+   * @returns whether the record was written with the ignorable marker.
+   */
+  function appendMapRecord(session, payload) {
+    const appendRecord = cfg.appendPluginRecord !== undefined ? cfg.appendPluginRecord : dshSession.appendPluginRecord
+    if (typeof appendRecord === 'function') {
+      appendRecord(session, 'plugin:memory/map', payload)
+      return true
+    }
+    const logged = session.append('memory/map', payload, { ignorable: true })
+    return logged?.ignorable === true
+  }
+
   function maybeSync() {
     if (syncInFlight || !git.available()) return
     if (!scheduler.shouldCommit() && !scheduler.shouldSync()) return