@martian-engineering/lossless-claw 0.13.0 → 0.13.2

package/docs/configuration.md

@@ -84,6 +84,7 @@ Most installations only need to override a handful of keys. If you want a comple
   "enableSummaryThinking": true,
   "maxAssemblyTokenBudget": 30000,
   "summaryMaxOverageFactor": 3,
+  "fallbackMaxTokens": 512,
   "customInstructions": "",
   "circuitBreakerThreshold": 5,
   "circuitBreakerCooldownMs": 1800000,
@@ -141,13 +142,19 @@ Notes on the example:
 Install with OpenClaw's plugin installer:
 
 ```bash
-openclaw plugins install @martian-engineering/lossless-claw
+openclaw plugins install @martian-engineering/lossless-claw@latest
 ```
 
 If you are running from a local OpenClaw checkout:
 
 ```bash
-pnpm openclaw plugins install @martian-engineering/lossless-claw
+pnpm openclaw plugins install @martian-engineering/lossless-claw@latest
+```
+
+Use exact versions only for rollback or reproducible canary testing. OpenClaw treats an exact install spec such as `@martian-engineering/lossless-claw@0.12.0` as pinned, so plugin update sync will not follow newer LCM releases until you return to the moving track:
+
+```bash
+openclaw plugins update @martian-engineering/lossless-claw@latest
 ```
 
 For local plugin development, link a working copy:
@@ -176,17 +183,17 @@ openclaw plugins install --link /path/to/lossless-claw
 | `enableSummaryThinking` | `boolean` | `true` | `LCM_ENABLE_SUMMARY_THINKING` | When true, requests low reasoning budget from the model during summarization calls. Set to false to disable reasoning and keep summarization output concise. |
 | `proactiveThresholdCompactionMode` | `"deferred" \| "inline"` | `"deferred"` | `LCM_PROACTIVE_THRESHOLD_COMPACTION_MODE` | Controls whether proactive threshold compaction is deferred into maintenance debt by default or run inline for legacy behavior. |
 | `autoRotateSessionFiles.enabled` | `boolean` | `true` | `LCM_AUTO_ROTATE_SESSION_FILES_ENABLED` | Enables automatic rotation for oversized LCM-managed session JSONL files. |
-| `autoRotateSessionFiles.createBackups` | `boolean` | `false` | `LCM_AUTO_ROTATE_SESSION_FILES_CREATE_BACKUPS` | Creates or replaces the rolling `rotate-latest` SQLite backup before automatic session-file rotation. Manual `/lcm rotate` backups are always created. |
+| `autoRotateSessionFiles.createBackups` | `boolean` | `false` | `LCM_AUTO_ROTATE_SESSION_FILES_CREATE_BACKUPS` | Creates or replaces the rolling `rotate-latest` SQLite backup before automatic session-file rotation. Manual `/lossless rotate` backups are always created. |
 | `autoRotateSessionFiles.sizeBytes` | `integer` | `2097152` | `LCM_AUTO_ROTATE_SESSION_FILES_SIZE_BYTES` | Byte threshold that triggers automatic session-file rotation. |
 | `autoRotateSessionFiles.startup` | `"rotate" \| "warn" \| "off"` | `"rotate"` | `LCM_AUTO_ROTATE_SESSION_FILES_STARTUP` | Startup behavior for oversized indexed OpenClaw session transcripts that also have active LCM bootstrap state. |
-| `autoRotateSessionFiles.runtime` | `"rotate" \| "warn" \| "off"` | `"rotate"` | `LCM_AUTO_ROTATE_SESSION_FILES_RUNTIME` | Runtime behavior after post-turn checks. Runtime `rotate` logs deferral for active session JSONL rewrites and leaves direct rotation to startup or manual `/lcm rotate`. |
+| `autoRotateSessionFiles.runtime` | `"rotate" \| "warn" \| "off"` | `"rotate"` | `LCM_AUTO_ROTATE_SESSION_FILES_RUNTIME` | Runtime behavior after post-turn checks. Runtime `rotate` logs deferral for active session JSONL rewrites and leaves direct rotation to startup or manual `/lossless rotate`. |
 | `independentLogFile.enabled` | `boolean` | `true` | `LCM_LOG_FILE_ENABLED` | Writes lossless-claw JSONL logs to an independent plugin-owned file in addition to OpenClaw's runtime logger. |
 | `independentLogFile.file` | `string` | `/tmp/openclaw/lossless-claw-YYYY-MM-DD.log` | `LCM_LOG_FILE` | Optional log path. A dated `lossless-claw-YYYY-MM-DD.log` path rolls over daily. |
 | `independentLogFile.maxFileBytes` | `integer` | `104857600` | `LCM_LOG_MAX_FILE_BYTES` | Size threshold for rotating the active lossless-claw log file to `.1.log` through `.5.log`. |
 
 > **Multi-profile note:** `OPENCLAW_STATE_DIR` (set by the host OpenClaw gateway) controls where state is stored. When two gateways run on the same host (e.g. separate bot personas), each gateway sets its own `OPENCLAW_STATE_DIR` and lossless-claw automatically uses that directory for the database, large-file payloads, auth-profile lookups, and legacy secrets — no per-profile plugin config is needed.
 
-Automatic session-file rotation rewrites only the live session transcript, keeps the active LCM conversation and durable history intact, and refreshes the bootstrap checkpoint. Before manual or startup rewrites, rotation forces leaf-only compaction for raw context outside the preserved tail so trimmed transcript messages are covered by LCM summaries without running unrelated summary-condensation passes. Startup rotation first scans OpenClaw's current indexed session stores for configured agents, then intersects those candidates with active LCM conversations and matching bootstrap file mappings. Runtime rotation checks from `afterTurn()` and `maintain()` intentionally do not directly rewrite active session JSONL because embedded prompt-lock fences can still be open while tool-call loops and host background maintenance overlap; runtime `rotate` logs a deferral until startup, manual `/lcm rotate`, or a future host-owned full-transcript rewrite primitive is available. Automatic rotation does not create a SQLite backup by default; set `autoRotateSessionFiles.createBackups` to `true` to make startup rotation create one pre-rotation LCM database backup for the batch before any transcript is rewritten. Manual `/lcm rotate` always keeps its backup-backed behavior regardless of this flag. Rotation never runs for ignored sessions, stateless sessions, or sessions without active LCM state. The preserved JSONL tail follows the existing rotate behavior, which is controlled by `freshTailCount`. Transcript GC uses the host-provided `rewriteTranscriptEntries` primitive and defers until host-approved background maintenance when `transcriptGcEnabled` is enabled.
+Automatic session-file rotation rewrites only the live session transcript, keeps the active LCM conversation and durable history intact, and refreshes the bootstrap checkpoint. Before manual or startup rewrites, rotation forces leaf-only compaction for raw context outside the preserved tail so trimmed transcript messages are covered by LCM summaries without running unrelated summary-condensation passes. Startup rotation first scans OpenClaw's current indexed session stores for configured agents, then intersects those candidates with active LCM conversations and matching bootstrap file mappings. Runtime rotation checks from `afterTurn()` and `maintain()` intentionally do not directly rewrite active session JSONL because embedded prompt-lock fences can still be open while tool-call loops and host background maintenance overlap; runtime `rotate` logs a deferral until startup, manual `/lossless rotate`, or a future host-owned full-transcript rewrite primitive is available. Automatic rotation does not create a SQLite backup by default; set `autoRotateSessionFiles.createBackups` to `true` to make startup rotation create one pre-rotation LCM database backup for the batch before any transcript is rewritten. Manual `/lossless rotate` always keeps its backup-backed behavior regardless of this flag. Rotation never runs for ignored sessions, stateless sessions, or sessions without active LCM state. The preserved JSONL tail follows the existing rotate behavior, which is controlled by `freshTailCount`. Transcript GC uses the host-provided `rewriteTranscriptEntries` primitive and defers until host-approved background maintenance when `transcriptGcEnabled` is enabled.
 
 Lossless-claw writes routine operational JSONL logs by default at `/tmp/openclaw/lossless-claw-YYYY-MM-DD.log`, beside OpenClaw's `/tmp/openclaw/openclaw-YYYY-MM-DD.log`. Routine info and debug lines go to the independent file instead of the shared OpenClaw log. Startup banners and warning/error lines still go through OpenClaw's runtime logger so gateway-level startup and failure diagnostics remain visible. The independent file follows the same practical rotation model as OpenClaw: a dated filename rolls over when the local date changes, stale dated files are pruned after 3 days, and an oversized active file is rotated through `.1.log` to `.5.log`.
 
@@ -216,6 +223,7 @@ Every automatic decision emits grep-able log lines prefixed with `[lcm] auto-rot
 | `leafTargetTokens` | `integer` | `2400` | `LCM_LEAF_TARGET_TOKENS` | Prompt target for leaf summary size. |
 | `condensedTargetTokens` | `integer` | `2000` | `LCM_CONDENSED_TARGET_TOKENS` | Prompt target for condensed summary size. |
 | `summaryMaxOverageFactor` | `number` | `3` | `LCM_SUMMARY_MAX_OVERAGE_FACTOR` | Hard ceiling multiplier before oversized summaries are deterministically truncated. |
+| `fallbackMaxTokens` | `integer` | `512` | `LCM_FALLBACK_MAX_TOKENS` | Maximum token budget for deterministic fallback summaries when the LLM summarizer is unavailable. Values below 64 are ignored. |
 | `largeFileThresholdTokens` | `integer` | `25000` | `LCM_LARGE_FILE_TOKEN_THRESHOLD` | Preferred key for the token threshold that routes text attachments into large-file summarization. |
 | `largeFileTokenThreshold` | `integer` | alias of `largeFileThresholdTokens` | `LCM_LARGE_FILE_TOKEN_THRESHOLD` | Legacy alias accepted by the runtime. Prefer `largeFileThresholdTokens` in new config. |
 | `maxAssemblyTokenBudget` | `integer` | unset | `LCM_MAX_ASSEMBLY_TOKEN_BUDGET` | Optional hard cap for assembly and threshold evaluation, useful with smaller-context models. |
@@ -380,19 +388,19 @@ Lossless-claw now defaults `proactiveThresholdCompactionMode` to `deferred`.
 - `assemble()` leaves pending threshold debt for after-turn background drain or host-approved `maintain()` while the live prompt is still within budget
 - `assemble()` only consumes pending threshold debt synchronously as an emergency safeguard when the live prompt estimate is already over the active token budget
 - old non-threshold debt from earlier builds is revalidated; if the conversation is no longer over threshold, it is cleared as a no-op
-- `/lcm status` / `/lossless status` shows the current maintenance state, including pending/running/last-failure details
+- `/lossless status` (`/lcm status` alias) shows the current maintenance state, including pending/running/last-failure details
 - status output also surfaces the latest API/cache telemetry as diagnostics, not as a deferral gate
 - set `proactiveThresholdCompactionMode` to `inline` only if you need the legacy inline proactive compaction behavior for compatibility
 
-### `/lcm rotate`
+### `/lossless rotate`
 
-`/lcm rotate` exists for a different use case than `/new` or `/reset`:
+`/lossless rotate` exists for a different use case than `/new` or `/reset`:
 
 - `/new` keeps the same active LCM conversation row and only prunes context.
 - `/reset` changes OpenClaw session flow, which is sometimes more disruptive than users want.
-- `/lcm rotate` keeps the live OpenClaw session identity and the same active LCM conversation row, but rewrites the backing transcript into a compact preserved-tail form.
+- `/lossless rotate` keeps the live OpenClaw session identity and the same active LCM conversation row, but rewrites the backing transcript into a compact preserved-tail form.
 
-Before rotating, Lossless-claw replaces one rolling `rotate-latest` SQLite backup. It then rewrites the current session transcript and checkpoints the same conversation at the new transcript frontier so bootstrap does not replay the dropped transcript history. Existing summaries, context items, and conversation identity stay in place. If you want additional timestamped snapshots, run `/lcm backup` explicitly before `/lcm rotate`.
+Before rotating, Lossless-claw replaces one rolling `rotate-latest` SQLite backup. It then rewrites the current session transcript and checkpoints the same conversation at the new transcript frontier so bootstrap does not replay the dropped transcript history. Existing summaries, context items, and conversation identity stay in place. If you want additional timestamped snapshots, run `/lossless backup` explicitly before `/lossless rotate`.
 
 ## Environment-only knobs outside plugin config
 
@@ -428,7 +436,7 @@ sqlite3 "${OPENCLAW_STATE_DIR:-$HOME/.openclaw}/lcm.db" ".backup ${OPENCLAW_STAT
 Or from a supported OpenClaw chat/native command surface:
 
 ```text
-/lcm backup
+/lossless backup
 ```
 
 ## Disabling lossless-claw

package/openclaw.plugin.json

@@ -5,6 +5,16 @@
   "activation": {
     "onStartup": true
   },
+  "commandAliases": [
+    {
+      "name": "lossless",
+      "kind": "runtime-slash"
+    },
+    {
+      "name": "lcm",
+      "kind": "runtime-slash"
+    }
+  ],
   "skills": [
     "skills/lossless-claw"
   ],
@@ -193,6 +203,10 @@
       "label": "Summary Max Overage Factor",
       "help": "Maximum allowed overage factor for summaries relative to target tokens (default 3). Summaries exceeding this are deterministically truncated."
     },
+    "fallbackMaxTokens": {
+      "label": "Fallback Max Tokens",
+      "help": "Maximum token budget for deterministic fallback summaries when the LLM summarizer fails (default 512, minimum 64)"
+    },
     "customInstructions": {
       "label": "Custom Instructions",
       "help": "Natural language instructions injected into all summarization prompts (e.g., formatting rules, tone control)"
@@ -515,6 +529,10 @@
         "type": "number",
         "minimum": 1
       },
+      "fallbackMaxTokens": {
+        "type": "integer",
+        "minimum": 64
+      },
       "customInstructions": {
         "type": "string"
       },

package/package.json

@@ -1,9 +1,12 @@
 {
   "name": "@martian-engineering/lossless-claw",
-  "version": "0.13.0",
+  "version": "0.13.2",
   "description": "Lossless Context Management plugin for OpenClaw — DAG-based conversation summarization with threshold compaction",
   "type": "module",
   "main": "dist/index.js",
+  "bin": {
+    "lossless-claw-migrate-sessions": "dist/migrate-sessions.js"
+  },
   "license": "MIT",
   "author": "Josh Lehman <josh@martian.engineering>",
   "keywords": [
@@ -16,7 +19,7 @@
     "dag"
   ],
   "scripts": {
-    "build": "esbuild index.ts --bundle --platform=node --target=node22 --format=esm --outfile=dist/index.js --external:openclaw --external:\"@earendil-works/*\" --minify-whitespace",
+    "build": "esbuild index.ts --bundle --platform=node --target=node22 --format=esm --outfile=dist/index.js --external:openclaw --external:\"@earendil-works/*\" --minify-whitespace && esbuild src/cli/migrate-sessions.ts --bundle --platform=node --target=node22 --format=esm --outfile=dist/migrate-sessions.js --external:openclaw --external:\"@earendil-works/*\" --minify-whitespace",
     "changeset": "changeset",
     "plugin-inspector:ci": "npm exec --yes --package @openclaw/plugin-inspector@0.3.11 -- plugin-inspector ci --plugin-root . --out plugin-inspector-reports",
     "release:verify": "npm run typecheck && npm run build && npm test && npm pack --dry-run",
@@ -39,13 +42,13 @@
   },
   "devDependencies": {
     "@changesets/changelog-github": "^0.6.0",
-    "@changesets/cli": "^2.30.0",
-    "@earendil-works/pi-agent-core": "^0.79.0",
-    "@earendil-works/pi-ai": "^0.79.0",
-    "@earendil-works/pi-coding-agent": "^0.79.0",
-    "esbuild": "^0.28.0",
+    "@changesets/cli": "^2.31.0",
+    "@earendil-works/pi-agent-core": "^0.79.8",
+    "@earendil-works/pi-ai": "^0.79.8",
+    "@earendil-works/pi-coding-agent": "^0.79.8",
+    "esbuild": "^0.28.1",
     "typescript": "^5.7.0",
-    "vitest": "^3.0.0"
+    "vitest": "^4.1.9"
   },
   "peerDependencies": {
     "openclaw": ">=2026.5.28"

package/skills/lossless-claw/SKILL.md

@@ -15,7 +15,8 @@ Start here:
 4. If they suspect summary corruption or truncation, use `/lossless doctor`.
 5. If they want high-confidence junk/session cleanup guidance, use `/lossless doctor clean` before recommending any deletes.
 6. If they ask how `/new`, `/reset`, or `/lossless rotate` interacts with LCM, read the session-lifecycle reference before answering.
-7. Load the relevant reference file instead of improvising details from memory.
+7. If they ask how to import old or past OpenClaw conversation data into Lossless, read the session-lifecycle reference before answering.
+8. Load the relevant reference file instead of improvising details from memory.
 
 Reference map:
 
@@ -23,13 +24,13 @@ Reference map:
 - Internal model and data flow: `references/architecture.md`
 - Diagnostics and summary-health workflow: `references/diagnostics.md`
 - Recall tools and when to use them: `references/recall-tools.md`
-- `/new`, `/reset`, and `/lossless rotate` behavior with current lossless-claw session mapping: `references/session-lifecycle.md`
+- `/new`, `/reset`, `/lossless rotate`, and past-session import behavior with current lossless-claw session mapping: `references/session-lifecycle.md`
 
 Working rules:
 
 - Prioritize explaining why a setting matters, not just what it does.
 - Prefer the native plugin command surface for MVP workflows (`/lossless`, with `/lcm` as alias).
 - Do not assume the Go TUI is installed.
-- Do not recommend advanced rewrite/backfill/transplant/dissolve flows unless the user explicitly asks for non-MVP internals.
+- Do not recommend advanced rewrite/backfill/transplant/dissolve flows unless the user explicitly asks for non-MVP internals. If the user specifically asks to import old or past OpenClaw conversation data into Lossless, recommend the packaged session migration CLI instead of TUI backfill/surgery flows.
 - For exact evidence retrieval from compacted history, guide the user toward recall tools instead of guessing from summaries.
 - When users compare `/lossless` to `/status`, explain that they report different layers: `/lossless` shows LCM-side frontier/summary metrics, while `/status` shows the last assembled runtime prompt snapshot.

package/skills/lossless-claw/references/architecture.md

@@ -29,7 +29,7 @@ Bad summaries do not stay local:
 
 That is why configuration choices around compaction thresholds and summary model quality matter operationally.
 
-## What `/lcm` tells you
+## What `/lossless` tells you
 
 The MVP command surface focuses on operational facts:
 
@@ -40,7 +40,7 @@ The MVP command surface focuses on operational facts:
 - total summarized source-token coverage when available
 - broken or truncated summary presence
 
-## What `/lcm doctor` tells you
+## What `/lossless doctor` tells you
 
 The MVP doctor flow is diagnostic only.
 
@@ -51,7 +51,7 @@ It looks for known summary-health markers that indicate:
 
 This gives users one place to answer the question “is my summary graph healthy?” without introducing a broader mutation surface.
 
-## What `/lcm doctor clean` tells you
+## What `/lossless doctor clean` tells you
 
 The cleaners flow is also diagnostic first.
 

package/skills/lossless-claw/references/config.md

@@ -347,7 +347,7 @@ Why it matters:
 - `deferred` is the default and avoids foreground turn stalls by recording one coalesced maintenance row per conversation
 - `deferred` also stores provider/model/cache telemetry so Anthropic-family sessions can avoid rewriting a still-hot prompt cache
 - `inline` preserves the legacy foreground compaction path for hosts that do not yet support deferred execution
-- `/lossless status` and `/lcm status` surface pending/running/last-failure maintenance state so operators can see when compaction is queued
+- `/lossless status` (`/lcm status` alias) surfaces pending/running/last-failure maintenance state so operators can see when compaction is queued
 - after-turn background drain and host-approved `maintain()` consume routine threshold debt; `assemble()` only drains pending threshold debt synchronously as an emergency safeguard when the live prompt estimate is already over budget
 
 ### `autoRotateSessionFiles`
@@ -365,7 +365,7 @@ Defaults:
 Why it matters:
 
 - prevents very large OpenClaw session JSONL files from choking fallback/gateway startup while LCM owns the durable context
-- runtime rotation only creates or replaces the rolling `rotate-latest` DB backup when `createBackups` is `true`; manual `/lossless rotate` / `/lcm rotate` always keeps its backup-backed behavior
+- runtime rotation only creates or replaces the rolling `rotate-latest` DB backup when `createBackups` is `true`; manual `/lossless rotate` always keeps its backup-backed behavior
 - runtime JSONL rewrites run from `afterTurn()` after the host turn completes; `maintain()` skips rotation and leaves it to `afterTurn()` or startup because background maintenance can overlap an embedded model call
 - startup scans OpenClaw's current indexed session stores for configured agents, intersects those candidates with active LCM bootstrap state, and creates one pre-rotation DB backup for the startup batch only when `createBackups` is `true`
 - only runs for active, writable LCM conversations; ignored sessions, stateless sessions, sessions outside the indexed startup candidate set, and sessions without active LCM state are skipped
@@ -721,6 +721,25 @@ Why it matters:
 - guards against runaway summaries that are much larger than their target budget
 - useful when summary models are verbose or unstable
 
+### `fallbackMaxTokens`
+
+| | |
+| --- | --- |
+| Type | `integer` |
+| Default | `512` |
+| Minimum | `64` |
+| Env | `LCM_FALLBACK_MAX_TOKENS` |
+
+Maximum token budget for deterministic fallback summaries when the LLM summarizer is unavailable.
+
+Why it matters:
+
+- when the LLM summarizer fails (auth errors, timeout, empty output), Lossless falls back to a purely local truncation-based summary
+- this fallback is bounded by `fallbackMaxTokens` so it cannot balloon the context
+- values below `64` are ignored so the required fallback marker can fit inside the configured budget
+- lower values produce more aggressive truncation; higher values preserve more source text at the cost of larger fallback summaries
+- the default `512` is conservative; raise it if you prefer richer fallback summaries over more aggressive truncation
+
 ### `summaryMaxCallsPerWindow`, `summaryCallWindowMs`, and `summarySpendBackoffMs`
 
 Bounds model-backed compaction and large-file summarization calls per session.
@@ -788,7 +807,7 @@ Design note: stripping happens at compaction time, not at message ingestion.  Th
 
 Useful interpretation notes:
 
-- `tokens in context` is the current LCM frontier token count in the live LCM state.
+- `LCM frontier tokens` is the current LCM frontier token count in the live LCM state.
 - `compression ratio` is shown as a rounded `1:N`, which is easier to read than a tiny percentage for heavily compacted conversations.
 - `/status` may still show a different context number because it reflects the runtime prompt that was actually assembled and sent on the last turn.
 

package/skills/lossless-claw/references/session-lifecycle.md

@@ -73,6 +73,40 @@ When answering users:
 - explain that cron scheduler session keys are isolated per runtime run while preserving archived prior runs
 - if they need a truly separate LCM history, use a different session key context (for example a different chat/thread/binding) or explicit non-MVP migration/surgery tools
 
+## Importing past conversation data
+
+If a user explicitly asks how to import old or past OpenClaw conversation data into Lossless, recommend the packaged session migration CLI:
+
+```bash
+npx --package @martian-engineering/lossless-claw@latest lossless-claw-migrate-sessions --state-dir ~/.openclaw
+```
+
+That command name, `lossless-claw-migrate-sessions`, is the npm package executable declared by lossless-claw. It backfills OpenClaw JSONL session files into `lcm.db`.
+
+Use it when:
+
+- lossless-claw was installed after useful OpenClaw sessions already existed
+- lossless-claw was disabled or missing for a period
+- the user wants historical JSONL sessions imported into Lossless storage
+
+Do not present it as:
+
+- a background replay loop
+- a summarization or embedding migration
+- a replacement for normal startup bootstrap/crash recovery
+- a general rewrite/transplant/surgery tool
+
+Safe recommendation pattern:
+
+1. Run the dry-run command first and inspect the output.
+2. Apply only after the user confirms the target state directory and import set:
+
+```bash
+npx --package @martian-engineering/lossless-claw@latest lossless-claw-migrate-sessions --state-dir ~/.openclaw --apply
+```
+
+The CLI defaults to `${OPENCLAW_STATE_DIR:-~/.openclaw}` and `${OPENCLAW_STATE_DIR:-~/.openclaw}/lcm.db`. On `--apply`, it creates a timestamped SQLite backup before writing when the database already exists. For narrow imports, suggest `--file <path>`, repeatable `--sessions-dir <path>`, `--since <iso-date>`, or `--limit <n>`.
+
 ## Relation to `/status`
 
 This session behavior is separate from `/status` metrics.