@martian-engineering/lossless-claw 0.5.2 → 0.6.0

package/README.md

@@ -7,6 +7,7 @@ Lossless Context Management plugin for [OpenClaw](https://github.com/openclaw/op
 - [What it does](#what-it-does)
 - [Quick start](#quick-start)
 - [Configuration](#configuration)
+- [Commands And Skill](#commands-and-skill)
 - [Documentation](#documentation)
 - [Development](#development)
 - [License](#license)
@@ -27,6 +28,16 @@ Nothing is lost. Raw messages stay in the database. Summaries link back to their
 
 **It feels like talking to an agent that never forgets. Because it doesn't. In normal operation, you'll never need to think about compaction again.**
 
+## Commands And Skill
+
+The plugin now ships a bundled `lossless-claw` skill plus a small native command surface:
+
+- `/lcm` shows version, enablement/selection state, DB path and size, summary counts, and summary-health status
+- `/lcm doctor` scans for broken or truncated summaries
+- `/lossless` is an alias for `/lcm` on native command surfaces
+
+The bundled skill focuses on configuration, diagnostics, architecture, and recall-tool usage. Its reference set lives under `skills/lossless-claw/references/`.
+
 ## Quick start
 
 ### Prerequisites
@@ -59,6 +70,8 @@ openclaw plugins install --link /path/to/lossless-claw
 
 The install command records the plugin, enables it, and applies compatible slot selection (including `contextEngine` when applicable).
 
+> **Note:** If your OpenClaw config uses `plugins.allow`, make sure both `lossless-claw` and any active plugins you rely on remain allowlisted. In some setups, narrowing the allowlist can prevent plugin-backed integrations from loading, even if `lossless-claw` itself is installed correctly. Restart the gateway after plugin config changes.
+
 ### Configure OpenClaw
 
 In most cases, no manual JSON edits are needed after `openclaw plugins install`.
@@ -92,14 +105,17 @@ Add a `lossless-claw` entry under `plugins.entries` in your OpenClaw config:
       "lossless-claw": {
         "enabled": true,
         "config": {
-          "freshTailCount": 32,
+          "freshTailCount": 64,
+          "leafChunkTokens": 80000,
+          "newSessionRetainDepth": 2,
           "contextThreshold": 0.75,
-          "incrementalMaxDepth": -1,
+          "incrementalMaxDepth": 1,
           "ignoreSessionPatterns": [
             "agent:*:cron:**"
           ],
           "summaryModel": "anthropic/claude-haiku-4-5",
-          "expansionModel": "anthropic/claude-haiku-4-5"
+          "expansionModel": "anthropic/claude-haiku-4-5",
+          "delegationTimeoutMs": 300000
         }
       }
     }
@@ -107,7 +123,7 @@ Add a `lossless-claw` entry under `plugins.entries` in your OpenClaw config:
 }
 ```
 
-`summaryModel` and `summaryProvider` let you pin compaction summarization to a cheaper or faster model than your main OpenClaw session model. `expansionModel` does the same for `lcm_expand_query` sub-agent calls (drilling into summaries to recover detail). When unset, both fall back to OpenClaw's configured default model/provider. See [Expansion model override requirements](#expansion-model-override-requirements) for the required `subagent` trust policy when using `expansionModel`.
+`leafChunkTokens` controls how many source tokens can accumulate in a leaf compaction chunk before summarization is triggered. The default is `20000`, but quota-limited summary providers may benefit from a larger value to reduce compaction frequency. `summaryModel` and `summaryProvider` let you pin compaction summarization to a cheaper or faster model than your main OpenClaw session model. `expansionModel` does the same for `lcm_expand_query` sub-agent calls (drilling into summaries to recover detail). `delegationTimeoutMs` controls how long `lcm_expand_query` waits for that delegated sub-agent to finish before returning a timeout error; it defaults to `120000` (120s). When unset, the model settings still fall back to OpenClaw's configured default model/provider. See [Expansion model override requirements](#expansion-model-override-requirements) for the required `subagent` trust policy when using `expansionModel`.
 
 ### Environment variables
 
@@ -119,11 +135,12 @@ Add a `lossless-claw` entry under `plugins.entries` in your OpenClaw config:
 | `LCM_STATELESS_SESSION_PATTERNS` | `""` | Comma-separated glob patterns for session keys that may read from LCM but never write to it |
 | `LCM_SKIP_STATELESS_SESSIONS` | `true` | Enable stateless-session write skipping for matching session keys |
 | `LCM_CONTEXT_THRESHOLD` | `0.75` | Fraction of context window that triggers compaction (0.0–1.0) |
-| `LCM_FRESH_TAIL_COUNT` | `32` | Number of recent messages protected from compaction |
+| `LCM_FRESH_TAIL_COUNT` | `64` | Number of recent messages protected from compaction |
+| `LCM_NEW_SESSION_RETAIN_DEPTH` | `2` | Context retained after `/new` (`-1` keeps all context, `2` keeps d2+) |
 | `LCM_LEAF_MIN_FANOUT` | `8` | Minimum raw messages per leaf summary |
 | `LCM_CONDENSED_MIN_FANOUT` | `4` | Minimum summaries per condensed node |
 | `LCM_CONDENSED_MIN_FANOUT_HARD` | `2` | Relaxed fanout for forced compaction sweeps |
-| `LCM_INCREMENTAL_MAX_DEPTH` | `0` | How deep incremental compaction goes (0 = leaf only, -1 = unlimited) |
+| `LCM_INCREMENTAL_MAX_DEPTH` | `1` | How deep incremental compaction goes (0 = leaf only, 1 = one condensed pass, -1 = unlimited) |
 | `LCM_LEAF_CHUNK_TOKENS` | `20000` | Max source tokens per leaf compaction chunk |
 | `LCM_LEAF_TARGET_TOKENS` | `1200` | Target token count for leaf summaries |
 | `LCM_CONDENSED_TARGET_TOKENS` | `2000` | Target token count for condensed summaries |
@@ -136,7 +153,7 @@ Add a `lossless-claw` entry under `plugins.entries` in your OpenClaw config:
 | `LCM_SUMMARY_BASE_URL` | *(from OpenClaw / provider default)* | Base URL override for summarization API calls |
 | `LCM_EXPANSION_MODEL` | *(from OpenClaw)* | Model override for `lcm_expand_query` sub-agent (e.g. `anthropic/claude-haiku-4-5`) |
 | `LCM_EXPANSION_PROVIDER` | *(from OpenClaw)* | Provider override for `lcm_expand_query` sub-agent |
-| `LCM_AUTOCOMPACT_DISABLED` | `false` | Disable automatic compaction after turns |
+| `LCM_DELEGATION_TIMEOUT_MS` | `120000` | Max time to wait for delegated `lcm_expand_query` sub-agent completion |
 | `LCM_PRUNE_HEARTBEAT_OK` | `false` | Retroactively delete `HEARTBEAT_OK` turn cycles from LCM storage |
 
 ### Expansion model override requirements
@@ -177,8 +194,10 @@ Plugin config equivalents:
 - `ignoreSessionPatterns`
 - `statelessSessionPatterns`
 - `skipStatelessSessions`
+- `newSessionRetainDepth`
 - `summaryModel`
 - `summaryProvider`
+- `delegationTimeoutMs`
 
 Environment variables still win over plugin config when both are set.
 
@@ -196,17 +215,36 @@ If `summaryModel` already includes a provider prefix such as `anthropic/claude-s
 ### Recommended starting configuration
 
 ```
-LCM_FRESH_TAIL_COUNT=32
-LCM_INCREMENTAL_MAX_DEPTH=-1
+LCM_FRESH_TAIL_COUNT=64
+LCM_LEAF_CHUNK_TOKENS=20000
+LCM_INCREMENTAL_MAX_DEPTH=1
 LCM_CONTEXT_THRESHOLD=0.75
 ```
 
-- **freshTailCount=32** protects the last 32 messages from compaction, giving the model enough recent context for continuity.
-- **incrementalMaxDepth=-1** enables unlimited automatic condensation after each compaction pass — the DAG cascades as deep as needed. Set to `0` (default) for leaf-only, or a positive integer for a specific depth cap.
+- **freshTailCount=64** protects the last 64 messages from compaction, giving the model more recent context for continuity.
+- **leafChunkTokens=20000** limits how large each leaf compaction chunk can grow before LCM summarizes it. Increase this when your summary provider is quota-limited and frequent leaf compactions are exhausting that quota.
+- **incrementalMaxDepth=1** runs one condensed pass after each leaf compaction by default. Set to `0` for leaf-only behavior, a larger positive integer for a deeper cap, or `-1` for unlimited cascading.
 - **contextThreshold=0.75** triggers compaction when context reaches 75% of the model's window, leaving headroom for the model's response.
 
 ### Session exclusion patterns
 
+### Session reset semantics
+
+Lossless-claw distinguishes OpenClaw's two session-reset commands:
+
+- `/new` keeps the active conversation row and all stored summaries, but prunes `context_items` so the next turn rebuilds context from retained summaries instead of the fresh tail.
+- `/reset` archives the active conversation row and creates a new active row for the same stable `sessionKey`, giving the next turn a clean LCM conversation while preserving prior history.
+
+`newSessionRetainDepth` (or `LCM_NEW_SESSION_RETAIN_DEPTH`) controls how much summary structure survives `/new`:
+
+- `-1`: keep all existing context items
+- `0`: keep all summaries, drop only fresh-tail messages
+- `1`: keep d1+ summaries
+- `2`: keep d2+ summaries; recommended default
+- `3+`: keep only deeper, more abstract summaries
+
+Lossless-claw currently applies these storage semantics through the `before_reset` hook only. User-facing confirmation text after `/new` or `/reset` must be emitted by OpenClaw's command handlers.
+
 Use `ignoreSessionPatterns` or `LCM_IGNORE_SESSION_PATTERNS` to keep low-value sessions completely out of LCM. Matching sessions do not create conversations, do not store messages, and do not participate in compaction or delegated expansion grants.
 
 Pattern rules:

package/docs/configuration.md

@@ -26,6 +26,7 @@ Set recommended environment variables:
 
 ```bash
 export LCM_FRESH_TAIL_COUNT=32
+export LCM_NEW_SESSION_RETAIN_DEPTH=2
 export LCM_INCREMENTAL_MAX_DEPTH=-1
 ```
 
@@ -51,6 +52,18 @@ For most use cases, 0.75 is a good balance.
 
 For coding conversations with tool calls (which generate many messages per logical turn), 32 is recommended.
 
+### /new retain depth
+
+`LCM_NEW_SESSION_RETAIN_DEPTH` (default `2`) controls what survives OpenClaw's `/new` command.
+
+- `-1` keeps all existing context items, making `/new` a transcript-only reset from lossless-claw's perspective.
+- `0` drops only fresh-tail message items and keeps all summaries.
+- `1` drops d0 summaries and keeps d1+.
+- `2` drops d0 and d1 summaries, keeping d2+ project-arc context. This is the recommended default.
+- `3+` keeps only deeper, more abstract summaries.
+
+`/new` never deletes the summaries themselves. It only prunes `context_items`, so the summary DAG remains available for later retrieval and expansion.
+
 ### Leaf fanout
 
 `LCM_LEAF_MIN_FANOUT` (default `8`) is the minimum number of raw messages that must be available outside the fresh tail before a leaf pass runs.
@@ -91,6 +104,25 @@ The actual summary size depends on the LLM's output; these values are guidelines
 - Smaller chunks create summaries more frequently from less material.
 - This also affects the condensed minimum input threshold (10% of this value).
 
+### Maximum assembly token budget
+
+`LCM_MAX_ASSEMBLY_TOKEN_BUDGET` (default: none) caps the token budget used for context assembly and compaction threshold evaluation. When set, this takes precedence over both the 128k fallback and runtime-provided budgets.
+
+Set this if you're using a model with a smaller context window:
+
+- **8k models:** `LCM_MAX_ASSEMBLY_TOKEN_BUDGET=7000`
+- **32k models:** `LCM_MAX_ASSEMBLY_TOKEN_BUDGET=30000`
+- **128k+ models:** No need to set (128k fallback is appropriate)
+
+### Summary size cap
+
+`LCM_SUMMARY_MAX_OVERAGE_FACTOR` (default: `3`) controls the hard ceiling on summary sizes relative to the target tokens (`leafTargetTokens` for leaf summaries, `condensedTargetTokens` for condensed summaries).
+
+If a summary exceeds `overage_factor * target_tokens`, it is deterministically truncated. A warning is logged when any summary exceeds `1.5 * target_tokens`.
+
+- **Lower values** (e.g., 2) enforce tighter summaries but may truncate more often with weaker summarizer models.
+- **Higher values** (e.g., 5) allow more LLM flexibility but risk storing oversized summaries.
+
 ## Model selection
 
 LCM uses the same model as the parent OpenClaw session for summarization by default. You can override this:
@@ -113,10 +145,22 @@ When more than one source is present, compaction summarization resolves in this
 
 If `summaryModel` already includes a provider prefix such as `anthropic/claude-sonnet-4-20250514`, `summaryProvider` is ignored for that choice.
 
+For delegated `lcm_expand_query` runs, you can extend the sub-agent wait window with `delegationTimeoutMs` (plugin config) or `LCM_DELEGATION_TIMEOUT_MS` (environment variable). The default is `120000` milliseconds.
+
 ## Session controls
 
 ### Excluding sessions entirely
 
+### `/new` vs `/reset`
+
+Lossless-claw treats the two OpenClaw reset commands differently:
+
+- `/new` keeps the active LCM conversation and prunes active context according to `newSessionRetainDepth`.
+- `/reset` archives the active conversation row and creates a fresh active row for the same stable `sessionKey`.
+
+This preserves lossless history while still giving users a real clean-slate command.
+OpenClaw's command handlers still own the user-facing post-command disclosure text; lossless-claw applies only the underlying storage transition through `before_reset`.
+
 Use `ignoreSessionPatterns` or `LCM_IGNORE_SESSION_PATTERNS` to keep low-value sessions completely out of LCM. Matching sessions do not create conversations, do not store messages, and do not participate in compaction or delegated expansion grants.
 
 - Matching uses the full session key.

package/openclaw.plugin.json

@@ -1,5 +1,8 @@
 {
   "id": "lossless-claw",
+  "skills": [
+    "skills/lossless-claw"
+  ],
   "uiHints": {
     "contextThreshold": {
       "label": "Context Threshold",
@@ -13,6 +16,30 @@
       "label": "Fresh Tail Count",
       "help": "Number of recent messages protected from compaction"
     },
+    "leafChunkTokens": {
+      "label": "Leaf Chunk Tokens",
+      "help": "Maximum source tokens per leaf compaction chunk before summarization"
+    },
+    "bootstrapMaxTokens": {
+      "label": "Bootstrap Max Tokens",
+      "help": "Maximum raw parent-history tokens imported into a brand-new conversation bootstrap; oldest turns are dropped first"
+    },
+    "newSessionRetainDepth": {
+      "label": "New Session Retain Depth",
+      "help": "Context retained after /new (-1 keeps all context, 2 keeps d2+)"
+    },
+    "leafTargetTokens": {
+      "label": "Leaf Target Tokens",
+      "help": "Target token count for leaf summaries"
+    },
+    "condensedTargetTokens": {
+      "label": "Condensed Target Tokens",
+      "help": "Target token count for condensed summaries"
+    },
+    "maxExpandTokens": {
+      "label": "Max Expand Tokens",
+      "help": "Token cap for lcm_expand_query expansion calls"
+    },
     "dbPath": {
       "label": "Database Path",
       "help": "Path to LCM SQLite database (default: ~/.openclaw/lcm.db)"
@@ -37,6 +64,14 @@
       "label": "Summary Provider",
       "help": "Provider override used only when summaryModel is a bare model name (e.g., 'openai-resp')"
     },
+    "largeFileSummaryModel": {
+      "label": "Large File Summary Model",
+      "help": "Model override for large-file summarization"
+    },
+    "largeFileSummaryProvider": {
+      "label": "Large File Summary Provider",
+      "help": "Provider override for large-file summarization"
+    },
     "expansionModel": {
       "label": "Expansion Model",
       "help": "Model override for lcm_expand_query sub-agent (e.g., 'anthropic/claude-haiku-4-5')"
@@ -44,6 +79,30 @@
     "expansionProvider": {
       "label": "Expansion Provider",
       "help": "Provider override for lcm_expand_query sub-agent (e.g., 'anthropic')"
+    },
+    "delegationTimeoutMs": {
+      "label": "Delegation Timeout (ms)",
+      "help": "Maximum time to wait for delegated lcm_expand_query sub-agent completion before timing out"
+    },
+    "maxAssemblyTokenBudget": {
+      "label": "Max Assembly Token Budget",
+      "help": "Hard ceiling for assembly token budget — caps runtime-provided and fallback budgets. Set for smaller context-window models (e.g., 30000 for 32k models)"
+    },
+    "summaryMaxOverageFactor": {
+      "label": "Summary Max Overage Factor",
+      "help": "Maximum allowed overage factor for summaries relative to target tokens (default 3). Summaries exceeding this are deterministically truncated."
+    },
+    "customInstructions": {
+      "label": "Custom Instructions",
+      "help": "Natural language instructions injected into all summarization prompts (e.g., formatting rules, tone control)"
+    },
+    "timezone": {
+      "label": "Timezone",
+      "help": "IANA timezone used for summary timestamps"
+    },
+    "pruneHeartbeatOk": {
+      "label": "Prune HEARTBEAT_OK",
+      "help": "Retroactively delete HEARTBEAT_OK turn cycles from LCM storage"
     }
   },
   "configSchema": {
@@ -66,6 +125,30 @@
         "type": "integer",
         "minimum": 1
       },
+      "leafChunkTokens": {
+        "type": "integer",
+        "minimum": 1
+      },
+      "bootstrapMaxTokens": {
+        "type": "integer",
+        "minimum": 1
+      },
+      "newSessionRetainDepth": {
+        "type": "integer",
+        "minimum": -1
+      },
+      "leafTargetTokens": {
+        "type": "integer",
+        "minimum": 1
+      },
+      "condensedTargetTokens": {
+        "type": "integer",
+        "minimum": 1
+      },
+      "maxExpandTokens": {
+        "type": "integer",
+        "minimum": 1
+      },
       "leafMinFanout": {
         "type": "integer",
         "minimum": 2
@@ -106,11 +189,42 @@
       "summaryProvider": {
         "type": "string"
       },
+      "largeFileSummaryModel": {
+        "type": "string"
+      },
+      "largeFileSummaryProvider": {
+        "type": "string"
+      },
       "expansionModel": {
         "type": "string"
       },
       "expansionProvider": {
         "type": "string"
+      },
+      "delegationTimeoutMs": {
+        "type": "integer",
+        "minimum": 1
+      },
+      "maxAssemblyTokenBudget": {
+        "type": "integer",
+        "minimum": 1000
+      },
+      "summaryMaxOverageFactor": {
+        "type": "number",
+        "minimum": 1
+      },
+      "customInstructions": {
+        "type": "string"
+      },
+      "timezone": {
+        "type": "string"
+      },
+      "pruneHeartbeatOk": {
+        "type": "boolean"
+      },
+      "databasePath": {
+        "description": "Path to LCM SQLite database (alias for dbPath)",
+        "type": "string"
       }
     }
   }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@martian-engineering/lossless-claw",
-  "version": "0.5.2",
+  "version": "0.6.0",
   "description": "Lossless Context Management plugin for OpenClaw — DAG-based conversation summarization with incremental compaction",
   "type": "module",
   "main": "index.ts",
@@ -24,6 +24,7 @@
   "files": [
     "index.ts",
     "src/**/*.ts",
+    "skills/",
     "openclaw.plugin.json",
     "docs/",
     "README.md",

package/skills/lossless-claw/SKILL.md

@@ -0,0 +1,33 @@
+---
+name: lossless-claw
+description: Configure, diagnose, and use lossless-claw effectively in OpenClaw, with emphasis on key settings, summary health, and recall-tool usage.
+---
+
+# Lossless Claw
+
+Use this skill when the task is about operating, tuning, or debugging the `lossless-claw` OpenClaw plugin.
+
+Start here:
+
+1. Confirm whether the user needs configuration help, diagnostics, recall-tool guidance, or session-lifecycle guidance.
+2. If they need a quick health check, tell them to run `/lossless` (`/lcm` is the shorter alias).
+3. If they suspect summary corruption or truncation, use `/lossless doctor`.
+4. If they ask how `/new` or `/reset` interacts with LCM, read the session-lifecycle reference before answering.
+5. Load the relevant reference file instead of improvising details from memory.
+
+Reference map:
+
+- Configuration (complete config surface on current main): `references/config.md`
+- Internal model and data flow: `references/architecture.md`
+- Diagnostics and summary-health workflow: `references/diagnostics.md`
+- Recall tools and when to use them: `references/recall-tools.md`
+- `/new` and `/reset` behavior with current lossless-claw session mapping: `references/session-lifecycle.md`
+
+Working rules:
+
+- Prioritize explaining why a setting matters, not just what it does.
+- Prefer the native plugin command surface for MVP workflows (`/lossless`, with `/lcm` as alias).
+- Do not assume the Go TUI is installed.
+- Do not recommend advanced rewrite/backfill/transplant/dissolve flows unless the user explicitly asks for non-MVP internals.
+- For exact evidence retrieval from compacted history, guide the user toward recall tools instead of guessing from summaries.
+- When users compare `/lossless` to `/status`, explain that they report different layers: `/lossless` shows LCM-side frontier/summary metrics, while `/status` shows the last assembled runtime prompt snapshot.

package/skills/lossless-claw/references/architecture.md

@@ -0,0 +1,52 @@
+# Architecture
+
+`lossless-claw` stores full conversation history in SQLite and uses summaries to keep active context within model limits.
+
+## Core flow
+
+1. Messages are persisted into the LCM database.
+2. Older messages are compacted into leaf summaries.
+3. Leaf summaries can be condensed into higher-depth summaries.
+4. Context assembly mixes summaries with the fresh raw tail.
+5. Recall tools let agents drill back into compacted material when precision matters.
+
+## Mental model
+
+Think of LCM as two layers:
+
+- durable storage of the full conversation record
+- a summary DAG used to present compacted context efficiently
+
+The summary DAG is not the source of truth. Raw messages remain the ground truth.
+
+## Why summary quality matters
+
+Bad summaries do not stay local:
+
+- poor leaf summaries degrade condensed summaries
+- poor condensed summaries degrade future recall
+- aggressive truncation reduces the precision of downstream answers
+
+That is why configuration choices around compaction thresholds and summary model quality matter operationally.
+
+## What `/lcm` tells you
+
+The MVP command surface focuses on operational facts:
+
+- package version
+- whether the plugin is enabled and selected
+- database path and size
+- summary counts
+- total summarized source-token coverage when available
+- broken or truncated summary presence
+
+## What `/lcm doctor` tells you
+
+The MVP doctor flow is diagnostic only.
+
+It looks for known summary-health markers that indicate:
+
+- deterministic fallback summaries
+- truncated summary artifacts near the end of stored content
+
+This gives users one place to answer the question “is my summary graph healthy?” without introducing a broader mutation surface.