@martian-engineering/lossless-claw 0.5.3 → 0.6.1

package/README.md

@@ -7,6 +7,7 @@ Lossless Context Management plugin for [OpenClaw](https://github.com/openclaw/op
 - [What it does](#what-it-does)
 - [Quick start](#quick-start)
 - [Configuration](#configuration)
+- [Commands And Skill](#commands-and-skill)
 - [Documentation](#documentation)
 - [Development](#development)
 - [License](#license)
@@ -27,6 +28,16 @@ Nothing is lost. Raw messages stay in the database. Summaries link back to their
 
 **It feels like talking to an agent that never forgets. Because it doesn't. In normal operation, you'll never need to think about compaction again.**
 
+## Commands And Skill
+
+The plugin now ships a bundled `lossless-claw` skill plus a small native command surface:
+
+- `/lcm` shows version, enablement/selection state, DB path and size, summary counts, and summary-health status
+- `/lcm doctor` scans for broken or truncated summaries
+- `/lossless` is an alias for `/lcm` on native command surfaces
+
+The bundled skill focuses on configuration, diagnostics, architecture, and recall-tool usage. Its reference set lives under `skills/lossless-claw/references/`.
+
 ## Quick start
 
 ### Prerequisites
@@ -96,6 +107,7 @@ Add a `lossless-claw` entry under `plugins.entries` in your OpenClaw config:
         "config": {
           "freshTailCount": 64,
           "leafChunkTokens": 80000,
+          "newSessionRetainDepth": 2,
           "contextThreshold": 0.75,
           "incrementalMaxDepth": 1,
           "ignoreSessionPatterns": [
@@ -124,6 +136,7 @@ Add a `lossless-claw` entry under `plugins.entries` in your OpenClaw config:
 | `LCM_SKIP_STATELESS_SESSIONS` | `true` | Enable stateless-session write skipping for matching session keys |
 | `LCM_CONTEXT_THRESHOLD` | `0.75` | Fraction of context window that triggers compaction (0.0–1.0) |
 | `LCM_FRESH_TAIL_COUNT` | `64` | Number of recent messages protected from compaction |
+| `LCM_NEW_SESSION_RETAIN_DEPTH` | `2` | Context retained after `/new` (`-1` keeps all context, `2` keeps d2+) |
 | `LCM_LEAF_MIN_FANOUT` | `8` | Minimum raw messages per leaf summary |
 | `LCM_CONDENSED_MIN_FANOUT` | `4` | Minimum summaries per condensed node |
 | `LCM_CONDENSED_MIN_FANOUT_HARD` | `2` | Relaxed fanout for forced compaction sweeps |
@@ -141,7 +154,6 @@ Add a `lossless-claw` entry under `plugins.entries` in your OpenClaw config:
 | `LCM_EXPANSION_MODEL` | *(from OpenClaw)* | Model override for `lcm_expand_query` sub-agent (e.g. `anthropic/claude-haiku-4-5`) |
 | `LCM_EXPANSION_PROVIDER` | *(from OpenClaw)* | Provider override for `lcm_expand_query` sub-agent |
 | `LCM_DELEGATION_TIMEOUT_MS` | `120000` | Max time to wait for delegated `lcm_expand_query` sub-agent completion |
-| `LCM_AUTOCOMPACT_DISABLED` | `false` | Disable automatic compaction after turns |
 | `LCM_PRUNE_HEARTBEAT_OK` | `false` | Retroactively delete `HEARTBEAT_OK` turn cycles from LCM storage |
 
 ### Expansion model override requirements
@@ -182,6 +194,7 @@ Plugin config equivalents:
 - `ignoreSessionPatterns`
 - `statelessSessionPatterns`
 - `skipStatelessSessions`
+- `newSessionRetainDepth`
 - `summaryModel`
 - `summaryProvider`
 - `delegationTimeoutMs`
@@ -215,6 +228,23 @@ LCM_CONTEXT_THRESHOLD=0.75
 
 ### Session exclusion patterns
 
+### Session reset semantics
+
+Lossless-claw distinguishes OpenClaw's two session-reset commands:
+
+- `/new` keeps the active conversation row and all stored summaries, but prunes `context_items` so the next turn rebuilds context from retained summaries instead of the fresh tail.
+- `/reset` archives the active conversation row and creates a new active row for the same stable `sessionKey`, giving the next turn a clean LCM conversation while preserving prior history.
+
+`newSessionRetainDepth` (or `LCM_NEW_SESSION_RETAIN_DEPTH`) controls how much summary structure survives `/new`:
+
+- `-1`: keep all existing context items
+- `0`: keep all summaries, drop only fresh-tail messages
+- `1`: keep d1+ summaries
+- `2`: keep d2+ summaries; recommended default
+- `3+`: keep only deeper, more abstract summaries
+
+Lossless-claw currently applies these storage semantics through the `before_reset` hook only. User-facing confirmation text after `/new` or `/reset` must be emitted by OpenClaw's command handlers.
+
 Use `ignoreSessionPatterns` or `LCM_IGNORE_SESSION_PATTERNS` to keep low-value sessions completely out of LCM. Matching sessions do not create conversations, do not store messages, and do not participate in compaction or delegated expansion grants.
 
 Pattern rules:

package/docs/configuration.md

@@ -26,6 +26,7 @@ Set recommended environment variables:
 
 ```bash
 export LCM_FRESH_TAIL_COUNT=32
+export LCM_NEW_SESSION_RETAIN_DEPTH=2
 export LCM_INCREMENTAL_MAX_DEPTH=-1
 ```
 
@@ -51,6 +52,18 @@ For most use cases, 0.75 is a good balance.
 
 For coding conversations with tool calls (which generate many messages per logical turn), 32 is recommended.
 
+### /new retain depth
+
+`LCM_NEW_SESSION_RETAIN_DEPTH` (default `2`) controls what survives OpenClaw's `/new` command.
+
+- `-1` keeps all existing context items, making `/new` a transcript-only reset from lossless-claw's perspective.
+- `0` drops only fresh-tail message items and keeps all summaries.
+- `1` drops d0 summaries and keeps d1+.
+- `2` drops d0 and d1 summaries, keeping d2+ project-arc context. This is the recommended default.
+- `3+` keeps only deeper, more abstract summaries.
+
+`/new` never deletes the summaries themselves. It only prunes `context_items`, so the summary DAG remains available for later retrieval and expansion.
+
 ### Leaf fanout
 
 `LCM_LEAF_MIN_FANOUT` (default `8`) is the minimum number of raw messages that must be available outside the fresh tail before a leaf pass runs.
@@ -138,6 +151,16 @@ For delegated `lcm_expand_query` runs, you can extend the sub-agent wait window
 
 ### Excluding sessions entirely
 
+### `/new` vs `/reset`
+
+Lossless-claw treats the two OpenClaw reset commands differently:
+
+- `/new` keeps the active LCM conversation and prunes active context according to `newSessionRetainDepth`.
+- `/reset` archives the active conversation row and creates a fresh active row for the same stable `sessionKey`.
+
+This preserves lossless history while still giving users a real clean-slate command.
+OpenClaw's command handlers still own the user-facing post-command disclosure text; lossless-claw applies only the underlying storage transition through `before_reset`.
+
 Use `ignoreSessionPatterns` or `LCM_IGNORE_SESSION_PATTERNS` to keep low-value sessions completely out of LCM. Matching sessions do not create conversations, do not store messages, and do not participate in compaction or delegated expansion grants.
 
 - Matching uses the full session key.

package/openclaw.plugin.json

@@ -1,5 +1,8 @@
 {
   "id": "lossless-claw",
+  "skills": [
+    "skills/lossless-claw"
+  ],
   "uiHints": {
     "contextThreshold": {
       "label": "Context Threshold",
@@ -17,6 +20,26 @@
       "label": "Leaf Chunk Tokens",
       "help": "Maximum source tokens per leaf compaction chunk before summarization"
     },
+    "bootstrapMaxTokens": {
+      "label": "Bootstrap Max Tokens",
+      "help": "Maximum raw parent-history tokens imported into a brand-new conversation bootstrap; oldest turns are dropped first"
+    },
+    "newSessionRetainDepth": {
+      "label": "New Session Retain Depth",
+      "help": "Context retained after /new (-1 keeps all context, 2 keeps d2+)"
+    },
+    "leafTargetTokens": {
+      "label": "Leaf Target Tokens",
+      "help": "Target token count for leaf summaries"
+    },
+    "condensedTargetTokens": {
+      "label": "Condensed Target Tokens",
+      "help": "Target token count for condensed summaries"
+    },
+    "maxExpandTokens": {
+      "label": "Max Expand Tokens",
+      "help": "Token cap for lcm_expand_query expansion calls"
+    },
     "dbPath": {
       "label": "Database Path",
       "help": "Path to LCM SQLite database (default: ~/.openclaw/lcm.db)"
@@ -41,6 +64,14 @@
       "label": "Summary Provider",
       "help": "Provider override used only when summaryModel is a bare model name (e.g., 'openai-resp')"
     },
+    "largeFileSummaryModel": {
+      "label": "Large File Summary Model",
+      "help": "Model override for large-file summarization"
+    },
+    "largeFileSummaryProvider": {
+      "label": "Large File Summary Provider",
+      "help": "Provider override for large-file summarization"
+    },
     "expansionModel": {
       "label": "Expansion Model",
       "help": "Model override for lcm_expand_query sub-agent (e.g., 'anthropic/claude-haiku-4-5')"
@@ -64,6 +95,14 @@
     "customInstructions": {
       "label": "Custom Instructions",
       "help": "Natural language instructions injected into all summarization prompts (e.g., formatting rules, tone control)"
+    },
+    "timezone": {
+      "label": "Timezone",
+      "help": "IANA timezone used for summary timestamps"
+    },
+    "pruneHeartbeatOk": {
+      "label": "Prune HEARTBEAT_OK",
+      "help": "Retroactively delete HEARTBEAT_OK turn cycles from LCM storage"
     }
   },
   "configSchema": {
@@ -90,6 +129,26 @@
         "type": "integer",
         "minimum": 1
       },
+      "bootstrapMaxTokens": {
+        "type": "integer",
+        "minimum": 1
+      },
+      "newSessionRetainDepth": {
+        "type": "integer",
+        "minimum": -1
+      },
+      "leafTargetTokens": {
+        "type": "integer",
+        "minimum": 1
+      },
+      "condensedTargetTokens": {
+        "type": "integer",
+        "minimum": 1
+      },
+      "maxExpandTokens": {
+        "type": "integer",
+        "minimum": 1
+      },
       "leafMinFanout": {
         "type": "integer",
         "minimum": 2
@@ -130,6 +189,12 @@
       "summaryProvider": {
         "type": "string"
       },
+      "largeFileSummaryModel": {
+        "type": "string"
+      },
+      "largeFileSummaryProvider": {
+        "type": "string"
+      },
       "expansionModel": {
         "type": "string"
       },
@@ -150,6 +215,16 @@
       },
       "customInstructions": {
         "type": "string"
+      },
+      "timezone": {
+        "type": "string"
+      },
+      "pruneHeartbeatOk": {
+        "type": "boolean"
+      },
+      "databasePath": {
+        "description": "Path to LCM SQLite database (alias for dbPath)",
+        "type": "string"
       }
     }
   }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@martian-engineering/lossless-claw",
-  "version": "0.5.3",
+  "version": "0.6.1",
   "description": "Lossless Context Management plugin for OpenClaw — DAG-based conversation summarization with incremental compaction",
   "type": "module",
   "main": "index.ts",
@@ -24,6 +24,7 @@
   "files": [
     "index.ts",
     "src/**/*.ts",
+    "skills/",
     "openclaw.plugin.json",
     "docs/",
     "README.md",

package/skills/lossless-claw/SKILL.md

@@ -0,0 +1,33 @@
+---
+name: lossless-claw
+description: Configure, diagnose, and use lossless-claw effectively in OpenClaw, with emphasis on key settings, summary health, and recall-tool usage.
+---
+
+# Lossless Claw
+
+Use this skill when the task is about operating, tuning, or debugging the `lossless-claw` OpenClaw plugin.
+
+Start here:
+
+1. Confirm whether the user needs configuration help, diagnostics, recall-tool guidance, or session-lifecycle guidance.
+2. If they need a quick health check, tell them to run `/lossless` (`/lcm` is the shorter alias).
+3. If they suspect summary corruption or truncation, use `/lossless doctor`.
+4. If they ask how `/new` or `/reset` interacts with LCM, read the session-lifecycle reference before answering.
+5. Load the relevant reference file instead of improvising details from memory.
+
+Reference map:
+
+- Configuration (complete config surface on current main): `references/config.md`
+- Internal model and data flow: `references/architecture.md`
+- Diagnostics and summary-health workflow: `references/diagnostics.md`
+- Recall tools and when to use them: `references/recall-tools.md`
+- `/new` and `/reset` behavior with current lossless-claw session mapping: `references/session-lifecycle.md`
+
+Working rules:
+
+- Prioritize explaining why a setting matters, not just what it does.
+- Prefer the native plugin command surface for MVP workflows (`/lossless`, with `/lcm` as alias).
+- Do not assume the Go TUI is installed.
+- Do not recommend advanced rewrite/backfill/transplant/dissolve flows unless the user explicitly asks for non-MVP internals.
+- For exact evidence retrieval from compacted history, guide the user toward recall tools instead of guessing from summaries.
+- When users compare `/lossless` to `/status`, explain that they report different layers: `/lossless` shows LCM-side frontier/summary metrics, while `/status` shows the last assembled runtime prompt snapshot.

package/skills/lossless-claw/references/architecture.md

@@ -0,0 +1,52 @@
+# Architecture
+
+`lossless-claw` stores full conversation history in SQLite and uses summaries to keep active context within model limits.
+
+## Core flow
+
+1. Messages are persisted into the LCM database.
+2. Older messages are compacted into leaf summaries.
+3. Leaf summaries can be condensed into higher-depth summaries.
+4. Context assembly mixes summaries with the fresh raw tail.
+5. Recall tools let agents drill back into compacted material when precision matters.
+
+## Mental model
+
+Think of LCM as two layers:
+
+- durable storage of the full conversation record
+- a summary DAG used to present compacted context efficiently
+
+The summary DAG is not the source of truth. Raw messages remain the ground truth.
+
+## Why summary quality matters
+
+Bad summaries do not stay local:
+
+- poor leaf summaries degrade condensed summaries
+- poor condensed summaries degrade future recall
+- aggressive truncation reduces the precision of downstream answers
+
+That is why configuration choices around compaction thresholds and summary model quality matter operationally.
+
+## What `/lcm` tells you
+
+The MVP command surface focuses on operational facts:
+
+- package version
+- whether the plugin is enabled and selected
+- database path and size
+- summary counts
+- total summarized source-token coverage when available
+- broken or truncated summary presence
+
+## What `/lcm doctor` tells you
+
+The MVP doctor flow is diagnostic only.
+
+It looks for known summary-health markers that indicate:
+
+- deterministic fallback summaries
+- truncated summary artifacts near the end of stored content
+
+This gives users one place to answer the question “is my summary graph healthy?” without introducing a broader mutation surface.

package/skills/lossless-claw/references/config.md

@@ -0,0 +1,263 @@
+# Configuration
+
+This reference covers the current `lossless-claw` config surface on `main`, based on `openclaw.plugin.json`.
+
+`lossless-claw` is most effective when the operator understands which settings change compaction behavior and why.
+
+## First checks
+
+- Ensure the plugin is installed and enabled.
+- Ensure the context-engine slot points at `lossless-claw` when you want it to own compaction.
+- Run `/lossless` (`/lcm` alias) to confirm the plugin is active and see the live DB path.
+
+## High-impact settings
+
+These are the settings most operators should understand first.
+
+### `contextThreshold`
+
+Controls how full the model context can get before LCM compacts older material.
+
+- Lower values compact earlier.
+- Higher values compact later.
+
+Why it matters:
+
+- Too low increases summarization cost and churn.
+- Too high risks hitting the model window with large tool output or long replies.
+
+Good default:
+
+- `0.75`
+
+### `freshTailCount`
+
+Keeps the newest messages raw instead of compacting them.
+
+Why it matters:
+
+- Higher values preserve near-term conversational nuance.
+- Lower values free context budget sooner.
+
+Good starting range:
+
+- `32` to `64`
+
+### `leafChunkTokens`
+
+Caps how much raw material gets summarized into one leaf summary.
+
+Why it matters:
+
+- Larger chunks reduce summarization frequency.
+- Smaller chunks create more summaries and more DAG fragmentation.
+
+Use this when:
+
+- Your summarizer is rate-limited or expensive.
+- You want fewer but broader leaf summaries.
+
+### `incrementalMaxDepth`
+
+Controls how far automatic condensation cascades after leaf compaction.
+
+Why it matters:
+
+- `0` keeps only leaf summaries moving automatically.
+- `1` is a practical default for long-running sessions.
+- `-1` allows unlimited cascading, which can be useful for very long histories but is more aggressive.
+
+### `summaryModel` and `summaryProvider`
+
+Override the model used for compaction summarization.
+
+Why they matter:
+
+- Summary quality compounds upward in the DAG.
+- Cheaper models can reduce cost, but weak summaries create weak recalled context later.
+
+Guidance:
+
+- Pick a cheaper model only if it remains reliably structured and faithful.
+- `summaryProvider` only matters when `summaryModel` is a bare model name rather than a canonical provider/model ref.
+
+### `expansionModel` and `expansionProvider`
+
+Override the model used by delegated recall flows such as `lcm_expand_query`.
+
+Why they matter:
+
+- This lets recall-heavy work use a different cost/latency profile than normal compaction.
+- These are recall-path settings, not compaction-path settings.
+
+## Complete config surface
+
+## Core enablement and storage
+
+### `enabled`
+
+Boolean on/off switch for the plugin entry.
+
+Use this when:
+
+- you need the plugin installed but temporarily disabled
+- you want to distinguish “installed” from “selected and active”
+
+### `dbPath`
+
+Overrides the SQLite DB location.
+
+Why it matters:
+
+- useful for custom deployments, testing, or isolating environments
+- wrong path selection is a common reason operators think LCM is empty or not growing
+
+### `largeFileThresholdTokens`
+
+Threshold for externalizing oversized tool/file payloads out of the main transcript into large-file storage.
+
+Why it matters:
+
+- lower values externalize more aggressively
+- higher values keep more payload inline but can bloat storage and compaction inputs
+
+## Compaction timing and shape
+
+### `contextThreshold`
+
+See high-impact settings above.
+
+### `freshTailCount`
+
+See high-impact settings above.
+
+### `leafChunkTokens`
+
+See high-impact settings above.
+
+### `leafMinFanout`
+
+Minimum number of leaf items required before creating a leaf compaction grouping.
+
+Why it matters:
+
+- higher values avoid tiny leaf summaries
+- lower values compact sooner but can create overly granular summaries
+
+### `condensedMinFanout`
+
+Preferred minimum fanout for condensed summaries during normal condensation.
+
+Why it matters:
+
+- controls how eagerly summaries get grouped upward
+- affects DAG breadth and readability of higher-level summaries
+
+### `condensedMinFanoutHard`
+
+Hard lower bound for condensed fanout decisions.
+
+Why it matters:
+
+- acts as the guardrail when normal fanout preferences cannot be met cleanly
+- mostly useful for advanced tuning or pathological summary-tree shapes
+
+### `incrementalMaxDepth`
+
+See high-impact settings above.
+
+## Session-selection controls
+
+### `ignoreSessionPatterns`
+
+Glob-style session-key patterns that should never enter LCM.
+
+Why it matters:
+
+- keeps low-value automation or noisy sessions out of the DB
+- useful for excluding certain agent lanes or ephemeral traffic entirely
+
+### `statelessSessionPatterns`
+
+Patterns for sessions that may read from LCM but should not write to it.
+
+Why it matters:
+
+- useful for sub-agents and ephemeral workers
+- prevents recall helpers from polluting the main history
+
+### `skipStatelessSessions`
+
+Boolean that changes how stateless matches are treated.
+
+Why it matters:
+
+- when enabled, matching stateless sessions skip LCM persistence entirely
+- use carefully, because it affects whether those sessions behave as readers only or are effectively bypassed for writes
+
+## Recall-path and delegation controls
+
+### `expansionModel`
+
+See high-impact settings above.
+
+### `expansionProvider`
+
+See high-impact settings above.
+
+### `delegationTimeoutMs`
+
+Maximum time to wait for delegated recall completion.
+
+Why it matters:
+
+- lower values fail faster under slow sub-agent paths
+- higher values tolerate deeper recall but can make calls feel stuck longer
+
+### `maxAssemblyTokenBudget`
+
+Hard ceiling for assembled LCM token budget.
+
+Why it matters:
+
+- useful when the runtime model window is smaller than the surrounding system assumes
+- can prevent oversized assembly on smaller-context models
+
+## Summary quality and prompt controls
+
+### `summaryMaxOverageFactor`
+
+Maximum allowed overage factor before an oversized summary is truncated/downgraded.
+
+Why it matters:
+
+- guards against runaway summaries that are much larger than their target budget
+- useful when summary models are verbose or unstable
+
+### `customInstructions`
+
+Natural-language instructions injected into summarization prompts.
+
+Why it matters:
+
+- lets operators steer formatting or emphasis without patching code
+- should be used sparingly; low-quality instructions can degrade summary quality system-wide
+
+## Practical operator workflow
+
+1. Install and enable the plugin.
+2. Set the context-engine slot to `lossless-claw`.
+3. Start with conservative defaults.
+4. Run `/lossless` after startup to confirm path, size, and summary health.
+5. If recall feels weak, revisit `freshTailCount`, `leafChunkTokens`, and summarizer model quality before changing anything else.
+6. Touch advanced knobs like fanout, large-file thresholds, custom instructions, and assembly caps only after a concrete symptom appears.
+
+## Reading the status output
+
+`/lossless` is the right command for LCM-local metrics.
+
+Useful interpretation notes:
+
+- `tokens in context` is the current LCM frontier token count in the live LCM state.
+- `compression ratio` is shown as a rounded `1:N`, which is easier to read than a tiny percentage for heavily compacted conversations.
+- `/status` may still show a different context number because it reflects the runtime prompt that was actually assembled and sent on the last turn.

package/skills/lossless-claw/references/diagnostics.md

@@ -0,0 +1,79 @@
+# Diagnostics
+
+For the MVP, use the native command surface first.
+
+## Fast path
+
+### `/lossless` (`/lcm` alias)
+
+Use this when you need a quick health snapshot.
+
+It should answer:
+
+- Is `lossless-claw` enabled?
+- Is it selected as the context engine?
+- Which DB is active?
+- Is the DB growing as expected?
+- Are summaries present?
+- Are broken or truncated summaries present?
+
+### `/lossless doctor`
+
+Use this when summary corruption or truncation is suspected.
+
+It is the single user-facing diagnostic entrypoint for summary-health issues in the MVP.
+
+What it should help confirm:
+
+- whether broken summaries exist
+- whether truncation markers exist
+- which conversations are affected most
+
+## Interpreting common states
+
+### `/lossless` tokens vs `/status` context
+
+These numbers are related, but they are not the same metric.
+
+- `/lossless` reports LCM-side conversation metrics such as the current frontier token count and compression ratio.
+- `/status` reports the last assembled runtime prompt snapshot for the active model.
+
+Why they can differ:
+
+- runtime assembly can trim or omit frontier material before the request is sent
+- model-specific token budgeting and packing happen after LCM frontier selection
+- `/status` reflects a last-run snapshot, while `/lossless` reads live LCM state from the DB
+
+Treat `/lossless` as the LCM health/shape view, and `/status` as the runtime request view.
+
+### No summaries yet
+
+Usually means one of:
+
+- the conversation has not crossed compaction thresholds yet
+- the plugin is not selected as the context engine
+- writes are being skipped because the session matches stateless or ignored patterns
+
+### DB exists but stays tiny
+
+Usually means one of:
+
+- the plugin is not receiving traffic
+- the wrong DB path is configured
+- the plugin is enabled but not selected
+
+### Broken or truncated summaries detected
+
+Treat this as a signal to inspect summary health before trusting compacted context heavily.
+
+For MVP guidance:
+
+- keep the user on `/lossless doctor`
+- explain the count and affected conversations
+- avoid advertising separate repair-vs-doctor command families
+
+## Safe operator advice
+
+- Do not guess exact historical details from compacted context alone.
+- When a user wants a fact pattern verified, use recall tools to recover evidence.
+- Prefer changing one configuration knob at a time and then re-checking `/lossless`.