@martian-engineering/lossless-claw 0.9.1 → 0.9.3

package/docs/architecture.md

@@ -125,7 +125,7 @@ The assembler runs before each model turn and builds the message array:
 2. Resolve each item — summaries become user messages with XML wrappers; messages are reconstructed from parts.
 3. Split into evictable prefix and protected fresh tail (last `freshTailCount` raw messages).
 4. Compute fresh tail token cost (always included, even if over budget).
-5. Fill remaining budget from the evictable set, keeping newest items and dropping oldest.
+5. Fill remaining budget from the evictable set. By default this keeps newest older items and drops the oldest; when `promptAwareEviction` is enabled and a searchable prompt is present, the evictable prefix is ranked by prompt relevance first and then restored to chronological order.
 6. Normalize assistant content to array blocks (Anthropic API compatibility).
 7. Sanitize tool-use/result pairing (ensures every tool_result has a matching tool_use).
 

package/docs/configuration.md

@@ -23,6 +23,7 @@ Most installations only need to override a handful of keys. If you want a comple
   "contextThreshold": 0.75,
   "freshTailCount": 64,
   "freshTailMaxTokens": 24000,
+  "promptAwareEviction": false,
   "newSessionRetainDepth": 2,
   "leafMinFanout": 8,
   "condensedMinFanout": 4,
@@ -54,9 +55,12 @@ Most installations only need to override a handful of keys. If you want a comple
   "proactiveThresholdCompactionMode": "deferred",
   "cacheAwareCompaction": {
     "enabled": true,
+    "cacheTTLSeconds": 300,
     "maxColdCacheCatchupPasses": 2,
     "hotCachePressureFactor": 4,
-    "hotCacheBudgetHeadroomRatio": 0.2
+    "hotCacheBudgetHeadroomRatio": 0.2,
+    "coldCacheObservationThreshold": 3,
+    "criticalBudgetPressureRatio": 0.70
   },
   "dynamicLeafChunkTokens": {
     "enabled": true,
@@ -123,6 +127,7 @@ openclaw plugins install --link /path/to/lossless-claw
 | `contextThreshold` | `number` | `0.75` | `LCM_CONTEXT_THRESHOLD` | Fraction of the active model context window that triggers compaction. |
 | `freshTailCount` | `integer` | `64` | `LCM_FRESH_TAIL_COUNT` | Number of newest messages always kept raw. |
 | `freshTailMaxTokens` | `integer` | unset | `LCM_FRESH_TAIL_MAX_TOKENS` | Optional token cap for the protected fresh tail. The newest message is always preserved even if it exceeds the cap. |
+| `promptAwareEviction` | `boolean` | `false` | `LCM_PROMPT_AWARE_EVICTION_ENABLED` | When enabled, budget-constrained assembly keeps older evictable items by prompt relevance instead of pure chronology. This improves retrieval under tight budgets, but it can reduce prompt-cache hit rates because the preserved prefix changes as prompts change. |
 | `leafMinFanout` | `integer` | `8` | `LCM_LEAF_MIN_FANOUT` | Minimum number of raw messages required before a leaf pass runs. |
 | `condensedMinFanout` | `integer` | `4` | `LCM_CONDENSED_MIN_FANOUT` | Number of same-depth summaries needed before condensation is attempted. |
 | `condensedMinFanoutHard` | `integer` | `2` | `LCM_CONDENSED_MIN_FANOUT_HARD` | Hard floor for condensation grouping during maintenance and repair flows. |
@@ -171,6 +176,7 @@ openclaw plugins install --link /path/to/lossless-claw
 | `cacheAwareCompaction.hotCachePressureFactor` | `number` | `4` | `LCM_HOT_CACHE_PRESSURE_FACTOR` | Multiplier applied to the hot-cache leaf trigger before raw-history pressure overrides cache preservation. |
 | `cacheAwareCompaction.hotCacheBudgetHeadroomRatio` | `number` | `0.2` | `LCM_HOT_CACHE_BUDGET_HEADROOM_RATIO` | Minimum fraction of the real token budget that must remain free before hot-cache incremental compaction is skipped entirely. |
 | `cacheAwareCompaction.coldCacheObservationThreshold` | `integer` | `3` | `LCM_COLD_CACHE_OBSERVATION_THRESHOLD` | Consecutive cold observations required before non-explicit cache misses are treated as truly cold. This dampens one-off routing noise and provider failover blips. |
+| `cacheAwareCompaction.criticalBudgetPressureRatio` | `number` | `0.70` | `LCM_CRITICAL_BUDGET_PRESSURE_RATIO` | Fraction of the token budget at which deferred compaction bypasses hot-cache delay so prompt-mutating debt can run before overflow. Set to `1` to disable this bypass. |
 
 #### `dynamicLeafChunkTokens`
 
@@ -187,9 +193,25 @@ When cache-aware compaction is enabled:
 - hot cache skips incremental maintenance entirely when the assembled context is still comfortably below the real token budget
 - hot cache also gets a short hysteresis window so one ambiguous turn does not immediately discard a recently healthy cache signal
 - cold cache still allows bounded catch-up passes via `cacheAwareCompaction.maxColdCacheCatchupPasses`
+- once `currentTokenCount >= criticalBudgetPressureRatio * tokenBudget`, deferred compaction bypasses hot-cache delay so prompt-mutating debt can run before emergency overflow handling
 
 When incremental leaf compaction still runs on a hot cache, follow-on condensed passes are suppressed so the maintenance cycle only pays for the leaf pass that was explicitly justified.
 
+### Prompt-aware eviction
+
+When `promptAwareEviction` is enabled:
+
+- the protected fresh tail is still preserved exactly as usual
+- only the older evictable prefix is affected
+- if the evictable prefix does not fit and the current prompt has searchable terms, lossless-claw keeps the most relevant older items instead of just the newest older items
+
+Tradeoff:
+
+- this can improve retrieval quality when the prompt is asking about an older topic and the assembled context is tight
+- it also makes the assembled prefix less stable for providers with prefix-based prompt caching, because different prompts can keep different older items
+
+If Anthropic prompt-cache stability matters more than topical recall under pressure, set `promptAwareEviction: false`.
+
 ## Behavior notes
 
 ### Summary model resolution

package/docs/tui.md

@@ -237,6 +237,34 @@ The confirmation screen shows:
 
 Each interactive operation also has a standalone CLI equivalent for scripting and batch operations.
 
+### `lcm-tui doctor`
+
+Scans for genuinely truncated summaries and can rewrite them in place. This is narrower than `repair`: it looks for specific truncation marker shapes instead of the generic fallback-summary marker.
+
+```bash
+# Preview repairs for one conversation
+lcm-tui doctor 44 --show-diff
+
+# Apply repairs through Codex CLI OAuth after `codex login`
+lcm-tui doctor 44 --apply --provider openai-codex --model gpt-5.3-codex
+
+# Scan only across every conversation
+lcm-tui doctor --all
+```
+
+| Flag | Description |
+|------|-------------|
+| `--apply` | Write repaired summaries to the database |
+| `--summary` | Scan only and show counts |
+| `--all` | Scan all conversations (discovery mode only) |
+| `--provider <id>` | API provider (default: anthropic) |
+| `--model <model>` | API model (default: `claude-haiku-4-5`) |
+| `--base-url <url>` | Custom API base URL (overrides config and env) |
+| `--show-diff` | Show unified diff for each fix |
+| `--timestamps` | Inject timestamps into rewrite source text |
+
+Use `--provider openai-codex` when you want ChatGPT Plus/Pro OAuth from the Codex CLI. Keep `--provider openai` for direct OpenAI-compatible HTTP calls with a raw `OPENAI_API_KEY`, including custom `--base-url` proxies.
+
 ### `lcm-tui repair`
 
 Finds and fixes corrupted summaries (those containing the `[LCM fallback summary]` marker from failed summarization attempts).
@@ -253,6 +281,12 @@ lcm-tui repair 44 --apply
 
 # Repair a specific summary
 lcm-tui repair 44 --summary-id sum_abc123 --apply
+
+# Repair through Codex CLI OAuth after `codex login`
+lcm-tui repair 44 --apply --provider openai-codex --model gpt-5.3-codex
+
+# Repair through a custom OpenAI-compatible proxy with a raw API key
+lcm-tui repair 44 --apply --provider openai --model gpt-5.3-codex --base-url https://proxy.example.com/openai
 ```
 
 The repair process:
@@ -260,7 +294,7 @@ The repair process:
 2. Orders them bottom-up: leaves first (in context ordinal order), then condensed nodes by ascending depth
 3. Reconstructs source material from linked messages (leaves) or child summaries (condensed)
 4. Resolves `previous_context` for each node (for deduplication in the prompt)
-5. Sends to Anthropic API with the appropriate depth prompt
+5. Sends to the resolved provider API with the appropriate depth prompt
 6. Updates the database in a single transaction
 
 | Flag | Description |
@@ -268,6 +302,9 @@ The repair process:
 | `--apply` | Write repairs to database (default: dry run) |
 | `--all` | Scan all conversations |
 | `--summary-id <id>` | Target a specific summary |
+| `--provider <id>` | API provider (inferred from `--model` when omitted) |
+| `--model <model>` | API model (default depends on provider) |
+| `--base-url <url>` | Custom API base URL (overrides config and env) |
 | `--verbose` | Show content hashes and previews |
 
 ### `lcm-tui rewrite`
@@ -284,10 +321,10 @@ lcm-tui rewrite 44 --depth 0 --apply
 # Rewrite everything bottom-up
 lcm-tui rewrite 44 --all --apply --diff
 
-# Rewrite with OpenAI Responses API
-lcm-tui rewrite 44 --summary sum_abc123 --provider openai --model gpt-5.3-codex --apply
+# Rewrite with Codex CLI OAuth after `codex login`
+lcm-tui rewrite 44 --summary sum_abc123 --provider openai-codex --model gpt-5.3-codex --apply
 
-# Rewrite through a custom OpenAI-compatible proxy
+# Rewrite through a custom OpenAI-compatible proxy with a raw API key
 lcm-tui rewrite 44 --summary sum_abc123 --provider openai --model gpt-5.3-codex --base-url https://proxy.example.com/openai --apply
 
 # Use custom prompt templates
@@ -380,10 +417,10 @@ lcm-tui backfill my-agent session_abc123 --apply --recompact --single-root
 # Import + compact + transplant into an active conversation
 lcm-tui backfill my-agent session_abc123 --apply --transplant-to 653
 
-# Backfill using OpenAI
-lcm-tui backfill my-agent session_abc123 --apply --provider openai --model gpt-5.3-codex
+# Backfill using Codex CLI OAuth after `codex login`
+lcm-tui backfill my-agent session_abc123 --apply --provider openai-codex --model gpt-5.3-codex
 
-# Backfill through a custom OpenAI-compatible proxy
+# Backfill through a custom OpenAI-compatible proxy with a raw API key
 lcm-tui backfill my-agent session_abc123 --apply --provider openai --model gpt-5.3-codex --base-url https://proxy.example.com/openai
 ```
 
@@ -484,14 +521,15 @@ Resolution order:
 
 If the provider auth profile mode is `oauth` (not `api_key`), set the provider API key environment variable explicitly.
 
-Interactive rewrite (`w`/`W`) can be configured with:
+Summary-producing operations (`doctor`, `repair`, `rewrite`, `backfill`, and interactive rewrite `w`/`W`) can be configured with:
 - `LCM_TUI_SUMMARY_PROVIDER`
 - `LCM_TUI_SUMMARY_MODEL`
 - `LCM_TUI_SUMMARY_BASE_URL`
-- `LCM_TUI_CONVERSATION_WINDOW_SIZE` (default `200`)
 
 It also honors `LCM_SUMMARY_PROVIDER` / `LCM_SUMMARY_MODEL` / `LCM_SUMMARY_BASE_URL` as fallback.
 
+Separately, the conversation browser window size uses `LCM_TUI_CONVERSATION_WINDOW_SIZE` (default `200`).
+
 ## Database
 
 The TUI operates directly on the SQLite database at `~/.openclaw/lcm.db`. All write operations (rewrite, dissolve, repair, transplant, backfill) use transactions. Changes take effect on the next conversation turn — the running OpenClaw instance picks up database changes automatically.

package/openclaw.plugin.json

@@ -4,6 +4,14 @@
   "skills": [
     "skills/lossless-claw"
   ],
+  "contracts": {
+    "tools": [
+      "lcm_grep",
+      "lcm_describe",
+      "lcm_expand",
+      "lcm_expand_query"
+    ]
+  },
   "uiHints": {
     "enabled": {
       "label": "Enabled",
@@ -25,6 +33,10 @@
       "label": "Fresh Tail Max Tokens",
       "help": "Optional token cap for the protected fresh tail; the newest message is always preserved"
     },
+    "promptAwareEviction": {
+      "label": "Prompt-Aware Eviction",
+      "help": "When enabled, budget-constrained assembly keeps older context by prompt relevance instead of pure chronology. This can improve recall under tight budgets, but it can also reduce provider prompt-cache hit rates because the preserved prefix changes as prompts change."
+    },
     "leafChunkTokens": {
       "label": "Leaf Chunk Tokens",
       "help": "Maximum source tokens per leaf compaction chunk before summarization"
@@ -173,6 +185,10 @@
       "label": "Cold Cache Observation Threshold",
       "help": "Consecutive cold observations required before non-explicit cache misses are treated as truly cold"
     },
+    "cacheAwareCompaction.criticalBudgetPressureRatio": {
+      "label": "Critical Budget Pressure Ratio",
+      "help": "Fraction of token budget at which deferred compaction fires regardless of prompt-cache state. Defaults to 0.70 — set to 1 to disable the override and let cache-aware throttling fully control deferral."
+    },
     "dynamicLeafChunkTokens.enabled": {
       "label": "Dynamic Leaf Chunk Tokens",
       "help": "When enabled, incremental compaction uses a larger working leaf chunk in busy sessions and keeps the static floor in quieter sessions"
@@ -226,6 +242,9 @@
         "type": "integer",
         "minimum": 0
       },
+      "promptAwareEviction": {
+        "type": "boolean"
+      },
       "leafChunkTokens": {
         "type": "integer",
         "minimum": 1
@@ -363,6 +382,11 @@
           "coldCacheObservationThreshold": {
             "type": "integer",
             "minimum": 1
+          },
+          "criticalBudgetPressureRatio": {
+            "type": "number",
+            "minimum": 0,
+            "maximum": 1
           }
         }
       },

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@martian-engineering/lossless-claw",
-  "version": "0.9.1",
+  "version": "0.9.3",
   "description": "Lossless Context Management plugin for OpenClaw — DAG-based conversation summarization with incremental compaction",
   "type": "module",
   "main": "dist/index.js",
@@ -31,27 +31,49 @@
     "LICENSE"
   ],
   "dependencies": {
-    "@mariozechner/pi-agent-core": "*",
-    "@mariozechner/pi-ai": "*",
     "@sinclair/typebox": "0.34.48"
   },
   "devDependencies": {
     "@changesets/changelog-github": "^0.6.0",
     "@changesets/cli": "^2.30.0",
+    "@mariozechner/pi-agent-core": "0.66.1",
+    "@mariozechner/pi-ai": "0.66.1",
+    "@mariozechner/pi-coding-agent": "0.66.1",
     "esbuild": "^0.28.0",
     "typescript": "^5.7.0",
     "vitest": "^3.0.0"
   },
   "peerDependencies": {
+    "@mariozechner/pi-agent-core": "*",
+    "@mariozechner/pi-ai": "*",
+    "@mariozechner/pi-coding-agent": "*",
     "openclaw": "*"
   },
+  "peerDependenciesMeta": {
+    "@mariozechner/pi-agent-core": {
+      "optional": true
+    },
+    "@mariozechner/pi-ai": {
+      "optional": true
+    },
+    "@mariozechner/pi-coding-agent": {
+      "optional": true
+    }
+  },
   "publishConfig": {
     "access": "public"
   },
   "openclaw": {
     "extensions": [
       "./dist/index.js"
-    ]
+    ],
+    "compat": {
+      "pluginApi": ">=2026.2.17",
+      "minGatewayVersion": "2026.2.17"
+    },
+    "build": {
+      "openclawVersion": "2026.2.17"
+    }
   },
   "repository": {
     "type": "git",

package/skills/lossless-claw/references/config.md

@@ -57,6 +57,21 @@ Good starting range:
 - Leave unset unless large tool outputs are forcing avoidable cost or overflow.
 - Start around `12000` to `32000` when you want a softer, size-aware fresh tail.
 
+### `promptAwareEviction`
+
+Controls whether budget-constrained assembly keeps older context by prompt relevance or pure chronology.
+
+Why it matters:
+
+- when enabled, lossless-claw can keep an older but on-topic summary instead of a newer irrelevant one
+- this can improve retrieval quality when the assembled context is tight
+- it also makes the preserved prompt prefix less stable, which can reduce prefix-based prompt-cache hit rates
+
+Good default:
+
+- `false`
+- enable it only when topical older-context recall under tight budgets matters more than prompt-cache stability
+
 ### `leafChunkTokens`
 
 Caps how much raw material gets summarized into one leaf summary.
@@ -88,12 +103,14 @@ Good defaults:
 - `hotCachePressureFactor: 4`
 - `hotCacheBudgetHeadroomRatio: 0.2`
 - `coldCacheObservationThreshold: 3`
+- `criticalBudgetPressureRatio: 0.70`
 
 Operationally:
 
 - hot cache stretches the incremental leaf trigger to `dynamicLeafChunkTokens.max`
 - hot cache skips incremental maintenance entirely when the assembled context is comfortably below the real token budget
 - hot cache gets a short hysteresis window so a recent cache hit stays "hot" briefly unless telemetry shows a break
+- critical token-budget pressure bypasses hot-cache delay once the live prompt reaches `criticalBudgetPressureRatio * tokenBudget`
 - if hot-cache maintenance still runs, it stays leaf-only and suppresses follow-on condensed passes
 
 ### `dynamicLeafChunkTokens`
@@ -232,6 +249,21 @@ See high-impact settings above.
 
 See high-impact settings above.
 
+### `promptAwareEviction`
+
+Boolean toggle for prompt-sensitive selection inside the evictable prefix during assembly.
+
+Why it matters:
+
+- only applies when the older evictable prefix does not fit the token budget
+- the protected fresh tail is unaffected
+- `true` keeps the most relevant older items for the current prompt
+- `false` falls back to pure chronological retention for the older prefix
+
+Env override:
+
+- `LCM_PROMPT_AWARE_EVICTION_ENABLED`
+
 ### `leafChunkTokens`
 
 See high-impact settings above.
@@ -398,6 +430,24 @@ Default:
 
 - `3`
 
+#### `cacheAwareCompaction.criticalBudgetPressureRatio`
+
+Fraction of the token budget at which deferred compaction bypasses hot-cache delay.
+
+Why it matters:
+
+- lets prompt-mutating deferred compaction run before the runtime falls back to emergency overflow handling
+- preserves cache-aware throttling below the pressure threshold
+- can be set to `1` to disable this pressure bypass
+
+Default:
+
+- `0.70`
+
+Env override:
+
+- `LCM_CRITICAL_BUDGET_PRESSURE_RATIO`
+
 ### `dynamicLeafChunkTokens`
 
 #### `dynamicLeafChunkTokens.enabled`