@martian-engineering/lossless-claw 0.11.3 → 0.13.0

package/skills/lossless-claw/references/config.md

@@ -30,6 +30,44 @@ Good default:
 
 - `0.75`
 
+### `contextThresholdOverrides`
+
+Optional ordered rules that choose a different compaction threshold for matching runtime contexts.
+
+Supported match fields:
+
+- `model`: exact runtime model id, such as `openai/gpt-5.5`
+- `modelContextWindowMin`: match models/windows at or above this token count
+- `modelContextWindowMax`: match models/windows at or below this token count
+- `sessionPattern`: session-key glob, using the same `*` and `**` semantics as ignored/stateless sessions
+
+Rules are AND-matched: if a rule includes both `model` and `sessionPattern`, both must match. If multiple rules match, Lossless picks the highest-specificity rule, then the earliest rule in the array for ties. If no rule matches, it falls back to global `contextThreshold`.
+
+Example:
+
+```json
+{
+  "contextThreshold": 0.75,
+  "contextThresholdOverrides": [
+    {
+      "name": "large-context-models",
+      "match": { "modelContextWindowMin": 900000 },
+      "contextThreshold": 0.15
+    },
+    {
+      "name": "telegram-sessions",
+      "match": { "sessionPattern": "agent:*:telegram:**" },
+      "contextThreshold": 0.3
+    }
+  ]
+}
+```
+
+Debugging:
+
+- threshold-selection logs include the selected threshold, source, rule index/name, token budget, threshold tokens, model, context-window value, and match reason
+- there is no env-var override for `contextThresholdOverrides`; use plugin config for structured rules
+
 ### `freshTailCount`
 
 Keeps the newest messages raw instead of compacting them.
@@ -286,6 +324,20 @@ Why it matters:
 - keep this off unless you want transcript GC to mutate the live session file during maintenance
 - the default is `false`
 
+### `enableSummaryThinking`
+
+Controls whether the summarization model receives a low reasoning budget.
+
+Why it matters:
+
+- when `true` (default), summarization calls request `reasoningIfSupported: "low"`, allowing the model to think before producing summaries — this is the current default behavior
+- when `false`, no explicit reasoning budget is requested, which can reduce cost and keep summarization output more concise when reasoning is not needed for faithful summaries
+- set to `false` when you want to minimize token spend on reasoning during compaction, especially with reasoning-capable models
+
+Env override:
+
+- `LCM_ENABLE_SUMMARY_THINKING`
+
 ### `proactiveThresholdCompactionMode`
 
 Controls whether proactive threshold compaction is deferred into maintenance debt or kept inline for legacy behavior.
@@ -296,7 +348,7 @@ Why it matters:
 - `deferred` also stores provider/model/cache telemetry so Anthropic-family sessions can avoid rewriting a still-hot prompt cache
 - `inline` preserves the legacy foreground compaction path for hosts that do not yet support deferred execution
 - `/lossless status` and `/lcm status` surface pending/running/last-failure maintenance state so operators can see when compaction is queued
-- background `maintain()` can still do non-prompt-mutating work, but prompt-mutating debt is consumed pre-assembly once cache is cold or the next turn is already approaching overflow
+- after-turn background drain and host-approved `maintain()` consume routine threshold debt; `assemble()` only drains pending threshold debt synchronously as an emergency safeguard when the live prompt estimate is already over budget
 
 ### `autoRotateSessionFiles`
 
@@ -314,6 +366,7 @@ Why it matters:
 
 - prevents very large OpenClaw session JSONL files from choking fallback/gateway startup while LCM owns the durable context
 - runtime rotation only creates or replaces the rolling `rotate-latest` DB backup when `createBackups` is `true`; manual `/lossless rotate` / `/lcm rotate` always keeps its backup-backed behavior
+- runtime JSONL rewrites run from `afterTurn()` after the host turn completes; `maintain()` skips rotation and leaves it to `afterTurn()` or startup because background maintenance can overlap an embedded model call
 - startup scans OpenClaw's current indexed session stores for configured agents, intersects those candidates with active LCM bootstrap state, and creates one pre-rotation DB backup for the startup batch only when `createBackups` is `true`
 - only runs for active, writable LCM conversations; ignored sessions, stateless sessions, sessions outside the indexed startup candidate set, and sessions without active LCM state are skipped
 - the preserved transcript tail follows the normal rotate behavior controlled by `freshTailCount`
@@ -325,6 +378,28 @@ Operational logging:
 - rotate logs include `phase`, `action`, `sessionId`, `sessionKey`, `sessionFile`, `sizeBytes`, `thresholdBytes`, `durationMs`, `backupPath`, `bytesRemoved`, `preservedTailMessageCount`, and `checkpointSize`
 - real warning logs include the same available context plus `reason` or `error`; quiet startup skips such as missing files, missing bootstrap mappings, and below-threshold files are counted in the summary instead of logged per candidate
 
+### `independentLogFile`
+
+Writes lossless-claw JSONL logs to an independent plugin-owned file in addition to OpenClaw's runtime logger.
+
+Defaults:
+
+- `enabled: true`
+- `file: /tmp/openclaw/lossless-claw-YYYY-MM-DD.log`
+- `maxFileBytes: 104857600`
+
+Why it matters:
+
+- keeps high-volume `[lcm]` operational traces separate from the shared OpenClaw gateway log
+- still sends startup banners and warning/error lines through OpenClaw's runtime logger, so gateway-level startup and failure diagnostics remain visible
+- a dated `lossless-claw-YYYY-MM-DD.log` path rolls over daily, stale dated files are pruned after 3 days, and oversized files rotate through `.1.log` to `.5.log`
+
+Env overrides:
+
+- `LCM_LOG_FILE_ENABLED`
+- `LCM_LOG_FILE`
+- `LCM_LOG_MAX_FILE_BYTES`
+
 ## Compaction timing and shape
 
 ### `contextThreshold`
@@ -468,6 +543,7 @@ Why it matters:
 
 - keeps low-value automation or noisy sessions out of the DB
 - useful for excluding certain agent lanes or ephemeral traffic entirely
+- cron scheduler keys are already isolated per runtime run, so ignore them only when they should bypass LCM compaction
 
 ### `statelessSessionPatterns`
 
@@ -515,6 +591,29 @@ Why it matters:
 - useful when the runtime model window is smaller than the surrounding system assumes
 - can prevent oversized assembly on smaller-context models
 
+## Anti-replay flood guard
+
+The ingest path runs `assertNoReplayTimestampFlood` to refuse batches that look like webhook-style replay attacks (many replay-like user messages or many identical internal messages at the same `created_at`). Because SQLite `datetime('now')` is second-granularity, legitimate idempotent bursts from sub-agents can also trip the guard if it is single-threshold. The role-aware thresholds below split the budget by message origin.
+
+### `replayFloodThresholdExternal`
+
+Max replay-like messages allowed in a single SQLite-second for `role=user` before the guard refuses the batch. Defaults to `3`.
+
+Why it matters:
+
+- preserves replay defense for third-partyly-rebroadcastable input
+- lower values are stricter but risk rejecting legitimate dedup retries from upstream channels
+
+### `replayFloodThresholdInternal`
+
+Max identical messages allowed in a single SQLite-second for `role=tool/assistant/system` before the guard refuses the batch. Defaults to `32`.
+
+Why it matters:
+
+- absorbs legitimate same-second idempotent tool returns (for example, sub-agents emitting many `{"status":"ok"}` results)
+- still bounded so a pathological loop cannot ingest unboundedly under the same timestamp
+- raise it if you operate cron sub-agents that emit very tight bursts; lower it if you want stricter sanity protection
+
 ## Nested objects
 
 ### `cacheAwareCompaction`
@@ -622,6 +721,28 @@ Why it matters:
 - guards against runaway summaries that are much larger than their target budget
 - useful when summary models are verbose or unstable
 
+### `summaryMaxCallsPerWindow`, `summaryCallWindowMs`, and `summarySpendBackoffMs`
+
+Bounds model-backed compaction and large-file summarization calls per session.
+
+Defaults:
+
+- `summaryMaxCallsPerWindow`: `24`
+- `summaryCallWindowMs`: `600000`
+- `summarySpendBackoffMs`: `1800000`
+
+Env overrides:
+
+- `LCM_SUMMARY_MAX_CALLS_PER_WINDOW`
+- `LCM_SUMMARY_CALL_WINDOW_MS`
+- `LCM_SUMMARY_SPEND_BACKOFF_MS`
+
+Why they matter:
+
+- prevents non-auth provider failures, ineffective compaction, or repeated deferred debt from spending unbounded summarization calls
+- keeps provider-auth failures on the separate auth circuit breaker path
+- direct deterministic fallbacks remain available when model-backed large-file summaries are throttled
+
 ### `customInstructions`
 
 Natural-language instructions injected into summarization prompts.
@@ -631,6 +752,25 @@ Why it matters:
 - lets operators steer formatting or emphasis without patching code
 - should be used sparingly; low-quality instructions can degrade summary quality system-wide
 
+### `stripInjectedContextTags`
+
+| | |
+| --- | --- |
+| Type | `string[]` |
+| Default | `["active_memory_plugin", "relevant-memories", "relevant_memories", "hindsight_memories"]` |
+| Env | `LCM_STRIP_INJECTED_CONTEXT_TAGS` (comma-separated) |
+
+XML tag names whose blocks are stripped from message content before compaction summarization.
+
+Why it matters:
+
+- Memory and context plugins (active-memory, memory-lancedb, hindsight-openclaw) prepend XML-tagged blocks to user messages via the `prependContext` hook.  These blocks are ephemeral retrieval context — they helped the model on that specific turn but are not part of the actual conversation.
+- Without stripping, the summarizer treats injected memories as real conversation content, permanently corrupting compacted summaries with auto-retrieved context that the user never said.
+- The default list covers well-known OpenClaw memory plugin tags.  Add custom tag names if you use plugins that inject context via other tags.
+- Set to `[]` (or empty env string) to disable stripping.
+
+Design note: stripping happens at compaction time, not at message ingestion.  The raw message stored in the LCM database still contains the original injected blocks, so `lcm_expand` and `lcm_grep` can still surface the full context the model saw on any given turn.  Only the summarizer input is cleaned.
+
 ## Practical operator workflow
 
 1. Install and enable the plugin.

package/skills/lossless-claw/references/diagnostics.md

@@ -1,9 +1,34 @@
 # Diagnostics
 
-For the MVP, use the native command surface first.
+For the MVP, use the native command surface first. For debugging lossless-claw behavior or failures, inspect the independent Lossless log before the shared OpenClaw gateway log.
 
 ## Fast path
 
+### Independent Lossless log
+
+Check this first when lossless-claw needs to debug itself, because routine `[lcm]` info and debug lines are written here instead of the shared OpenClaw gateway log.
+
+Default path:
+
+```bash
+/tmp/openclaw/lossless-claw-YYYY-MM-DD.log
+```
+
+For today's local log, use:
+
+```bash
+tail -n 200 "/tmp/openclaw/lossless-claw-$(date +%F).log"
+```
+
+Useful patterns:
+
+```bash
+rg -n "\\[lcm\\] (auto-rotate|rotate|runtime\\.llm\\.complete|summary|compact|assembly)" /tmp/openclaw/lossless-claw-*.log
+rg -n "warn|error|failed|truncated|deterministic|fallback" /tmp/openclaw/lossless-claw-*.log
+```
+
+The dated default log rolls over daily. Dated files are pruned after 3 days, and oversized active logs rotate through `.1.log` to `.5.log`. Startup banners and warning/error lines are also sent to OpenClaw's runtime logger, so check `/tmp/openclaw/openclaw-YYYY-MM-DD.log` after the Lossless log when you need gateway-level startup or failure context.
+
 ### `/lossless` (`/lcm` alias)
 
 Use this when you need a quick health snapshot.

package/skills/lossless-claw/references/session-lifecycle.md

@@ -8,25 +8,28 @@ For stock `lossless-claw` on current main:
 
 - OpenClaw handles `/new` and `/reset` as session-reset operations.
 - `lossless-claw` handles `/lossless rotate` (`/lcm rotate`) as transcript maintenance on the current conversation.
-- `lossless-claw` does **not** currently register its own `before_reset` hook or a custom reset policy.
 - `lossless-claw` prefers **`sessionKey`** as the stable identity for an LCM conversation.
-- When the same `sessionKey` reappears with a new `sessionId`, `lossless-claw` updates the stored `sessionId` on the existing LCM conversation row instead of creating a brand-new LCM conversation.
+- `/reset` archives the active conversation and creates a fresh active row for the same stable `sessionKey`.
+- Cron scheduler keys (`agent:<agent>:cron:<job>...`) are isolated per runtime run when a new `sessionId` reuses the same `sessionKey`.
+- For ordinary non-cron session keys, continuity still follows the stable `sessionKey`.
 
 ## What that means in practice
 
-If a user asks whether `/new` or `/reset` gives them a fresh LCM conversation, the answer is usually **no** under the current implementation.
+If a user asks whether `/new` or `/reset` gives them a fresh LCM conversation, distinguish the commands.
 
-They get a fresh OpenClaw session runtime, but LCM continuity still follows the stable `sessionKey` when one is available.
+They get a fresh OpenClaw session runtime, but LCM continuity usually still follows the stable `sessionKey` when one is available.
 
 So today:
 
-- `/new` and `/reset` can reset the runtime session
-- but LCM history may continue in the same conversation row if the chat/thread keeps the same `sessionKey`
+- `/new` prunes active context but keeps the same LCM conversation row
+- `/reset` archives the active LCM conversation row and creates a fresh active row
+- ordinary chat/thread LCM history may continue in the same row across runtime `sessionId` changes when the stable `sessionKey` continues
+- cron scheduler keys create fresh LCM rows per runtime run so prior runs do not enter the new run's assembled context
 - `/lossless rotate` keeps that same conversation row, summaries, and context items in place while compacting only the live transcript backing
 
 ## Why
 
-Current lossless-claw conversation resolution does this:
+Current lossless-claw conversation resolution generally does this:
 
 1. look up by `sessionKey` first
 2. fall back to `sessionId` only when no `sessionKey` match exists
@@ -34,6 +37,8 @@ Current lossless-claw conversation resolution does this:
 
 That behavior preserves continuity across session resets for the same chat identity.
 
+Cron keys are the exception: when an active cron conversation exists for the same `sessionKey` but a different runtime `sessionId`, lossless-claw archives the prior active row and starts a fresh one for the new run. Prior messages remain persisted on the archived conversation.
+
 ## `/lossless rotate`
 
 `/lossless rotate` is distinct from `/new` and `/reset`.
@@ -49,22 +54,23 @@ This makes rotate the lightweight option when the problem is transcript bloat ra
 
 ## Important limitation
 
-There is still **no plugin-specific `/new` vs `/reset` split** in stock lossless-claw docs or runtime behavior.
+There is a plugin-specific `/new` vs `/reset` split in current lossless-claw behavior.
 
 If someone is asking for semantics like:
 
 - `/new` gives them a fresh LCM conversation row
-- `/reset` archives old LCM conversation and starts a new one for the same stable `sessionKey`
 
-that is a **design/spec topic**, not current stock behavior.
+that remains a **design/spec topic**, not current stock behavior.
 
 ## Safe operator guidance
 
 When answering users:
 
-- do not promise that `/new` or `/reset` clears LCM history
+- do not promise that `/new` clears LCM history
+- explain that `/reset` archives the active LCM row and starts a fresh one for the same stable `sessionKey`
 - explain that `/lossless rotate` compacts the current transcript without splitting the LCM conversation
-- explain that current stock behavior follows `sessionKey` continuity
+- explain that ordinary current stock behavior follows `sessionKey` continuity
+- explain that cron scheduler session keys are isolated per runtime run while preserving archived prior runs
 - if they need a truly separate LCM history, use a different session key context (for example a different chat/thread/binding) or explicit non-MVP migration/surgery tools
 
 ## Relation to `/status`