@psiclawops/hypermem 0.8.5 → 0.9.0

package/CHANGELOG.md

@@ -2,6 +2,32 @@
 
 All notable changes to hypermem are documented here.
 
+## 0.9.0 - adaptive context lifecycle
+
+- **Adaptive lifecycle is now production behavior.** Compose, afterTurn, recall, trim, compaction, and eviction share the same pressure-band policy across bootstrap, warmup, steady, elevated, high, and critical states.
+- **Smart recall and adaptive eviction landed.** `/new` and confident topic shifts widen recall, high-pressure turns gate recall down, and topic-centroid-guided eviction activates only at elevated pressure or worse.
+- **Lifecycle telemetry is release-gated.** Trim and compose reports classify lifecycle bands, divergence, and metadata-only topic signal without exposing topic names, prompt text, document text, or user content.
+- **Deterministic topic evidence replaces live-sample gating.** The 0.9.0 topic-bearing compose gate is covered by deterministic fixtures and report tests, while live topic-bearing samples remain future tuning evidence only.
+- **Forked-context integration is wired.** Forked subagent children inherit bounded parent hot-window context and start warmup or steady instead of cold bootstrap unless `/new` is explicit.
+- **Vector coverage is repaired.** Active facts, knowledge, and eligible episodes reached 100% vector coverage before the release candidate validation pass.
+
+## 0.8.8 - release hardening, diagnostics, lifecycle visibility
+
+- **Release packaging aligned across packages.** Core, hypercompositor, and memory plugin versions align at 0.8.8, with version parity validation and bump-script hardening to prevent stale plugin dependencies or lockfile drift.
+- **Installer path simplified.** The shell installer now follows the npm-first path, stages the runtime with `hypermem-install`, preserves existing config/data, backs up existing staged runtime when confirmed, and prints merge-safe OpenClaw activation commands.
+- **Integration validation documented.** `docs/INTEGRATION_VALIDATION.md` defines the install state machine, fresh install checks, upgrade checks, package dry-run validation, and common integration failure signatures.
+- **Diagnostics documented.** `docs/DIAGNOSTICS.md` covers `hypermem-status`, `hypermem-model-audit`, compose/trim reports, version parity, release-path checks, runtime logs, warm restore diagnostics, adaptive lifecycle diagnostics, and the runtime diagnostics API allowlist blocker.
+- **Adaptive lifecycle visibility landed.** The pure lifecycle policy kernel, compose diagnostics, and afterTurn gradient cap are wired while leaving recall breadth, eviction tuning, and telemetry tuning deferred to 0.9.0.
+- **Warm restore hardening is included.** Snapshot integrity, repaired restore paths, provider/parity gates, repair notices, and repair-depth caps are covered by validation guidance and release tests.
+- **Reranker and embedding fixes are included.** Reranker wiring, ZeroEntropy endpoint handling, sqlite-vec native runtime packaging, and provider/model diagnostics are part of the 0.8.8 operational release train.
+
+## 0.8.6 — docs cleanup, model audit config parsing, validator fix
+
+- **`hypermem-model-audit` now understands object-shaped OpenClaw model config.** It correctly reads `model.primary` plus fallback arrays from modern agent config instead of reporting an empty model set.
+- **Docs release notes no longer drift on package version text.** README and INSTALL now point operators to the next npm release instead of hard-coding conflicting minimum versions.
+- **README and INSTALL roles are clearer.** INSTALL is explicitly the canonical operator bring-up guide, while README stays as the shorter install-state overview.
+- **Docs validator false positives fixed.** `scripts/validate-docs.mjs` now checks only `openclaw config set plugins.slots.* ...` lines, so `config get` examples no longer trigger bogus plugin-id mismatch warnings.
+
 ## 0.8.5 — provider/model-aware failover detection, release parity
 
 - **Provider + model identity is now tracked explicitly in HyperCompositor model state.** Mid-session routing changes are detected on the full `provider/model` key, not budget alone, so `github-copilot/claude-sonnet-4-6` and `anthropic/claude-sonnet-4-6` are treated as different operational envelopes.

package/INSTALL.md

@@ -1,5 +1,7 @@
 # hypermem — Installation Guide
 
+This is the canonical install procedure. Keep README shorter and point operators here for the full bring-up path.
+
 ## Prerequisites
 
 - **Node.js 22+** (uses built-in `node:sqlite`)
@@ -15,11 +17,41 @@ openclaw config get gateway # should return gateway config, not an error
 
 If `gateway status` shows "disabled" or "not configured", complete OpenClaw onboarding first. `openclaw gateway restart` only works when the gateway service is already set up. On a brand-new OpenClaw install that has never been started, you need `openclaw gateway start` (or the full onboarding flow) before installing plugins.
 
+## Non-OpenClaw usage
+
+HyperMem can also be used as a normal Node.js library without OpenClaw plugins. This mode is useful for tests, custom agents, migration tooling, and experiments with the memory/composition API.
+
+```bash
+npm install @psiclawops/hypermem
+```
+
+```typescript
+import { HyperMem } from '@psiclawops/hypermem';
+import { join } from 'node:path';
+import { homedir } from 'node:os';
+
+const hm = await HyperMem.create({
+  dataDir: join(homedir(), '.openclaw', 'hypermem'),
+  embedding: { provider: 'none' },
+});
+
+await hm.recordUserMessage('my-agent', 'session-1', 'Hello');
+const composed = await hm.compose({
+  agentId: 'my-agent',
+  sessionKey: 'session-1',
+  prompt: 'Hello',
+  tokenBudget: 4000,
+  provider: 'anthropic',
+});
+```
+
+No gateway, plugin load path, or OpenClaw config is required in library mode. OpenClaw-specific setup starts below.
+
 ## Quick Start
 
 This guide is deliberately declarative. Follow the steps in order and verify each install state before moving on.
 
-> **Release note:** if the npm package you installed does not contain `hypermem-install` or `install:runtime`, you are on an older public release. Use the source-clone path in this guide or wait for `0.8.4+`.
+> **Release note:** if the npm package you installed does not contain `hypermem-install`, `install:runtime`, and `hypermem-model-audit`, you are on an older public release. Use the source-clone path in this guide or wait for the next npm release.
 
 ```bash
 npm install @psiclawops/hypermem
@@ -28,6 +60,16 @@ npx hypermem-install
 
 `hypermem-install` stages the plugin runtime into `~/.openclaw/plugins/hypermem`. It does **not** modify your OpenClaw config and does **not** restart the gateway. That means a successful `npx hypermem-install` is **not** a completed install. It is only a completed staging step.
 
+The shell installer is now a thin npm-first wrapper around this same path:
+
+```bash
+curl -fsSL https://raw.githubusercontent.com/PsiClawOps/hypermem/main/install.sh | bash
+```
+
+It installs the npm package into `~/.hypermem`, backs up any existing staged runtime when confirmed, stages the runtime with `hypermem-install`, writes a lightweight starter config only if no config exists, and prints merge-safe OpenClaw activation commands. It does not edit OpenClaw config and does not restart the gateway.
+
+Release validation details live in [docs/INTEGRATION_VALIDATION.md](./docs/INTEGRATION_VALIDATION.md). Diagnostic surfaces live in [docs/DIAGNOSTICS.md](./docs/DIAGNOSTICS.md).
+
 > **Prerequisites:** OpenClaw must be installed and onboarded. Run `openclaw gateway status` to confirm. If the gateway is not configured, complete OpenClaw setup first.
 >
 > **Config merge warning:** if you already have values in `plugins.load.paths` or `plugins.allow`, merge them instead of overwriting blindly.
@@ -118,7 +160,17 @@ OpenClaw loads the plugin runtime from `~/.openclaw/plugins/hypermem/`.
 
 ### Verification checkpoints
 
-Walk the install state machine explicitly:
+Run the installed-system doctor first:
+
+```bash
+hypermem-doctor --fix-plan
+```
+
+`hypermem-doctor` is read-only. It inspects OpenClaw config, HyperMem config, plugin wiring, recommended OpenClaw runtime settings, data directory shape, runtime plugin load state, and active model context-window risk. It prints exact `openclaw config set ...` commands when something needs review. It does not edit config or restart the gateway.
+
+Expected result after a complete install: no required failures. Recommendation warnings should be reviewed before production use, especially context-window warnings for GPT/OpenAI-compatible/local gateways.
+
+Walk the install state machine explicitly if you need a manual check:
 
 1. **Runtime staged**
    ```bash
@@ -142,29 +194,73 @@ Walk the install state machine explicitly:
    Expected: both `hypercompositor` and `hypermem` show as loaded.
 
 4. **Runtime healthy**
-   Run from the repo clone directory, because `bin/` is a relative path:
    ```bash
-   node bin/hypermem-status.mjs --health
+   hypermem-status --health
    ```
    Expected on fresh installs: the plugin may report `no sessions ingested` or empty counts. That means healthy but unused, not broken.
 
-5. **Runtime active**
+5. **Model budget audited**
+   ```bash
+   hypermem-model-audit --strict
+   ```
+   Expected: every configured model either matches a known context-window pattern or has explicit `contextWindowOverrides`. For GPT/OpenAI-compatible/local gateways, prefer explicit overrides unless logs prove OpenClaw passes a correct runtime `tokenBudget`.
+
+6. **Runtime active**
 
    Send a message to any agent, then verify:
 
    ```bash
-   openclaw logs --limit 100 | grep -E 'hypermem|context-engine'
+   openclaw logs --limit 100 | grep -E 'hypermem|context-engine|budget source'
    ```
 
    Expected lightweight-mode lines:
    - `[hypermem] hypermem initialized`
    - `[hypermem] Embedding provider: none — semantic search disabled, using FTS5 fallback`
    - `[hypermem:compose]`
+   - `budget source: runtime tokenBudget=...` or `budget source: contextWindowOverrides[...]` for the active model
 
 If you see a fallback like `falling back to default engine "legacy"`, the install is **not** fully active yet even if staging and wiring succeeded.
 
 ---
 
+## Upgrade Path
+
+Upgrades preserve the HyperMem data directory and existing config. The runtime staging directory is replaceable.
+
+```bash
+cp -a ~/.openclaw/plugins/hypermem ~/.openclaw/plugins/hypermem.backup.$(date +%Y%m%d-%H%M%S) 2>/dev/null || true
+npm install @psiclawops/hypermem@latest
+npx hypermem-install
+openclaw gateway restart
+```
+
+Then validate:
+
+```bash
+openclaw plugins list
+openclaw logs --limit 100 | grep -E 'hypermem|context-engine|falling back'
+hypermem-status --health
+hypermem-model-audit --strict
+```
+
+Pass criteria:
+
+- `~/.openclaw/hypermem/config.json` is preserved unless the operator edits it intentionally.
+- existing `~/.openclaw/hypermem/agents/*/messages.db` files remain present.
+- `openclaw plugins list` shows both `hypercompositor` and `hypermem`.
+- logs do not show `falling back to default engine "legacy"`.
+- health output is clean, or reports only healthy-empty state on unused installs.
+
+Rollback disables HyperMem without deleting data:
+
+```bash
+openclaw config set plugins.slots.contextEngine legacy
+openclaw config set plugins.slots.memory none
+openclaw gateway restart
+```
+
+---
+
 ## What hypermem Does
 
 hypermem replaces OpenClaw's default context assembly with a four-layer SQLite-backed memory system. Every turn, it queries all layers in parallel and composes context within a fixed token budget. No transcript accumulates. No lossy summarization. Content that doesn't fit this turn stays in storage instead of being destroyed.
@@ -716,6 +812,16 @@ Solo installs can skip this.
 
 ## Token Budget Tuning
 
+**Lookup paths for operators and agents.** To inspect the active config at any time:
+
+```bash
+cat ~/.openclaw/hypermem/config.json
+openclaw config get plugins.entries.hypercompositor.config
+openclaw config get plugins.slots.contextEngine
+```
+
+Resolution order is: (1) `plugins.entries.hypercompositor.config` in `openclaw.json`, (2) `~/.openclaw/hypermem/config.json`, (3) code defaults. See [docs/TUNING.md](./docs/TUNING.md) for the full tuning reference.
+
 These settings live in `~/.openclaw/hypermem/config.json` under the `compositor` key. All fields are optional — omit any knob to get the code-level default. Gateway restart required after changes.
 
 The recommended starting config for a standard single-agent deployment is intentionally lean on turn-1 warming. Semantic recall and fact triggers fire against each incoming message, so topic-relevant context surfaces as the conversation takes shape. This produces a steadier pressure profile than aggressive pre-loading and avoids the warm→trim→compact cycling you see when every session starts near the top of the budget.
@@ -785,6 +891,10 @@ HyperMem sizes the token budget from the model string using an internal pattern
 - `warmHistoryBudgetFraction` × *wrong budget* → wrong warm load on first turn
 - Trim tiers and compaction thresholds fire against the wrong ceiling
 
+**Treat this as mandatory tuning, not optional polish.** If the runtime does not pass the real model budget, you must supply the correct window yourself. In practice that means OpenAI-compatible surfaces, Codex/OpenRouter-style providers, custom provider prefixes, and local model gateways often need an explicit override because the model string alone is not a trustworthy source of truth.
+
+**When you know both numbers, set both.** Each override entry accepts `contextTokens` and `contextWindow`. HyperMem resolves from `contextTokens` first, then `contextWindow`, and the validator enforces `contextTokens <= contextWindow`. Setting both makes your intended usable budget explicit and documents the full advertised window for future operators.
+
 The two symptoms that indicate window-detection failure:
 
 1. **Undersized window detected** (you have a 200k model, HyperMem thinks it's 90k): every turn warms near the top of the misdetected budget, trim fires constantly, semantic recall and facts get starved. You see continuous `warm→trim→compact` cycling even on short sessions.
@@ -800,6 +910,16 @@ The two symptoms that indicate window-detection failure:
 
 If you see `fallback contextWindowSize` for your model, detection failed and you need an override.
 
+**Packaged audit helper.** HyperMem now ships `hypermem-model-audit`, which inspects your configured models plus any existing `contextWindowOverrides` and flags models that are running on risky autodetect paths:
+
+```bash
+hypermem-model-audit
+hypermem-model-audit --strict
+hypermem-model-audit --models openai-codex/gpt-5.4,ollama/llama-3.3-70b
+```
+
+Use it during install and after model changes. `--strict` exits non-zero if a model is missing explicit metadata or is only partially overridden.
+
 **Apply an override.** Add a `contextWindowOverrides` block to `~/.openclaw/hypermem/config.json`. The key is `"provider/model"` as it appears in your agent's model string (lowercase, exact match):
 
 ```json
@@ -809,9 +929,10 @@ If you see `fallback contextWindowSize` for your model, detection failed and you
     "contextWindowReserve": 0.25,
     "warmHistoryBudgetFraction": 0.27,
     "contextWindowOverrides": {
-      "ollama/llama-3.3-70b":      { "contextTokens": 131072 },
-      "copilot-local/custom-sft":  { "contextTokens": 32768 },
-      "vllm/qwen3-coder-ft":       { "contextTokens": 262144 }
+      "ollama/llama-3.3-70b":      { "contextTokens": 131072, "contextWindow": 131072 },
+      "openai-codex/gpt-5.4":      { "contextTokens": 200000, "contextWindow": 200000 },
+      "copilot-local/custom-sft":  { "contextTokens": 32768,  "contextWindow": 32768 },
+      "vllm/qwen3-coder-ft":       { "contextTokens": 262144, "contextWindow": 262144 }
     }
   }
 }
@@ -826,6 +947,8 @@ Resolution order, highest-to-lowest priority:
 
 Gateway restart required after editing overrides. Invalid override entries (malformed keys, impossible ranges, empty values) are dropped on load with a warning; the sanitizer will not let a bad override poison the resolver.
 
+**OpenAI-family warning.** Do not assume `gpt-*`, `openai/*`, `openai-codex/*`, or OpenAI-compatible hosted endpoints will always arrive with correct runtime budget metadata. If you cannot prove the runtime is logging `runtime tokenBudget=...` for that exact model, add an explicit override and verify it in logs before tuning anything else.
+
 **Interaction with warming and trimming.** Once the correct window is in place:
 
 - First-turn warm load = `detectedWindow × budgetFraction × (1 - contextWindowReserve) × warmHistoryBudgetFraction`