@psiclawops/hypermem 0.8.2 → 0.8.3

package/docs/PHASE1-VALIDATION.md

@@ -0,0 +1,132 @@
+# HyperMem Phase 1 Validation Runbook
+
+Operator-facing guide for running and interpreting the Phase 1 validation suite.
+
+---
+
+## Quick Start
+
+Run the full Phase 1 validation flow:
+
+```bash
+npm run build && node scripts/validate-compose.mjs && node scripts/validate-config-surface.mjs
+```
+
+Or run individual validations:
+
+```bash
+# Compose validation (facts, library, budget pressure)
+node scripts/validate-compose.mjs
+
+# Config surface parity (install.sh, docs, README)
+node scripts/validate-config-surface.mjs
+
+# Config key resolution tests
+node test/config-validation.mjs
+
+# Retrieval regression harness
+node test/retrieval-regression.mjs
+
+# Compositor integration (includes budget-pressure fixture)
+node test/compositor.mjs
+
+# Plugin pipeline (requires plugin build)
+npm run validate:plugin-pipeline
+
+# Startup fleet seeding on cold boot
+node test/fleet-startup-seeding.mjs
+
+# Release path hardening harness (builds core + plugin)
+npm run validate:release-path
+
+# Compose report (operator-readable diagnostics)
+node scripts/compose-report.mjs
+```
+
+---
+
+## What Each Validation Covers
+
+| Validation | What it proves |
+|---|---|
+| `validate-compose.mjs` | End-to-end compose with seeded facts, knowledge retrieval, and budget pressure |
+| `validate-config-surface.mjs` | Config keys present in install.sh, INSTALL.md, TUNING.md, README |
+| `config-validation.mjs` | contextWindowOverrides sanitization, budget resolution, maintenance defaults |
+| `retrieval-regression.mjs` | Scope isolation, superseded-fact filtering, budget pressure, knowledge retrieval |
+| `compositor.mjs` | Four-layer compose, trigger routing, keystone injection, budget-pressure filtering |
+| `plugin-pipeline.mjs` | Real plugin assemble() path with seeded L4 memory, tight-budget proof |
+| `fleet-startup-seeding.mjs` | Cold-start fleet population from workspace identity files plus idempotent repeat-boot behavior |
+| `release-gateway-path.mjs` | Real plugin release-path proof: tool-chain ejection counters, ArtifactRef, replay marker, and degradation telemetry |
+| `compose-report.mjs` | Operator-readable diagnostics showing layer counts and budget decisions |
+
+---
+
+## Interpreting Healthy Output
+
+A passing run shows all checks green:
+
+```
+  ALL 12 CHECKS PASSED ✅
+```
+
+Key diagnostics in the compose report:
+
+- **factsIncluded > 0**: facts were retrieved for the prompt
+- **tokenCount <= budget**: compositor respected the token ceiling
+- **retrievalMode**: `trigger`, `fallback_knn`, or `fts_only` — shows which retrieval path fired
+- **scopeFiltered >= 0**: cross-session facts correctly filtered
+
+Maintenance diagnostics (when `verboseLogging` is enabled):
+
+```
+[indexer] Maintenance: considered=5 skipped=2 scanned=3 mutated=0 duration=12ms exit=complete
+```
+
+- **considered**: conversations examined
+- **skipped**: conversations within cooldown window
+- **scanned**: conversations where sweeps ran
+- **mutated**: total messages deleted or truncated
+- **exit**: `complete`, `cap-reached`, `cooldown`, or `no-conversations`
+
+---
+
+## Common Cases
+
+### Empty context (no facts/knowledge seeded)
+
+Expected: `factsIncluded=0`, `contextBlock` may be empty or contain only history. This is normal for fresh installs or agents with no indexed conversations.
+
+### Missing fact in context
+
+Check: is the fact's `superseded_by` column NULL? Superseded facts are filtered. Is the fact's `agent_id` correct for the composing agent? Cross-agent facts are scope-filtered.
+
+### Over-budget compose
+
+The compositor respects token budgets with a small tolerance (5-15%). If `tokenCount` significantly exceeds `tokenBudget`, check `compositor.budgetFraction` and `contextWindowReserve` in config.
+
+### Maintenance not running
+
+Check that `maintenance.periodicInterval` is set in config.json. Default is 300000ms (5 min). If `verboseLogging` is enabled, you should see maintenance diagnostics every tick.
+
+---
+
+## What Remains Unverified After Phase 1
+
+- **Vector/semantic retrieval**: all Phase 1 tests run FTS-only (no Ollama dependency)
+- **Full hot-cache plugin path inside Phase 1 itself**: these checks use the direct compositor, not the full cache-layer assemble lifecycle. Use `npm run validate:release-path` for that proof.
+- **Multi-agent fleet interactions**: scope isolation is tested, but fleet-wide maintenance behavior is not
+- **Provider-specific formatting**: tests use `provider: 'anthropic'` only
+- **Real model token counting**: tests use the char/4 heuristic estimator, not tiktoken
+
+---
+
+## Maintenance Tuning Reference
+
+See [TUNING.md](./TUNING.md#background-maintenance) for the full knob reference. Key defaults:
+
+| Setting | Default | Effect |
+|---|---|---|
+| `maintenance.periodicInterval` | 300000ms | Background tick cadence |
+| `maintenance.maxActiveConversations` | 5 | Conversations processed per agent per tick |
+| `maintenance.recentConversationCooldownMs` | 30000ms | Skip recently processed conversations |
+| `maintenance.maxCandidatesPerPass` | 200 | Cap on mutations per tick |

package/docs/RELEASE_0.8.0_VALIDATION.md

@@ -0,0 +1,70 @@
+# HyperMem 0.8.0 Release Path Validation
+
+This is the operator runbook for the release hardening harness.
+
+## Command
+
+```bash
+npm run validate:release-path
+```
+
+That command builds core + plugin, then runs `test/release-gateway-path.mjs` against the built plugin dist in an isolated temporary HOME.
+
+## What it proves
+
+The harness exercises the real context-engine plugin path, not just direct compositor helpers.
+
+It verifies four release-path behaviors in one run:
+
+1. **normal compose path** returns assembled context through `engine.assemble()`
+2. **artifact degradation** emits a canonical `[artifact:...]` reference in system context
+3. **tool-chain ejection** is recorded through real compose-path co-ejection counters in telemetry
+4. **tool-loop replay recovery** emits the canonical `[replay state=entering ...]` marker when runtime history is hot and the hot cache is cold
+
+## Telemetry contract
+
+When `HYPERMEM_TELEMETRY=1`, the plugin now emits a `degradation` JSONL event alongside the existing `assemble`, `trim`, and `trim-guard` events.
+
+Per event fields:
+
+- `agentId`
+- `sessionKey`
+- `turnId`
+- `path` (`compose` or `toolLoop`)
+- `toolChainCoEjections`
+- `toolChainStubReplacements`
+- `artifactDegradations`
+- `artifactOversizeThresholdTokens`
+- `replayState`
+- `replayReason` (legacy machine reason strings may still contain `redis` for compatibility)
+
+The release harness asserts those counters against prompt-visible behavior, so the telemetry is not just emitted, it is verified.
+
+## Inspecting artifacts manually
+
+By default the harness deletes its temp HOME on success.
+
+To keep the temp workspace and telemetry file:
+
+```bash
+HYPERMEM_KEEP_RELEASE_TMP=1 npm run validate:release-path
+```
+
+The script will print the preserved temp path. The telemetry file lives at:
+
+```text
+<tmp>/release-telemetry.jsonl
+```
+
+## Healthy result
+
+```text
+ALL 12 CHECKS PASSED ✅
+```
+
+A healthy run means the built plugin can prove the Phase C prompt-path contracts that matter for `0.8.0`:
+
+- degraded content uses the canonical visible shapes where it is prompt-visible
+- degradation counters line up with what entered the model path
+- replay recovery is visible at the plugin boundary
+- the proof runs against the real assemble lifecycle, not a mocked helper only

package/docs/RELEASE_PROCESS.md

@@ -0,0 +1,10 @@
+# HyperMem Release Process
+
+**Canonical source:** [PsiClawOps/publication_process_internal](https://github.com/PsiClawOps/publication_process_internal)
+
+- **Universal process:** `PROCESS.md` (5-stage publication pipeline)
+- **HyperMem-specific:** `repos/hypermem.md` (scrub lists, stubs, build checks)
+- **Install verification:** `INSTALL_VERIFICATION.md` (independent vetting gate)
+- **Style:** `STYLE_GUIDE.md` + `FLEET_OUTPUT.md`
+
+Do not duplicate process content here. If something is missing from the canonical repo, add it there.

package/docs/ROADMAP.md

@@ -0,0 +1,39 @@
+# HyperMem Roadmap — Post-0.8.0
+
+Items that are designed but not yet implemented, or explicitly deferred for future releases.
+For shipped capabilities, see [CHANGELOG.md](../CHANGELOG.md) and [ARCHITECTURE.md](../ARCHITECTURE.md).
+
+---
+
+## Open Items
+
+| Item | WQ | Status | Notes |
+|---|---|---|---|
+| Cross-session context boundary markers | WQ-20260402-001 | 🟡 OPEN | `buildCrossSessionContext()` renders flat previews, no per-message boundaries or sender identity. Incident 6. |
+| Cursor durability (SQLite dual-write) | — | 🟡 DEFERRED | Cursor TTL = 24h. Dual-write to SQLite required before background indexer reads cursor reliably across restarts. |
+| Plugin type unification | — | 🟡 DEFERRED | Plugin uses dynamic imports; can't use TS types from core. Shims are intentional. Structural change needed. |
+| Strict topic mode: legacy NULL backfill | — | 🟡 DEFERRED | After ≥2 weeks of topic detection in production, run backfill to assign `topic_id` to legacy NULL messages, then narrow `getRecentMessagesByTopic()` to exclude NULL. Gate: topic detection stable, coverage >80% of new messages. Tracked in `specs/DEFERRED.md`. |
+| ACA Step 4 — retrieval stubs replace static files | — | 🔲 PENDING | `systemPromptAddition` carries governance doc chunks instead of embedding full workspace files. Blocked on Step 3 ✅ |
+| ACA Step 5 — governance context assembly | — | 🔲 PENDING | Full on-demand assembly replaces static prompt injection. Requires Step 4. |
+
+---
+
+## Cross-Agent Registry — Live Load
+
+**Current state:** `visibilityFilter()` in `cross-agent.ts` uses a hardcoded `defaultOrgRegistry()` to resolve agent tiers, orgs, and capabilities.
+
+**Known limitation:** This duplicates fleet structure that lives authoritatively in `fleet_agents` + `fleet_orgs` in library.db.
+
+**Planned:** Replace with live-loaded registry from library.db on gateway startup, with the hardcoded version as cold-start fallback only. This eliminates the need to maintain two copies of fleet topology.
+
+---
+
+## Write Authorization for Global-Scope Facts
+
+**Current state:** Facts written with `scope='global'` are readable fleet-wide. The write path has no authorization gate — any agent with HyperMem API access can write a global-scope fact.
+
+**Impact:** Acceptable for trusted single-operator deployments. All agents share the same trust boundary.
+
+**Planned:** Write-authority model that gates global-scope writes to designated agents (council seats or explicitly allowlisted agent IDs).
+
+**Workaround:** Restrict HyperMem API access to trusted agents only.

package/docs/SLASH_COMMANDS.md

@@ -0,0 +1,93 @@
+# Slash Commands
+
+hypermem supports operator-defined slash commands that hook into session lifecycle management. These are not built into the core runtime — they are wiring points you implement in your OpenClaw plugin or agent config.
+
+---
+
+## `/fresh` — Start an Unwarmed Session
+
+Flushes the current session's hot cache and starts fresh. Long-term memory (facts, vectors, episodes, knowledge graph) is preserved and will re-warm naturally on the next bootstrap.
+
+**Use case:** A user wants to start a new conversation without any session warmth bleeding in from a previous context.
+
+### What gets cleared
+
+| Slot | Cleared |
+|---|---|
+| `system` | ✅ |
+| `identity` | ✅ |
+| `history` | ✅ |
+| `window` | ✅ |
+| `cursor` | ✅ |
+| `context` | ✅ |
+| `facts` | ✅ |
+| `tools` | ✅ |
+| `meta` | ✅ |
+| Active sessions set | ✅ |
+| SQLite facts / knowledge | ❌ preserved |
+| Vector store | ❌ preserved |
+| Episodes | ❌ preserved |
+| Knowledge graph | ❌ preserved |
+
+### Wiring it in (OpenClaw plugin)
+
+```typescript
+import { flushSession } from 'hypermem';
+import type { CacheLayer } from 'hypermem';
+
+// In your slash command handler:
+if (input.trim() === '/fresh') {
+  const result = await flushSession(cache, agentId, sessionKey);
+
+  if (result.success) {
+    return `Session cache cleared. Starting fresh — long-term memory is preserved.\nFlushed at: ${result.flushedAt}`;
+  } else {
+    return `Failed to flush session: ${result.error}`;
+  }
+}
+```
+
+### Using the `SessionFlusher` class
+
+If you need a bound helper (e.g. inside an agent that always operates as a fixed agentId):
+
+```typescript
+import { SessionFlusher } from 'hypermem';
+
+const flusher = new SessionFlusher(cache, 'my-agent');
+
+// Later, when /fresh is received:
+const result = await flusher.flush(sessionKey);
+```
+
+### Aliases
+
+You may want to register multiple names for the same command:
+
+```
+/fresh
+/newsession
+/clearcache
+/restart
+```
+
+All of these are just convention. hypermem does not register slash command names — that is up to your OpenClaw plugin config.
+
+---
+
+## Planned Commands
+
+| Command | Status | Description |
+|---|---|---|
+| `/fresh` | ✅ Available | Flush hot cache, preserve long-term memory |
+| `/memory` | planned | Show what hypermem currently has in context |
+| `/forget <topic>` | planned | Suppress a topic from future context injection |
+| `/recall <query>` | planned | Manually trigger a vector search and display results |
+
+---
+
+## See Also
+
+- [TUNING.md](./TUNING.md) — full operator knobs reference
+- [MIGRATION.md](./MIGRATION.md) — schema version compatibility table
+- `SessionFlusher` and `flushSession` exports in `src/session-flusher.ts`