pi-mega-compact 0.13.4 → 0.13.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +11 -8
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # pi-mega-compact
2
2
 
3
- A local context compressor for the [pi coding agent](https://github.com/earendil-works/pi). Keeps long sessions running without overflowing the context window. All on your machine — no cloud, no API calls, no telemetry.
3
+ A local-first context compressor for the [pi coding agent](https://github.com/earendil-works/pi). Keeps long sessions running without overflowing the context window. Local by default — no cloud, no API calls, no telemetry. Bring your own localhost embedder (Ollama, ONNX, TEI) for better semantic matches, or opt in to a remote endpoint if you need to.
4
4
 
5
5
  ## Features
6
6
 
@@ -13,10 +13,10 @@ A local context compressor for the [pi coding agent](https://github.com/earendil
13
13
  - **Durable memory** — on a cadence the store auto-reviews and safe-keeps decisions, facts, and preferences as first-class RAG memories, so long-running projects remember what mattered.
14
14
  - **Prompt-cache optimization** — message separation + cache striping (default OFF, opt-in). Targets 82-90% cache hit rate by structuring context around provider cache boundaries. Enables with `MEGACOMPACT_MESSAGE_SEPARATION` and `MEGACOMPACT_CACHE_STRIPING`.
15
15
  - **Context health + KV cache poison validation** (v0.12) — a real-time composite 0-1 health score per turn from five sub-scores (drift, output quality, error rate, cache health, cache poison). Catches garbled/hallucinated output and provider-side KV-cache corruption *before* they waste tokens — the failure mode where a large-context model (e.g. DeepSeek V4 Flash, 1M window) degrades at <1% usage. Tri-layer cache poison validation: prefix hash (L1 FNV-1a), output-quality-by-cache-hit (L2 semantic), error-rate correlation (L3 behavioral). The dashboard **Health tab** shows a gauge, sparkline, sub-score bars, alerts, and per-model breakdown. Auto-mitigation (force compaction on degraded context, prefix break on poisoned cache) is default OFF — toggle it in the Maintenance tab.
16
- - **RAG suite** — query reformulation (TF-IDF + RRF), tiered routing (L0 cache -> L1 FTS5 -> L2 PGlite), and recall-quality metrics (CRAG). All default OFF, opt-in via flags `MEGACOMPACT_QUERY_REFORMULATION`, `MEGACOMPACT_TIERED_ROUTER`, `MEGACOMPACT_RECALL_METRICS`.
16
+ - **RAG suite** — query reformulation (TF-IDF + RRF), tiered routing (L0 cache -> L1 FTS5 -> L2 PGlite), recall-quality metrics (CRAG), memory graph, and HyDE (hypothetical document embeddings). All default ON with graceful fallback — opt out via `MEGACOMPACT_<NAME>_DISABLED=true`. HyDE auto-activates only when an LLM embedder (Ollama/HTTP) is configured — it's a no-op with the default TrigramEmbedder. Toggle any flag from the dashboard Setup tab.
17
17
  - **Stacked memory graph** — the dashboard shows memory composition over time from 3 content sources (turns, durable memories, wiki) with a 9-gate validation system and a graph-health indicator. Per-model provider cache breakdown in the Cache tab.
18
18
  - **Debug bundle** — Maintenance tab in the dashboard has a **Gather Debug Logs** button that collects events, config, and store state into a shareable archive for bug reports.
19
- - **Fully local** — node:sqlite + trigram embeddings by default. Bring your own localhost embedder (ONNX, Ollama, TEI) for better semantic matches. Zero calls off your machine except the optional, localhost-only dashboard.
19
+ - **Local-first by default** — node:sqlite + trigram embeddings by default, zero calls off your machine. Bring your own localhost embedder (Ollama, ONNX, TEI) for better semantic matches — the embedder endpoint is loopback-only by default (`MEGACOMPACT_ALLOW_REMOTE_EMBEDDER=1` opts in to a remote/third-party endpoint). The optional dashboard is localhost-only. No cloud, no API calls, no telemetry.
20
20
  - **Team-run aware** — fine-grained durable trim fires at agent settle during sub-agent runs, so long multi-agent work doesn't just collapse at the end.
21
21
  - **Multi-pi dashboard** — one dashboard tab per active pi process with the context stack, per-repo stats, and a live SSE feed across all of them. The React SPA has 15 lazy-loaded tabs — **Overview**, **Cache**, **Sessions**, **Turns** (per-turn memory + recall + rewind), and **Health** (context-health gauge + cache-poison alerts) are primary; **Repos**, **Events**, **Config**, **Setup** (embedding wizard), **Metrics** (perf latency/TPS/CPU), **Game**, **Achievements**, **Topics** (wiki), **Memory Map** (D3 graph), and **Maintenance** (debug bundle) are advanced.
22
22
  - **Auto-categorizing wiki.** Every 3 compactions (seeds from turns before that), the store clusters your real memory embeddings (k-means) and labels each cluster with its most discriminative terms (TF-IDF) — no LLM, no Ollama, fully local. The dashboard **Wiki tab** browses topics, searches by label or term, and drills down into the member memories of each cluster.
@@ -67,13 +67,16 @@ Set env vars before starting pi. Defaults are in `src/config/dedup.ts`.
67
67
  | `MEGACOMPACT_AUTO` | `true` | Enable auto-compaction |
68
68
  | `MEGACOMPACT_DEDUP_SIM` | `0.90` | Cosine threshold for near-dup collapse |
69
69
  | `MEGACOMPACT_CROSSREPO_ENABLED` | `true` | Cross-repo recall on resume |
70
- | `MEGACOMPACT_EMBEDDING_URL` | _(unset)_ | BYO localhost embedder endpoint |
70
+ | `MEGACOMPACT_EMBEDDING_URL` | _(unset)_ | BYO localhost embedder endpoint (loopback-only by default) |
71
+ | `MEGACOMPACT_ALLOW_REMOTE_EMBEDDER` | `false` | Opt-in: allow non-loopback embedder endpoint (e.g. hosted API) |
71
72
  | `MEGACOMPACT_TUI_WIDGET` | `true` | Render the above-editor panel |
72
73
  | `MEGACOMPACT_MESSAGE_SEPARATION` | `false` | Opt-in: separate messages at provider cache boundaries |
73
74
  | `MEGACOMPACT_CACHE_STRIPING` | `false` | Opt-in: stripe cache stripes across conversation |
74
- | `MEGACOMPACT_QUERY_REFORMULATION` | `false` | Opt-in: TF-IDF + RRF query reformulation |
75
- | `MEGACOMPACT_TIERED_ROUTER` | `false` | Opt-in: L0->L1->L2 recall routing |
76
- | `MEGACOMPACT_RECALL_METRICS` | `false` | Opt-in: CRAG recall quality metrics |
75
+ | `MEGACOMPACT_QUERY_REFORMULATION_DISABLED` | `false` | Default ON — set `true` to disable TF-IDF + RRF query reformulation |
76
+ | `MEGACOMPACT_TIERED_ROUTER_DISABLED` | `false` | Default ON — set `true` to disable L0->L1->L2 recall routing |
77
+ | `MEGACOMPACT_RECALL_METRICS_DISABLED` | `false` | Default ON — set `true` to disable CRAG recall quality metrics |
78
+ | `MEGACOMPACT_MEMORY_GRAPH_DISABLED` | `false` | Default ON — set `true` to disable memory graph |
79
+ | `MEGACOMPACT_HYDE_DISABLED` | `false` | Default ON — set `true` to disable HyDE (auto-activates only with LLM embedder) |
77
80
  | `MEGACOMPACT_MEMORY_GRAPH_SEED_TURNS` | `true` | Seed memory graph from turns (default ON) |
78
81
  | `MEGACOMPACT_WIKI_SEED_FROM_TURNS` | `true` | Seed wiki from turns (default ON) |
79
82
  | `MEGACOMPACT_RAPTOR_INCREMENTAL` | `true` | Incremental RAPTOR updates (default ON) |
@@ -104,7 +107,7 @@ Detailed architecture: [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)
104
107
 
105
108
  ```bash
106
109
  npm run build # TypeScript compile
107
- npm test # Build + 1088 tests
110
+ npm test # Build + 1145 tests
108
111
  npm run lint # Type check + guardrails scan
109
112
  ```
110
113
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-mega-compact",
3
- "version": "0.13.4",
3
+ "version": "0.13.5",
4
4
  "description": "Layered, local, vector-backed context compressor for pi — supersede/collapse/cluster compaction with deduped inline recall.",
5
5
  "type": "module",
6
6
  "license": "BSD-3-Clause",