pi-mega-compact 0.7.8 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -12
- package/dist/extensions/dashboard-server/html.js +1023 -0
- package/dist/extensions/dashboard-server/html.test.js +41 -0
- package/dist/extensions/dashboard-server/index-reader.js +133 -0
- package/dist/extensions/dashboard-server/server.js +530 -0
- package/dist/extensions/dashboard-server/server.test.js +120 -0
- package/dist/extensions/dashboard-server/snapshot.js +43 -0
- package/dist/extensions/dashboard-server/state.js +30 -0
- package/dist/extensions/dashboard-server/types.js +5 -0
- package/dist/extensions/dashboard-server-s32.test.js +181 -0
- package/dist/extensions/dashboard-server.js +7 -1315
- package/dist/extensions/mega-commands.js +162 -134
- package/dist/extensions/mega-compact.js +3 -0
- package/dist/extensions/mega-compact.test.js +90 -21
- package/dist/extensions/mega-conflict-cmds.js +5 -1
- package/dist/extensions/mega-dashboard-cmds.js +29 -22
- package/dist/extensions/mega-db-cmds.js +11 -2
- package/dist/extensions/mega-events/agent-handlers.js +222 -0
- package/dist/extensions/mega-events/compact-handlers.js +162 -0
- package/dist/extensions/mega-events/context-handler.js +249 -0
- package/dist/extensions/mega-events/register.js +21 -0
- package/dist/extensions/mega-events/session-handlers.js +142 -0
- package/dist/extensions/mega-events.js +15 -699
- package/dist/extensions/mega-game-cmds.js +106 -0
- package/dist/extensions/mega-game-cmds.test.js +113 -0
- package/dist/extensions/mega-pipeline/compact.js +324 -0
- package/dist/extensions/mega-pipeline/memory-review.js +38 -0
- package/dist/extensions/mega-pipeline/recall.js +147 -0
- package/dist/extensions/mega-pipeline.js +9 -480
- package/dist/extensions/mega-runtime/helpers.js +40 -0
- package/dist/extensions/mega-runtime/query.js +29 -0
- package/dist/extensions/mega-runtime/state.js +877 -0
- package/dist/extensions/mega-runtime/state.test.js +171 -0
- package/dist/extensions/mega-runtime/widget.js +270 -0
- package/dist/extensions/mega-runtime/widget.test.js +160 -0
- package/dist/extensions/mega-runtime.js +15 -947
- package/dist/src/config/themes.js +84 -0
- package/dist/src/config/themes.test.js +94 -0
- package/dist/src/game/scoring.js +105 -0
- package/dist/src/game/scoring.test.js +98 -0
- package/dist/src/store/sqlite/checkpoints.js +145 -0
- package/dist/src/store/sqlite/dedup-mirror.js +64 -0
- package/dist/src/store/sqlite/foundation.js +38 -0
- package/dist/src/store/sqlite/game-achievements.js +111 -0
- package/dist/src/store/sqlite/game-achievements.test.js +67 -0
- package/dist/src/store/sqlite/game-scores.js +105 -0
- package/dist/src/store/sqlite/game-scores.test.js +106 -0
- package/dist/src/store/sqlite/game-state.js +54 -0
- package/dist/src/store/sqlite/game-state.test.js +76 -0
- package/dist/src/store/sqlite/global-index.js +224 -0
- package/dist/src/store/sqlite/maintenance.js +235 -0
- package/dist/src/store/sqlite/memories.js +164 -0
- package/dist/src/store/sqlite/meta.js +82 -0
- package/dist/src/store/sqlite/model-snapshots.js +47 -0
- package/dist/src/store/sqlite/raptor.js +57 -0
- package/dist/src/store/sqlite/raw-transcript.js +134 -0
- package/dist/src/store/sqlite/schema.js +294 -0
- package/dist/src/store/sqlite/session-state.js +28 -0
- package/dist/src/store/sqlite/stats.js +66 -0
- package/dist/src/store/sqlite/utils.js +120 -0
- package/dist/src/store/sqlite.js +23 -1607
- package/extensions/dashboard-server/html.test.ts +50 -0
- package/extensions/dashboard-server/html.ts +1026 -0
- package/extensions/dashboard-server/index-reader.ts +130 -0
- package/extensions/dashboard-server/server.test.ts +131 -0
- package/extensions/dashboard-server/server.ts +505 -0
- package/extensions/dashboard-server/snapshot.ts +44 -0
- package/extensions/dashboard-server/state.ts +33 -0
- package/extensions/dashboard-server/types.ts +134 -0
- package/extensions/dashboard-server-s32.test.ts +195 -0
- package/extensions/dashboard-server.ts +7 -1431
- package/extensions/mega-commands.ts +33 -10
- package/extensions/mega-compact.test.ts +198 -43
- package/extensions/mega-compact.ts +3 -0
- package/extensions/mega-conflict-cmds.ts +6 -2
- package/extensions/mega-dashboard-cmds.ts +30 -23
- package/extensions/mega-db-cmds.ts +11 -3
- package/extensions/mega-events/agent-handlers.ts +262 -0
- package/extensions/mega-events/compact-handlers.ts +192 -0
- package/extensions/mega-events/context-handler.ts +290 -0
- package/extensions/mega-events/register.ts +37 -0
- package/extensions/mega-events/session-handlers.ts +165 -0
- package/extensions/mega-events.ts +15 -780
- package/extensions/mega-game-cmds.test.ts +137 -0
- package/extensions/mega-game-cmds.ts +122 -0
- package/extensions/mega-pipeline/compact.ts +366 -0
- package/extensions/mega-pipeline/memory-review.ts +46 -0
- package/extensions/mega-pipeline/recall.ts +165 -0
- package/extensions/mega-pipeline.ts +9 -537
- package/extensions/mega-runtime/helpers.ts +68 -0
- package/extensions/mega-runtime/query.ts +29 -0
- package/extensions/mega-runtime/state.test.ts +171 -0
- package/extensions/mega-runtime/state.ts +967 -0
- package/extensions/mega-runtime/widget.test.ts +185 -0
- package/extensions/mega-runtime/widget.ts +359 -0
- package/extensions/mega-runtime.ts +15 -1093
- package/package.json +4 -3
- package/src/config/themes.test.ts +116 -0
- package/src/config/themes.ts +124 -0
- package/src/game/scoring.test.ts +103 -0
- package/src/game/scoring.ts +158 -0
- package/src/store/sqlite/checkpoints.ts +204 -0
- package/src/store/sqlite/dedup-mirror.ts +114 -0
- package/src/store/sqlite/foundation.ts +63 -0
- package/src/store/sqlite/game-achievements.test.ts +80 -0
- package/src/store/sqlite/game-achievements.ts +147 -0
- package/src/store/sqlite/game-scores.test.ts +132 -0
- package/src/store/sqlite/game-scores.ts +168 -0
- package/src/store/sqlite/game-state.test.ts +89 -0
- package/src/store/sqlite/game-state.ts +87 -0
- package/src/store/sqlite/global-index.ts +305 -0
- package/src/store/sqlite/maintenance.ts +294 -0
- package/src/store/sqlite/memories.ts +217 -0
- package/src/store/sqlite/meta.ts +108 -0
- package/src/store/sqlite/model-snapshots.ts +83 -0
- package/src/store/sqlite/raptor.ts +107 -0
- package/src/store/sqlite/raw-transcript.ts +221 -0
- package/src/store/sqlite/schema.ts +305 -0
- package/src/store/sqlite/session-state.ts +38 -0
- package/src/store/sqlite/stats.ts +127 -0
- package/src/store/sqlite/utils.ts +125 -0
- package/src/store/sqlite.ts +23 -2204
package/README.md
CHANGED
|
@@ -6,7 +6,7 @@ sessions into a **local SQLite store** and offers **deduped inline recall** —
|
|
|
6
6
|
running **locally inside the extension**, with **no remote MCP server** and
|
|
7
7
|
**zero network calls at runtime** (PREVENT-PI-004).
|
|
8
8
|
|
|
9
|
-
> **Status - v0.7.
|
|
9
|
+
> **Status - v0.7.9.** Storage uses the built-in `node:sqlite` backend
|
|
10
10
|
> (`DatabaseSync`, Node >=22.13): zero native build step, fully local, and zero
|
|
11
11
|
> network at runtime. See [RELEASE_NOTES.md](RELEASE_NOTES.md) for the full
|
|
12
12
|
> changelog.
|
|
@@ -96,7 +96,7 @@ Marker insert compact-marker; dedupe so repeated triggers cost ~0 tokens
|
|
|
96
96
|
**One store, three ways to read it back — one dedup engine:**
|
|
97
97
|
|
|
98
98
|
| Entry point | Trigger | Behavior |
|
|
99
|
-
|
|
99
|
+
| --- | --- | --- |
|
|
100
100
|
| **Auto-inline** (Layer 5) | `session_start` / `session_tree` | Resume → `recallAndInline(source:"resume")` prepends the most relevant checkpoints, deduped against current context. |
|
|
101
101
|
| **On-demand recall** | `/mega-recall [query]` | Semantic search the store, dedupe, and inline the top-K. |
|
|
102
102
|
| **Dedup sentinel** | every compact | A lightweight `mega-compact-marker` entry lets auto-inline and recall skip re-injecting / re-vectorizing already-present regions. |
|
|
@@ -243,7 +243,7 @@ The commands (slash commands inside pi):
|
|
|
243
243
|
### Commands
|
|
244
244
|
|
|
245
245
|
| Command | Description |
|
|
246
|
-
|
|
246
|
+
| --- | --- |
|
|
247
247
|
| `/mega-compact [summary...]` | Manually compact the current session. A summary arg is used verbatim; otherwise the COLLAPSE heuristics build one. Persists a `chkpt_xxx`. |
|
|
248
248
|
| `/mega-compact off` | Disable auto-compaction for this session. |
|
|
249
249
|
| `/mega-status` | Show config + current context usage + store stats (checkpoint count, dedup rate, tokens saved) + the **installed version**. |
|
|
@@ -267,13 +267,12 @@ The commands (slash commands inside pi):
|
|
|
267
267
|
|
|
268
268
|
The **tier** you see in the toolbar and dashboard is a *live pressure band* (`low` → `medium` → `high` → `ultra` → `mega`) that climbs automatically as your context window fills and falls back as it's relieved — it is driven by `currentTokens / effectiveThreshold`, not a manual setting. The base compaction *threshold* is set by `MEGACOMPACT_TIER` at startup as a **% of the model context window** (`low` 50% · `medium` 60% · `high` 70% · `ultra` 70% · `mega` 75%; default `low`) — the fire point is `tierPct × contextWindow`, so it always lands below pi's native ~80% auto-compaction (any model size). The old static token amounts (50k/100k/200k/1M/10M) are now only the boot fallback used before the first context event reports a window. `/mega-tier` was removed in v0.7.6. Higher pressure also deepens the live trim and reviews durable memory more often — the whole system reacts as one.
|
|
269
269
|
|
|
270
|
-
|
|
271
270
|
### Live stats widget
|
|
272
271
|
|
|
273
272
|
Above the pi editor the extension shows a compact widget:
|
|
274
273
|
|
|
275
274
|
```
|
|
276
|
-
⚡ high·low v0.7.
|
|
275
|
+
⚡ high·low v0.7.9 │ 142k/200k tokens (71%) │ 3 chkpts │ 🤖 2 agents │ turn 5
|
|
277
276
|
◐ armed │ dedup: 92% │ saved: 45k tok
|
|
278
277
|
```
|
|
279
278
|
|
|
@@ -304,17 +303,17 @@ before starting pi.
|
|
|
304
303
|
### Core settings
|
|
305
304
|
|
|
306
305
|
| Variable | Default | Meaning |
|
|
307
|
-
|
|
306
|
+
| --- | --- | --- |
|
|
308
307
|
| `MEGACOMPACT_FAST_GATE_PCT` | `70` | Context-usage % that arms the auto-trigger. Defaults to the tier's % of window (`tierPct*100`): low 50 · med 60 · high 70 · ultra 70 · mega 75. Override raises the arming floor. |
|
|
309
308
|
| `MEGACOMPACT_TIER` | `low` | Named trigger preset — sets the compaction threshold as a **% of the model context window**: `low`(50%) `medium`(60%) `high`(70%) `ultra`(70%) `mega`(75%). Fire point = `tierPct × contextWindow`, so it always fires below pi's native ~80% auto-compaction (any model size). The old static token amounts (50k/100k/200k/1M/10M) are now only the **boot fallback** used before the first context event reports a window. Default `low`. |
|
|
310
|
-
| `MEGACOMPACT_THRESHOLD_TOKENS` |
|
|
309
|
+
| `MEGACOMPACT_THRESHOLD_TOKENS` | *(tier default)* | Explicit **absolute** token budget (the `custom` tier). Overrides `MEGACOMPACT_TIER` when set and is **never percent-scaled** — use this to pin an exact token fire point regardless of model window. |
|
|
311
310
|
| `MEGACOMPACT_ANCHOR_USER_MESSAGES` | `3` | Never drop the most recent N user messages (anchor floor). |
|
|
312
311
|
| `MEGACOMPACT_PRESERVE_RECENT` | `4` | Preserve the most recent N messages verbatim. |
|
|
313
312
|
| `MEGACOMPACT_AUTO` | `true` | Enable the auto-trigger. |
|
|
314
313
|
| `MEGACOMPACT_AUTO_INLINE` | `true` | Auto-inline on resume / branch. |
|
|
315
314
|
| `MEGACOMPACT_AUTO_INLINE_K` | `3` | Top-K checkpoints to auto-inline. |
|
|
316
315
|
| `MEGACOMPACT_DEDUP_SIM` | `0.90` | Cosine threshold to collapse near-dupes. |
|
|
317
|
-
| `MEGACOMPACT_STATE_DIR` |
|
|
316
|
+
| `MEGACOMPACT_STATE_DIR` | *(none — per-repo default)* | Override the store location. By default state is per-repo at `<repo>/.pi/mega-compact/`; this env var forces a single explicit dir (used as the fallback for non-git cwds). |
|
|
318
317
|
|
|
319
318
|
### Dedup pipeline flags
|
|
320
319
|
|
|
@@ -323,7 +322,7 @@ behavior. `MARK_ONLY_*` tiers run + record their decision but never
|
|
|
323
322
|
collapse (safe partial-rollout / auto-degrade state).
|
|
324
323
|
|
|
325
324
|
| Variable | Default | Meaning |
|
|
326
|
-
|
|
325
|
+
| --- | --- | --- |
|
|
327
326
|
| `MEGACOMPACT_L0_ENABLED` | `true` | L0 exact content-hash dedup. |
|
|
328
327
|
| `MEGACOMPACT_L1_ENABLED` | `true` | L1 MinHash/LSH near-dup verification. |
|
|
329
328
|
| `MEGACOMPACT_L2_ENABLED` | `true` | L2 semantic cosine dedup + MMR retrieval diversity. |
|
|
@@ -332,7 +331,7 @@ collapse (safe partial-rollout / auto-degrade state).
|
|
|
332
331
|
| `MEGACOMPACT_MARK_ONLY_L1` | `false` | L1: record, don't collapse. |
|
|
333
332
|
| `MEGACOMPACT_MARK_ONLY_L2` | `false` | L2: record, don't collapse. |
|
|
334
333
|
| `MEGACOMPACT_MINILM` | `false` | MiniLM embedder flag — **off; not shipped** (see Embedding). BYO via `MEGACOMPACT_EMBEDDING_URL`. |
|
|
335
|
-
| `MEGACOMPACT_EMBEDDING_URL` |
|
|
334
|
+
| `MEGACOMPACT_EMBEDDING_URL` | *(unset)* | BYO localhost embedder endpoint (loopback-only; enables `HttpEmbedder`). |
|
|
336
335
|
| `MEGACOMPACT_L2_THRESHOLD` | `0.85` | L2 cosine firing point (trigram-honest; set higher for semantic backends). |
|
|
337
336
|
| `MEGACOMPACT_L1_JACCARD` | `0.8` | L1 MinHash/LSH near-dup Jaccard threshold. |
|
|
338
337
|
| `MEGACOMPACT_MMR_LAMBDA` | `0.5` | MMR retrieval-diversity weight (λ·relevance − (1−λ)·maxSim). |
|
|
@@ -349,13 +348,13 @@ checklist, MARK_ONLY degrade) and `docs/RETENTION_POLICY.md` for TTL / soft-dele
|
|
|
349
348
|
### Continuity & memory
|
|
350
349
|
|
|
351
350
|
| Variable | Default | Meaning |
|
|
352
|
-
|
|
351
|
+
| --- | --- | --- |
|
|
353
352
|
| `MEGACOMPACT_LEGACY_DURABLE_TRIM` | `false` | Restore the legacy auto-trigger (`ctx.compact()` stops the agent). One-release rollback; default uses live context-event trim + pi native auto-compaction (compact-and-continue). |
|
|
354
353
|
| `MEGACOMPACT_CROSSREPO_ENABLED` | `true` | Cross-repo recall on resume + `/mega-recall --cross-repo` (HNSW index over every repo). |
|
|
355
354
|
| `MEGACOMPACT_CROSSREPO_COSINE` | `0.90` | Stricter cosine floor for cross-repo hits (vs `0.85` same-repo). |
|
|
356
355
|
| `MEGACOMPACT_MEMORY_AUTO_REVIEW` | `true` | Auto-review the conversation every `MEGACOMPACT_MEMORY_REVIEW_INTERVAL` turns → durable memories. |
|
|
357
356
|
| `MEGACOMPACT_MEMORY_REVIEW_INTERVAL` | `10` | Turns between auto-review cycles. |
|
|
358
|
-
| `MEGACOMPACT_PGLITE_DISABLED` |
|
|
357
|
+
| `MEGACOMPACT_PGLITE_DISABLED` | *(unset — index on)* | Kill-switch for the PGlite/HNSW cross-repo index; set `1`/`true` to disable (falls back to sync per-session scan). |
|
|
359
358
|
|
|
360
359
|
## Dashboard
|
|
361
360
|
|