pi-mega-compact 0.7.8 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (122) hide show
  1. package/README.md +11 -12
  2. package/dist/extensions/dashboard-server/html.js +1023 -0
  3. package/dist/extensions/dashboard-server/html.test.js +41 -0
  4. package/dist/extensions/dashboard-server/index-reader.js +133 -0
  5. package/dist/extensions/dashboard-server/server.js +530 -0
  6. package/dist/extensions/dashboard-server/server.test.js +120 -0
  7. package/dist/extensions/dashboard-server/snapshot.js +43 -0
  8. package/dist/extensions/dashboard-server/state.js +30 -0
  9. package/dist/extensions/dashboard-server/types.js +5 -0
  10. package/dist/extensions/dashboard-server-s32.test.js +181 -0
  11. package/dist/extensions/dashboard-server.js +7 -1315
  12. package/dist/extensions/mega-commands.js +162 -134
  13. package/dist/extensions/mega-compact.js +3 -0
  14. package/dist/extensions/mega-compact.test.js +90 -21
  15. package/dist/extensions/mega-conflict-cmds.js +5 -1
  16. package/dist/extensions/mega-dashboard-cmds.js +29 -22
  17. package/dist/extensions/mega-db-cmds.js +11 -2
  18. package/dist/extensions/mega-events/agent-handlers.js +222 -0
  19. package/dist/extensions/mega-events/compact-handlers.js +162 -0
  20. package/dist/extensions/mega-events/context-handler.js +249 -0
  21. package/dist/extensions/mega-events/register.js +21 -0
  22. package/dist/extensions/mega-events/session-handlers.js +142 -0
  23. package/dist/extensions/mega-events.js +15 -699
  24. package/dist/extensions/mega-game-cmds.js +106 -0
  25. package/dist/extensions/mega-game-cmds.test.js +113 -0
  26. package/dist/extensions/mega-pipeline/compact.js +324 -0
  27. package/dist/extensions/mega-pipeline/memory-review.js +38 -0
  28. package/dist/extensions/mega-pipeline/recall.js +147 -0
  29. package/dist/extensions/mega-pipeline.js +9 -480
  30. package/dist/extensions/mega-runtime/helpers.js +40 -0
  31. package/dist/extensions/mega-runtime/query.js +29 -0
  32. package/dist/extensions/mega-runtime/state.js +877 -0
  33. package/dist/extensions/mega-runtime/state.test.js +171 -0
  34. package/dist/extensions/mega-runtime/widget.js +270 -0
  35. package/dist/extensions/mega-runtime/widget.test.js +160 -0
  36. package/dist/extensions/mega-runtime.js +15 -947
  37. package/dist/src/config/themes.js +84 -0
  38. package/dist/src/config/themes.test.js +94 -0
  39. package/dist/src/game/scoring.js +105 -0
  40. package/dist/src/game/scoring.test.js +98 -0
  41. package/dist/src/store/sqlite/checkpoints.js +145 -0
  42. package/dist/src/store/sqlite/dedup-mirror.js +64 -0
  43. package/dist/src/store/sqlite/foundation.js +38 -0
  44. package/dist/src/store/sqlite/game-achievements.js +111 -0
  45. package/dist/src/store/sqlite/game-achievements.test.js +67 -0
  46. package/dist/src/store/sqlite/game-scores.js +105 -0
  47. package/dist/src/store/sqlite/game-scores.test.js +106 -0
  48. package/dist/src/store/sqlite/game-state.js +54 -0
  49. package/dist/src/store/sqlite/game-state.test.js +76 -0
  50. package/dist/src/store/sqlite/global-index.js +224 -0
  51. package/dist/src/store/sqlite/maintenance.js +235 -0
  52. package/dist/src/store/sqlite/memories.js +164 -0
  53. package/dist/src/store/sqlite/meta.js +82 -0
  54. package/dist/src/store/sqlite/model-snapshots.js +47 -0
  55. package/dist/src/store/sqlite/raptor.js +57 -0
  56. package/dist/src/store/sqlite/raw-transcript.js +134 -0
  57. package/dist/src/store/sqlite/schema.js +294 -0
  58. package/dist/src/store/sqlite/session-state.js +28 -0
  59. package/dist/src/store/sqlite/stats.js +66 -0
  60. package/dist/src/store/sqlite/utils.js +120 -0
  61. package/dist/src/store/sqlite.js +23 -1607
  62. package/extensions/dashboard-server/html.test.ts +50 -0
  63. package/extensions/dashboard-server/html.ts +1026 -0
  64. package/extensions/dashboard-server/index-reader.ts +130 -0
  65. package/extensions/dashboard-server/server.test.ts +131 -0
  66. package/extensions/dashboard-server/server.ts +505 -0
  67. package/extensions/dashboard-server/snapshot.ts +44 -0
  68. package/extensions/dashboard-server/state.ts +33 -0
  69. package/extensions/dashboard-server/types.ts +134 -0
  70. package/extensions/dashboard-server-s32.test.ts +195 -0
  71. package/extensions/dashboard-server.ts +7 -1431
  72. package/extensions/mega-commands.ts +33 -10
  73. package/extensions/mega-compact.test.ts +198 -43
  74. package/extensions/mega-compact.ts +3 -0
  75. package/extensions/mega-conflict-cmds.ts +6 -2
  76. package/extensions/mega-dashboard-cmds.ts +30 -23
  77. package/extensions/mega-db-cmds.ts +11 -3
  78. package/extensions/mega-events/agent-handlers.ts +262 -0
  79. package/extensions/mega-events/compact-handlers.ts +192 -0
  80. package/extensions/mega-events/context-handler.ts +290 -0
  81. package/extensions/mega-events/register.ts +37 -0
  82. package/extensions/mega-events/session-handlers.ts +165 -0
  83. package/extensions/mega-events.ts +15 -780
  84. package/extensions/mega-game-cmds.test.ts +137 -0
  85. package/extensions/mega-game-cmds.ts +122 -0
  86. package/extensions/mega-pipeline/compact.ts +366 -0
  87. package/extensions/mega-pipeline/memory-review.ts +46 -0
  88. package/extensions/mega-pipeline/recall.ts +165 -0
  89. package/extensions/mega-pipeline.ts +9 -537
  90. package/extensions/mega-runtime/helpers.ts +68 -0
  91. package/extensions/mega-runtime/query.ts +29 -0
  92. package/extensions/mega-runtime/state.test.ts +171 -0
  93. package/extensions/mega-runtime/state.ts +967 -0
  94. package/extensions/mega-runtime/widget.test.ts +185 -0
  95. package/extensions/mega-runtime/widget.ts +359 -0
  96. package/extensions/mega-runtime.ts +15 -1093
  97. package/package.json +4 -3
  98. package/src/config/themes.test.ts +116 -0
  99. package/src/config/themes.ts +124 -0
  100. package/src/game/scoring.test.ts +103 -0
  101. package/src/game/scoring.ts +158 -0
  102. package/src/store/sqlite/checkpoints.ts +204 -0
  103. package/src/store/sqlite/dedup-mirror.ts +114 -0
  104. package/src/store/sqlite/foundation.ts +63 -0
  105. package/src/store/sqlite/game-achievements.test.ts +80 -0
  106. package/src/store/sqlite/game-achievements.ts +147 -0
  107. package/src/store/sqlite/game-scores.test.ts +132 -0
  108. package/src/store/sqlite/game-scores.ts +168 -0
  109. package/src/store/sqlite/game-state.test.ts +89 -0
  110. package/src/store/sqlite/game-state.ts +87 -0
  111. package/src/store/sqlite/global-index.ts +305 -0
  112. package/src/store/sqlite/maintenance.ts +294 -0
  113. package/src/store/sqlite/memories.ts +217 -0
  114. package/src/store/sqlite/meta.ts +108 -0
  115. package/src/store/sqlite/model-snapshots.ts +83 -0
  116. package/src/store/sqlite/raptor.ts +107 -0
  117. package/src/store/sqlite/raw-transcript.ts +221 -0
  118. package/src/store/sqlite/schema.ts +305 -0
  119. package/src/store/sqlite/session-state.ts +38 -0
  120. package/src/store/sqlite/stats.ts +127 -0
  121. package/src/store/sqlite/utils.ts +125 -0
  122. package/src/store/sqlite.ts +23 -2204
package/README.md CHANGED
@@ -6,7 +6,7 @@ sessions into a **local SQLite store** and offers **deduped inline recall** —
6
6
  running **locally inside the extension**, with **no remote MCP server** and
7
7
  **zero network calls at runtime** (PREVENT-PI-004).
8
8
 
9
- > **Status - v0.7.7.** Storage uses the built-in `node:sqlite` backend
9
+ > **Status - v0.7.9.** Storage uses the built-in `node:sqlite` backend
10
10
  > (`DatabaseSync`, Node >=22.13): zero native build step, fully local, and zero
11
11
  > network at runtime. See [RELEASE_NOTES.md](RELEASE_NOTES.md) for the full
12
12
  > changelog.
@@ -96,7 +96,7 @@ Marker insert compact-marker; dedupe so repeated triggers cost ~0 tokens
96
96
  **One store, three ways to read it back — one dedup engine:**
97
97
 
98
98
  | Entry point | Trigger | Behavior |
99
- |---|---|---|
99
+ | --- | --- | --- |
100
100
  | **Auto-inline** (Layer 5) | `session_start` / `session_tree` | Resume → `recallAndInline(source:"resume")` prepends the most relevant checkpoints, deduped against current context. |
101
101
  | **On-demand recall** | `/mega-recall [query]` | Semantic search the store, dedupe, and inline the top-K. |
102
102
  | **Dedup sentinel** | every compact | A lightweight `mega-compact-marker` entry lets auto-inline and recall skip re-injecting / re-vectorizing already-present regions. |
@@ -243,7 +243,7 @@ The commands (slash commands inside pi):
243
243
  ### Commands
244
244
 
245
245
  | Command | Description |
246
- |---|---|
246
+ | --- | --- |
247
247
  | `/mega-compact [summary...]` | Manually compact the current session. A summary arg is used verbatim; otherwise the COLLAPSE heuristics build one. Persists a `chkpt_xxx`. |
248
248
  | `/mega-compact off` | Disable auto-compaction for this session. |
249
249
  | `/mega-status` | Show config + current context usage + store stats (checkpoint count, dedup rate, tokens saved) + the **installed version**. |
@@ -267,13 +267,12 @@ The commands (slash commands inside pi):
267
267
 
268
268
  The **tier** you see in the toolbar and dashboard is a *live pressure band* (`low` → `medium` → `high` → `ultra` → `mega`) that climbs automatically as your context window fills and falls back as it's relieved — it is driven by `currentTokens / effectiveThreshold`, not a manual setting. The base compaction *threshold* is set by `MEGACOMPACT_TIER` at startup as a **% of the model context window** (`low` 50% · `medium` 60% · `high` 70% · `ultra` 70% · `mega` 75%; default `low`) — the fire point is `tierPct × contextWindow`, so it always lands below pi's native ~80% auto-compaction (any model size). The old static token amounts (50k/100k/200k/1M/10M) are now only the boot fallback used before the first context event reports a window. `/mega-tier` was removed in v0.7.6. Higher pressure also deepens the live trim and reviews durable memory more often — the whole system reacts as one.
269
269
 
270
-
271
270
  ### Live stats widget
272
271
 
273
272
  Above the pi editor the extension shows a compact widget:
274
273
 
275
274
  ```
276
- ⚡ high·low v0.7.7 │ 142k/200k tokens (71%) │ 3 chkpts │ 🤖 2 agents │ turn 5
275
+ ⚡ high·low v0.7.9 │ 142k/200k tokens (71%) │ 3 chkpts │ 🤖 2 agents │ turn 5
277
276
  ◐ armed │ dedup: 92% │ saved: 45k tok
278
277
  ```
279
278
 
@@ -304,17 +303,17 @@ before starting pi.
304
303
  ### Core settings
305
304
 
306
305
  | Variable | Default | Meaning |
307
- |---|---|---|
306
+ | --- | --- | --- |
308
307
  | `MEGACOMPACT_FAST_GATE_PCT` | `70` | Context-usage % that arms the auto-trigger. Defaults to the tier's % of window (`tierPct*100`): low 50 · med 60 · high 70 · ultra 70 · mega 75. Override raises the arming floor. |
309
308
  | `MEGACOMPACT_TIER` | `low` | Named trigger preset — sets the compaction threshold as a **% of the model context window**: `low`(50%) `medium`(60%) `high`(70%) `ultra`(70%) `mega`(75%). Fire point = `tierPct × contextWindow`, so it always fires below pi's native ~80% auto-compaction (any model size). The old static token amounts (50k/100k/200k/1M/10M) are now only the **boot fallback** used before the first context event reports a window. Default `low`. |
310
- | `MEGACOMPACT_THRESHOLD_TOKENS` | _(tier default)_ | Explicit **absolute** token budget (the `custom` tier). Overrides `MEGACOMPACT_TIER` when set and is **never percent-scaled** — use this to pin an exact token fire point regardless of model window. |
309
+ | `MEGACOMPACT_THRESHOLD_TOKENS` | *(tier default)* | Explicit **absolute** token budget (the `custom` tier). Overrides `MEGACOMPACT_TIER` when set and is **never percent-scaled** — use this to pin an exact token fire point regardless of model window. |
311
310
  | `MEGACOMPACT_ANCHOR_USER_MESSAGES` | `3` | Never drop the most recent N user messages (anchor floor). |
312
311
  | `MEGACOMPACT_PRESERVE_RECENT` | `4` | Preserve the most recent N messages verbatim. |
313
312
  | `MEGACOMPACT_AUTO` | `true` | Enable the auto-trigger. |
314
313
  | `MEGACOMPACT_AUTO_INLINE` | `true` | Auto-inline on resume / branch. |
315
314
  | `MEGACOMPACT_AUTO_INLINE_K` | `3` | Top-K checkpoints to auto-inline. |
316
315
  | `MEGACOMPACT_DEDUP_SIM` | `0.90` | Cosine threshold to collapse near-dupes. |
317
- | `MEGACOMPACT_STATE_DIR` | _(none — per-repo default)_ | Override the store location. By default state is per-repo at `<repo>/.pi/mega-compact/`; this env var forces a single explicit dir (used as the fallback for non-git cwds). |
316
+ | `MEGACOMPACT_STATE_DIR` | *(none — per-repo default)* | Override the store location. By default state is per-repo at `<repo>/.pi/mega-compact/`; this env var forces a single explicit dir (used as the fallback for non-git cwds). |
318
317
 
319
318
  ### Dedup pipeline flags
320
319
 
@@ -323,7 +322,7 @@ behavior. `MARK_ONLY_*` tiers run + record their decision but never
323
322
  collapse (safe partial-rollout / auto-degrade state).
324
323
 
325
324
  | Variable | Default | Meaning |
326
- |---|---|---|
325
+ | --- | --- | --- |
327
326
  | `MEGACOMPACT_L0_ENABLED` | `true` | L0 exact content-hash dedup. |
328
327
  | `MEGACOMPACT_L1_ENABLED` | `true` | L1 MinHash/LSH near-dup verification. |
329
328
  | `MEGACOMPACT_L2_ENABLED` | `true` | L2 semantic cosine dedup + MMR retrieval diversity. |
@@ -332,7 +331,7 @@ collapse (safe partial-rollout / auto-degrade state).
332
331
  | `MEGACOMPACT_MARK_ONLY_L1` | `false` | L1: record, don't collapse. |
333
332
  | `MEGACOMPACT_MARK_ONLY_L2` | `false` | L2: record, don't collapse. |
334
333
  | `MEGACOMPACT_MINILM` | `false` | MiniLM embedder flag — **off; not shipped** (see Embedding). BYO via `MEGACOMPACT_EMBEDDING_URL`. |
335
- | `MEGACOMPACT_EMBEDDING_URL` | _(unset)_ | BYO localhost embedder endpoint (loopback-only; enables `HttpEmbedder`). |
334
+ | `MEGACOMPACT_EMBEDDING_URL` | *(unset)* | BYO localhost embedder endpoint (loopback-only; enables `HttpEmbedder`). |
336
335
  | `MEGACOMPACT_L2_THRESHOLD` | `0.85` | L2 cosine firing point (trigram-honest; set higher for semantic backends). |
337
336
  | `MEGACOMPACT_L1_JACCARD` | `0.8` | L1 MinHash/LSH near-dup Jaccard threshold. |
338
337
  | `MEGACOMPACT_MMR_LAMBDA` | `0.5` | MMR retrieval-diversity weight (λ·relevance − (1−λ)·maxSim). |
@@ -349,13 +348,13 @@ checklist, MARK_ONLY degrade) and `docs/RETENTION_POLICY.md` for TTL / soft-dele
349
348
  ### Continuity & memory
350
349
 
351
350
  | Variable | Default | Meaning |
352
- |---|---|---|
351
+ | --- | --- | --- |
353
352
  | `MEGACOMPACT_LEGACY_DURABLE_TRIM` | `false` | Restore the legacy auto-trigger (`ctx.compact()` stops the agent). One-release rollback; default uses live context-event trim + pi native auto-compaction (compact-and-continue). |
354
353
  | `MEGACOMPACT_CROSSREPO_ENABLED` | `true` | Cross-repo recall on resume + `/mega-recall --cross-repo` (HNSW index over every repo). |
355
354
  | `MEGACOMPACT_CROSSREPO_COSINE` | `0.90` | Stricter cosine floor for cross-repo hits (vs `0.85` same-repo). |
356
355
  | `MEGACOMPACT_MEMORY_AUTO_REVIEW` | `true` | Auto-review the conversation every `MEGACOMPACT_MEMORY_REVIEW_INTERVAL` turns → durable memories. |
357
356
  | `MEGACOMPACT_MEMORY_REVIEW_INTERVAL` | `10` | Turns between auto-review cycles. |
358
- | `MEGACOMPACT_PGLITE_DISABLED` | _(unset — index on)_ | Kill-switch for the PGlite/HNSW cross-repo index; set `1`/`true` to disable (falls back to sync per-session scan). |
357
+ | `MEGACOMPACT_PGLITE_DISABLED` | *(unset — index on)* | Kill-switch for the PGlite/HNSW cross-repo index; set `1`/`true` to disable (falls back to sync per-session scan). |
359
358
 
360
359
  ## Dashboard
361
360