pi-memory-evolution 0.2.4 → 0.2.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -2,6 +2,20 @@
2
2
 
3
3
  All notable changes to pi-memory-evolution are documented here.
4
4
 
5
+ ## [0.2.6](https://github.com/btnalit/pi-memory-evolution/compare/v0.2.5...v0.2.6) (2026-09-09)
6
+
7
+
8
+ ### Bug Fixes
9
+
10
+ * reach an allowlisted sibling model and give claims real headroom ([#16](https://github.com/btnalit/pi-memory-evolution/issues/16)) ([ffc47d1](https://github.com/btnalit/pi-memory-evolution/commit/ffc47d148355d5cbf2d77bb5edd1d0111947b7e1))
11
+
12
+ ## [0.2.5](https://github.com/btnalit/pi-memory-evolution/compare/v0.2.4...v0.2.5) (2026-09-09)
13
+
14
+
15
+ ### Bug Fixes
16
+
17
+ * recover memory jobs with bounded cross-provider fallback ([#14](https://github.com/btnalit/pi-memory-evolution/issues/14)) ([76cfa6f](https://github.com/btnalit/pi-memory-evolution/commit/76cfa6fcd257866d6ed98a5b6a9fd14cc1b914a7))
18
+
5
19
  ## [0.2.4](https://github.com/btnalit/pi-memory-evolution/compare/v0.2.3...v0.2.4) (2026-09-09)
6
20
 
7
21
 
package/README.cn.md CHANGED
@@ -15,7 +15,7 @@
15
15
  - **相关内容注入**:结合当前问题和近期用户上下文选择记忆,过滤弱匹配和失效状态;没有匹配就不填充无关内容。
16
16
  - **跨会话召回**:在共享记忆库的不同会话、目录中查找背景;提供只读 `memory_recall` 工具,供助手在任务中途补查。
17
17
 
18
- 学习使用 Pi 当前模型和已有认证,无需另配 API Key、向量数据库或嵌入服务。
18
+ 学习默认使用 Pi agent 当前模型和已有认证;额度耗尽、限流或重复故障时,可切换到 Pi 已配置的其他供应商模型,不改变主对话模型。无需另配 API Key、向量数据库或嵌入服务。详见[重试、切换与预算](docs/recovery.md)。
19
19
 
20
20
  ## 安装
21
21
 
@@ -42,7 +42,7 @@ pi install https://github.com/btnalit/pi-memory-evolution
42
42
  /memory status
43
43
  ```
44
44
 
45
- 出现 `SQLite ok (schema 6)` 表示存储初始化成功。后续正常使用 Pi 即可,学习和召回会自动运行。
45
+ 出现 `SQLite ok (schema 7)` 表示存储初始化成功。后续正常使用 Pi 即可,学习和召回会自动运行。
46
46
 
47
47
  ## 使用
48
48
 
@@ -87,7 +87,7 @@ Git 安装请把上面的包来源替换为安装时使用的仓库 URL。操作
87
87
 
88
88
  数据默认保存在 `~/.pi/agent/agent-suite/memory-evolution/`,使用本地 SQLite。`PI_CODING_AGENT_DIR` 可改变存储前缀;不同工作目录默认共享记忆库。
89
89
 
90
- 学习会将经过过滤的来源内容发送给当前模型,并消耗相应额度。记忆不是经过独立验证的事实,匹配和敏感信息过滤也并非万无一失;重要内容仍需核实。详见[存储与隐私](docs/usage.md#local-storage-and-provenance)。
90
+ 学习会将经过过滤的来源内容发送给当前模型或允许使用的备用供应商,并消耗相应额度。默认启用跨供应商切换;可通过[恢复配置](docs/recovery.md)限制供应商模型名单或关闭切换。所有备用模型共享调用上限。记忆不是经过独立验证的事实,匹配和敏感信息过滤也并非万无一失;重要内容仍需核实。详见[存储与隐私](docs/usage.md#local-storage-and-provenance)。
91
91
 
92
92
  ## 开发
93
93
 
package/README.md CHANGED
@@ -22,8 +22,10 @@ assistant to remember them.
22
22
  - **Recall across sessions.** Find background from another session or directory.
23
23
  A read-only `memory_recall` tool lets the assistant look up missing context mid-task.
24
24
 
25
- Learning uses Pi's active model and existing authentication. No separate API key,
26
- embedding service or vector database is required.
25
+ Learning defaults to Pi's active model and existing authentication. Quota/rate limits
26
+ or repeated failures can use an available model from another configured provider, without
27
+ changing the foreground model. [Fallback and budgets](docs/recovery.md) are configurable;
28
+ no separate API key, embedding service or vector database is required.
27
29
 
28
30
  ## Install
29
31
 
@@ -46,7 +48,7 @@ Choose **one** source, then run these commands inside Pi:
46
48
  /memory status
47
49
  ```
48
50
 
49
- `SQLite ok (schema 6)` confirms storage initialization. Continue using Pi normally;
51
+ `SQLite ok (schema 7)` confirms storage initialization. Continue using Pi normally;
50
52
  learning and recall run automatically.
51
53
 
52
54
  ## Use
@@ -96,7 +98,9 @@ Local SQLite state lives in `~/.pi/agent/agent-suite/memory-evolution/`.
96
98
  `PI_CODING_AGENT_DIR` changes that prefix; different working directories share the
97
99
  same store by default.
98
100
 
99
- Learning sends filtered source content to the active model and consumes its quota.
101
+ Learning sends filtered source content to the active model, or an allowed fallback provider,
102
+ and consumes that provider's quota. Cross-provider fallback is enabled by default; restrict
103
+ its allowlist or disable it in [recovery configuration](docs/recovery.md).
100
104
  Memories are historical evidence, not independently verified facts. Matching and
101
105
  secret filtering are imperfect; verify important claims. See [privacy and storage](docs/usage.md#local-storage-and-provenance).
102
106
 
@@ -195,7 +195,7 @@ reset, automatic state revival or fabricated verification occurs.
195
195
  Reads validate evidence/feedback shape and tool-observation kind, and return independent
196
196
  copies of nested metadata. Status validates receipt fields in addition to existing
197
197
  record/job/history integrity. Unsupported schema versions fail before DDL. Old builds
198
- reject schema 6: do not manually downgrade its marker.
198
+ reject schema 7: do not manually downgrade its marker.
199
199
 
200
200
  Stop all Pi processes sharing the state directory and back up the complete state before
201
201
  activation. Update/restart all those processes together (or reload after a consistent
package/docs/design.md CHANGED
@@ -177,7 +177,7 @@ multiple projects. The prompt requires an explicitly identifiable same subject/f
177
177
  preservation of project/resource qualifications; matching cwd alone is not identity.
178
178
 
179
179
  Output is validated JSON (an outer Markdown fence is tolerated), at most 64,000 bytes
180
- and 16 claims of 4–480 UTF-16 code units each. Fields are restricted to `kind`, `content`,
180
+ and 16 claims of 4–800 UTF-16 code units each. Fields are restricted to `kind`, `content`,
181
181
  optional `replaces` and `searchTerms`. Aliases are at most 8 sanitized strings of 2–64
182
182
  characters, with total JSON <=1024 bytes. Malformed claims/aliases reject the batch.
183
183
  The prompt asks for concise Chinese/English aliases, never added facts. Existing text
@@ -194,9 +194,9 @@ not detected globally.
194
194
 
195
195
  Each attempt uses at most one model call, no tools, an 8,192-output-token cap (clamped
196
196
  against a smaller model limit), a fresh request session ID and `cacheRetention: "none"`.
197
- A 120-second outer deadline bounds waiting
198
- even when a provider ignores abort; remote computation/billing cannot be guaranteed to
199
- stop. Failed calls retain local summary claims. User-cue prose is saved but needs a
197
+ A 120-second per-attempt deadline bounds waiting even when a provider ignores abort;
198
+ remote computation/billing cannot be guaranteed to stop. A backup has a fresh deadline,
199
+ clamped by the source's remaining 300-second cumulative allowance. Failed calls retain local summary claims. User-cue prose is saved but needs a
200
200
  successful model attempt to become claims; it has no local extraction fallback.
201
201
 
202
202
  Global recall is **not global rewriting**. Cross-origin variants remain separate instead
@@ -223,6 +223,12 @@ One SQLite database, WAL + FULL synchronous mode and private file permissions.
223
223
  - `feedback_receipts`: exact source-ID/memory-ID idempotency keys, verdict and numeric
224
224
  timestamp, including redundant/late feedback receipts; undo never reopens them.
225
225
  - `metadata`: schema/import marker.
226
+ - `model_calls`: reserved attempts, selected model/provider, outcome, usage and estimated/reported cost.
227
+ - `route_health`: provider/model cooldowns, separate from source backoff.
228
+ - `recovery_notices`: bounded persistent notice deduplication keys.
229
+ Source-level call/model/time/correction counts survive receipt pruning; shared ceilings are
230
+ reserved transactionally. The active model is preferred; allowed backups come only from
231
+ other Pi-configured providers. [Recovery policy and privacy](recovery.md) define the bounds.
226
232
 
227
233
  The `scope` field records canonical cwd, not an inferred repository/branch/subject identity
228
234
  (or an explicit annotation of a legacy record). It is no longer a recall boundary. Reads
@@ -239,13 +245,16 @@ failure with backoff; attempt/state checks prevent late results or failures from
239
245
  a new owner's job. Model waiting and recurring recovery use the same serial task chain;
240
246
  queued capture/manual work prevents the timer from piling up duplicate tasks.
241
247
 
242
- Automatic selection/claim both enforce persisted due time and failure budget, ordered by
243
- retry time then oldest source. Each actual failure schedules 1 minute, 5 minutes, 15 minutes,
244
- then 1 hour of backoff. Five consecutive failures pause that source with a warning and
245
- status diagnostics; repeated crashes also consume the budget. Shutdown cancellation does
246
- not. Successful completion resets the failure fields. Other eligible work continues;
247
- there is no unbounded per-source model loop. `/memory evolve` selects one newest eligible
248
- source and can override the delay/cap for one explicit attempt (not reset the budget).
248
+ Automatic selection/claim enforce source backoff and call/time/failure caps. Least-recently
249
+ checked ordering prevents temporarily unroutable sources starving other work. Provider/model
250
+ cooldowns never contaminate a source's retry_at. Generic runtime backoff is 1 minute,
251
+ 5 minutes, 15 minutes, then 1 hour with up to 20% jitter; quota/auth/rate-limit errors can
252
+ immediately use an allowed other provider. Two recent transport failures pause a model.
253
+ Defaults allow 4 reserved calls / 2 models / 300 seconds per source, one format correction,
254
+ and 20 reservations/hour shared across providers and processes. Unsafe writes/refusals
255
+ pause immediately. Three output failures or five generic failures also pause work.
256
+ Shutdown adds no failures, but an already reserved request may still cost money.
257
+ `/memory evolve` overrides source delay/caps for one attempt, never shared ceilings.
249
258
  Completed/retired jobs are never forced to run again. A source resumed in another directory
250
259
  retains its original provenance. While Pi is closed no polling occurs.
251
260
 
@@ -266,9 +275,9 @@ Memory reads validate indexed identity/origin/hash against JSON. Undo validates
266
275
  unique before/after IDs and only succeeds when the current records still equal the event's
267
276
  after state. New records become tombstones rather than being physically erased. Status
268
277
  also validates source jobs/history and the schema marker. Unsupported schema versions
269
- are rejected before DDL. Schemas 2/3/4 upgrade transactionally to 5 without rewriting
270
- claims/history or resetting evidence timestamps; missing retry columns/indexes and the
271
- feedback receipt table are added. Old evidence metadata stays absent/unknown.
278
+ are rejected before DDL. Schemas 2–6 upgrade transactionally to 7 without rewriting
279
+ claims/history or resetting evidence timestamps. Missing retry/feedback tables and routing
280
+ accounting are added; known v6 model waits are separated from source backoff. Old evidence metadata stays absent/unknown.
272
281
  Old failures below the cap are due immediately, with unknown cause/time explicitly labeled.
273
282
  The source/alias/retry contract is validated on read.
274
283
  These detect structural corruption, not all well-formed edits by an owner of the database.
@@ -278,7 +287,7 @@ Undo does not clear suppression hashes or reopen jobs. Forget/undo is not secure
278
287
 
279
288
  Earlier 0.2 SQLite records, IDs, histories and origin labels stay intact and become
280
289
  eligible for global relevance-based recall, including existing `legacy` claims. The
281
- schema-2/3/4-to-5 upgrade requires no data copying, JSONL re-import or manual reset.
290
+ schema-2-through-6-to-7 upgrade requires no data copying, JSONL re-import or manual reset.
282
291
  Stop/back up before upgrading, reload all processes sharing the DB, and restore a matching
283
292
  backup for rollback; older code must not be pointed at a manually downgraded marker.
284
293
 
@@ -288,6 +297,9 @@ facts. Parent corrections preserve unchanged children and derive new facts witho
288
297
  reintroducing explicitly suppressed child content. Missing origins retain the `legacy`
289
298
  label. Adoption is an optional annotation, not a recall prerequisite. Completed imports
290
299
  are not replayed; earlier discarded revision information is not automatically reconstructed.
300
+ Empty/missing ledgers do not consume the import opportunity. Only saved digests proving a
301
+ recognized empty v6 snapshot (without a migration event) can reopen old completed markers;
302
+ zero derived claims from a real consumed ledger are not sufficient.
291
303
  Old signals/proposals/execution plans remain historical files, never automatic actions.
292
304
 
293
305
  List defaults to all origins, 20 per page sorted by update time then ID; `all` is an alias.
@@ -0,0 +1,134 @@
1
+ # Recovery, provider fallback and budgets
2
+
3
+ Memory learning defaults to **Pi's active agent model**. With no session override this
4
+ is Pi's configured default. The extension never sets the foreground model, creates a
5
+ second credential store, changes provider configuration, or probes models to rank them.
6
+ It discovers available models through Pi's registry and uses registry completion/auth.
7
+
8
+ ## Provider fallback
9
+
10
+ Cross-provider fallback is enabled by default. If the default route is unavailable,
11
+ learning may send the sanitized source and selected candidate memories to another
12
+ **already-configured, available Pi provider**. This is a privacy/cost boundary: configure
13
+ an explicit allowlist or disable fallback below if some configured providers must not
14
+ receive memory data. Installing this update does not require per-request approvals.
15
+
16
+ The active model is always first when eligible. Backups must support text. Another model of
17
+ the **same** provider is never chosen automatically, because it shares that provider's
18
+ credentials and account quota. Naming one in `fallbackModels` overrides that: it is the only
19
+ redundancy available when Pi has a single configured provider, and it is then used for
20
+ model-specific failures (`rate_limit`, `invalid_output`, `output_limit`, `context_limit`)
21
+ but still skipped for `auth` and `quota`, which no sibling can escape. An explicit
22
+ fallback list defines priority; otherwise known catalog input/output price, non-reasoning
23
+ preference and stable model ID order choose among available backups. Unknown/zero prices
24
+ are not assumed free. Catalog availability is not proof of working credentials or credit.
25
+ At most two models may receive any one source by default, including the primary.
26
+
27
+ All source/model budgets are checked in a SQLite write transaction. Selection does not
28
+ grant call authority. Provider cooldowns are shared across sources and Pi processes.
29
+ A cooldown for A does **not** change a source's retry timestamp or prevent B from working.
30
+ Last-checked ordering moves temporarily unroutable sources behind other eligible work.
31
+
32
+ ## Error policy
33
+
34
+ | Result | Automatic action |
35
+ | --- | --- |
36
+ | HTTP 401/403 | Cool down that provider for 15 minutes; try an allowed other provider. Pi owns auth resolution/refresh. |
37
+ | HTTP 402 or fixed quota/billing code | Cool down the provider for 1 hour or a later supplied reset; try another provider. |
38
+ | HTTP 429 without a known quota code | Cool down that **model** for at least 60 seconds and at least `Retry-After`, since tpm/rpm ceilings are usually per model; try another provider, or an allowlisted model of the same provider. |
39
+ | HTTP 408, timeout, network/5xx failure | Delayed retry; two recent transport failures cool down that model for 15 minutes and permit fallback. |
40
+ | Context overflow or HTTP 400/404/422 | Do not resend unchanged requests to that model immediately; cool it down for 1 hour and permit a compatible backup. |
41
+ | Invalid JSON / output truncation | Initial attempt, at most one corrective prompt, then an alternate model. Three output failures pause the source; two failures do not authorize repeatedly probing the same model. |
42
+ | Unsafe/unauthorized write or recognized safety/refusal | Reject and pause that source. No fallback to circumvent safety or write guards. |
43
+ | Stale result | Re-read on a bounded delayed retry; not counted as a provider-health failure. |
44
+ | Cancellation / shutdown / reload | Release the lease without adding a failure; already-reserved requests may still have consumed quota. |
45
+ | Unknown error | Safe generic category and bounded transport retry; no guessing that arbitrary error prose means insufficient credit. |
46
+
47
+ Generic source backoff is 1 minute, 5 minutes, 15 minutes, then 1 hour, with up to 20%
48
+ positive jitter on runtime failures. Route failures can try one alternate immediately;
49
+ there are at most two immediate attempts per queue item, not a sleep/retry loop. The
50
+ 120-second deadline is **per attempt**, not an outer timeout that also kills its backup.
51
+ Request deadlines are clamped to the source's remaining cumulative time budget. A provider
52
+ ignoring cancellation cannot commit late output, but remote execution/billing may continue.
53
+
54
+ HTTP status/error codes are observed before the underlying SDK consumes failed responses,
55
+ using request-local fetch only for verified supporting APIs: OpenAI completions/responses,
56
+ Azure responses, Codex responses, Anthropic messages and Mistral conversations. Codex memory
57
+ requests use HTTP SSE; foreground transport is unchanged. Other adapters retain their normal
58
+ transport. Complete structured JSON errors (including Google SDK errors) can also supply a
59
+ fixed status/code; arbitrary text is never searched for status-like numbers or echoed.
60
+ Only known code values, numeric `Retry-After`/reset metadata and raw stop enums are used.
61
+ Unsupported/opaque SDK errors can still be generic `provider`; no universal balance API is
62
+ claimed. Error-body inspection is transient, bounded to 8192 bytes and 200 ms, never stored.
63
+
64
+ Input planning drops lower-priority existing candidates when a conservative byte/token
65
+ estimate exceeds the chosen context window. It does not truncate facts or progress JSON to
66
+ pretend they fit. If the original evidence still cannot fit, that route is rejected. Model
67
+ mistakes and provider-specific tokenization remain possible; this is not exact token counting.
68
+
69
+ ## Optional configuration
70
+
71
+ Create `recovery.json` **in the memory state directory**, not in Pi's settings file. Reload
72
+ all Pi processes sharing that directory after changing policy. Defaults are:
73
+
74
+ ```json
75
+ {
76
+ "crossProviderFallback": true,
77
+ "fallbackModels": [],
78
+ "callsPerHour": 20,
79
+ "sourceCalls": 4,
80
+ "sourceModels": 2,
81
+ "timeoutMs": 120000,
82
+ "sourceTimeMs": 300000,
83
+ "dailyEstimatedUsd": null
84
+ }
85
+ ```
86
+
87
+ `fallbackModels` optionally contains exact `provider/model-id` strings, in priority order.
88
+ Only models Pi reports available are considered. It does not override the default model
89
+ or configure authentication. `crossProviderFallback: false` confines learning to the active
90
+ model. Invalid policy fails closed with a fixed error, never prints configuration contents.
91
+
92
+ Limits apply together:
93
+
94
+ - **20 reservations per rolling hour across all models/providers/processes**, not 20 per
95
+ fallback model. An exhausted shared budget cannot be bypassed by switching providers.
96
+ - **4 reserved calls, 2 models and 300 seconds per source** by default. Reservations include
97
+ failed/preflight/cancelled attempts conservatively; they are not exact server billing counts.
98
+ - One persisted format-correction allowance survives cancellation/reload. Safety validation
99
+ and transactional all-or-nothing writes apply equally to every backup model.
100
+ - The older five-failure safety cap is also retained, including crashes and pre-upgrade work.
101
+ - `/memory evolve [source-id]` may override source backoff/route cooldown/source caps for
102
+ **one** explicit attempt, without clearing counters. It cannot bypass shared request or
103
+ estimated-cost ceilings and does not loop through backup models. Settled jobs are not rerun.
104
+
105
+ `dailyEstimatedUsd` is disabled by default because provider catalogs can contain absent,
106
+ zero, outdated or non-billing prices (subscriptions and local models are common). Set a
107
+ positive value to enforce a rolling 24-hour **catalog-estimated** spending ceiling.
108
+ Unknown pricing fails closed under this optional ceiling. Reservation uses a conservative
109
+ input-byte bound, maximum configured pricing tier and output allowance; returned usage/cost
110
+ replaces it when available. Missing usage and cancelled/error responses are not assumed
111
+ free. These estimates are not account balance, billing receipts, or a guaranteed invoice cap.
112
+ Actual provider accounting can differ. Token counts, estimates and unknown-cost counts are
113
+ shown separately from Pi's foreground session totals.
114
+
115
+ ## Diagnostics and upgrade
116
+
117
+ `/memory status` shows the current default, allowed routes, provider/model cooldowns,
118
+ source attempts/calls/output failures/time, shared budget and recent selected-model outcomes.
119
+ A successful fallback can show one informational notice per route pair/hour. Failures and
120
+ pause summaries are also deduplicated across reloads, not re-armed by unrelated successes.
121
+ No exception bodies, credentials or failed model text are persisted in diagnostics.
122
+
123
+ Schema **2–6 upgrade to 7**. Stop Pi and back up the whole state directory first; restart or
124
+ reload all processes after updating. Claims, IDs, histories, forgetting and evidence dates
125
+ are preserved. Existing v6 call receipts seed source counts/model history where available;
126
+ missing history is not invented. Known v6 route-contaminated waits are separated from the
127
+ original source backoff. Earlier paused five-failure jobs are not indiscriminately reset.
128
+ Older code rejects schema 7; rollback requires a matching backup, not editing the marker.
129
+
130
+ Empty/missing legacy input no longer consumes the one-time import opportunity. Supply valid
131
+ ledgers and use `/memory import [directory]`. A v6 completed/zero-count marker is reopened
132
+ only when its saved digest proves a recognized empty snapshot and no migration event exists.
133
+ A real consumed ledger with no derived claims remains completed; zero count alone cannot
134
+ justify replay over later corrections/forgetting. Originals and copy-only archives are kept.
package/docs/testing.md CHANGED
@@ -17,6 +17,17 @@ helper scripts and memory state must not ship in the package.
17
17
 
18
18
  `check:package` inspects `npm pack --dry-run --json`; it does not publish to npm.
19
19
 
20
+ ## Recovery and routing regressions
21
+
22
+ `http-diagnostics.test.ts` and `fallback-http.test.ts` use the real Pi ModelRuntime/Registry
23
+ with loopback HTTP responses, not mocks that simply invoke `onResponse`. They verify HTTP
24
+ 401/403/400/429/503, quota/context/safety codes, Retry-After, Google SDK JSON errors, and
25
+ cross-provider fallback with the backup's own authentication. No live provider is contacted.
26
+ `scheduler.test.ts` covers bounded correction, per-attempt timeout/late-output rejection,
27
+ provider cooldowns, allowlists, shared hard ceilings and persistent source limits.
28
+ `reliability-regressions.test.ts` covers empty ledgers, safe v6-marker repair, preserving
29
+ zero-claim real imports, schema-6 scheduling migration, and case-sensitive resource identity.
30
+
20
31
  ## npm CLI compatibility
21
32
 
22
33
  All consumers of `npm pack --json` share `scripts/lib/npm-pack.mjs`. npm 10/11
@@ -67,7 +78,7 @@ host explicitly. The script prints the tested host version. The test:
67
78
  1. Builds and extracts the actual npm tarball, outside the development checkout.
68
79
  2. Uses real `pi install`, `pi list` and default package discovery—no explicit `-e`
69
80
  entry or memory-extension wrapper. Checks `/memory status`, `/memory learning`
70
- and `/memory explain`, including schema 6, with no checkout `node_modules`.
81
+ and `/memory explain`, including schema 7, with no checkout `node_modules`.
71
82
  3. Verifies repeated installation does not duplicate the package setting, then
72
83
  removes it and confirms the command disappears while records/history remain.
73
84
  4. Creates a local Git origin from the packed files and the real lockfile. A
@@ -75,7 +86,7 @@ host explicitly. The script prints the tested host version. The test:
75
86
  `file` transport is permitted. No GitHub access is needed.
76
87
  5. Exercises the native Git installer and its real npm dependency step, updates to
77
88
  a new commit, switches from an old pinned tag back to the default branch, and
78
- verifies source switching and removal preserve schema-6 state.
89
+ verifies source switching and removal preserve schema-7 state.
79
90
  6. Serves the actual tarball through a loopback npm registry, with a fresh cache,
80
91
  then checks native `pi install npm:pi-memory-evolution`, repeat installation,
81
92
  normal loading and removal. No host peer packages are served or installed.
package/docs/usage.md CHANGED
@@ -10,6 +10,7 @@ This reference covers detailed behavior and optional controls; normal use is aut
10
10
  - [Commands](#commands)
11
11
  - [Migration](#migration-from-01)
12
12
  - [Recovery and troubleshooting](#recovery-and-troubleshooting)
13
+ - [Provider fallback, error policies and budgets](recovery.md)
13
14
 
14
15
  ## Installation and updates
15
16
 
@@ -110,9 +111,11 @@ rename the tool to hide the conflict; an old installation would still run its ho
110
111
  - Each processing attempt makes at most one background model call, using up to 32
111
112
  recently updated active memories from that source's capture origin. This is a
112
113
  conservative automatic-replacement safeguard, **not a recall restriction**.
113
- It uses **the current Pi session model and Pi's own provider/auth resolution**.
114
- With no model override in the session, this is Pi's configured default model.
115
- There is no extra API key, provider setting, subagent, or alternate-model fallback.
114
+ It defaults to **the current Pi session model and Pi's own provider/auth resolution**.
115
+ With no session override, this is Pi's configured default. Quota/rate limits or repeated
116
+ failures may switch to an available model from another configured provider, not a sibling
117
+ sharing the default provider's quota. The foreground model is never changed. See
118
+ [fallback privacy, error handling and budgets](recovery.md); no extra credentials are stored.
116
119
  - Valid additions/replacements commit immediately, with provenance and before/after
117
120
  history. Inferred memories remain labeled `provisional`, but are recallable without
118
121
  approval. Pinned memories cannot be automatically replaced.
@@ -123,14 +126,17 @@ rename the tool to hide the conflict; an old installation would still run its ho
123
126
  learning its claims requires a successful model attempt.
124
127
  - Session start checks persisted work across **all origins**. While Pi remains running,
125
128
  a local recovery timer checks every **15 seconds** (after the previous check/call ends),
126
- gradually draining eligible pending/failed work one source per check. Selection favors
127
- the earliest retry time, then oldest capture; a failed source does not block other work.
128
- - Failed attempts retry automatically after **1 minute, 5 minutes, 15 minutes, then 1 hour**.
129
- After **5 consecutive failures**, that source pauses and a warning points to diagnostics;
130
- there are no indefinite paid probes. Retry times/budgets survive reloads and restarts.
131
- `/memory status` shows safe error categories, failure times, attempt counts, next retry
132
- times and paused counts. `/memory evolve` remains an optional one-off override of the
133
- delay/limit, not the normal recovery path; successful/retired jobs are never rerun.
129
+ gradually draining eligible pending/failed work one source per check (at most one immediate
130
+ backup per check). Selection favors least recently checked work, then retry time and oldest
131
+ capture, so unavailable routes do not indefinitely block other sources.
132
+ - Recovery is **error-specific**, with persisted source backoff, separate provider/model
133
+ cooldowns, at most one output correction, and bounded cross-provider fallback. Generic
134
+ backoff is 1 minute, 5 minutes, 15 minutes, then 1 hour, plus up to 20% jitter. Defaults
135
+ cap each source at **4 reserved calls / 2 models / 300 seconds**, and all models and
136
+ processes share **20 calls per rolling hour**. Three output failures or five generic
137
+ failures also pause work; unsafe writes/refusals pause immediately. `/memory status`
138
+ explains routes, budgets and actual outcomes. `/memory evolve` overrides source limits
139
+ for one attempt, never shared ceilings or settled jobs. [Full policy/configuration](recovery.md).
134
140
  - A job lease lasts **150 seconds** (120-second deadline plus 30-second grace). The timer
135
141
  detects expired running jobs and schedules them with the same bounded backoff. Shutdown/
136
142
  reload cancellation returns work to pending without consuming the failure budget.
@@ -158,7 +164,7 @@ rename the tool to hide the conflict; an old installation would still run its ho
158
164
  recall regardless of its original directory.
159
165
  - Query coverage, evidence-based document frequency, field weights, mild length
160
166
  normalization and a relative cutoff reject weak secondary matches. Unseen query words
161
- no longer receive the highest rarity weight. Exact paths must match, and a quoted
167
+ no longer receive the highest rarity weight. Exact paths must match, including case; `/srv/Atlas` and `/srv/atlas` are distinct. A quoted
162
168
  question in a replay/incident note is weaker than evidence answering it. Redundancy
163
169
  filtering cannot let a project-state note hide a preference of the same origin.
164
170
  Source IDs/cwd have no authority bonus. After relevance gates, host-assigned evidence,
@@ -181,8 +187,10 @@ There is no additional call on ordinary recall. Eligible work turns may now incu
181
187
  additional background call each; no related tracked state or no tool observation means
182
188
  no progress call. A mixed statement/work turn may additionally incur a separate learning
183
189
  call; interrupted turns with usable observations may also learn through automatic recovery.
184
- Automatic retries may add up to four calls per source after the initial
185
- failure; every attempt uses the session's then-current model/authentication. Model mistakes
190
+ Automatic retries/fallback share a default total of four reserved calls per source;
191
+ they do not get separate budgets per model. Each request uses Pi's authentication for the
192
+ selected provider. Cross-provider data sharing and optional estimated-cost limits are
193
+ explained in [recovery configuration](recovery.md). Model mistakes
186
194
  remain possible; tool observations and model-generated aliases are not proof of truth. Use
187
195
  history, correction, pinning and undo rather than treating generated claims as verified facts.
188
196
 
@@ -270,7 +278,9 @@ Sensitive lines/blocks are suppressed before capture, edits, model submission an
270
278
  recall. This covers common token/password/JSON/Chinese/Bearer/private-key formats,
271
279
  including quoted multiline values, indented YAML blocks and control-character cleanup,
272
280
  but not every possible secret. Do not rely on a regex as a complete DLP system. Sanitized
273
- sources and selected existing memories go to the already-configured Pi model provider.
281
+ sources and selected existing memories go to the active Pi model provider or an allowed
282
+ already-configured fallback provider. Fallback is enabled by default; restrict the model
283
+ allowlist or disable it in [recovery configuration](recovery.md) to constrain data sharing.
274
284
 
275
285
  ## Commands
276
286
 
@@ -291,10 +301,10 @@ These are optional direct controls, **not approval gates**:
291
301
  /memory evolve # optional one-off retry, overriding delay/failure limit
292
302
  /memory evolve <source-id> # retry one named source without clearing other failure counts
293
303
  /memory import [directory] # explicit, repeat-safe legacy JSONL import; never replays a completed one
294
- /memory archive-legacy # copy inactive legacy plan files to a timestamped archive; originals kept
304
+ /memory archive-legacy # copy inactive legacy plan files to a private unique archive; originals kept
295
305
  /memory undo <event-id> # reverse actual changes, if not modified since
296
306
  /memory feedback <id> <verdict> # useful | unhelpful | accurate | incorrect
297
- /memory correct <id> <replacement> # literal replacement, 4–480 characters
307
+ /memory correct <id> <replacement> # literal replacement, 4–800 characters
298
308
  /memory forget <id>
299
309
  /memory pin <id>
300
310
  /memory unpin <id>
@@ -350,18 +360,23 @@ preserve unchanged children of corrected summaries and carry superseded-content
350
360
  when corrected legacy claims are adopted into a project.
351
361
 
352
362
  **Upgrading from earlier 0.2 development builds:** stop Pi and back up the state directory
353
- first, then update/reload all Pi processes sharing it. Schema markers **2, 3 and 4 upgrade
354
- transactionally to 5**, adding replay-safe feedback receipts and the optional evidence/feedback
355
- contract (plus missing retry fields for older schemas). Existing records, IDs, histories and
363
+ first, then update/reload all Pi processes sharing it. Schema markers **2–6 upgrade
364
+ transactionally to 7**, adding missing recovery/feedback fields, source call/time/model
365
+ budgets, provider cooldowns and usage receipts. Known v6 route waits are separated from
366
+ source failure backoff; retained call history seeds counters without inventing missing calls. Existing records, IDs, histories and
356
367
  evidence dates remain unchanged. Missing evidence stays unknown, with no fabricated backfill.
357
368
  Existing failures below the limit become automatically eligible; their old error cause/time
358
369
  remain labeled unknown rather than invented. No copying, manual marker reset or JSONL
359
- re-import is needed. Older builds reject schema 6; rollback requires a matching backup,
370
+ re-import is needed. Older builds reject schema 7; rollback requires a matching backup,
360
371
  not editing the marker. Existing records are immediately eligible for global relevance-based recall.
361
372
  Legacy claims previously excluded by cwd filtering become eligible too. This shares
362
373
  relevant stored claims with the active Pi session/provider, not raw session archives.
363
374
 
364
- These migration fixes do not replay an already completed import or retroactively erase
375
+ Empty/missing ledgers do not mark an import completed. After supplying files, use
376
+ `/memory import [directory]`. Provably empty v6 snapshots can reopen safely; a zero claim
377
+ count alone never authorizes replay of a real consumed ledger. See [migration details](recovery.md).
378
+
379
+ These migration fixes do not replay an already completed real import or retroactively erase
365
380
  previously stored sensitive data. If an older import already lost revision information,
366
381
  use the preserved ledgers/backup to review and correct affected records; do not reset the
367
382
  migration marker or replace a populated database blindly.
@@ -395,10 +410,11 @@ JSONL, not changes made in the new database.
395
410
  automatically processes due work. `/memory status` distinguishes scheduled retries from
396
411
  paused sources and shows the next eligible time (the next poll may be up to 15 seconds
397
412
  later when no local work is queued). After a crash, the 150-second lease must expire first.
398
- If a source reaches 5 failures, inspect its category: `timeout`, `output_limit`,
399
- `invalid_output`, `write_rejected`, `stale`, `provider`, `unavailable`, `interrupted` or
400
- `unknown`. Fix model/authentication or investigate repeated output/write rejection;
401
- `/memory evolve` can then make one extra attempt without resetting the automatic budget.
413
+ Inspect source call/time/output limits as well as failure count. Status distinguishes
414
+ `auth`, `quota`, `rate_limit`, `context_limit`, `request`, `safety`, transport, output and
415
+ stale-write failures. Provider cooldowns are separate from source retry times; switching
416
+ providers cannot bypass the shared budget. `/memory evolve` makes one explicit source
417
+ attempt without resetting counters or overriding shared ceilings. See [error policy](recovery.md).
402
418
  A bad source remains saved/visible rather than being deleted or labeled successful.
403
419
  - **Persistent storage/import errors:** stop all Pi processes using that agent directory
404
420
  and back up the **entire** state directory, including any SQLite sidecars and legacy
@@ -411,4 +427,4 @@ JSONL, not changes made in the new database.
411
427
  stopped; do not mix one backup's database with another's WAL/SHM files.
412
428
 
413
429
  Diagnostics intentionally do not echo provider error bodies, which may contain secrets.
414
- `SQLite ok (schema 6)` checks database structure/record validity, not the truth of model claims.
430
+ `SQLite ok (schema 7)` checks database structure/record validity, not the truth of model claims.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-memory-evolution",
3
- "version": "0.2.4",
3
+ "version": "0.2.6",
4
4
  "description": "Memory that maintains itself. Pi learns what matters, injects what this session needs, and recalls the rest — nothing to configure, no commands to learn.",
5
5
  "license": "MIT",
6
6
  "publishConfig": {
@@ -0,0 +1,76 @@
1
+ import type { Diagnostic } from '../memory/diagnostics.ts';
2
+ import type { FailureCode } from '../memory/recovery.ts';
3
+
4
+ // Only adapters verified to support request-local fetch injection. Never monkey-patch global fetch,
5
+ // replace auth/proxy composition, or pass this option to SDKs (e.g. Google/Bedrock) that reject it.
6
+ export const OBSERVABLE_HTTP_APIS = new Set(['openai-completions', 'openai-responses', 'azure-openai-responses',
7
+ 'openai-codex-responses', 'anthropic-messages', 'mistral-conversations']);
8
+ const ERROR_CODES: Record<string, FailureCode> = {
9
+ insufficient_quota: 'quota', quota_exceeded: 'quota', billing_hard_limit_reached: 'quota', insufficient_balance: 'quota',
10
+ usage_limit_reached: 'quota', usage_not_included: 'quota',
11
+ context_length_exceeded: 'context_limit', max_context_length_exceeded: 'context_limit',
12
+ content_filter: 'safety', content_policy_violation: 'safety', safety: 'safety',
13
+ };
14
+ export function httpFailure(d: Diagnostic): FailureCode {
15
+ if (d.errorClass) return d.errorClass;
16
+ return d.httpStatus === 401 || d.httpStatus === 403 ? 'auth' : d.httpStatus === 402 ? 'quota'
17
+ : d.httpStatus === 429 ? 'rate_limit' : d.httpStatus === 408 ? 'timeout'
18
+ : [400, 404, 422].includes(d.httpStatus ?? 0) ? 'request' : 'provider';
19
+ }
20
+ export function observeStatus(d: Diagnostic, status: number, headers?: Record<string, string>, now = Date.now()): void {
21
+ if (Number.isSafeInteger(status) && status >= 100 && status <= 599) d.httpStatus = status;
22
+ const retry = Object.entries(headers ?? {}).find(([key]) => key.toLowerCase() === 'retry-after')?.[1];
23
+ if (retry) {
24
+ const delay = /^\d+(?:\.\d+)?$/u.test(retry.trim()) ? Number(retry) * 1000 : Date.parse(retry) - now;
25
+ // Preserve long waits (up to the JS timestamp range); never shorten a server's requested pause.
26
+ if (Number.isFinite(delay) && delay > 0 && now + delay <= 8_640_000_000_000_000) d.retryAfterMs = Math.ceil(delay);
27
+ }
28
+ }
29
+ /** Some SDKs (notably Google) return a complete JSON error document as errorMessage.
30
+ * Decode only that structural shape, not status-looking numbers or prose substrings. */
31
+ export function observeStructuredError(d: Diagnostic, message?: string): void {
32
+ if (!message || Buffer.byteLength(message) > 8192) return;
33
+ try {
34
+ const value = JSON.parse(message);
35
+ const status = value?.error?.code ?? value?.status;
36
+ if (d.httpStatus === undefined && Number.isInteger(status) && status >= 400 && status <= 599) observeStatus(d, status);
37
+ const code = [value?.error?.code, value?.error?.type, value?.code].find(c => typeof c === 'string' && Object.hasOwn(ERROR_CODES, c));
38
+ if (code) d.errorClass = ERROR_CODES[code] as Diagnostic['errorClass'];
39
+ } catch { /* Opaque provider errors stay unknown; no raw text is retained or guessed from. */ }
40
+ }
41
+ export function diagnosticFetch(d: Diagnostic, signal: AbortSignal): typeof globalThis.fetch {
42
+ const underlying = globalThis.fetch;
43
+ return async (input, init) => {
44
+ const response = await underlying(input, init);
45
+ observeStatus(d, response.status, Object.fromEntries(response.headers));
46
+ if (response.status < 400) return response;
47
+ // Inspect only a bounded error JSON clone, without delaying the original body indefinitely.
48
+ // No free-form message, header, URL, credential or body is copied into the diagnostic.
49
+ const reader = response.clone().body?.getReader();
50
+ if (!reader) return response;
51
+ let timer: ReturnType<typeof setTimeout> | undefined;
52
+ const inspect = async () => {
53
+ const chunks: Uint8Array[] = []; let bytes = 0;
54
+ while (!signal.aborted) {
55
+ const { done, value } = await reader.read(); if (done) break;
56
+ bytes += value.byteLength; if (bytes > 8192) return;
57
+ chunks.push(value);
58
+ }
59
+ if (signal.aborted) return;
60
+ try {
61
+ const value = JSON.parse(Buffer.concat(chunks).toString('utf8'));
62
+ const code = [value?.error?.code, value?.error?.type, value?.code].find(c => typeof c === 'string' && Object.hasOwn(ERROR_CODES, c));
63
+ if (code) {
64
+ d.errorClass = ERROR_CODES[code] as Diagnostic['errorClass'];
65
+ const reset = value?.error?.resets_at;
66
+ if (d.errorClass === 'quota' && Number.isSafeInteger(reset) && reset * 1000 > Date.now() && reset * 1000 <= 8_640_000_000_000_000)
67
+ d.retryAfterMs = Math.max(d.retryAfterMs ?? 0, reset * 1000 - Date.now());
68
+ }
69
+ } catch { /* Unknown/HTML/oversized bodies retain only the status. */ }
70
+ };
71
+ try { await Promise.race([inspect(), new Promise<void>(resolve => { timer = setTimeout(resolve, 200); })]); }
72
+ catch { /* Diagnostics must not change the provider's response handling. */ }
73
+ finally { clearTimeout(timer); void reader.cancel().catch(() => {}); }
74
+ return response;
75
+ };
76
+ }