pi-memory-evolution 0.2.4 → 0.2.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +7 -0
- package/README.cn.md +3 -3
- package/README.md +8 -4
- package/docs/core-quality.md +1 -1
- package/docs/design.md +26 -14
- package/docs/recovery.md +130 -0
- package/docs/testing.md +13 -2
- package/docs/usage.md +42 -26
- package/package.json +1 -1
- package/src/adapter/http-diagnostics.ts +76 -0
- package/src/adapter/pi-api.ts +18 -8
- package/src/index.ts +11 -6
- package/src/memory/diagnostics.ts +9 -2
- package/src/memory/evolution.ts +19 -6
- package/src/memory/legacy.ts +7 -2
- package/src/memory/memory-store.ts +114 -49
- package/src/memory/processing-state.ts +53 -12
- package/src/memory/progress-targets.ts +1 -1
- package/src/memory/recovery.ts +3 -3
- package/src/memory/routing-policy.ts +31 -0
- package/src/memory/scheduler.ts +64 -0
- package/src/memory/search.ts +2 -2
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to pi-memory-evolution are documented here.
|
|
4
4
|
|
|
5
|
+
## [0.2.5](https://github.com/btnalit/pi-memory-evolution/compare/v0.2.4...v0.2.5) (2026-09-09)
|
|
6
|
+
|
|
7
|
+
|
|
8
|
+
### Bug Fixes
|
|
9
|
+
|
|
10
|
+
* recover memory jobs with bounded cross-provider fallback ([#14](https://github.com/btnalit/pi-memory-evolution/issues/14)) ([76cfa6f](https://github.com/btnalit/pi-memory-evolution/commit/76cfa6fcd257866d6ed98a5b6a9fd14cc1b914a7))
|
|
11
|
+
|
|
5
12
|
## [0.2.4](https://github.com/btnalit/pi-memory-evolution/compare/v0.2.3...v0.2.4) (2026-09-09)
|
|
6
13
|
|
|
7
14
|
|
package/README.cn.md
CHANGED
|
@@ -15,7 +15,7 @@
|
|
|
15
15
|
- **相关内容注入**:结合当前问题和近期用户上下文选择记忆,过滤弱匹配和失效状态;没有匹配就不填充无关内容。
|
|
16
16
|
- **跨会话召回**:在共享记忆库的不同会话、目录中查找背景;提供只读 `memory_recall` 工具,供助手在任务中途补查。
|
|
17
17
|
|
|
18
|
-
|
|
18
|
+
学习默认使用 Pi agent 当前模型和已有认证;额度耗尽、限流或重复故障时,可切换到 Pi 已配置的其他供应商模型,不改变主对话模型。无需另配 API Key、向量数据库或嵌入服务。详见[重试、切换与预算](docs/recovery.md)。
|
|
19
19
|
|
|
20
20
|
## 安装
|
|
21
21
|
|
|
@@ -42,7 +42,7 @@ pi install https://github.com/btnalit/pi-memory-evolution
|
|
|
42
42
|
/memory status
|
|
43
43
|
```
|
|
44
44
|
|
|
45
|
-
出现 `SQLite ok (schema
|
|
45
|
+
出现 `SQLite ok (schema 7)` 表示存储初始化成功。后续正常使用 Pi 即可,学习和召回会自动运行。
|
|
46
46
|
|
|
47
47
|
## 使用
|
|
48
48
|
|
|
@@ -87,7 +87,7 @@ Git 安装请把上面的包来源替换为安装时使用的仓库 URL。操作
|
|
|
87
87
|
|
|
88
88
|
数据默认保存在 `~/.pi/agent/agent-suite/memory-evolution/`,使用本地 SQLite。`PI_CODING_AGENT_DIR` 可改变存储前缀;不同工作目录默认共享记忆库。
|
|
89
89
|
|
|
90
|
-
|
|
90
|
+
学习会将经过过滤的来源内容发送给当前模型或允许使用的备用供应商,并消耗相应额度。默认启用跨供应商切换;可通过[恢复配置](docs/recovery.md)限制供应商模型名单或关闭切换。所有备用模型共享调用上限。记忆不是经过独立验证的事实,匹配和敏感信息过滤也并非万无一失;重要内容仍需核实。详见[存储与隐私](docs/usage.md#local-storage-and-provenance)。
|
|
91
91
|
|
|
92
92
|
## 开发
|
|
93
93
|
|
package/README.md
CHANGED
|
@@ -22,8 +22,10 @@ assistant to remember them.
|
|
|
22
22
|
- **Recall across sessions.** Find background from another session or directory.
|
|
23
23
|
A read-only `memory_recall` tool lets the assistant look up missing context mid-task.
|
|
24
24
|
|
|
25
|
-
Learning
|
|
26
|
-
|
|
25
|
+
Learning defaults to Pi's active model and existing authentication. Quota/rate limits
|
|
26
|
+
or repeated failures can use an available model from another configured provider, without
|
|
27
|
+
changing the foreground model. [Fallback and budgets](docs/recovery.md) are configurable;
|
|
28
|
+
no separate API key, embedding service or vector database is required.
|
|
27
29
|
|
|
28
30
|
## Install
|
|
29
31
|
|
|
@@ -46,7 +48,7 @@ Choose **one** source, then run these commands inside Pi:
|
|
|
46
48
|
/memory status
|
|
47
49
|
```
|
|
48
50
|
|
|
49
|
-
`SQLite ok (schema
|
|
51
|
+
`SQLite ok (schema 7)` confirms storage initialization. Continue using Pi normally;
|
|
50
52
|
learning and recall run automatically.
|
|
51
53
|
|
|
52
54
|
## Use
|
|
@@ -96,7 +98,9 @@ Local SQLite state lives in `~/.pi/agent/agent-suite/memory-evolution/`.
|
|
|
96
98
|
`PI_CODING_AGENT_DIR` changes that prefix; different working directories share the
|
|
97
99
|
same store by default.
|
|
98
100
|
|
|
99
|
-
Learning sends filtered source content to the active model
|
|
101
|
+
Learning sends filtered source content to the active model, or an allowed fallback provider,
|
|
102
|
+
and consumes that provider's quota. Cross-provider fallback is enabled by default; restrict
|
|
103
|
+
its allowlist or disable it in [recovery configuration](docs/recovery.md).
|
|
100
104
|
Memories are historical evidence, not independently verified facts. Matching and
|
|
101
105
|
secret filtering are imperfect; verify important claims. See [privacy and storage](docs/usage.md#local-storage-and-provenance).
|
|
102
106
|
|
package/docs/core-quality.md
CHANGED
|
@@ -195,7 +195,7 @@ reset, automatic state revival or fabricated verification occurs.
|
|
|
195
195
|
Reads validate evidence/feedback shape and tool-observation kind, and return independent
|
|
196
196
|
copies of nested metadata. Status validates receipt fields in addition to existing
|
|
197
197
|
record/job/history integrity. Unsupported schema versions fail before DDL. Old builds
|
|
198
|
-
reject schema
|
|
198
|
+
reject schema 7: do not manually downgrade its marker.
|
|
199
199
|
|
|
200
200
|
Stop all Pi processes sharing the state directory and back up the complete state before
|
|
201
201
|
activation. Update/restart all those processes together (or reload after a consistent
|
package/docs/design.md
CHANGED
|
@@ -194,9 +194,9 @@ not detected globally.
|
|
|
194
194
|
|
|
195
195
|
Each attempt uses at most one model call, no tools, an 8,192-output-token cap (clamped
|
|
196
196
|
against a smaller model limit), a fresh request session ID and `cacheRetention: "none"`.
|
|
197
|
-
A 120-second
|
|
198
|
-
|
|
199
|
-
|
|
197
|
+
A 120-second per-attempt deadline bounds waiting even when a provider ignores abort;
|
|
198
|
+
remote computation/billing cannot be guaranteed to stop. A backup has a fresh deadline,
|
|
199
|
+
clamped by the source's remaining 300-second cumulative allowance. Failed calls retain local summary claims. User-cue prose is saved but needs a
|
|
200
200
|
successful model attempt to become claims; it has no local extraction fallback.
|
|
201
201
|
|
|
202
202
|
Global recall is **not global rewriting**. Cross-origin variants remain separate instead
|
|
@@ -223,6 +223,12 @@ One SQLite database, WAL + FULL synchronous mode and private file permissions.
|
|
|
223
223
|
- `feedback_receipts`: exact source-ID/memory-ID idempotency keys, verdict and numeric
|
|
224
224
|
timestamp, including redundant/late feedback receipts; undo never reopens them.
|
|
225
225
|
- `metadata`: schema/import marker.
|
|
226
|
+
- `model_calls`: reserved attempts, selected model/provider, outcome, usage and estimated/reported cost.
|
|
227
|
+
- `route_health`: provider/model cooldowns, separate from source backoff.
|
|
228
|
+
- `recovery_notices`: bounded persistent notice deduplication keys.
|
|
229
|
+
Source-level call/model/time/correction counts survive receipt pruning; shared ceilings are
|
|
230
|
+
reserved transactionally. The active model is preferred; allowed backups come only from
|
|
231
|
+
other Pi-configured providers. [Recovery policy and privacy](recovery.md) define the bounds.
|
|
226
232
|
|
|
227
233
|
The `scope` field records canonical cwd, not an inferred repository/branch/subject identity
|
|
228
234
|
(or an explicit annotation of a legacy record). It is no longer a recall boundary. Reads
|
|
@@ -239,13 +245,16 @@ failure with backoff; attempt/state checks prevent late results or failures from
|
|
|
239
245
|
a new owner's job. Model waiting and recurring recovery use the same serial task chain;
|
|
240
246
|
queued capture/manual work prevents the timer from piling up duplicate tasks.
|
|
241
247
|
|
|
242
|
-
Automatic selection/claim
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
248
|
+
Automatic selection/claim enforce source backoff and call/time/failure caps. Least-recently
|
|
249
|
+
checked ordering prevents temporarily unroutable sources starving other work. Provider/model
|
|
250
|
+
cooldowns never contaminate a source's retry_at. Generic runtime backoff is 1 minute,
|
|
251
|
+
5 minutes, 15 minutes, then 1 hour with up to 20% jitter; quota/auth/rate-limit errors can
|
|
252
|
+
immediately use an allowed other provider. Two recent transport failures pause a model.
|
|
253
|
+
Defaults allow 4 reserved calls / 2 models / 300 seconds per source, one format correction,
|
|
254
|
+
and 20 reservations/hour shared across providers and processes. Unsafe writes/refusals
|
|
255
|
+
pause immediately. Three output failures or five generic failures also pause work.
|
|
256
|
+
Shutdown adds no failures, but an already reserved request may still cost money.
|
|
257
|
+
`/memory evolve` overrides source delay/caps for one attempt, never shared ceilings.
|
|
249
258
|
Completed/retired jobs are never forced to run again. A source resumed in another directory
|
|
250
259
|
retains its original provenance. While Pi is closed no polling occurs.
|
|
251
260
|
|
|
@@ -266,9 +275,9 @@ Memory reads validate indexed identity/origin/hash against JSON. Undo validates
|
|
|
266
275
|
unique before/after IDs and only succeeds when the current records still equal the event's
|
|
267
276
|
after state. New records become tombstones rather than being physically erased. Status
|
|
268
277
|
also validates source jobs/history and the schema marker. Unsupported schema versions
|
|
269
|
-
are rejected before DDL. Schemas 2
|
|
270
|
-
claims/history or resetting evidence timestamps
|
|
271
|
-
|
|
278
|
+
are rejected before DDL. Schemas 2–6 upgrade transactionally to 7 without rewriting
|
|
279
|
+
claims/history or resetting evidence timestamps. Missing retry/feedback tables and routing
|
|
280
|
+
accounting are added; known v6 model waits are separated from source backoff. Old evidence metadata stays absent/unknown.
|
|
272
281
|
Old failures below the cap are due immediately, with unknown cause/time explicitly labeled.
|
|
273
282
|
The source/alias/retry contract is validated on read.
|
|
274
283
|
These detect structural corruption, not all well-formed edits by an owner of the database.
|
|
@@ -278,7 +287,7 @@ Undo does not clear suppression hashes or reopen jobs. Forget/undo is not secure
|
|
|
278
287
|
|
|
279
288
|
Earlier 0.2 SQLite records, IDs, histories and origin labels stay intact and become
|
|
280
289
|
eligible for global relevance-based recall, including existing `legacy` claims. The
|
|
281
|
-
schema-2
|
|
290
|
+
schema-2-through-6-to-7 upgrade requires no data copying, JSONL re-import or manual reset.
|
|
282
291
|
Stop/back up before upgrading, reload all processes sharing the DB, and restore a matching
|
|
283
292
|
backup for rollback; older code must not be pointed at a manually downgraded marker.
|
|
284
293
|
|
|
@@ -288,6 +297,9 @@ facts. Parent corrections preserve unchanged children and derive new facts witho
|
|
|
288
297
|
reintroducing explicitly suppressed child content. Missing origins retain the `legacy`
|
|
289
298
|
label. Adoption is an optional annotation, not a recall prerequisite. Completed imports
|
|
290
299
|
are not replayed; earlier discarded revision information is not automatically reconstructed.
|
|
300
|
+
Empty/missing ledgers do not consume the import opportunity. Only saved digests proving a
|
|
301
|
+
recognized empty v6 snapshot (without a migration event) can reopen old completed markers;
|
|
302
|
+
zero derived claims from a real consumed ledger are not sufficient.
|
|
291
303
|
Old signals/proposals/execution plans remain historical files, never automatic actions.
|
|
292
304
|
|
|
293
305
|
List defaults to all origins, 20 per page sorted by update time then ID; `all` is an alias.
|
package/docs/recovery.md
ADDED
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
# Recovery, provider fallback and budgets
|
|
2
|
+
|
|
3
|
+
Memory learning defaults to **Pi's active agent model**. With no session override this
|
|
4
|
+
is Pi's configured default. The extension never sets the foreground model, creates a
|
|
5
|
+
second credential store, changes provider configuration, or probes models to rank them.
|
|
6
|
+
It discovers available models through Pi's registry and uses registry completion/auth.
|
|
7
|
+
|
|
8
|
+
## Provider fallback
|
|
9
|
+
|
|
10
|
+
Cross-provider fallback is enabled by default. If the default route is unavailable,
|
|
11
|
+
learning may send the sanitized source and selected candidate memories to another
|
|
12
|
+
**already-configured, available Pi provider**. This is a privacy/cost boundary: configure
|
|
13
|
+
an explicit allowlist or disable fallback below if some configured providers must not
|
|
14
|
+
receive memory data. Installing this update does not require per-request approvals.
|
|
15
|
+
|
|
16
|
+
The active model is always first when eligible. Backups must support text; other models
|
|
17
|
+
of the active provider are not used to evade account-wide quota/rate limits. An explicit
|
|
18
|
+
fallback list defines priority; otherwise known catalog input/output price, non-reasoning
|
|
19
|
+
preference and stable model ID order choose among available backups. Unknown/zero prices
|
|
20
|
+
are not assumed free. Catalog availability is not proof of working credentials or credit.
|
|
21
|
+
At most two models may receive any one source by default, including the primary.
|
|
22
|
+
|
|
23
|
+
All source/model budgets are checked in a SQLite write transaction. Selection does not
|
|
24
|
+
grant call authority. Provider cooldowns are shared across sources and Pi processes.
|
|
25
|
+
A cooldown for A does **not** change a source's retry timestamp or prevent B from working.
|
|
26
|
+
Last-checked ordering moves temporarily unroutable sources behind other eligible work.
|
|
27
|
+
|
|
28
|
+
## Error policy
|
|
29
|
+
|
|
30
|
+
| Result | Automatic action |
|
|
31
|
+
| --- | --- |
|
|
32
|
+
| HTTP 401/403 | Cool down that provider for 15 minutes; try an allowed other provider. Pi owns auth resolution/refresh. |
|
|
33
|
+
| HTTP 402 or fixed quota/billing code | Cool down the provider for 1 hour or a later supplied reset; try another provider. |
|
|
34
|
+
| HTTP 429 without a known quota code | Provider cooldown of at least 60 seconds and at least `Retry-After`; try another provider. |
|
|
35
|
+
| HTTP 408, timeout, network/5xx failure | Delayed retry; two recent transport failures cool down that model for 15 minutes and permit fallback. |
|
|
36
|
+
| Context overflow or HTTP 400/404/422 | Do not resend unchanged requests to that model immediately; cool it down for 1 hour and permit a compatible backup. |
|
|
37
|
+
| Invalid JSON / output truncation | Initial attempt, at most one corrective prompt, then an alternate model. Three output failures pause the source; two failures do not authorize repeatedly probing the same model. |
|
|
38
|
+
| Unsafe/unauthorized write or recognized safety/refusal | Reject and pause that source. No fallback to circumvent safety or write guards. |
|
|
39
|
+
| Stale result | Re-read on a bounded delayed retry; not counted as a provider-health failure. |
|
|
40
|
+
| Cancellation / shutdown / reload | Release the lease without adding a failure; already-reserved requests may still have consumed quota. |
|
|
41
|
+
| Unknown error | Safe generic category and bounded transport retry; no guessing that arbitrary error prose means insufficient credit. |
|
|
42
|
+
|
|
43
|
+
Generic source backoff is 1 minute, 5 minutes, 15 minutes, then 1 hour, with up to 20%
|
|
44
|
+
positive jitter on runtime failures. Route failures can try one alternate immediately;
|
|
45
|
+
there are at most two immediate attempts per queue item, not a sleep/retry loop. The
|
|
46
|
+
120-second deadline is **per attempt**, not an outer timeout that also kills its backup.
|
|
47
|
+
Request deadlines are clamped to the source's remaining cumulative time budget. A provider
|
|
48
|
+
ignoring cancellation cannot commit late output, but remote execution/billing may continue.
|
|
49
|
+
|
|
50
|
+
HTTP status/error codes are observed before the underlying SDK consumes failed responses,
|
|
51
|
+
using request-local fetch only for verified supporting APIs: OpenAI completions/responses,
|
|
52
|
+
Azure responses, Codex responses, Anthropic messages and Mistral conversations. Codex memory
|
|
53
|
+
requests use HTTP SSE; foreground transport is unchanged. Other adapters retain their normal
|
|
54
|
+
transport. Complete structured JSON errors (including Google SDK errors) can also supply a
|
|
55
|
+
fixed status/code; arbitrary text is never searched for status-like numbers or echoed.
|
|
56
|
+
Only known code values, numeric `Retry-After`/reset metadata and raw stop enums are used.
|
|
57
|
+
Unsupported/opaque SDK errors can still be generic `provider`; no universal balance API is
|
|
58
|
+
claimed. Error-body inspection is transient, bounded to 8192 bytes and 200 ms, never stored.
|
|
59
|
+
|
|
60
|
+
Input planning drops lower-priority existing candidates when a conservative byte/token
|
|
61
|
+
estimate exceeds the chosen context window. It does not truncate facts or progress JSON to
|
|
62
|
+
pretend they fit. If the original evidence still cannot fit, that route is rejected. Model
|
|
63
|
+
mistakes and provider-specific tokenization remain possible; this is not exact token counting.
|
|
64
|
+
|
|
65
|
+
## Optional configuration
|
|
66
|
+
|
|
67
|
+
Create `recovery.json` **in the memory state directory**, not in Pi's settings file. Reload
|
|
68
|
+
all Pi processes sharing that directory after changing policy. Defaults are:
|
|
69
|
+
|
|
70
|
+
```json
|
|
71
|
+
{
|
|
72
|
+
"crossProviderFallback": true,
|
|
73
|
+
"fallbackModels": [],
|
|
74
|
+
"callsPerHour": 20,
|
|
75
|
+
"sourceCalls": 4,
|
|
76
|
+
"sourceModels": 2,
|
|
77
|
+
"timeoutMs": 120000,
|
|
78
|
+
"sourceTimeMs": 300000,
|
|
79
|
+
"dailyEstimatedUsd": null
|
|
80
|
+
}
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
`fallbackModels` optionally contains exact `provider/model-id` strings, in priority order.
|
|
84
|
+
Only models Pi reports available are considered. It does not override the default model
|
|
85
|
+
or configure authentication. `crossProviderFallback: false` confines learning to the active
|
|
86
|
+
model. Invalid policy fails closed with a fixed error, never prints configuration contents.
|
|
87
|
+
|
|
88
|
+
Limits apply together:
|
|
89
|
+
|
|
90
|
+
- **20 reservations per rolling hour across all models/providers/processes**, not 20 per
|
|
91
|
+
fallback model. An exhausted shared budget cannot be bypassed by switching providers.
|
|
92
|
+
- **4 reserved calls, 2 models and 300 seconds per source** by default. Reservations include
|
|
93
|
+
failed/preflight/cancelled attempts conservatively; they are not exact server billing counts.
|
|
94
|
+
- One persisted format-correction allowance survives cancellation/reload. Safety validation
|
|
95
|
+
and transactional all-or-nothing writes apply equally to every backup model.
|
|
96
|
+
- The older five-failure safety cap is also retained, including crashes and pre-upgrade work.
|
|
97
|
+
- `/memory evolve [source-id]` may override source backoff/route cooldown/source caps for
|
|
98
|
+
**one** explicit attempt, without clearing counters. It cannot bypass shared request or
|
|
99
|
+
estimated-cost ceilings and does not loop through backup models. Settled jobs are not rerun.
|
|
100
|
+
|
|
101
|
+
`dailyEstimatedUsd` is disabled by default because provider catalogs can contain absent,
|
|
102
|
+
zero, outdated or non-billing prices (subscriptions and local models are common). Set a
|
|
103
|
+
positive value to enforce a rolling 24-hour **catalog-estimated** spending ceiling.
|
|
104
|
+
Unknown pricing fails closed under this optional ceiling. Reservation uses a conservative
|
|
105
|
+
input-byte bound, maximum configured pricing tier and output allowance; returned usage/cost
|
|
106
|
+
replaces it when available. Missing usage and cancelled/error responses are not assumed
|
|
107
|
+
free. These estimates are not account balance, billing receipts, or a guaranteed invoice cap.
|
|
108
|
+
Actual provider accounting can differ. Token counts, estimates and unknown-cost counts are
|
|
109
|
+
shown separately from Pi's foreground session totals.
|
|
110
|
+
|
|
111
|
+
## Diagnostics and upgrade
|
|
112
|
+
|
|
113
|
+
`/memory status` shows the current default, allowed routes, provider/model cooldowns,
|
|
114
|
+
source attempts/calls/output failures/time, shared budget and recent selected-model outcomes.
|
|
115
|
+
A successful fallback can show one informational notice per route pair/hour. Failures and
|
|
116
|
+
pause summaries are also deduplicated across reloads, not re-armed by unrelated successes.
|
|
117
|
+
No exception bodies, credentials or failed model text are persisted in diagnostics.
|
|
118
|
+
|
|
119
|
+
Schema **2–6 upgrade to 7**. Stop Pi and back up the whole state directory first; restart or
|
|
120
|
+
reload all processes after updating. Claims, IDs, histories, forgetting and evidence dates
|
|
121
|
+
are preserved. Existing v6 call receipts seed source counts/model history where available;
|
|
122
|
+
missing history is not invented. Known v6 route-contaminated waits are separated from the
|
|
123
|
+
original source backoff. Earlier paused five-failure jobs are not indiscriminately reset.
|
|
124
|
+
Older code rejects schema 7; rollback requires a matching backup, not editing the marker.
|
|
125
|
+
|
|
126
|
+
Empty/missing legacy input no longer consumes the one-time import opportunity. Supply valid
|
|
127
|
+
ledgers and use `/memory import [directory]`. A v6 completed/zero-count marker is reopened
|
|
128
|
+
only when its saved digest proves a recognized empty snapshot and no migration event exists.
|
|
129
|
+
A real consumed ledger with no derived claims remains completed; zero count alone cannot
|
|
130
|
+
justify replay over later corrections/forgetting. Originals and copy-only archives are kept.
|
package/docs/testing.md
CHANGED
|
@@ -17,6 +17,17 @@ helper scripts and memory state must not ship in the package.
|
|
|
17
17
|
|
|
18
18
|
`check:package` inspects `npm pack --dry-run --json`; it does not publish to npm.
|
|
19
19
|
|
|
20
|
+
## Recovery and routing regressions
|
|
21
|
+
|
|
22
|
+
`http-diagnostics.test.ts` and `fallback-http.test.ts` use the real Pi ModelRuntime/Registry
|
|
23
|
+
with loopback HTTP responses, not mocks that simply invoke `onResponse`. They verify HTTP
|
|
24
|
+
401/403/400/429/503, quota/context/safety codes, Retry-After, Google SDK JSON errors, and
|
|
25
|
+
cross-provider fallback with the backup's own authentication. No live provider is contacted.
|
|
26
|
+
`scheduler.test.ts` covers bounded correction, per-attempt timeout/late-output rejection,
|
|
27
|
+
provider cooldowns, allowlists, shared hard ceilings and persistent source limits.
|
|
28
|
+
`reliability-regressions.test.ts` covers empty ledgers, safe v6-marker repair, preserving
|
|
29
|
+
zero-claim real imports, schema-6 scheduling migration, and case-sensitive resource identity.
|
|
30
|
+
|
|
20
31
|
## npm CLI compatibility
|
|
21
32
|
|
|
22
33
|
All consumers of `npm pack --json` share `scripts/lib/npm-pack.mjs`. npm 10/11
|
|
@@ -67,7 +78,7 @@ host explicitly. The script prints the tested host version. The test:
|
|
|
67
78
|
1. Builds and extracts the actual npm tarball, outside the development checkout.
|
|
68
79
|
2. Uses real `pi install`, `pi list` and default package discovery—no explicit `-e`
|
|
69
80
|
entry or memory-extension wrapper. Checks `/memory status`, `/memory learning`
|
|
70
|
-
and `/memory explain`, including schema
|
|
81
|
+
and `/memory explain`, including schema 7, with no checkout `node_modules`.
|
|
71
82
|
3. Verifies repeated installation does not duplicate the package setting, then
|
|
72
83
|
removes it and confirms the command disappears while records/history remain.
|
|
73
84
|
4. Creates a local Git origin from the packed files and the real lockfile. A
|
|
@@ -75,7 +86,7 @@ host explicitly. The script prints the tested host version. The test:
|
|
|
75
86
|
`file` transport is permitted. No GitHub access is needed.
|
|
76
87
|
5. Exercises the native Git installer and its real npm dependency step, updates to
|
|
77
88
|
a new commit, switches from an old pinned tag back to the default branch, and
|
|
78
|
-
verifies source switching and removal preserve schema-
|
|
89
|
+
verifies source switching and removal preserve schema-7 state.
|
|
79
90
|
6. Serves the actual tarball through a loopback npm registry, with a fresh cache,
|
|
80
91
|
then checks native `pi install npm:pi-memory-evolution`, repeat installation,
|
|
81
92
|
normal loading and removal. No host peer packages are served or installed.
|
package/docs/usage.md
CHANGED
|
@@ -10,6 +10,7 @@ This reference covers detailed behavior and optional controls; normal use is aut
|
|
|
10
10
|
- [Commands](#commands)
|
|
11
11
|
- [Migration](#migration-from-01)
|
|
12
12
|
- [Recovery and troubleshooting](#recovery-and-troubleshooting)
|
|
13
|
+
- [Provider fallback, error policies and budgets](recovery.md)
|
|
13
14
|
|
|
14
15
|
## Installation and updates
|
|
15
16
|
|
|
@@ -110,9 +111,11 @@ rename the tool to hide the conflict; an old installation would still run its ho
|
|
|
110
111
|
- Each processing attempt makes at most one background model call, using up to 32
|
|
111
112
|
recently updated active memories from that source's capture origin. This is a
|
|
112
113
|
conservative automatic-replacement safeguard, **not a recall restriction**.
|
|
113
|
-
It
|
|
114
|
-
With no
|
|
115
|
-
|
|
114
|
+
It defaults to **the current Pi session model and Pi's own provider/auth resolution**.
|
|
115
|
+
With no session override, this is Pi's configured default. Quota/rate limits or repeated
|
|
116
|
+
failures may switch to an available model from another configured provider, not a sibling
|
|
117
|
+
sharing the default provider's quota. The foreground model is never changed. See
|
|
118
|
+
[fallback privacy, error handling and budgets](recovery.md); no extra credentials are stored.
|
|
116
119
|
- Valid additions/replacements commit immediately, with provenance and before/after
|
|
117
120
|
history. Inferred memories remain labeled `provisional`, but are recallable without
|
|
118
121
|
approval. Pinned memories cannot be automatically replaced.
|
|
@@ -123,14 +126,17 @@ rename the tool to hide the conflict; an old installation would still run its ho
|
|
|
123
126
|
learning its claims requires a successful model attempt.
|
|
124
127
|
- Session start checks persisted work across **all origins**. While Pi remains running,
|
|
125
128
|
a local recovery timer checks every **15 seconds** (after the previous check/call ends),
|
|
126
|
-
gradually draining eligible pending/failed work one source per check
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
129
|
+
gradually draining eligible pending/failed work one source per check (at most one immediate
|
|
130
|
+
backup per check). Selection favors least recently checked work, then retry time and oldest
|
|
131
|
+
capture, so unavailable routes do not indefinitely block other sources.
|
|
132
|
+
- Recovery is **error-specific**, with persisted source backoff, separate provider/model
|
|
133
|
+
cooldowns, at most one output correction, and bounded cross-provider fallback. Generic
|
|
134
|
+
backoff is 1 minute, 5 minutes, 15 minutes, then 1 hour, plus up to 20% jitter. Defaults
|
|
135
|
+
cap each source at **4 reserved calls / 2 models / 300 seconds**, and all models and
|
|
136
|
+
processes share **20 calls per rolling hour**. Three output failures or five generic
|
|
137
|
+
failures also pause work; unsafe writes/refusals pause immediately. `/memory status`
|
|
138
|
+
explains routes, budgets and actual outcomes. `/memory evolve` overrides source limits
|
|
139
|
+
for one attempt, never shared ceilings or settled jobs. [Full policy/configuration](recovery.md).
|
|
134
140
|
- A job lease lasts **150 seconds** (120-second deadline plus 30-second grace). The timer
|
|
135
141
|
detects expired running jobs and schedules them with the same bounded backoff. Shutdown/
|
|
136
142
|
reload cancellation returns work to pending without consuming the failure budget.
|
|
@@ -158,7 +164,7 @@ rename the tool to hide the conflict; an old installation would still run its ho
|
|
|
158
164
|
recall regardless of its original directory.
|
|
159
165
|
- Query coverage, evidence-based document frequency, field weights, mild length
|
|
160
166
|
normalization and a relative cutoff reject weak secondary matches. Unseen query words
|
|
161
|
-
no longer receive the highest rarity weight. Exact paths must match, and
|
|
167
|
+
no longer receive the highest rarity weight. Exact paths must match, including case; `/srv/Atlas` and `/srv/atlas` are distinct. A quoted
|
|
162
168
|
question in a replay/incident note is weaker than evidence answering it. Redundancy
|
|
163
169
|
filtering cannot let a project-state note hide a preference of the same origin.
|
|
164
170
|
Source IDs/cwd have no authority bonus. After relevance gates, host-assigned evidence,
|
|
@@ -181,8 +187,10 @@ There is no additional call on ordinary recall. Eligible work turns may now incu
|
|
|
181
187
|
additional background call each; no related tracked state or no tool observation means
|
|
182
188
|
no progress call. A mixed statement/work turn may additionally incur a separate learning
|
|
183
189
|
call; interrupted turns with usable observations may also learn through automatic recovery.
|
|
184
|
-
Automatic retries
|
|
185
|
-
|
|
190
|
+
Automatic retries/fallback share a default total of four reserved calls per source;
|
|
191
|
+
they do not get separate budgets per model. Each request uses Pi's authentication for the
|
|
192
|
+
selected provider. Cross-provider data sharing and optional estimated-cost limits are
|
|
193
|
+
explained in [recovery configuration](recovery.md). Model mistakes
|
|
186
194
|
remain possible; tool observations and model-generated aliases are not proof of truth. Use
|
|
187
195
|
history, correction, pinning and undo rather than treating generated claims as verified facts.
|
|
188
196
|
|
|
@@ -270,7 +278,9 @@ Sensitive lines/blocks are suppressed before capture, edits, model submission an
|
|
|
270
278
|
recall. This covers common token/password/JSON/Chinese/Bearer/private-key formats,
|
|
271
279
|
including quoted multiline values, indented YAML blocks and control-character cleanup,
|
|
272
280
|
but not every possible secret. Do not rely on a regex as a complete DLP system. Sanitized
|
|
273
|
-
sources and selected existing memories go to the
|
|
281
|
+
sources and selected existing memories go to the active Pi model provider or an allowed
|
|
282
|
+
already-configured fallback provider. Fallback is enabled by default; restrict the model
|
|
283
|
+
allowlist or disable it in [recovery configuration](recovery.md) to constrain data sharing.
|
|
274
284
|
|
|
275
285
|
## Commands
|
|
276
286
|
|
|
@@ -291,7 +301,7 @@ These are optional direct controls, **not approval gates**:
|
|
|
291
301
|
/memory evolve # optional one-off retry, overriding delay/failure limit
|
|
292
302
|
/memory evolve <source-id> # retry one named source without clearing other failure counts
|
|
293
303
|
/memory import [directory] # explicit, repeat-safe legacy JSONL import; never replays a completed one
|
|
294
|
-
/memory archive-legacy # copy inactive legacy plan files to a
|
|
304
|
+
/memory archive-legacy # copy inactive legacy plan files to a private unique archive; originals kept
|
|
295
305
|
/memory undo <event-id> # reverse actual changes, if not modified since
|
|
296
306
|
/memory feedback <id> <verdict> # useful | unhelpful | accurate | incorrect
|
|
297
307
|
/memory correct <id> <replacement> # literal replacement, 4–480 characters
|
|
@@ -350,18 +360,23 @@ preserve unchanged children of corrected summaries and carry superseded-content
|
|
|
350
360
|
when corrected legacy claims are adopted into a project.
|
|
351
361
|
|
|
352
362
|
**Upgrading from earlier 0.2 development builds:** stop Pi and back up the state directory
|
|
353
|
-
first, then update/reload all Pi processes sharing it. Schema markers **2
|
|
354
|
-
transactionally to
|
|
355
|
-
|
|
363
|
+
first, then update/reload all Pi processes sharing it. Schema markers **2–6 upgrade
|
|
364
|
+
transactionally to 7**, adding missing recovery/feedback fields, source call/time/model
|
|
365
|
+
budgets, provider cooldowns and usage receipts. Known v6 route waits are separated from
|
|
366
|
+
source failure backoff; retained call history seeds counters without inventing missing calls. Existing records, IDs, histories and
|
|
356
367
|
evidence dates remain unchanged. Missing evidence stays unknown, with no fabricated backfill.
|
|
357
368
|
Existing failures below the limit become automatically eligible; their old error cause/time
|
|
358
369
|
remain labeled unknown rather than invented. No copying, manual marker reset or JSONL
|
|
359
|
-
re-import is needed. Older builds reject schema
|
|
370
|
+
re-import is needed. Older builds reject schema 7; rollback requires a matching backup,
|
|
360
371
|
not editing the marker. Existing records are immediately eligible for global relevance-based recall.
|
|
361
372
|
Legacy claims previously excluded by cwd filtering become eligible too. This shares
|
|
362
373
|
relevant stored claims with the active Pi session/provider, not raw session archives.
|
|
363
374
|
|
|
364
|
-
|
|
375
|
+
Empty/missing ledgers do not mark an import completed. After supplying files, use
|
|
376
|
+
`/memory import [directory]`. Provably empty v6 snapshots can reopen safely; a zero claim
|
|
377
|
+
count alone never authorizes replay of a real consumed ledger. See [migration details](recovery.md).
|
|
378
|
+
|
|
379
|
+
These migration fixes do not replay an already completed real import or retroactively erase
|
|
365
380
|
previously stored sensitive data. If an older import already lost revision information,
|
|
366
381
|
use the preserved ledgers/backup to review and correct affected records; do not reset the
|
|
367
382
|
migration marker or replace a populated database blindly.
|
|
@@ -395,10 +410,11 @@ JSONL, not changes made in the new database.
|
|
|
395
410
|
automatically processes due work. `/memory status` distinguishes scheduled retries from
|
|
396
411
|
paused sources and shows the next eligible time (the next poll may be up to 15 seconds
|
|
397
412
|
later when no local work is queued). After a crash, the 150-second lease must expire first.
|
|
398
|
-
|
|
399
|
-
`
|
|
400
|
-
|
|
401
|
-
|
|
413
|
+
Inspect source call/time/output limits as well as failure count. Status distinguishes
|
|
414
|
+
`auth`, `quota`, `rate_limit`, `context_limit`, `request`, `safety`, transport, output and
|
|
415
|
+
stale-write failures. Provider cooldowns are separate from source retry times; switching
|
|
416
|
+
providers cannot bypass the shared budget. `/memory evolve` makes one explicit source
|
|
417
|
+
attempt without resetting counters or overriding shared ceilings. See [error policy](recovery.md).
|
|
402
418
|
A bad source remains saved/visible rather than being deleted or labeled successful.
|
|
403
419
|
- **Persistent storage/import errors:** stop all Pi processes using that agent directory
|
|
404
420
|
and back up the **entire** state directory, including any SQLite sidecars and legacy
|
|
@@ -411,4 +427,4 @@ JSONL, not changes made in the new database.
|
|
|
411
427
|
stopped; do not mix one backup's database with another's WAL/SHM files.
|
|
412
428
|
|
|
413
429
|
Diagnostics intentionally do not echo provider error bodies, which may contain secrets.
|
|
414
|
-
`SQLite ok (schema
|
|
430
|
+
`SQLite ok (schema 7)` checks database structure/record validity, not the truth of model claims.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-memory-evolution",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.5",
|
|
4
4
|
"description": "Memory that maintains itself. Pi learns what matters, injects what this session needs, and recalls the rest — nothing to configure, no commands to learn.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"publishConfig": {
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
import type { Diagnostic } from '../memory/diagnostics.ts';
|
|
2
|
+
import type { FailureCode } from '../memory/recovery.ts';
|
|
3
|
+
|
|
4
|
+
// Only adapters verified to support request-local fetch injection. Never monkey-patch global fetch,
|
|
5
|
+
// replace auth/proxy composition, or pass this option to SDKs (e.g. Google/Bedrock) that reject it.
|
|
6
|
+
export const OBSERVABLE_HTTP_APIS = new Set(['openai-completions', 'openai-responses', 'azure-openai-responses',
|
|
7
|
+
'openai-codex-responses', 'anthropic-messages', 'mistral-conversations']);
|
|
8
|
+
const ERROR_CODES: Record<string, FailureCode> = {
|
|
9
|
+
insufficient_quota: 'quota', quota_exceeded: 'quota', billing_hard_limit_reached: 'quota', insufficient_balance: 'quota',
|
|
10
|
+
usage_limit_reached: 'quota', usage_not_included: 'quota',
|
|
11
|
+
context_length_exceeded: 'context_limit', max_context_length_exceeded: 'context_limit',
|
|
12
|
+
content_filter: 'safety', content_policy_violation: 'safety', safety: 'safety',
|
|
13
|
+
};
|
|
14
|
+
export function httpFailure(d: Diagnostic): FailureCode {
|
|
15
|
+
if (d.errorClass) return d.errorClass;
|
|
16
|
+
return d.httpStatus === 401 || d.httpStatus === 403 ? 'auth' : d.httpStatus === 402 ? 'quota'
|
|
17
|
+
: d.httpStatus === 429 ? 'rate_limit' : d.httpStatus === 408 ? 'timeout'
|
|
18
|
+
: [400, 404, 422].includes(d.httpStatus ?? 0) ? 'request' : 'provider';
|
|
19
|
+
}
|
|
20
|
+
export function observeStatus(d: Diagnostic, status: number, headers?: Record<string, string>, now = Date.now()): void {
|
|
21
|
+
if (Number.isSafeInteger(status) && status >= 100 && status <= 599) d.httpStatus = status;
|
|
22
|
+
const retry = Object.entries(headers ?? {}).find(([key]) => key.toLowerCase() === 'retry-after')?.[1];
|
|
23
|
+
if (retry) {
|
|
24
|
+
const delay = /^\d+(?:\.\d+)?$/u.test(retry.trim()) ? Number(retry) * 1000 : Date.parse(retry) - now;
|
|
25
|
+
// Preserve long waits (up to the JS timestamp range); never shorten a server's requested pause.
|
|
26
|
+
if (Number.isFinite(delay) && delay > 0 && now + delay <= 8_640_000_000_000_000) d.retryAfterMs = Math.ceil(delay);
|
|
27
|
+
}
|
|
28
|
+
}
|
|
29
|
+
/** Some SDKs (notably Google) return a complete JSON error document as errorMessage.
|
|
30
|
+
* Decode only that structural shape, not status-looking numbers or prose substrings. */
|
|
31
|
+
export function observeStructuredError(d: Diagnostic, message?: string): void {
|
|
32
|
+
if (!message || Buffer.byteLength(message) > 8192) return;
|
|
33
|
+
try {
|
|
34
|
+
const value = JSON.parse(message);
|
|
35
|
+
const status = value?.error?.code ?? value?.status;
|
|
36
|
+
if (d.httpStatus === undefined && Number.isInteger(status) && status >= 400 && status <= 599) observeStatus(d, status);
|
|
37
|
+
const code = [value?.error?.code, value?.error?.type, value?.code].find(c => typeof c === 'string' && Object.hasOwn(ERROR_CODES, c));
|
|
38
|
+
if (code) d.errorClass = ERROR_CODES[code] as Diagnostic['errorClass'];
|
|
39
|
+
} catch { /* Opaque provider errors stay unknown; no raw text is retained or guessed from. */ }
|
|
40
|
+
}
|
|
41
|
+
export function diagnosticFetch(d: Diagnostic, signal: AbortSignal): typeof globalThis.fetch {
|
|
42
|
+
const underlying = globalThis.fetch;
|
|
43
|
+
return async (input, init) => {
|
|
44
|
+
const response = await underlying(input, init);
|
|
45
|
+
observeStatus(d, response.status, Object.fromEntries(response.headers));
|
|
46
|
+
if (response.status < 400) return response;
|
|
47
|
+
// Inspect only a bounded error JSON clone, without delaying the original body indefinitely.
|
|
48
|
+
// No free-form message, header, URL, credential or body is copied into the diagnostic.
|
|
49
|
+
const reader = response.clone().body?.getReader();
|
|
50
|
+
if (!reader) return response;
|
|
51
|
+
let timer: ReturnType<typeof setTimeout> | undefined;
|
|
52
|
+
const inspect = async () => {
|
|
53
|
+
const chunks: Uint8Array[] = []; let bytes = 0;
|
|
54
|
+
while (!signal.aborted) {
|
|
55
|
+
const { done, value } = await reader.read(); if (done) break;
|
|
56
|
+
bytes += value.byteLength; if (bytes > 8192) return;
|
|
57
|
+
chunks.push(value);
|
|
58
|
+
}
|
|
59
|
+
if (signal.aborted) return;
|
|
60
|
+
try {
|
|
61
|
+
const value = JSON.parse(Buffer.concat(chunks).toString('utf8'));
|
|
62
|
+
const code = [value?.error?.code, value?.error?.type, value?.code].find(c => typeof c === 'string' && Object.hasOwn(ERROR_CODES, c));
|
|
63
|
+
if (code) {
|
|
64
|
+
d.errorClass = ERROR_CODES[code] as Diagnostic['errorClass'];
|
|
65
|
+
const reset = value?.error?.resets_at;
|
|
66
|
+
if (d.errorClass === 'quota' && Number.isSafeInteger(reset) && reset * 1000 > Date.now() && reset * 1000 <= 8_640_000_000_000_000)
|
|
67
|
+
d.retryAfterMs = Math.max(d.retryAfterMs ?? 0, reset * 1000 - Date.now());
|
|
68
|
+
}
|
|
69
|
+
} catch { /* Unknown/HTML/oversized bodies retain only the status. */ }
|
|
70
|
+
};
|
|
71
|
+
try { await Promise.race([inspect(), new Promise<void>(resolve => { timer = setTimeout(resolve, 200); })]); }
|
|
72
|
+
catch { /* Diagnostics must not change the provider's response handling. */ }
|
|
73
|
+
finally { clearTimeout(timer); void reader.cancel().catch(() => {}); }
|
|
74
|
+
return response;
|
|
75
|
+
};
|
|
76
|
+
}
|