pi-memory-evolution 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,188 @@
1
+ # Progress pipeline follow-up
2
+
3
+ This **unreleased 0.2.0** follow-up fixes reproduced capture/nomination failures after
4
+ `a3c5943`. It keeps schema 5, ordinary recall's relevance/identity gates, automatic model
5
+ selection/authentication, and the existing transactional write boundary.
6
+
7
+ ## Reproduced failure chain
8
+
9
+ A read-only investigation found a stale project-state claim absent from all five saved
10
+ progress target lists in the diagnostic snapshot; all five corresponding model events
11
+ had zero changed records. The model could not update an ID it never received. `done`
12
+ was processing status, not proof of learning.
13
+
14
+ Replay of the tool-heavy development session showed:
15
+
16
+ 1. A long implementation run ended with an assistant error, so no progress source was
17
+ captured despite completed tool operations.
18
+ 2. In its successful continuation, a 64-entry context window retained only “continue”.
19
+ The last-eight-unique-paths policy also dropped the repository root. No targets meant
20
+ no source/job, even though a full test result existed.
21
+ 3. In the later commit/push/restart turn, only the last eight tool results survived;
22
+ restart inspection displaced the real commit/push result.
23
+ 4. Reusing answer retrieval for update nomination required an exact absolute repository
24
+ path. A project-state claim containing only its bare repository name could not qualify.
25
+ Per-path top-2 and answer redundancy gates further restricted update opportunities.
26
+ 5. The unmodified state was less than one day old. Its freshness factor was about 0.914,
27
+ so time-based decay did not exclude it. Age is not evidence of task completion.
28
+ 6. The user's explicit three-part requirements statement did not match the learning cue
29
+ regex and no compaction occurred, so it was not captured as a user statement.
30
+
31
+ This does not prove every old clause was false. Test completion and a commit do not imply
32
+ full product acceptance; updates must preserve unverified or still-pending clauses.
33
+
34
+ ## User context and natural learning
35
+
36
+ The public active-branch context facade remains the only context source. A bounded scan
37
+ now considers up to 4096 entries and 4096 messages, retaining at most six sanitized user
38
+ texts of 2048 bytes each. Repeated consecutive topic-less continuations share a slot.
39
+ Reset and unknown-topic barriers are kept, not replaced by an older successful topic.
40
+ Assistant/tool text, raw compaction summaries, injected digests and slash commands do not
41
+ supply user topics. Retained user tails remain supported. No full-session archive replay,
42
+ cross-branch traversal or persistent topic cache was introduced.
43
+
44
+ `memory/learning.ts` distinguishes explicit cues and natural declarations such as:
45
+
46
+ - 我比较在意的三大功能……这些你觉得做得怎么样?
47
+ - 我们的核心需求是……
48
+ - 我希望系统每次都能自动……
49
+ - 我喜欢茶。
50
+ - Our priorities are …
51
+ - Our project must …
52
+
53
+ A statement may precede a question seeking feedback. A plain recall question, quoted
54
+ example, vague continuation or one-off execution request is not a new durable requirement.
55
+ This is conservative pattern-based intent recognition, not universal language understanding.
56
+ The active model still extracts provisional claims and may return no update.
57
+
58
+ A mixed statement/work turn can capture a separate user source and progress source.
59
+ Their model calls use the serial queue and retain separate write authority: tool output
60
+ cannot create preferences, and a user request is not proof an operation succeeded. This
61
+ can cost two background calls rather than the old single cue call that discarded progress.
62
+ Source idempotency and existing retry caps still apply.
63
+
64
+ ## Evidence selection and interruptions
65
+
66
+ The progress inspector considers only the current user turn, with at most 4096 trailing
67
+ messages. It requires linked call/result IDs, a work request and at least one recognized
68
+ work operation. Allowed observation tool names are bash/write/edit/read/grep/find/ls;
69
+ unknown/custom tools are not automatically trusted as work observations.
70
+
71
+ Operation classes prioritize evidence instead of blindly taking the newest eight:
72
+
73
+ | Priority | Examples |
74
+ |---|---|
75
+ | 5 | git commit/push/merge/rebase/cherry-pick |
76
+ | 4 | test/build/process runners; git status/log/show/diff/ls-remote |
77
+ | 3 | file edits/writes and recognized mutation/deployment commands |
78
+ | 0 | supporting inspection |
79
+
80
+ The priority is a retention hint, not a success/verification classifier. Git arguments,
81
+ Python/Node scripts or exit code alone do not prove a particular task completed. The
82
+ model must inspect the actual operation/output, failures and qualifications. Ties favor
83
+ newer observations; the chosen set is restored to chronological order for the model.
84
+ The last eight most important observations fit within a 28,000-byte JSON budget; budget
85
+ pressure removes lower-priority observations first. Each stored operation is at most
86
+ 1024 bytes, output at most 2048 bytes, with head/tail previews and redaction. Request/report
87
+ remain bounded. The payload exposes omitted counts and scan limits, not an illusion of
88
+ complete evidence. More than eight important distinct outcomes can still be omitted.
89
+
90
+ `completion` is either `completed` or `interrupted`. Error/aborted final assistant responses
91
+ retain already observed tool operations but omit the assistant report. This never certifies
92
+ the whole work item as finished. A request, unmatched result, unfinished tool-use response
93
+ or assistant-only report cannot substitute for observed work. Captures made while the
94
+ foreground signal is cancelled remain eligible for existing automatic recovery; they do
95
+ not require another compaction or a manual evolve command.
96
+
97
+ `memory_recall`/other internal memory tools and operation arguments referencing the owned
98
+ state directory are excluded. A mixed shell command mentioning that directory may be
99
+ conservatively omitted rather than feeding the extension's own records back as independent
100
+ corroboration. This is not a universal detector of indirect self-reference through renamed
101
+ files or custom tools.
102
+
103
+ There is **no new per-tool disk journal**. Hard kills before `agent_end`, work outside the
104
+ bounded scan, lost tool-result pairs, unsupported commands and missing retained context
105
+ remain limitations. Extending those guarantees requires a separate persistence design,
106
+ not calling an interrupted task complete.
107
+
108
+ ## Operation resources and update nomination
109
+
110
+ A small non-executing shell lexer extracts hints from explicit `cd` and `git -C`, including
111
+ quoted paths and simple environment prefixes. It does not treat quoted semicolons or
112
+ heredoc source bodies as independent commands. Dynamic paths/substitutions/globs and
113
+ unsupported shell syntax lose hints rather than being executed or guessed.
114
+
115
+ File operations can discover a real checkout root by walking parent `.git` markers
116
+ without running git. Operation resources come from arguments, not output or the current
117
+ cwd alone. At most 16 resources are retained, in operation-priority order; up to eight
118
+ complete paths of at most 512 bytes each are included in the model payload as host hints.
119
+ Longer paths can still participate locally, but are not truncated into a fictional model
120
+ resource path.
121
+
122
+ `memory/progress-targets.ts` is separate from question-answer retrieval:
123
+
124
+ - Candidates are existing project states in the capture origin, excluding pinned,
125
+ forgotten, conflicted or explicitly incorrect records. Expired states remain eligible
126
+ for fresh evidence.
127
+ - Operation resource paths or explicit bare project names provide subject hints; a
128
+ generic cwd/origin is not authority. Distinct absolute resources with the same basename
129
+ cannot qualify by falling back to generic commit/push words.
130
+ - User-topic matches are a separate fallback, not tool-output-derived subject inference.
131
+ - Pending states receive nomination priority over historical completed notes.
132
+ - Up to eight IDs are nominated without per-path top-2, answer deduplication or an
133
+ answer-relative score cutoff. Ordinary recall's path and topic gates are unchanged.
134
+
135
+ Current nomination scores are resource path 60 / explicit project name 50, plus two per
136
+ matched user feature (up to eight), plus 20 for recognizable pending state wording. They
137
+ are deterministic candidate heuristics, not learned confidence. Oldest evidence/ID break
138
+ ties. Eight candidates are still a cap; large/multi-project tasks may need additional
139
+ observations and cannot be assumed fully updated in one pass.
140
+
141
+ The model still must identify the same subject/fact and use exact `replaces` IDs. Store
142
+ checks still enforce candidate authority, source time, generation, pinning and origin.
143
+ A nominated resource is not proof of completion. New preferences/facts from progress are
144
+ rejected, and unsupported compound clauses must remain unchanged.
145
+
146
+ ## Diagnostics and activation
147
+
148
+ `/memory learning` shows a bounded, sanitized transient capture/nomination snapshot:
149
+
150
+ - natural/explicit learning intent and number of captured user statements;
151
+ - no-work-request/no-work-observation/unfinished-response versus observed work;
152
+ - scan limit, linked/ignored/retained/omitted observation counts and completion type;
153
+ - no-update-targets, already-captured or progress-captured;
154
+ - operation resources, selected IDs and candidate reasons, without memory bodies.
155
+
156
+ It also shows recent actual model transaction changed-record counts. `/memory status`
157
+ includes those persistent outcomes and explicitly says `done` means processed/retired,
158
+ not necessarily learned. Zero changes remain distinguishable from a provider failure.
159
+ Diagnostics do not persist raw provider responses or claim to know why the model chose
160
+ not to change a fact. User feedback and aliases are not inferred from lookup frequency.
161
+
162
+ No schema migration or old-record rewrite is needed. Reload/restart activates the new
163
+ code. Updating alone does not reopen completed jobs, replay old transcripts, backfill
164
+ previously missed requirements or invent completed states. New observations/statements
165
+ and compaction use the repaired pipeline. Back up state before version changes as usual.
166
+
167
+ ## Validation
168
+
169
+ - **215 passing tests**, strict typecheck and package inspection via `npm run check`.
170
+ - Synthetic runtime tests cover long work/continuation, early commit/test preservation,
171
+ interruptions, cancelled foreground recovery, operation identity, ambiguous paths,
172
+ same-origin/pin/lifecycle boundaries, natural requirements, separate mixed-source
173
+ authority, replay idempotency, JSON expansion budgets and zero-change diagnostics.
174
+ - Real installed Pi/Bun with a loopback fake model performs an actual temporary Git
175
+ commit and failed push followed by **12** diagnostic tool results. The early result
176
+ reaches the model, the bare-project pending state is nominated and updated, and the
177
+ next provider payload contains the new partial state with full acceptance still open.
178
+ A natural priorities statement also learns without an explicit remember cue. Existing
179
+ recall/tool/feedback/recovery and model/auth tests remain enabled.
180
+ - Read-only replay of three relevant real-session turns (interrupted implementation,
181
+ successful continuation, commit/push/restart) now nominates both previously missed
182
+ stale states. The continuation retains test counts; the commit turn retains the actual
183
+ commit result. The connection reported `total_changes() = 0`.
184
+
185
+ The real-data replay exercises host selection only, not a paid model consolidation or
186
+ production update. It does not claim the historical records are already corrected. The
187
+ real-Pi update test uses synthetic data and a fake model, not a multi-day natural-language
188
+ accuracy benchmark. See [design.md](design.md) and [core-quality.md](core-quality.md).
@@ -0,0 +1,85 @@
1
+ # Recall and progress quality follow-up
2
+
3
+ This follows `f6599d0` (global cross-session recall), not the earlier directory-isolation
4
+ review. Version 0.2.0 remains unreleased. Validation was run on Node 26.8.1 and the
5
+ installed Pi 0.85 standalone Bun host.
6
+
7
+ ## Three causes and changes
8
+
9
+ 1. **Weak secondary recall:** partial CJK fragments and repository-name components could
10
+ qualify unrelated records. Raw overlap lacked enough query coverage and relative
11
+ relevance filtering. Retrieval now uses words/concepts, exact literals, document
12
+ frequency, coverage and redundant-facet filtering. Body/alias/origin-identifier field
13
+ factors are 1/0.8/0.2. Source IDs/current cwd have no authority bonus. Generic internal
14
+ `scope` fields are not treated as cross-context semantics: doing so distorted frequency
15
+ weights and could promote storage-directory facts over a cross-session preference.
16
+ 2. **Stale progress:** ordinary work turns did not enter the cue/compaction-only evolution
17
+ loop. Completed work can now supply linked tool observations, including failures,
18
+ targeting at most 8 existing same-origin project states. Expired states can be updated
19
+ with fresh evidence, but forgotten/conflicted/pinned states cannot. Explicit operation
20
+ paths are not crowded out by generic task words. Store checks prevent even accidentally
21
+ widened candidates from relabeling facts as project progress.
22
+ 3. **Cross-language misses:** literal matching could not connect an English question to
23
+ the Chinese preference. A bounded bilingual concept map supports existing records;
24
+ optional model-generated bilingual aliases extend matching without recall-time model
25
+ calls. Aliases are validated, persisted, copied defensively and undoable. Alias-only
26
+ changes do not refresh evidence dates; correction clears old aliases. Excerpts apply
27
+ the same literal/prose boundary, rather than locating a topic word inside an earlier
28
+ filename and omitting the actual relevant passage.
29
+
30
+ The detailed formula, caps, update restrictions and migration contract are in
31
+ [design.md](design.md). No project files/configuration are modified by memory evolution.
32
+
33
+ ## Validation
34
+
35
+ - **122/122 tests**, strict TypeScript checking, and package dry-run inspection.
36
+ - Runtime coverage: **100% lines / 91.44% branches / 98.68% functions**. Coverage is not
37
+ a correctness proof, and host-specific behavior also needs the real-Pi check below.
38
+ - Real Pi RPC tests passed repeatedly using fresh processes, separate working directories,
39
+ a shared temporary memory database and a loopback fake model. They verify active-model/
40
+ auth reuse, cross-session recall, contextual continuation, topic changes, Chinese queries,
41
+ learned aliases, provenance, forget and absence of approval prompts.
42
+ - The real-host test also performs a real local Git commit in a temporary repository,
43
+ followed by a push that fails because no remote exists. Actual tool events reach the
44
+ progress model input. The fake model replaces “not committed or pushed” with “commit
45
+ created; push pending”; the next turn receives the updated state. Three memory-model
46
+ calls are observed: two explicit learning inputs and one work observation.
47
+ - Schema-2-to-3 upgrade tests preserve records/history/dates. Malformed aliases/targets,
48
+ invalid model updates, forgetting queued targets and manual correction are covered.
49
+ - One synthetic 600-summary/1,800-record sample: capture 662 ms, reopen 1 ms, cold/warm
50
+ recall 196/99 ms. Segmentation is memoized only within a query for repeated text/origins;
51
+ this is a local timing sample, not a large-corpus performance guarantee.
52
+
53
+ ## Real-data replay, separately read-only
54
+
55
+ A direct SQLite read-only connection (`query_only`, no `MemoryStore` construction) read
56
+ 146 existing records. Nine query/context cases passed; the connection reported zero
57
+ changes and the existing schema marker stayed at 2. No migration, correction, deletion,
58
+ model evolution or paid call was triggered by this replay. The active Pi session may
59
+ independently capture new memories; these counts are a snapshot, not a frozen fixture.
60
+
61
+ - The Chinese project-boundary question selected the actual cross-session preference.
62
+ - The equivalent English question ranked that preference first, with a relevant provenance
63
+ qualification second; the weak storage-directory fact was no longer selected.
64
+ - The reload/injection question selected two relevant rules, not old CI/setup filler.
65
+ - The model/auth question retrieved corresponding implementation/reuse context.
66
+ - Context-free continuation and unrelated networking topics were empty. A memory-topic
67
+ continuation recalled the preference; a later topic switch did not revive it.
68
+
69
+ ## Limits and activation
70
+
71
+ These checks establish orchestration and bounded regression cases, not general live-model
72
+ accuracy. Tool results/reports and aliases remain untrusted evidence; generated states
73
+ are provisional. A quoted command is not execution, and a successful commit/test is not
74
+ proof of a successful push. The model can still make semantic mistakes.
75
+
76
+ The vocabulary is not universal translation; corpus-sensitive thresholds can miss useful
77
+ results. Cross-origin semantic identity is not automatically resolved. Upgrading does not
78
+ invent completion for existing stale entries or replay old tool transcripts. New eligible
79
+ observations/compactions can update tracked progress; exact-ID correction remains available.
80
+
81
+ The local installation references the checkout. Back up with Pi stopped before schema
82
+ upgrade, then reload/restart all instances sharing the database. Status should report
83
+ `SQLite ok (schema 3)` and global topic-based recall. Older builds require a matching
84
+ backup for rollback; do not manually downgrade the schema marker. No npm/tag release or
85
+ live paid-provider/multi-day TUI validation was performed.
@@ -0,0 +1,82 @@
1
+ # Follow-up review of memory evolution 0.2
2
+
3
+ **Historical implementation review at `038de65`.** Its directory-scoped recall assumption
4
+ was subsequently identified as a requirements error: memory must follow the conversation,
5
+ not the working directory. Current behavior is documented in [design.md](design.md):
6
+ all-origin recall, recent-user topic context, visible provenance and conservative writes.
7
+ The test counts below describe that earlier review, not the current test suite.
8
+
9
+ Baseline: `9c84010` on `main`. Scope: all implementation modules, tests, the real-Pi
10
+ smoke script, package metadata and documentation. This review keeps automatic memory
11
+ updates, Pi's active model/authentication and the no-tools/no-approval boundary.
12
+
13
+ ## Reproduced issues and fixes
14
+
15
+ The first nine isolated regression cases all failed against the baseline before fixes.
16
+ Additional tests were added while inspecting lifecycle sequences and host contracts.
17
+
18
+ | Area | Defect | Fix |
19
+ |---|---|---|
20
+ | Privacy | Removing controls after matching could assemble an unchecked `password` label. Multiline quoted/JSON values and indented YAML secrets leaked past line-only matching. | Normalize controls first; suppress quoted values and indented continuation blocks before capture, model submission and display. |
21
+ | Markdown extraction | Inner triple backticks could close a four-backtick fence. Bold stripping changed recursive globs, including globs inside code spans. Recognized nested headings caused later progress siblings to disappear. | Match fence character/length, protect code spans, only remove simple bold labels, and maintain a heading stack. |
22
+ | Identity and scope | Colon-concatenated scope/kind/content could collide for valid colon-containing paths. An undocumented `*` scope was included in every scoped read. | Hash a JSON tuple for new IDs and use exact scope lookup. Existing IDs remain valid. |
23
+ | Forget/retry | Forget retired only the first parent; other pending sources repeating the same fact could still feed a model and relearn a paraphrase. | Retire known repeating pending/failed sources, including formatted claims beyond the normal ingestion quota. Keep the current valid replacement transaction's source intact. |
24
+ | Legacy migration | Parent correction erased unchanged children; earlier child edits could prevent later corrected parents from deriving new facts. Corrected legacy claims lost old-content suppression when adopted. | Preserve unchanged children, distinguish newer parent corrections from obsolete child content, and carry suppression hashes through adoption. |
25
+ | Undo/integrity | A structurally corrupted before/after pair could write an unrelated record during undo. Indexed identity/hash and source job errors were incompletely checked; unsupported schemas could receive DDL before rejection. | Validate paired/unique IDs, record metadata and indexed values; validate sources/history in status; reject unsupported versions before DDL. |
26
+ | Batch deduplication | The same content with different kinds could be inserted twice within one local/model batch. | Deduplicate staged claims by exact content as well as persisted records. |
27
+ | Model provenance | Reading the live `ctx.model` again after awaiting completion could label an old response with the newly selected model or throw after context invalidation. | Capture the model and its identity before the call. |
28
+ | Async lifecycle/UI | A throwing context getter outside the queue's `try` could poison later work and shutdown. Notification failure could masquerade as rollback after a successful commit. Commands could reopen storage after shutdown. | Bound the entire queued task, isolate notifications from commits, retain truthful skipped/failed outcomes, and stop commands after shutdown. |
29
+ | Inspection commands | Only 20 unpaginated records were accessible; older legacy imports could not be reached for adoption. `show` omitted useful provenance. | Paginated current/all/legacy lists with stable ordering; `show` includes source and timestamps. Truncated previews are marked. |
30
+ | Excerpts | English sentences were not separated, and a long matching sentence could be clipped before its actual match. Tiny byte budgets could be exceeded by an ellipsis. | Sentence-aware and match-centered excerpts, code-point-safe clipping, and explicit tiny-budget handling. |
31
+ | Evidence dates | Pin/unpin, adoption and undo could make old project-state evidence appear fresh for another seven days. | Preserve or restore the prior evidence date; retain operation time separately in event history. |
32
+ | Smoke harness | Spawn failure and early process exit were not handled cleanly; cleanup could finish before the child exited. | Handle process/stdin errors, await close with bounded termination, and clean temporary state on failure. |
33
+
34
+ The wildcard case requires such a row to exist; there is no public command creating
35
+ one. The unsafe undo case requires corrupted/edited local history, not ordinary model
36
+ JSON. Neither is evidence that production data was attacked or overwritten.
37
+
38
+ ## Validation
39
+
40
+ - **88/88 tests passed**, including a deterministic 150-step mixed lifecycle sequence.
41
+ - Strict TypeScript check passed.
42
+ - Node coverage: **100% lines, 90.28% branches, 98.25% functions**. Coverage alone is not
43
+ a correctness guarantee; the baseline's high coverage did not catch these state-order bugs.
44
+ - Four real Node processes concurrently writing SQLite remain covered.
45
+ - Real Pi 0.85/Bun with a loopback fake OpenAI-compatible endpoint verified default-model
46
+ and authentication reuse, two automatic updates, recall injection, removal of the digest
47
+ after `/memory forget`, integrity checks and absence of approval dialogs.
48
+ - A nonexistent Pi executable fails without leaving new temporary state or a live server.
49
+ - Packaging is checked using `npm pack --dry-run`; tests and scripts are not runtime files.
50
+ - Synthetic performance sample: 600 summaries / 1,800 claims captured in about 1.05 s;
51
+ reopening about 2 ms; cold/warm recall about 169/69 ms. These are local observations,
52
+ not cross-machine benchmarks or a guaranteed performance budget.
53
+
54
+ All tests use temporary state and synthetic inputs. No paid model call, production-state
55
+ migration, cleanup or reset was performed as part of the validation.
56
+
57
+ Main regression files (links refer to a Git checkout; tests/scripts are not shipped in
58
+ an npm tarball):
59
+ Tests and maintenance scripts live in the Git checkout, not the runtime tarball:
60
+
61
+ - [State/privacy/parser regressions](https://github.com/btnalit/pi-memory-evolution/blob/main/src/memory/review-regressions.test.ts)
62
+ - [Pi lifecycle and command tests](https://github.com/btnalit/pi-memory-evolution/blob/main/src/index.test.ts)
63
+ - [Adapter tests](https://github.com/btnalit/pi-memory-evolution/blob/main/src/adapter/pi-api.test.ts)
64
+ - [Retriever tests](https://github.com/btnalit/pi-memory-evolution/blob/main/src/memory/retriever.test.ts)
65
+ - [Real-Pi smoke script](https://github.com/btnalit/pi-memory-evolution/blob/main/scripts/test-pi.mjs)
66
+
67
+ ## Remaining boundaries
68
+
69
+ - Semantic model accuracy, arbitrary paraphrase equivalence and exhaustive secret
70
+ detection are not guaranteed. Inferred claims remain provisional.
71
+ - Suppression conservatively skips an entire known repeating source's pending model
72
+ pass. Unrelated local claims remain, but unlearned prose may need a new explicit source.
73
+ - Forget/undo is logical, not secure erasure. Previous files/history are preserved.
74
+ - Migration fixes apply to new imports; completed imports are not replayed and previously
75
+ discarded revision information is not automatically reconstructed.
76
+ - Structural checks are not authentication of all well-formed local database edits.
77
+ - No live-provider quality evaluation, multi-day TUI trial or minimum-Node-version matrix
78
+ was run. Local validation used Node 26.8.1 and the real Pi 0.85 standalone binary.
79
+ - History retention/export and periodic backlog draining remain outside this small design.
80
+
81
+ See [README.md](../README.md) for current commands and recovery procedures, and
82
+ [design.md](design.md) for runtime invariants.
@@ -0,0 +1,102 @@
1
+ # Testing
2
+
3
+ Run these commands from a Git checkout after `npm ci --ignore-scripts`.
4
+ Tests use synthetic data and temporary directories, not production memories.
5
+
6
+ ## Regression and package checks
7
+
8
+ ```bash
9
+ npm run check
10
+ ```
11
+
12
+ This runs strict TypeScript checking, the `src/**/*.test.ts` regression suite and
13
+ `check:package`. Package checking asserts that the Pi manifest points to the current
14
+ entry, host APIs remain peer dependencies, all production TypeScript sources are
15
+ packed, and local documentation links resolve to packaged files. Test sources,
16
+ helper scripts and memory state must not ship in the package.
17
+
18
+ `check:package` inspects `npm pack --dry-run --json`; it does not publish to npm.
19
+
20
+ ## Installation smoke test
21
+
22
+ ```bash
23
+ npm run test:install
24
+ # Or select another installed Pi executable:
25
+ PI_TEST_BINARY=/path/to/pi npm run test:install
26
+ ```
27
+
28
+ Requires Pi 0.85+, Node.js 22.19+, npm, Git and `tar`. npm scripts can resolve the
29
+ bundled Pi from dev dependencies first; use `PI_TEST_BINARY` to select your installed
30
+ host explicitly. The script prints the tested host version. The test:
31
+
32
+ 1. Builds and extracts the actual npm tarball, outside the development checkout.
33
+ 2. Uses real `pi install`, `pi list` and default package discovery—no explicit `-e`
34
+ entry or memory-extension wrapper. Checks `/memory status`, `/memory learning`
35
+ and `/memory explain`, including schema 5, with no checkout `node_modules`.
36
+ 3. Verifies repeated installation does not duplicate the package setting, then
37
+ removes it and confirms the command disappears while records/history remain.
38
+ 4. Creates a local Git origin from the packed files and the real lockfile. A
39
+ fixture-only Git URL rewrite redirects an `.invalid` URL to this origin; only
40
+ `file` transport is permitted. No GitHub access is needed.
41
+ 5. Exercises the native Git installer and its real npm dependency step, updates to
42
+ a new commit, switches from an old pinned tag back to the default branch, and
43
+ verifies source switching and removal preserve schema-5 state.
44
+ 6. Serves the actual tarball through a loopback npm registry, with a fresh cache,
45
+ then checks native `pi install npm:pi-memory-evolution`, repeat installation,
46
+ normal loading and removal. No host peer packages are served or installed.
47
+
48
+ The test whitelists child environment variables, gives Pi a fresh agent directory
49
+ and HOME, disables startup network operations, and uses private npm caches/config with
50
+ lifecycle scripts, audit and update notifications disabled. npm is offline for Git
51
+ installation; only the npm fixture uses the loopback registry. Explicit Git updates
52
+ need `PI_OFFLINE=0` (otherwise Pi silently skips them), but file-only Git transport
53
+ and offline npm still prevent public network access. No real credentials are copied.
54
+ Only diagnostic slash commands are submitted; a model turn is a test failure.
55
+ Temporary files and child Pi processes are cleaned up.
56
+
57
+ This checks Pi's package-management and extension-loading path, **not** public
58
+ GitHub/public-registry availability, npm publication, every platform or the
59
+ behavior of dependency lifecycle scripts. There is no project-specific installer:
60
+ users install with Pi's native package manager. These tests are maintenance checks,
61
+ not an extra installation step for users.
62
+
63
+ ## Real host with a simulated model
64
+
65
+ ```bash
66
+ npm run test:pi
67
+ # PI_TEST_BINARY=/path/to/pi is also supported here.
68
+ ```
69
+
70
+ This uses actual Pi processes with a loopback fake provider, testing active-model
71
+ and auth plumbing, automatic replacement, fresh cross-directory sessions,
72
+ contextual follow-ups, unknown-topic barriers, bilingual aliases, provenance,
73
+ feedback, forget and the read-only recall tool without approval dialogs.
74
+
75
+ It executes real Git commits in temporary repositories and intentionally failed
76
+ pushes with no remote. Long tool streams verify that early work evidence still
77
+ nominates old pending states. Persisted failures, malformed model output, startup
78
+ recovery and the real 15-second timer are also exercised; only synthetic retry due
79
+ times are accelerated.
80
+
81
+ These are real-host integration tests, **not live-provider semantic accuracy tests**.
82
+
83
+ ## Live-provider validation
84
+
85
+ A live-provider run must use Pi's actual selected model/auth, synthetic fixtures and
86
+ an isolated memory state directory. Check actual records and transaction history,
87
+ not just an assistant acknowledgement or a job marked `done`. Inspect outgoing
88
+ provider context for recall, and retain unverified portions of composite states.
89
+
90
+ Use explicit time/call budgets and preserve failures. Do not replay production
91
+ transcripts, copy credentials, or hand-edit production claims to make a test pass.
92
+ `--offline` disables Pi startup probes, not model requests. Background learning can
93
+ consume provider quota and is not included in foreground session token totals.
94
+
95
+ There is no paid live-provider command in the repository's automatic checks.
96
+ A single successful live scenario does not establish multi-day reliability or full
97
+ product acceptance. Historical design/review documents describe their own tested
98
+ revisions; see [progress pipeline](progress-pipeline.md), [core quality](core-quality.md)
99
+ and [quality validation](quality-validation.md) for context.
100
+
101
+ No GitHub Actions workflow is configured. Run the local checks before submitting
102
+ changes; the Pi-dependent scripts are explicit commands, not part of `npm run check`.