instar 1.1.3 → 1.2.1

package/upgrades/side-effects/secret-drop-hardening.md

@@ -0,0 +1,64 @@
+# Side-effects review — Secret Drop hardening
+
+Per L6. Seven dimensions.
+
+## 1. Over-block / under-block
+
+**Before.** UNDER-block: a buggy consumer that called `/secrets/retrieve/<token>` once and dropped the response (parse error, exception in the writer step) lost the secret with no recovery path. The aggressive deletion-on-retrieve looked like a tight one-shot guarantee but actually created a silent-loss footgun, because the same 5-minute cleanup timer was already handling memory hygiene.
+
+**After.** No new over-block: the default retrieval shape returns the same value the previous code did. Existing callers that don't pass `?consume=true` simply gain the ability to retry within the cleanup window — a safer state, not a different one. Callers that genuinely need one-shot semantics opt in with the query parameter.
+
+The new stuck-consumer event closes the silent-failure gap: a submission that lingers unconsumed for 60s now produces a visible cue to the bound topic. Operators see the failure instead of waiting on a value that's already lost.
+
+## 2. Level-of-abstraction fit
+
+Three changes at deliberately different altitudes:
+
+- `SecretDrop.ts`: new methods (`peekReceived`, `consumeReceived`, `onStuckConsumer`) and the stuck-timer state machine. Pure logic, unit-tested.
+- `routes.ts`: the route honors `?consume=true`; one default listener wires the stuck-consumer event to the topic-bound session via `injectPasteNotification`. Thin orchestration only.
+- Back-compat alias: `getReceived` is kept as `consumeReceived`'s legacy synonym so no caller has to migrate in this PR. Callers migrate at their own pace by switching to `peekReceived` + `consumeReceived` explicitly.
+
+## 3. Signal vs Authority compliance
+
+The Secret Drop request remains the authoritative declaration that this token may receive a submission (operator-issued via `/secrets/request`). The submission, once made, remains the authoritative payload — no agent code can mint a fake submission. The new methods change only the lifecycle of an already-authorized value:
+
+- `peekReceived`: read-only, idempotent, returns the existing authority.
+- `consumeReceived`: terminates the authority's lifetime explicitly.
+- `onStuckConsumer`: signal-only emission to listeners; no listener gains write access to the submission.
+
+No new authority is created. Existing CSRF, rate-limit, expiry, and one-time submission semantics are unchanged.
+
+## 4. Interactions with adjacent systems
+
+- **The `/secrets/retrieve/<token>` route.** Default behavior shifts from destructive to non-destructive. Existing callers that don't add `?consume=true` retain the value across multiple calls — strictly safer; no callers depend on "second call returns 404" except as a footgun.
+- **The `/secrets/drop/<token>` form-side route.** Untouched. Operator-facing flow is byte-identical.
+- **`onReceive` callback on the request.** Untouched. Still fires once on submit.
+- **Existing 5-minute auto-cleanup.** Untouched. Still the safety net.
+- **System-message routing via `injectPasteNotification`.** Reused for the new stuck-consumer system message. Same pattern as `secret-drop-received`.
+- **Existing `SecretDrop.test.ts` suite.** All 25 cases continue to pass. The new test file adds 13 cases for the new surface.
+
+## 5. Rollback cost
+
+Low. Three files modified (`SecretDrop.ts`, `routes.ts`, one test file added). `git revert <merge-sha>` restores the pre-change atomic retrieve-and-delete behavior. Callers that adopted `?consume=true` continue to work after a revert (the parameter becomes a no-op against the pre-change atomic-delete path). The stuck-consumer event simply stops firing.
+
+No state migration, no deployed-agent impact, no schema change. In-memory only.
+
+## 6. Backwards compatibility / drift surface
+
+Fully backwards-compatible at the call-site level:
+
+- `getReceived(token)` is preserved as an alias for `consumeReceived(token)`.
+- The route default is non-destructive — strictly safer for existing callers.
+- Callers that depend on one-shot semantics (the bridge script in the parallel telegram-history-backfill PR being one example) opt in via `?consume=true` explicitly.
+
+Drift surface: small. Two new methods on `SecretDrop`. One new event interface. One new constant for the grace period. The grace period is a hard-coded constant for v1 ship; a future PR can make it configurable per-request if a real need surfaces.
+
+## 7. Authorization / Trust posture
+
+No new authority. The two new methods read/write the same in-memory `received` map the previous code touched. The new listener-registration surface (`onStuckConsumer`) accepts callbacks from the route layer only — there is no external API to register a listener over HTTP. The route layer's default listener routes only to the topic-bound session (the same authority the `secret-drop-received` path already uses).
+
+Trust ramifications: the operator gains a new visible signal (the stuck-consumer system message), which strengthens their ability to detect consumer-side failures. The agent gains a new structural primitive for safe-by-default retrieval, which reduces the surface for silent-loss bugs. Both directions of trust move in the safer direction.
+
+## Outcome
+
+Ship. Incident-driven, three independent changes, fully unit-tested, fully backwards-compatible at the call-site level. The single highest-impact change is the default-non-destructive retrieve — that alone makes the 2026-05-20 lost-SMS-code failure mode structurally impossible to repeat without an explicit `?consume=true` from the buggy consumer.

package/upgrades/side-effects/telegram-history-backfill.md

@@ -0,0 +1,131 @@
+# Side-effects review — Telegram history backfill (script tier)
+
+Per L6. Seven dimensions.
+
+## 1. Over-block / under-block
+
+**Before.** Echo had no path to retrieve Telegram history after a local
+topic-memory.db truncation. The Bot API is forward-only, so any data loss
+in the local mirror was permanent absent a Telegram-side export. The
+2026-05-20 native-module cascade exposed this gap.
+
+**After.** The script-tier backfill closes the gap for the post-recovery
+scenario. There is no over-block — the script runs only when invoked
+explicitly (`--chat` flag required), never silently. No new automatic
+behavior is added to any always-on path. Under-block protection: the
+importer's UNIQUE-constraint-guarded INSERT OR IGNORE prevents duplicates
+even under retries, FloodWait recoveries, and manual reruns.
+
+## 2. Level-of-abstraction fit
+
+Three modules at deliberately different altitudes:
+
+- `topic-memory-importer.mjs` — pure DB logic, unit-tested against the
+  real schema. No network, no auth, no gramjs dependency. Trivially testable.
+- `telegram-historian.mjs` — gramjs wrapper. Exposes the minimum surface
+  needed (connect, listForumTopics, iterTopicMessages). No business logic.
+- `telegram-history-backfill.mjs` — orchestration only. Parses CLI flags,
+  resolves credentials, ties the two libs together.
+
+Each module is replaceable without touching the others.
+
+## 3. Signal vs Authority compliance
+
+This is the central concern for this PR.
+
+`.instar/secrets/...` is the credential signal. The operator's sealed
+file is the authoritative declaration that this account may be accessed
+via MTProto. The script reads the signal and acts on it; it has no
+authority to mint or rotate credentials.
+
+The session string, once captured, is the long-lived authority. Storing
+it at mode 0600 outside any git path is the structural enforcement that
+no other process can read it. The operator's Telegram Devices page is
+the only revocation channel — exactly the right placement of authority,
+because the operator alone owns that surface.
+
+The script's read-only Telegram posture (iter/get only, no send/edit/
+delete) is the third layer: even if everything else were compromised,
+this code path cannot produce outbound messages or mutations against
+Telegram.
+
+## 4. Interactions with adjacent systems
+
+- **instar server (echo's own process).** Holds `topic-memory.db` open
+  under WAL mode. SQLite serializes writes across processes, so the
+  backfill can write while the server is reading without corruption. The
+  FTS triggers maintain the FTS index automatically; tested by
+  `tests/unit/topic-memory-importer.test.ts`'s "does not corrupt FTS
+  index across re-runs" case.
+- **Bot-API polling (TelegramAdapter).** Unaffected. Live messages
+  continue to arrive through the bot. The backfill writes the same
+  schema with the same constraints, so the live path's INSERT OR IGNORE
+  (if any) and the backfill's INSERT OR IGNORE converge to the same
+  final state regardless of ordering.
+- **Coherence gate / external-operation-gate hook.** The script is a
+  local Node process; it does not go through the MCP tool path the
+  external-operation-gate intercepts. The gate's authority remains over
+  agent-initiated network operations from the harness; this script is
+  operator-initiated from the CLI.
+- **Migration parity / PostUpdateMigrator.** No agent-installed file is
+  modified by this PR. No template change, no hook change, no config
+  default change. Existing agents are unaffected by the merge — they
+  pick up the script only if they explicitly run it.
+
+## 5. Rollback cost
+
+Trivial. Five files, two libraries, one test, one spec, one ELI16, one
+side-effects review. `git revert <merge-sha>` removes the code. The
+sealed credential file and the session string file are outside the repo
+and unaffected by the revert. The `topic-memory.db` is touched only by
+INSERT OR IGNORE, so already-imported rows are correct and stay — no
+data restoration needed.
+
+If the operator wants the imported rows undone too, a single SQL command
+removes them: `DELETE FROM messages WHERE timestamp < '<backfill cutoff>'`.
+This is safe because all writes go through the importer, which can be
+disabled by removing the script.
+
+## 6. Backwards compatibility / drift surface
+
+Fully backwards-compatible. The script is additive only — no existing
+behavior is modified. The `topic-memory.db` schema is read but never
+altered (no migrations). The CLI entry is a new file; nothing references
+it from existing code paths.
+
+Drift surface:
+- The script targets the schema as it exists 2026-05-20 (read live for
+  this PR). A future schema change (e.g. adding a `chat_id` column)
+  would require updating `topic-memory-importer.mjs`. The narrow surface
+  and unit-test coverage make this easy to spot in code review.
+- gramjs (`telegram` package) is a new dependency. Version pin is
+  `^2.26.22` — caret range allows non-breaking updates. If gramjs ever
+  ships a breaking 3.x, the import surface in `telegram-historian.mjs`
+  is small (4 imports, 3 method calls) so adaptation cost is bounded.
+
+## 7. Authorization / Trust posture
+
+The single point where new authority is granted is the operator's
+explicit hand-off of `api_id` + `api_hash` (2026-05-20T03:10:18Z via
+Telegram topic 10873) and the live SMS-code exchange during `--auth`.
+The script grants itself nothing — it only reads what the operator has
+already authorized.
+
+The session string, once captured, is the durable artifact of that
+authorization. The operator can revoke it via Telegram's Devices UI at
+any time; the script will then error on the next run and require a
+fresh `--auth`. This places revocation entirely in the operator's hands
+with no instar-side dependency.
+
+The "read-only on Telegram" posture is enforced by **not exposing any
+write method**, not by a guard the operator must trust. Adding a write
+method would require a spec change AND a fresh operator approval — the
+review-convergence + approved frontmatter tags would have to be
+regenerated, which is the structural enforcement here.
+
+## Outcome
+
+Ship. Incident-driven, single-purpose, read-only, idempotent, unit-tested.
+Promotion to a full `src/messaging/TelegramHistorian.ts` feature is a
+separate, follow-up PR after the recovery itself is verified end-to-end
+against the live lifeline supergroup.

package/upgrades/side-effects/test-env-isolation.md

@@ -0,0 +1,84 @@
+# Side-effects review — test-env-isolation
+
+Spec: `docs/specs/test-env-isolation.md`
+ELI16: `docs/specs/test-env-isolation.eli16.md`
+
+## Surface map
+
+| Layer | File(s) added / changed | Surface |
+|------|------------------------|---------|
+| 1 | `tests/vitest-setup.ts` | Process env at vitest startup. Every test inherits the strip. |
+| 2 | `tests/helpers/git-test-env.ts` (new); refactors to `tests/unit/worktree-monitor.test.ts`, `tests/unit/SafeGitExecutor.test.ts`, `tests/integration/rich-profile-integration.test.ts` | Three fixtures' git-spawn calls now pass an explicit sanitized env. |
+| 3 | `scripts/pre-push-fixture-guard.mjs` (new); `.husky/pre-push` (wire-in) | Pre-push hook gains a guard step that runs before the test suite. |
+| 4 | `scripts/check-repo-invariants.mjs` (new); `.github/workflows/ci.yml` (new `invariants` job) | Standalone CI job, no dependencies, runs in parallel with type check. |
+| 5 | Deletions: `file-0.txt`, `seed` | Repo root no longer carries fixture stowaways. |
+
+Tests: 14 new across `tests/unit/vitest-setup-git-env-strip.test.ts`, `tests/unit/scripts/pre-push-fixture-guard.test.ts`, `tests/unit/scripts/check-repo-invariants.test.ts`.
+
+## Over-block analysis
+
+**Layer 1 strip.** Could a legitimate test *need* the inherited `GIT_DIR`? In our codebase, no — every existing fixture that touched git either operates on a tmpdir (which doesn't need it) or already strips the vars locally. We grepped `tests/` for any consumer of `process.env.GIT_DIR` and found zero direct readers. The strip cannot regress a test that depends on the override, because no such test exists.
+
+If a future test needs to operate against the real repo (e.g., a meta-test that inspects the project's own git state), it can re-set the variable explicitly inside its own scope — the global strip removes the *inherited* value, not the *ability to set* one.
+
+**Layer 3 fixture-author guard.** Could it block a legitimate commit? Possible failure cases:
+
+- Author email of `test@test.com` is in the bad-author set. A real human who happens to use that email would be blocked. Mitigation: the email is in our internal fixture set; real contributors use their own emails. If a real case ever arises, `INSTAR_PRE_PUSH_FIXTURE_GUARD_SKIP=1` lets them through.
+- Subject line `init` matches the pattern. A real "init" commit from a new repo bootstrap would trip the guard. Mitigation: bootstrap commits live on their own branch and aren't pushed against an established `upstream/main`, so the guard's `<base>..HEAD` range returns empty for them. Tested in `pre-push-fixture-guard.test.ts`.
+
+**Layer 4 README line floor.** Could a legitimate README shrink below 100 lines? The current README is 271 lines and the project is growing, not shrinking. A deliberate slim-down would need `INSTAR_README_MIN_LINES` override, which is exactly the escape hatch.
+
+## Under-block analysis
+
+**Layers 1+2 don't catch process-spawning that re-injects the env.** If a future utility spawns `git` with an explicit `env: { GIT_DIR: '...' }`, layers 1+2 don't help — they only strip *inherited* env. Layer 3 catches the *consequence* (a pollution commit on the branch), so the failure class is still closed end-to-end, but layers 1+2 alone are not sufficient. This is acknowledged in the spec.
+
+**Layer 3 doesn't catch destructive changes that don't match the historical signature.** A future test fixture that wrote `# Different Stub` to README would land it if the commit message and author look normal. Layer 4 catches the resulting line-count regression; together, layers 3 and 4 cover both the "known-pattern pollution" and "novel pollution that drops content" failure modes.
+
+**Layer 4 doesn't enforce content correctness.** A README with 100+ lines of garbage would pass. We deliberately scoped Layer 4 to structural invariants only; content review is the job of human PR review.
+
+## Level-of-abstraction fit
+
+The fix lives at the right layer:
+
+- The root cause is in *test setup*, so the fix is in `vitest-setup.ts` — the central, single point that runs before every test. We did not push the strip down into individual fixtures (would require N changes and could be missed); we did not push it up into the test runner config (the var would still leak into test files that import their own helpers that ran first).
+- The push-time guard lives in `.husky/pre-push`, which is where push-time checks already live.
+- The CI invariant is a standalone job, not a step bolted onto an existing job that might be skipped under failure-routing rules.
+
+## Signal-vs-authority compliance
+
+Layer 3 is a brittle pattern matcher (authors + subjects) — it's a **signal**. It cannot block a sufficiently-disguised commit, and it doesn't try to. Authority for refusing a commit rests with the deterministic layers (1, 2, 4) and human review. The signal exists to make the *common* failure mode loud, not to be the load-bearing protection.
+
+The deterministic strip (Layer 1) is the load-bearing protection. The CI invariant (Layer 4) is the structural assertion. The signal (Layer 3) makes the failure visible early but is not the only line of defense.
+
+## Interactions with existing systems
+
+- **`pre-push-gate.js`** (NEXT.md / version check) runs first and remains unchanged.
+- **Working-tree integrity check** (post-test step in `ci.yml`) catches *uncommitted* mutations; the new invariants job catches *committed* state. Complementary, not duplicate.
+- **`lint-no-direct-destructive.js`** already requires `safe-git-allow:` on raw git use. Our new scripts carry it; the test files for those scripts carry it. No conflict.
+- **`PostUpdateMigrator`** is not affected — none of these changes touch agent-installed files. The Migration Parity Standard does not apply.
+
+## Rollback cost
+
+All five layers are net-additive. Each can be reverted independently:
+
+- Layer 1: revert five `delete` lines.
+- Layer 2: revert the helper file and three fixture edits.
+- Layer 3: comment out two lines in `.husky/pre-push` or delete the script.
+- Layer 4: comment out the `invariants` job in `ci.yml`.
+- Layer 5: re-create `file-0.txt` / `seed` — but they had no legitimate function, so no consumer breaks.
+
+No data migration. No schema change. No agent-state mutation. Reverting any layer leaves the repo in a valid state and the test suite green.
+
+## Risk summary
+
+- **Low risk of regression.** Layer 1 is a single-line process.env mutation in test setup that affects nothing in production code. Layers 2–5 are net-additive guards.
+- **Moderate risk of friction.** Layer 3 will be loud when triggered, including against legitimate edge cases (the documented bypass exists for this). Layer 4 will fail builds on cosmetic README slim-downs unless `INSTAR_README_MIN_LINES` is overridden.
+- **No risk of data loss.** Nothing in this change mutates state outside the worktree.
+
+## Verification done before commit
+
+- All targeted test files pass locally (45 in worktree-monitor + SafeGitExecutor, 14 in new tests).
+- `npm run lint` passes with the `safe-git-allow:` headers in place.
+- `node scripts/check-repo-invariants.mjs` against the prepared worktree returns clean.
+- `node scripts/pre-push-fixture-guard.mjs` against the prepared worktree returns clean.
+- Spec carries `approved: true` per direct principal authorization in Telegram topic 11235.