pi-longhaul 1.0.2 → 1.0.4

package/CHANGELOG.md

@@ -8,6 +8,70 @@ entries and log event types. Rendered state text, prompt wording and section lay
 
 ## [Unreleased]
 
+## [v1.0.4] - 2026-10-22
+
+### Fixed
+
+- The summarizer copied credentials into the state it wrote: a password from a script's output into Resume, an API
+  key or Bearer value into Now, decisions and the narrative. The state is sent with every later request, so the
+  value went out again at every turn, and states on real sessions held live ones. What the summarizer writes is now
+  scrubbed before it is recorded: each credential value becomes `<redacted>` and the rest of the line stays. A value
+  is one when the span showed it as a credential (a credential key's value, symbols and all, a command's password
+  argument, a value a secret store printed, `the password is …`), wherever the summarizer put it and however it
+  wrapped it; when the text names it
+  (`password=…`, `"api_key": "…"`, `Bearer …`, a `user:pass@` login, or a random-looking value just after a
+  credential word, so `password (64-hex): <hex>` and `the gateway key (<uuid>)`); or when its shape is one wherever
+  it stands (a JWT, a Vault token, AWS, GitHub, OpenAI-style, Slack and age keys). The span's credentials include
+  what the previous summary and a switchover's earlier messages showed. A digest (`sha256:…`) is never one, and
+  neither is a path, a count, a version or a date the text gives a key, nor, among the span's credentials, a name
+  (`app-token`) under a token or secret key, an id or hash its label names (`Evaluation ID:`, `plan_hash`,
+  `commit`), the rest of a command after a key whose value it computes (`TOKEN=$(vault kv get … | python3 …)`), a test
+  name after `--- PASS:`, or a value after a credential word that is part of a file name (`~/.vault-token`): on real
+  sessions the scrub took out the credentials and left versions, hashes, ids and names alone. A state written before this shows redacted from its session's next compaction
+  on (the summary, the summarizer's prompt, `longhaul_recall`, `npm run state`), by every rule but the span's own
+  credentials, which only the pass that wrote it had. Until then pi keeps sending the summary it stored, and a
+  transcript keeps every summary and record it already holds, so a credential found in a state must be rotated. The compaction log records how many values a pass redacted
+  (`redacted`), never the values. Both summarizer prompts also tell the model never to copy a credential's value
+  and to say where it lives instead, and `longhaul_remember` refuses a decision or dead end that carries one.
+- An error line naming a credential before its value (`ops-control password (64-hex): <value>`) was recorded as an
+  open error and shown in every state; no command-style shape matched it. Such a line is now recorded with the value
+  redacted, so the failure stays open, and one already carried is redacted at the next compaction.
+- The credential rules found too little: a password key glued to its prefix (`PGPASSWORD=`, `MYSQL_PWD=`), short
+  keys (`pw=`, `pass=`), `Authorization: Basic`, a private key's body past its first line, and GitLab, Google API,
+  Stripe, npm and Hugging Face tokens, now in both the scrub and the command ledger's credential shapes.
+
+## [v1.0.3] - 2026-10-22
+
+### Changed
+
+- Tested with pi 1.3.0, now recorded in `pi-compat.json`. No code changes to the extension for it.
+
+### Fixed
+
+- A span's record was lost whenever another extension wrote an entry between pi's compaction and Longhaul's span
+  record. pi-goal does this at every compaction (its `goal-state`, sometimes a `goal-contract` message), active
+  goal or not, so in a session with pi-goal installed most spans folded as narrative-less: the next state kept
+  only the span just closed, and every earlier span's work-log line, Now, decisions, dead ends, objective moves and
+  labelled rules were gone from it. The fold now passes over other extensions' entries until our record arrives; a
+  message, a compaction or one of Longhaul's own event entries arriving first still means the record never
+  landed. The records were always written, so an affected transcript gets its history back at its next
+  compaction. While its records were lost, each pass was shown earlier messages again and recorded their rules
+  again under new ids; with the records back, a rule recorded from the same message in near-duplicate words is
+  one rule, under its newest id, and a lift of any copy lifts it. A rule the user withdrew while its record was
+  lost could not be lifted then (the summarizer did not see it) and comes back now, so look over the
+  constraints the first state after the upgrade shows.
+- Spans whose records were written while earlier records were being lost can carry the same event seqs as an
+  earlier span (each pass numbered from a fold that had lost events). The fold now treats a seq as unique within
+  its span, so those spans fold too; a seq repeated inside one span is still refused as a duplicate. A correction
+  `longhaul_recall` shows as lifted is now the one the lift landed on, never another span's event with the same
+  seq.
+- A replayed event whose span or seq is not a whole number from 1 is refused as malformed. A seq of `1e999`
+  (Infinity) used to be accepted and broke the numbering of every event after it: a `longhaul_remember` write
+  was then stored without a seq and dropped, with no anomaly, when the transcript was read back.
+- The load smoke (`npm run load-smoke`, a development check that is not part of the package) resolved a
+  wildcard `exports` target by replacing only its first `*`; it now replaces every one, as Node does (CodeQL
+  `js/incomplete-sanitization`). No pi package uses a target with two, so the smoke's results are unchanged.
+
 ## [v1.0.2] - 2026-10-13
 
 ### Changed

package/CONTEXT.md

@@ -76,6 +76,12 @@ Values a later turn may need again, verbatim, with where each came from: those t
 that the session used again (a hash, a version, a long number).
 _Avoid_: facts (the scorer's measure of a related but different set)
 
+**Credential scrub**:
+Taking each credential value out of the text the summarizer writes into the state (the narrative, Now,
+decisions, dead ends), leaving `<redacted>` and the words around it. The value is gone; what it was and where it
+lives can stay.
+_Avoid_: sanitize (one-line.ts: control characters and newlines), redact (the ledger's word for dropping a whole command)
+
 ## Configuration
 
 **Agent dir**:

package/README.md

@@ -349,7 +349,16 @@ compaction than one that can.
   never from a line that looks like it holds a credential. A value the summarizer names is kept only when it
   stands in the turn it cites and no part of it stood on a credential line anywhere in the span; its name is
   shown as the summarizer's label.
-  That filter works on patterns: a bare token with no key or secret marker near it can still be learned and
+- What the summarizer writes into the state (the narrative, Now, decisions and dead ends) has each credential
+  value replaced by `<redacted>` before it is recorded: a whole value the span showed as a credential (a credential
+  key's value, a command's password argument, a secret store's output, `the password is …`), however it is wrapped, a value after a credential key,
+  `Bearer` or a credential word, a `user:pass@` password, and token shapes (JWTs, Vault, AWS, GitHub, OpenAI-style,
+  Slack, age). States written before this are shown the same way from their next compaction on, by everything but
+  the span's own credentials; a stored summary and the transcript keep what they hold, so rotate a credential found
+  in a state.
+  The prompt also tells the summarizer never to copy one, and `longhaul_remember` refuses a decision or dead end
+  that carries one. Error lines are recorded with the value redacted.
+  These filters work on patterns: a bare token with no key or secret marker near it can still be learned and
   carried. Treat a session's transcript as you would its tool output.
 
 The threat model is in [docs/security.md](docs/security.md).
@@ -361,9 +370,9 @@ listed in [pi-compat.json](pi-compat.json). On a listed line it runs silently. O
 per session and runs; a session that opted out of it (`PI_COMPACTOR=pi`) is not warned. If a compaction fails, pi's own compaction takes over. A nightly job tests against the
 newest pi and opens an issue when it breaks.
 
-Tested with pi 0.87.1, 0.99.0, 0.99.1, 0.99.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1.0 and 1.2.0. Upgrading or
-downgrading pi mid-project is covered too: CI hands one transcript through every listed pi, oldest to newest and
-back, with a Longhaul compaction by each, and each must continue what the others wrote.
+Tested with pi 0.87.1, 0.99.0, 0.99.1, 0.99.2, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.1.0, 1.2.0 and 1.3.0.
+Upgrading or downgrading pi mid-project is covered too: CI hands one transcript through every listed pi, oldest to
+newest and back, with a Longhaul compaction by each, and each must continue what the others wrote.
 
 ## Known issues
 

package/docs/security.md

@@ -39,6 +39,11 @@ extension.
   reading it into memory.
 - A timestamp a `Date` cannot hold is dropped rather than thrown on.
 - An id held by two entries (an old transcript can carry a re-minted id) resolves to the newest holder.
+- A rule recorded again from the message that gave it (near-duplicate words) folds into that rule; a copy of a
+  lifted rule stays lifted, so restored span records cannot bring back a rule the user withdrew from a copy. A lift
+  lands on the rule holding its id now, and on one that held it before only when none does: the store judges the
+  lift against that same rule, and `longhaul_recall` marks lifted what the state lifted.
+- A span or seq that is not a whole number from 1 (`1e999` parses as Infinity) makes the event malformed.
 - An entry from a newer state format is refused by name, so a downgrade starts clean rather than misreading.
 - Learned values read back from a transcript are checked again (length, printable ASCII, count). A name is
   checked as when it was given and dropped if it fails, and a named value must still have a value's shape.
@@ -212,6 +217,64 @@ the session (a web page, a README, another agent's message) could try to plant a
   replacement. A credential token anywhere in those turns refuses a value of the span as well, and so does one in
   the original words of an edited message, in a message another extension injected, or on a credential line of the
   summary, which is never a source itself.
+- What the summarizer writes into the state (the narrative, every Now item and Resume, decisions, dead ends) is
+  scrubbed before the span record is written (`credential-scrub.ts`): each credential value is replaced by
+  `<redacted>` and the words around it stay. A value counts when its shape is one wherever it stands (a JWT, a Vault
+  token, a private key's whole body, AWS, GitHub, GitLab, Google API, Stripe, npm, Hugging Face, OpenAI-style, Slack
+  and age keys); when the text names it, as a credential key's value (`DB_PASSWORD=`, `PGPASSWORD=`, `MYSQL_PWD=`,
+  `"api_key":`, escaped quotes too, symbols kept in the value up to a space or a quote; a password word may be glued
+  to its prefix, any other only after `_` or `-`, so `compass:` is no key; a value-like or opaque one of twelve
+  characters or more, never a path, and a number, count, version or date only as six digits and more under a key
+  that says password in full, so `token: missing`, `tests pass: 120/120` and `e2e_pass: 1234567` stay), a value-like
+  `Bearer` or `Basic` value, the password of a `user:pass@` URL (to the word's last `@`), or a random-looking value
+  (a UUID, hex from 20 characters, a long mixed-case run with digits) starting within 40 characters after a
+  credential word; or when it is one of the span's credentials (`span-credentials.ts`): a whole value the span
+  itself showed as a credential, found however the summarizer wrapped it (quoted, bold, between dashes or curly
+  quotes, after `=`, glued to a flag, in a login with or without a scheme: `app/x@db`, `app:x@db:5432`). The span's
+  credentials come from everything the summarizer was shown: the span, the previous summary, a switchover's earlier
+  user messages and, in warm mode, the session's last request. They are built from strong evidence only, because each
+  one is taken out everywhere: what the text rules above name; the value of a key named for one credential,
+  ending in its name or a short part after it (singular, so `tokens: 1200` and a list of `secrets` are not one, and
+  no key about one: `password_policy`, `token_env`), any shape under a password key
+  (`Summer2024.x`, `unmade-tremor-oxidize`, `48291375`), not a name under a token, secret or key key (`app-token`,
+  `APP_TOKEN`, a file or host name); a command's password argument (`mysql -pX`, `sshpass -p X`, `redis-cli -a
+  X`, `curl -u user:X`, `htpasswd -b FILE USER X`, `--db-password X`, `openssl -pass pass:X`, `keytool -storepass
+  X`); a random-looking word on another credential line; a value a secret store printed alone (or a random-looking
+  one among its other output); and `the password is …`, `the db pass is …`. Never a number (but a password's),
+  version, date or path, nor one plain lowercase word. A key holds its quoted string or its first word (after Go's
+  `:=` or a type, `db_password: str = "…"`; a YAML password's plain value runs to the line's end), never the rest of
+  a command line, and nothing when its value is computed (`TOKEN=$(vault kv get … | python3 -c "import sys,json")`
+  once made `python3` a credential); a random-looking word the word before it on its line labels as an id or a hash
+  (`Evaluation ID:`, `plan_hash`, `commit`) is that, not a secret, unless the label names a credential
+  (`secret_id`); and an upper-case `--- PASS: TestName` is a test runner's status, while `pass: 'x'` and `pass=x`
+  are keys. A credential word inside a path or file name (`/tmp/admin-bearer.txt`,
+  `~/.vault-token`) says where one lives, so the value after it is not taken for one. The broader set that refuses a named value
+  (narrative/named-values.ts, with the words after "pin", "card" or "login" and a secret store's whole output)
+  redacted versions, commit hashes and names in about a third of real fields, so the scrub does not use it. A
+  `sha256:` digest is never one, and a variable or placeholder (`$KEY`, `<from vault>`) stays. The projection applies
+  the text rules only, since only the pass that wrote a record had its span, so a record written before the scrub
+  shows redacted wherever the state is read from its session's next compaction on: the summary, the summarizer's
+  prompt (the open span's remembered items too), `longhaul_recall` (every hit, error lines and the span's narrative
+  included) and the state CLI. Nothing rewrites what is stored: pi re-sends the latest compaction's summary string
+  until the next compaction (on an older branch, whenever someone returns to it), and the transcript keeps the old
+  summaries, `details.state` checkpoints, span records, remember entries and the turns trace's model-written
+  reasons. A credential found in a state was sent with every request since; rotate it. `longhaul_remember` refuses
+  a decision or dead end that the scrub would change, with the credentials of every message still in context (the
+  kept window included). Measured on 24 real transcripts: the text rules redact 22 values over 6612 model-written
+  fields, at least one of them not a secret (an Ed25519 public key after "key"); the span's credentials, each span
+  against its own messages and the summary before it, redact 7 more over 2981 fields in 224 spans, every one a
+  credential or the visible part of one (a Wi-Fi passphrase, two admin passwords, a token's printed prefix, a rotated
+  secret's prefix). The summarizer prompts say never to
+  copy a credential's value; the scrub is what holds when a model does. An error signature with a command-style
+  shape is never recorded; any other is recorded as the scrub leaves it (`password (64-hex): <redacted>`), carried
+  ones too, so an ordinary failure that mentions a secret by id stays open. Every pattern is linear on hostile text
+  (T4); a private key without an END line is cut 16 KiB on. Not found: a credential the span never showed as one,
+  written in prose with no credential word near it; a token or secret key's value shaped as a name; without the
+  span, a value on the line after its key (`password:` then the value), a key in smart quotes or before a full-width
+  colon; a password given to `docker login -p` or piped to a `--password-stdin` command; a passphrase with spaces
+  (`correct horse battery staple`); a key about a password rather than holding one (`password_policy:`) gives no
+  value, by design; and a symbol at a value's edge (`Summer2024!` leaves `<redacted>!`). The user's own words (objective,
+  done-when and correction quotes) are verbatim by design and not scrubbed.
 - The ledger redacts commands that carry a secret, by the same patterns (a one-time-password seed passed to
   `oathtool -b` (or `-bv`, `--base32`, or a hex or base32 key of 16 or more characters), `ykman oath accounts add`,
   `ykman ... --key`, `sshpass -p`, `htpasswd -b`, `openssl -pass pass:`, `keytool -storepass`, `redis-cli -a`,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-longhaul",
-  "version": "1.0.2",
+  "version": "1.0.4",
   "description": "Keep pi sessions working all night. A compactor that carries your session forward instead of rewriting it: your rules kept in your own words, the state written by code, old reasoning left behind, and about 5x faster than pi's own compaction on a local model.",
   "type": "module",
   "license": "MIT",
@@ -32,7 +32,7 @@
     "extensions": [
       "./src/index.ts"
     ],
-    "image": "https://raw.githubusercontent.com/lobo235/pi-longhaul/v1.0.2/.github/assets/infographic.png"
+    "image": "https://raw.githubusercontent.com/lobo235/pi-longhaul/v1.0.4/.github/assets/infographic.png"
   },
   "files": [
     "src/**/*.ts",
@@ -75,10 +75,10 @@
     "typebox": "*"
   },
   "devDependencies": {
-    "@earendil-works/pi-coding-agent": "1.2.0",
-    "@types/node": "^24.19.1",
+    "@earendil-works/pi-coding-agent": "1.3.0",
+    "@types/node": "^24.19.2",
     "typebox": "1.3.7",
-    "typescript": "5.9.3"
+    "typescript": "7.0.2"
   },
   "author": "Justin Barlow"
 }

package/pi-compat.json

@@ -93,6 +93,15 @@
         "npm run smoke and npm run cross-version -- --extra 1.2.0: every listed pi continues 1.2.0's transcripts and it continues theirs",
         "one real compaction through pi --mode rpc on the setup-check fixture with a local model: a compaction record, no fallback"
       ]
+    },
+    {
+      "pi": "1.3.0",
+      "date": "2026-10-16",
+      "how": [
+        "npm test on the vendored pi (the version gate skipped): typecheck, unit suite, load smoke and the extension-API contract checks",
+        "npm run smoke and npm run cross-version -- --extra 1.3.0: every listed pi continues 1.3.0's transcripts and it continues theirs",
+        "one real compaction through pi --mode rpc on the setup-check fixture with a local model: a compaction record, no fallback"
+      ]
     }
   ]
 }

package/src/credential-scrub.ts

@@ -0,0 +1,319 @@
+/**
+ * Credential scrub (CONTEXT.md *Credential scrub*; security.md T3): text a model wrote into the state (the narrative,
+ * Now, decisions, dead ends) and the open-error lines, with each credential value replaced by REDACTED_VALUE. The
+ * state is rendered into every later request, so a password the summarizer copied out of a tool's output was sent
+ * again at every turn. Only the value goes: the words around it stay, so a Resume line still says what to do next.
+ *
+ * A value is a credential when
+ * - its shape is one wherever it stands: a JWT, a Vault token, an AWS, GitHub, OpenAI-style, Slack or age key;
+ * - the text names it: the value of a credential key (`password=x`, `"api_key": "x"`), a Bearer value, the password
+ *   of a `user:pass@` URL, or a random-looking value within CONTEXT_REACH characters after a credential word
+ *   (`password (64-hex): <hex>`, `the gateway key (<uuid>)`);
+ * - or it is one of the span's credentials (span-credentials.ts): a whole value the span itself showed as one
+ *   (`DB_PASSWORD=Ab3$x;9Kq2`, symbols and all), wherever the model put it and however it wrapped it (quoted, bold,
+ *   between dashes, glued to a flag or a login).
+ * A digest (`sha256:<hex>`) is never one. Every pattern is linear: a summarizer's answer is untrusted (security.md T4).
+ */
+
+import type { StateEvent } from "./store/events.ts";
+
+/** What a redacted value is replaced with. Holds no token character, so a second scrub leaves it alone. */
+export const REDACTED_VALUE = "<redacted>";
+
+/** The summarizer prompts' rule (narrative/prompt.ts, narrative/turns/conversation.ts); the scrub is what holds when a model ignores it. */
+export const CREDENTIAL_RULE = 'Never copy a password, key, token or other credential\'s value into the state, in any field: say what it is and where it lives ("the gateway key from its Vault secret"), never the value itself.';
+
+/** The span's credentials: whole values, each one judged a credential where the span showed it (span-credentials.ts). */
+export type SpanCredentials = ReadonlySet<string>;
+
+export interface Scrubbed {
+  text: string;
+  /** How many values were replaced; logged as a count, never the values. */
+  redacted: number;
+}
+
+/** A token: what a value is cut on, as named-values.ts TOKEN_SPLIT cuts a credential line. */
+const TOKEN = /[^\s=:"'`,;{}[\]()<>]+/g;
+/** A word: what a whole value is cut on. Space, and the dashes and quotes prose puts around a value (`—x—`, `“x”`). */
+export const WORD = /[^\s\u2013\u2014\u2018\u2019\u201c\u201d\u00ab\u00bb]+/g;
+/** How far after a credential word a random-looking value counts as its value. */
+const CONTEXT_REACH = 40;
+
+/** Shapes that are credentials wherever they stand (ledger/handoff-commands.ts SECRET_PATTERNS, and learned-secrets.ts TOKEN_SHAPES). */
+const CREDENTIAL_SHAPES = [
+  /(?<![\w-])eyJ[\w-]{4,}\.[\w-]{4,}\.[\w-]*/g,
+  /(?<![\w-])hv[sbr]\.[\w-]{8,}/g,
+  /\bAGE-SECRET-KEY-1[0-9A-Z]{20,}/g,
+  /\bAKIA[0-9A-Z]{16}\b/g,
+  /\b(?:gh[pousr]_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,}|sk-[A-Za-z0-9_-]{20,}|xox[abprs]-[A-Za-z0-9-]{10,})/g,
+  // GitLab, Google API, Stripe, npm and Hugging Face tokens; as ledger/handoff-commands.ts SECRET_PATTERNS, kept in step.
+  /\b(?:glpat-[A-Za-z0-9_-]{20,}|AIza[0-9A-Za-z_-]{35}|[sr]k_(?:live|test)_[A-Za-z0-9]{16,}|npm_[A-Za-z0-9]{36}|hf_[A-Za-z]{30,})/g,
+];
+const PEM_BEGIN = /-----BEGIN [A-Z ]{0,64}PRIVATE KEY(?: BLOCK)?-----/g;
+const PEM_END = "-----END ";
+/** A private key's body without an END line is cut here. */
+const PEM_CHARS = 16_384;
+
+/**
+ * The key part names a credential. A password word may be glued to its prefix (`PGPASSWORD`, `MYSQL_PWD`: a lazy
+ * bounded prefix from a word start, at most 48 tries per start, linear, security.md T4); any other word only after
+ * `_` or `-` (`DB_PASS`, `client-secret`), so `compass` and `bypass` are no key. No plural `tokens` (a count). A bare
+ * `pass` is one only before `=` or a quoted value (`pass=`, `pass: 'x'`): `--- PASS: TestName` is a test runner's status.
+ */
+const CREDENTIAL_KEY = String.raw`(?<![A-Za-z0-9])(?:[A-Za-z0-9_-]{0,48}?(?:passw(?:or)?ds?|passwd|pwd)|(?:[A-Za-z0-9]{1,32}[_-]){1,3}pass|pass(?=(?:\\{0,3}["'])?[ \t]{0,8}(?:=|:[ \t]{0,8}\\{0,3}["']))|(?:[A-Za-z0-9]{1,32}[_-]){0,3}(?:pw|passphrases?|passcodes?|secrets?|token|api[_-]?keys?|access[_-]?keys?|private[_-]?keys?|credentials?))`;
+/**
+ * A key that ends in a password word in full, whose value may be all digits (`ROOT_PASSWORD=48291375`); not `pass`
+ * or `pw` (`e2e_pass: 1234567`), nor a key about passwords (`password_min_length: 123456`).
+ */
+const NUMERIC_PASSWORD_KEY = /(?:passw(?:or)?d|passwd|pwd|passphrase|passcode)s?$/i;
+/** Whether `key` (a key part, its quotes and separator included or not) says password in full at its end. */
+export const isNumericPasswordKey = (key: string) => NUMERIC_PASSWORD_KEY.test(withoutTrailing(key, "\"' \t=:\\"));
+/** `key = value`, `key: value`, quoted as JSON or not, escaped quotes included; the value is group 2, up to a space or a quote, so symbols in it stay in it. */
+const KEYED_VALUE = new RegExp(String.raw`(${CREDENTIAL_KEY}(?:\\{0,3}["'])?[ \t]{0,8}[=:][ \t]{0,8}(?:\\{0,3}["'])?)([^\s"'\x60\\]{3,})`, "gi");
+const BEARER_VALUE = /(\b(?:bearer|basic)\s+)([^\s"'`,;{}[\]()<>]{8,})/gi;
+/** A URL login's prefix, `scheme://user:` (the user may be empty); its password runs to the word's last `@` (urlPasswordSpans). */
+const URL_LOGIN = /[a-z][a-z0-9+.-]{0,30}:\/\/[^\s/:@]{0,512}:/gi;
+const URL_PASSWORD_CHARS = 512;
+/** A credential named in prose: a random-looking value after it is its value. */
+const CREDENTIAL_WORD = /(?<![A-Za-z])(?:passw(?:or)?ds?|passwd|pwd|passphrases?|passcodes?|secrets?|tokens?|api[_ -]?keys?|keys?|bearer|credentials?|auth)(?![A-Za-z])/gi;
+
+/** A value the text points at rather than holds: a variable, a placeholder, a mask. */
+const REFERENCE = /^(?:\$|%|\*|\{|<|\.\.\.)/;
+const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
+/** As named-values.ts RANDOM_LOOKING: long hex with a digit and a letter, a long mixed-case run with digits, 13+ digits. */
+const RANDOM_LOOKING = [/^\d{13,}$/, /^(?=[0-9a-f]*\d)(?=[0-9a-f]*[a-f])[0-9a-f]{13,}$/i, /^(?=[A-Za-z0-9_+/-]*[a-z])(?=[A-Za-z0-9_+/-]*[A-Z])(?=[A-Za-z0-9_+/-]*\d)[A-Za-z0-9_+/-]{16,}={0,2}$/];
+const DIGEST_PREFIX = /sha(?:1|224|256|384|512):$/i;
+
+export const isRandomLooking = (token: string) => UUID.test(token) || RANDOM_LOOKING.some((shape) => shape.test(token));
+/** A token a credential could be: random-looking, or six characters or more with a digit. */
+export const isValueLike = (token: string) => isRandomLooking(token) || (token.length >= 6 && /\d/.test(token));
+/** A value a credential key holds: value-like, or opaque (twelve characters or more), never a path (`PWD=/home/app`); a word is a status ("token: missing"). A Bearer value must be value-like: "Bearer authentication" is prose. */
+const isHeldValue = (token: string) => (isValueLike(token) || token.length >= 12) && !/^(?:\/|~\/|\.\.?\/)/.test(token);
+/** A number, a count (`120/120`), a version, a time or a date: no credential, unless a password key holds it. */
+const NUMBERISH = /^(?:v?\d+(?:[.:_/-]\d+)*|\d{4}-\d{2}-\d{2}(?:[T ][\d:.]+Z?)?)$/i;
+export const isNumberish = (token: string) => NUMBERISH.test(token);
+/** A credential key's value (isHeldValue); a number only under a password key, so `tests pass: 120/120` stays. */
+const isKeyedHeld = (token: string, key: string) => {
+  const bare = withoutTrailing(token, ",.;:!?)]}");
+  return isHeldValue(token) && (!isNumberish(bare) || (isNumericPasswordKey(key) && /^\d{6,}$/.test(bare)));
+};
+/**
+ * A value prose names only by a nearby credential word: random-looking, hex only from LONG_HEX characters, since a
+ * plan or object id near "secret" is 16 (measured on real states: plan hashes, short ids).
+ */
+const LONG_HEX = 20;
+const isNamedByContext = (token: string) => isRandomLooking(token) && !(/^[0-9a-f]+$/i.test(token) && token.length < LONG_HEX);
+
+/** `token` without the trailing `marks`. A loop, not `/[.!?]+$/`: that regex retries from every mark of a long run (quadratic, security.md T4). */
+function withoutTrailing(token: string, marks: string): string {
+  let end = token.length;
+  while (end > 0 && marks.includes(token[end - 1])) end--;
+  return token.slice(0, end);
+}
+/** `token` without the sentence marks that end it. */
+const withoutClosingMarks = (token: string) => withoutTrailing(token, ".!?");
+
+interface Span { start: number; end: number }
+
+function shapeSpans(text: string): Span[] {
+  const spans: Span[] = [];
+  for (const shape of CREDENTIAL_SHAPES) for (const match of text.matchAll(shape)) spans.push({ start: match.index, end: match.index + match[0].length });
+  return [...spans, ...pemSpans(text)];
+}
+
+/** A private key from its BEGIN line through its END line, or PEM_CHARS on when it has none. One scan: each key starts after the last. */
+function pemSpans(text: string): Span[] {
+  const spans: Span[] = [];
+  let from = 0;
+  for (const begin of text.matchAll(PEM_BEGIN)) {
+    if (begin.index < from) continue;
+    // END is looked for inside the key's window only: the rest of the text, once per key, is quadratic (security.md T4).
+    const window = text.slice(begin.index, begin.index + PEM_CHARS);
+    const end = window.indexOf(PEM_END, begin[0].length);
+    const close = end < 0 ? -1 : window.indexOf("-----", end + PEM_END.length);
+    const stop = begin.index + (close < 0 ? window.length : close + 5);
+    spans.push({ start: begin.index, end: stop });
+    from = stop;
+  }
+  return spans;
+}
+
+/** The value group (2) of each match of `pattern`, unless it only points at a value or `holds` refuses it (given the key part, group 1). */
+function groupSpans(text: string, pattern: RegExp, holds: (value: string, key: string) => boolean = () => true): Span[] {
+  const spans: Span[] = [];
+  for (const match of text.matchAll(pattern)) {
+    if (REFERENCE.test(match[2]) || !holds(match[2], match[1])) continue;
+    const start = match.index + match[1].length;
+    spans.push({ start, end: start + match[2].length });
+  }
+  return spans;
+}
+
+/**
+ * Whether the credential word at [start, end) is part of a path or a file name (`/tmp/admin-bearer.txt`,
+ * `token.json`), which says where a credential lives, not that a value follows. Looks at most FILE_NAME_REACH back.
+ */
+const FILE_NAME_REACH = 256;
+function inFileName(text: string, start: number, end: number): boolean {
+  const before = text.slice(Math.max(0, start - FILE_NAME_REACH), start);
+  return /\/[^\s/]*$/.test(before) || /^[\w-]*\.[A-Za-z][A-Za-z0-9]{0,4}(?![\w-])/.test(text.slice(end, end + FILE_NAME_REACH));
+}
+
+/** Tokens named by context (isNamedByContext) that start within CONTEXT_REACH characters after a credential word, digests excepted. */
+function contextSpans(text: string, tokens: ReadonlyArray<{ token: string; start: number }>): Span[] {
+  const spans: Span[] = [];
+  let index = 0;
+  for (const word of text.matchAll(CREDENTIAL_WORD)) {
+    const after = word.index + word[0].length;
+    if (inFileName(text, word.index, after)) continue;
+    while (index < tokens.length && tokens[index].start < after) index++;
+    for (let next = index; next < tokens.length && tokens[next].start <= after + CONTEXT_REACH; next++) {
+      const { token, start } = tokens[next];
+      const bare = withoutClosingMarks(token);
+      if (isNamedByContext(bare) && !DIGEST_PREFIX.test(text.slice(Math.max(0, start - 7), start))) spans.push({ start, end: start + bare.length });
+    }
+  }
+  return spans;
+}
+
+/**
+ * The password of each `scheme://user:pass@host`: up to the last `@` of the word, so a password holding one is taken
+ * whole (`app:p@ss1234@db`). A port followed by a path is no login. Each login looks at most URL_PASSWORD_CHARS on.
+ */
+function urlPasswordSpans(text: string): Span[] {
+  const spans: Span[] = [];
+  for (const match of text.matchAll(URL_LOGIN)) {
+    const start = match.index + match[0].length;
+    const window = text.slice(start, start + URL_PASSWORD_CHARS + 1);
+    const space = window.search(/\s/);
+    const word = space < 0 ? window : window.slice(0, space);
+    const at = word.lastIndexOf("@");
+    if (at <= 0 || /^\d{1,5}\//.test(word) || REFERENCE.test(word)) continue;
+    spans.push({ start, end: start + at });
+  }
+  return spans;
+}
+
+const isWordChar = (char: string) => /[\p{L}\p{N}]/u.test(char);
+
+/** `word` without what wraps it (quotes, `**`, brackets, a closing mark), as offsets into `word`; empty when nothing is left. */
+export function credentialCore(word: string): { start: number; end: number } {
+  let start = 0;
+  let end = word.length;
+  while (start < end && !isWordChar(word[start])) start++;
+  while (end > start && !isWordChar(word[end - 1])) end--;
+  return { start, end };
+}
+
+/**
+ * Where a value may stand in a word, widest first: the word, after its first or last `=` or `:` (`DB_PASSWORD='x'`),
+ * after a short flag (`-px`), and each part between `/` and `@` (`app/x@db`), also after that part's first `:`
+ * (`app:x@db:5432`). Linear in the word.
+ */
+function placesIn(word: string): Array<[start: number, end: number]> {
+  const places: Array<[number, number]> = [[0, word.length]];
+  const first = word.search(/[=:]/);
+  const last = Math.max(word.lastIndexOf("="), word.lastIndexOf(":"));
+  if (first >= 0) places.push([first + 1, word.length]);
+  if (last > first) places.push([last + 1, word.length]);
+  const flag = /^-{1,2}[A-Za-z]/.exec(word);
+  if (flag) places.push([flag[0].length, word.length]);
+  // The next `:` at or after a part's start; the pointer only moves forward, so the scan stays linear.
+  let colon = -1;
+  const part = (from: number, to: number) => {
+    places.push([from, to]);
+    if (colon < from) {
+      const next = word.indexOf(":", from);
+      colon = next < 0 ? Infinity : next;
+    }
+    if (colon < to) places.push([colon + 1, to]);
+  };
+  let from = 0;
+  for (const separator of word.matchAll(/[/@]/g)) {
+    part(from, separator.index);
+    from = separator.index + 1;
+  }
+  if (from > 0) part(from, word.length);
+  return places;
+}
+
+/** Each span value in `text`, wrapping trimmed (credentialCore), at the widest place in its word that holds it. */
+function valueSpans(text: string, values: SpanCredentials): Span[] {
+  const spans: Span[] = [];
+  for (const match of text.matchAll(WORD)) {
+    const word = match[0];
+    for (const [from, to] of placesIn(word)) {
+      const core = credentialCore(word.slice(from, to));
+      if (core.end > core.start && values.has(word.slice(from + core.start, from + core.end))) {
+        spans.push({ start: match.index + from + core.start, end: match.index + from + core.end });
+        break;
+      }
+    }
+  }
+  return spans;
+}
+
+/** `text` with every span replaced, overlapping spans as one. */
+function replaceSpans(text: string, spans: Span[]): Scrubbed {
+  if (!spans.length) return { text, redacted: 0 };
+  spans.sort((a, b) => a.start - b.start);
+  let out = "";
+  let cursor = 0;
+  let redacted = 0;
+  for (const span of spans) {
+    if (span.end <= cursor) continue;
+    if (span.start >= cursor) {
+      out += text.slice(cursor, span.start) + REDACTED_VALUE;
+      redacted++;
+    }
+    cursor = span.end;
+  }
+  return { text: out + text.slice(cursor), redacted };
+}
+
+/** What the text itself names as a credential: a shape, a credential key's value, a Bearer value, a URL password, a value after a credential word. */
+function namedSpans(text: string): Span[] {
+  const tokens = [...text.matchAll(TOKEN)].map((match) => ({ token: match[0], start: match.index }));
+  return [
+    ...shapeSpans(text),
+    ...groupSpans(text, KEYED_VALUE, isKeyedHeld),
+    ...groupSpans(text, BEARER_VALUE, isValueLike),
+    ...urlPasswordSpans(text),
+    ...contextSpans(text, tokens),
+  ];
+}
+
+/** The values `text` names as credentials, wrapping trimmed: what scrubCredentials takes out of it on its own. */
+export function namedCredentialValues(text: string): string[] {
+  return namedSpans(text).map((span) => {
+    const value = text.slice(span.start, span.end);
+    const core = credentialCore(value);
+    return value.slice(core.start, core.end);
+  }).filter(Boolean);
+}
+
+/**
+ * `text` with its credential values replaced. `span` is the span's credentials; without them (text read back from a
+ * span record already written) only what the text itself names is found.
+ */
+export function scrubCredentials(text: string, span?: SpanCredentials): Scrubbed {
+  return replaceSpans(text, [...namedSpans(text), ...(span?.size ? valueSpans(text, span) : [])]);
+}
+
+/** What a pass wrote, scrubbed: the events whose text the summarizer chose, and the span's narrative. */
+export function scrubModelWritten(events: readonly StateEvent[], narrative: string | undefined, span: SpanCredentials): { events: StateEvent[]; narrative?: string; redacted: number } {
+  let redacted = 0;
+  const clean = (text: string) => {
+    const scrubbed = scrubCredentials(text, span);
+    redacted += scrubbed.redacted;
+    return scrubbed.text;
+  };
+  const scrubbedEvents = events.map((event): StateEvent => {
+    switch (event.kind) {
+      case "decision.add": case "deadEnd.add": return { ...event, text: clean(event.text) };
+      case "now.set": return { ...event, now: { inProgress: event.now.inProgress.map(clean), next: event.now.next.map(clean), blocked: event.now.blocked.map(clean), resume: clean(event.now.resume) } };
+      default: return event;
+    }
+  });
+  return { events: scrubbedEvents, ...(narrative === undefined ? {} : { narrative: clean(narrative) }), redacted };
+}

package/src/learned-secrets.ts

@@ -27,7 +27,7 @@ const KEY_CHARS = 64;
 const BLOCK_OPENER = /^(?:[|>](?:[-+]?\d?|\d[-+])|[{[])?\s*(?:#.*)?$/;
 
 /** A line read as `key = value`, `key: value` or `key    value` (aligned table), or undefined when it is not one. */
-function keyedLine(line: string): { key: string; value: string } | undefined {
+export function keyedLine(line: string): { key: string; value: string } | undefined {
   const text = line.trim();
   const separator = text.slice(0, KEY_CHARS + 1).search(/[=:]|\s{2,}|\t/);
   if (separator <= 0) return undefined;

package/src/ledger/handoff-commands.ts

@@ -74,6 +74,8 @@ const SECRET_PATTERNS: RegExp[] = [
   /\bAGE-SECRET-KEY-1[0-9A-Z]{20,}/,
   /\bAKIA[0-9A-Z]{16}\b/,
   /\b(?:gh[pousr]_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,}|sk-[A-Za-z0-9_-]{20,}|xox[abprs]-[A-Za-z0-9-]{10,})/,
+  // As credential-scrub.ts CREDENTIAL_SHAPES: GitLab, Google API, Stripe, npm and Hugging Face tokens.
+  /\b(?:glpat-[A-Za-z0-9_-]{20,}|AIza[0-9A-Za-z_-]{35}|[sr]k_(?:live|test)_[A-Za-z0-9]{16,}|npm_[A-Za-z0-9]{36}|hf_[A-Za-z]{30,})/,
 ];
 
 /** Whether `text` (a command, or a tool result's failure signature) carries a credential shape. */

package/src/ledger/handoff-errors.ts

@@ -7,6 +7,7 @@
 import type { ErrorRecord } from "./handoff.ts";
 import { containsSecret, normalizeCommand, redactCommand } from "./handoff-commands.ts";
 import type { ObservedError, ObservedRun } from "./handoff-observe.ts";
+import { scrubCredentials } from "../credential-scrub.ts";
 
 const VOLATILE_RES: RegExp[] = [
   /\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:?\d{2})?/g,
@@ -24,6 +25,14 @@ export function errorKey(signature: string): string {
   return line.replace(/\s+/g, " ").trim();
 }
 
+/**
+ * The signature a failure is recorded under: none when it has a command-style credential shape (dropped, as ever),
+ * else with each value the credential scrub finds redacted (`password (64-hex): <redacted>`). Redacted, not dropped:
+ * an ordinary failure ("secret db-creds version <uuid> not found") must stay open. The scrub is deterministic, so the
+ * same failure keys the same way pass after pass.
+ */
+const recordedSignature = (signature: string) => containsSecret(signature) ? undefined : scrubCredentials(signature).text;
+
 function idNumber(id: string): number {
   const match = /^E(\d+)$/.exec(id);
   return match ? Number(match[1]) : 0;
@@ -40,23 +49,28 @@ function succeededAfter(runs: ObservedRun[], command: string, after: number): bo
  * unresolved again and last seen now; a new one gets the next id. Then the mechanical rule: a
  * record whose source command exits 0 later in this pass is resolved. Records are copied. A
  * signature that carries a credential shape (a failing push or curl echoes its URL or header) is
- * never recorded, carried or new, because the exact half outlives the messages it came from; a
+ * never recorded, carried or new, because the exact half outlives the messages it came from; any
+ * other is recorded with the values the credential scrub finds redacted (recordedSignature); a
  * source command with one is stored as `<redacted>`, like its command record. Ids are never reused.
  */
 export function mergeErrors(previous: ErrorRecord[], observed: ObservedError[], runs: ObservedRun[], pass: number): ErrorRecord[] {
   let nextId = previous.reduce((max, record) => Math.max(max, idNumber(record.id)), 0) + 1;
-  const records = previous.filter((record) => !containsSecret(record.signature)).map((record) => ({ ...record, sourceCommand: redactCommand(record.sourceCommand) }));
+  const records = previous.flatMap((record) => {
+    const signature = recordedSignature(record.signature);
+    return signature === undefined ? [] : [{ ...record, signature, sourceCommand: redactCommand(record.sourceCommand) }];
+  });
   const lastSeenAt = new Map<string, number>();
   for (const error of observed) {
-    const key = errorKey(error.signature);
-    if (key === "" || containsSecret(error.signature)) continue;
+    const signature = recordedSignature(error.signature);
+    const key = signature === undefined ? "" : errorKey(signature);
+    if (signature === undefined || key === "") continue;
     let record = records.find((candidate) => errorKey(candidate.signature) === key);
     if (record) {
       record.resolved = false;
       record.lastSeenPass = pass;
       if (record.sourceCommand === undefined && error.sourceCommand !== undefined) record.sourceCommand = redactCommand(error.sourceCommand);
     } else {
-      record = { id: `E${nextId++}`, signature: error.signature, sourceCommand: redactCommand(error.sourceCommand), resolved: false, lastSeenPass: pass };
+      record = { id: `E${nextId++}`, signature, sourceCommand: redactCommand(error.sourceCommand), resolved: false, lastSeenPass: pass };
       records.push(record);
     }
     lastSeenAt.set(record.id, error.runIndex);