@raibitsi/tiresias 1.0.0 → 1.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,154 @@
1
+ # Changelog
2
+
3
+ Built from docs/SPEC.md, section 10, which has the full detail of every version.
4
+
5
+ ## v1.0.1 (2026-11-01)
6
+
7
+ - Cancelling a read while it waits its turn no longer adds extra delays to later requests.
8
+ - An unexpected robots.txt “not modified” response is retried once; browsing stops if it answers “not modified” again without providing rules.
9
+ - Malformed Retry-After headers keep the normal fallback cooldown.
10
+ - Relative HTML links follow the page's first base URL; receipts still name the page that was read.
11
+ - Rendered text with many lines stays readable when its final lines are visible. Incomplete geometry is reported as a partial inspection.
12
+ - Emoji and other Unicode characters remain intact when text is split into blocks.
13
+ - Large plain-format reads report how much text was omitted.
14
+ - Cached pages honor the current rendering setting and each read's rendering request.
15
+ - Markdown alternatives keep the original URL's query parameters.
16
+ - Office text is withheld with its receipts when the reply cannot fit both.
17
+ - Known archive signatures keep their archive warning even when the filename claims to be an Office document or application.
18
+ - Concurrent page saves cannot exceed the snapshot cap or push out earlier evidence.
19
+ - Replacing site restrictions no longer breaks unrelated browsing while confirmed restrictions are retained.
20
+ - Active calls keep their starting settings, including their scanner and translator hooks.
21
+ - Listing settings in private mode creates nothing and leaves folder permissions alone.
22
+ - Resuming a handoff works when an unrelated storage limit is invalid.
23
+ - History keeps questions up to 1,000 characters, marks longer cuts, and keeps questions that differ after character 500 separate in the needs report.
24
+
25
+ ## v1.0.0 (2026-11-01)
26
+
27
+ The first public release. Tiresias is a browser for AI agents with no screen: your agent calls it over MCP (or you run it from the command line), it reads a page into a short outline with numbered links, answers questions with exact quotes and receipts, and says honestly who is calling. It obeys robots.txt and every request to wait, stops at a CAPTCHA, a bot check or a login and hands the page to you, and runs on your machine under your keys, with no account and no telemetry. macOS and Linux; Windows is not supported yet.
28
+
29
+ What the versions before it built, in one place:
30
+
31
+ - **Reading (v0.1, v0.5.1).** Outline first, then one section or the best quoted passages, in pages that fit a context window; markdown asked for first, `llms.txt` and `llms-full.txt` used, MCP servers, API specs and feeds reported. Every quote has a receipt. Page content always sits inside `untrusted_content`; hidden text, invisible characters and instructions aimed at an AI are flagged.
32
+ - **Honest by default (v0.1, v0.2).** A plain user agent, robots.txt, Crawl-delay and Retry-After on every request, one second between requests to a site, and optional Web Bot Auth signing of every request with your own key. Raibit's own copy is verified by Cloudflare (2026-11-01).
33
+ - **Memory (v0.3).** What changed since the last visit, a cache shared by every agent on the machine, bookmarks and a watchlist, a needs report, a hand-off ticket to resume a read after a person clears a wall, and a phishing check before every fetch (optionally Google Web Risk with your own key).
34
+ - **Permission (v0.4).** Profiles, and forms sent only with a permission slip you grant at a terminal; login, password, CAPTCHA and payment forms are never sent.
35
+ - **Files (v0.5, v0.6).** A file is named by its first bytes and comes back as a warning with its fingerprint, never run; zip listings, Office text and calendar invites are read in memory; your own virus checker can be plugged in. Tabs, a downloads shelf, per-site settings and a settings page.
36
+ - **JavaScript pages and languages (v0.7).** Rendering in the Google Chrome you already have, off until you allow it, with every request the page makes fetched under Tiresias's rules; walls and hidden text judged by what the browser shows; the language of every read; your own translator.
37
+ - **Proof (v0.8).** Signed snapshots of exactly what a read received, and free Bitcoin timestamps (OpenTimestamps) that prove a receipt existed then and was not edited.
38
+ - **Hardening (v0.9, v0.9.1).** Risky settings take an all-caps confirmation phrase and are logged; a security audit, a fuzz script, a memory ceiling for the MCP server and an optional Docker box; then every deferred finding from the reviews closed, so v1.0 ships with no known bug.
39
+
40
+ In this release: the docs were rewritten for a first-time reader (README, QUICKSTART, SECURITY.md, and a new docs/DEMO.md with real output), the install is `npm install -g @raibitsi/tiresias`, two failures seen only on Linux were fixed (the anchor's receipt cursor after a file is written again under the same inode; a test whose waits ended during a slow run), and two refusal texts no longer say "v0.x". The key directory signing guidance now covers live signing, which Cloudflare asks for.
41
+
42
+ **Upgrading from v0.8:** if you had turned on saving, rendering or anchoring, each is held off after the upgrade until you confirm it once, with whichever of these apply:
43
+
44
+ ```
45
+ tiresias settings confirm saving --confirm "YES, TURN ON SAVING"
46
+ tiresias settings confirm rendering --confirm "YES, TURN ON RENDERING"
47
+ tiresias settings confirm anchoring --confirm "YES, TURN ON ANCHORING"
48
+ ```
49
+
50
+ (`rendering` at `auto` takes `"YES, TURN ON AUTOMATIC RENDERING"`.) The first reply after the upgrade lists what is waiting, `tiresias mcp` prints it on standard error at start, and `tiresias settings` shows it. Upgrading from v0.9.x needs nothing.
51
+
52
+ ## v0.9.1 (2026-10-31)
53
+
54
+ The pre-v1.0 sweep. Every MEDIUM and LOW finding deferred from v0.7.0, v0.8.0 and v0.9.0 (67 in all) was closed: 64 fixed, each with a test that fails without the fix, and 3 designed away (the behavior decided and written down). Two of them were decided by Travis: on a rendered page, text with `pointer-events: none` that cannot be measured for clipping now makes the inspection partial so the HTML checks also run; and any background image or gradient counts as covering the text under it (it can flag text under a see-through decorative layer, but never shows hidden text). Google Web Risk lookups stay exempt from site spacing (they are calls to Google, not site visits). Codex wrote the fixes and Claude reviewed each one; then Codex reviewed the whole sweep until a round came back clean. The full list is in SPEC.md, section 10, v0.9.1.
55
+
56
+ What you may notice:
57
+
58
+ - **`tiresias mcp` now tells you on standard error when a setting is held back.** If config.json turns on a risky setting that has not been confirmed (the usual case after upgrading from v0.8 with saving, rendering or anchoring on), the server prints one line at start with the exact `tiresias settings confirm ... --confirm '...'` command for each. Your AI app usually keeps that in its MCP server log.
59
+ - **Rendered pages:** walls and hidden text are judged more exactly (text in `display: contents` wrappers, dark-scheme pages, `color()` and `oklch()` colours, `xmp` and `textarea` text, page elements named like browser built-ins, CAPTCHA and password boxes past the first 2,000 candidates). A failed render keeps the last good rendered copy; an explicit `render` renders again unless a rendered copy is still fresh by the site's own caching rules (`fresh: true` always renders again); cached rendered replies keep their walls and hidden-text codes; a redirected script or style is handed to Chrome as a redirect; a large fixed layer above the text gets its own overlay note.
60
+ - **Save page and receipts:** a snapshot is never written that `saved verify` would refuse; quotes containing NUL and pages with odd characters verify; Office receipts name their snapshot; `clear` counts what it really removed. A failed `saved verify` or `receipts verify` now prints `status: "error"` with `reason: "verification_failed"` on standard error (exit codes unchanged).
61
+ - **Anchoring:** a bad answer from one calendar can no longer spoil a batch another calendar served well; calendars' Retry-After is honored; very large stores keep every batch searchable and resume long scans; `receipts prove --out` cleans up after itself on failure.
62
+ - **Confirmations:** every reply path carries the settings notice; a 24-hour notice is never cut short; always-allowed ports and the default spacing no longer ask for a phrase; `scanner off` and `translator off` work while `sites` is broken; suggested commands put `--confirm` in the right place.
63
+ - **Reading:** markdown code keeps its indentation; very deep pages and lists stay within the time budget; an unsolicited "not modified" answer is never served from an older parse; a JavaScript-only page read without rendering says so; `resume` keeps `render` and `translate`; a cancelled read writes nothing; `history --text` and `receipts --text` start with a count line. The parser version is 14, so a page cached by the older reader is parsed again once.
64
+ - **pdf.js's native canvas add-on** that an `npm install -g` puts on disk stays unbundled on purpose (it is never loaded; bundling would tie outside security fixes to Tiresias releases). README says so.
65
+
66
+ ## v0.9.0 (2026-10-31)
67
+
68
+ Risky settings need confirmation (W-32). Turning a risky setting on or looser (`saving`, `rendering`, `translator`, `scanner` with `--send-file`, `anchoring`, `anchor_calendars`, `safe_browsing` off, and per site more `ports`, lower `spacing_seconds`, `never_visit` or `hand_off` off) takes an exact all-caps phrase naming it, passed as `--confirm` (for example `--confirm "YES, TURN ON SAVING"`); a wrong or missing phrase changes nothing and prints the phrase. Turning things off or stricter never asks. Each change is recorded in history and the daily log with who made it (a terminal or not, best effort), and CLI and MCP replies carry `settings_notice` for 24 hours. A hand edit of `config.json` that loosens is not in force until confirmed (`tiresias settings confirm`); the last confirmed state lives in `confirmations.json`, and no record means the safe defaults (an owner upgrading from v0.8 with a risky setting on confirms it once). `TIRESIAS_RENDERING`, `TIRESIAS_SAFE_BROWSING=0` and `TIRESIAS_ALLOW_PORTS` loosen only with their phrase in `TIRESIAS_CONFIRM`. `tiresias settings confirmations off|on` (each with its own phrase) stops or resumes the asking; `tiresias settings` shows the state. MCP still cannot read or change any setting. It stops accidents and casual page-text tricks, not a fully fooled agent; the log makes that visible. Full record: `docs/v0.9/confirm.md`.
69
+
70
+ **Upgrading from v0.8:** if you had turned on saving, rendering or anchoring, each is held off after the upgrade until you confirm it once. Run whichever apply:
71
+
72
+ ```
73
+ tiresias settings confirm saving --confirm "YES, TURN ON SAVING"
74
+ tiresias settings confirm rendering --confirm "YES, TURN ON RENDERING"
75
+ tiresias settings confirm anchoring --confirm "YES, TURN ON ANCHORING"
76
+ ```
77
+
78
+ (If you had `rendering` at `auto`, its phrase is `"YES, TURN ON AUTOMATIC RENDERING"`.) Until then the first reply after the upgrade, CLI or MCP, lists each held-off setting with its exact command, and `tiresias settings` shows what is waiting. If you use Tiresias only through an AI app, run `tiresias settings` once at a terminal after upgrading, since the notice reaches you only if the agent passes it on.
79
+
80
+ Hardening audit of the whole codebase as it will be published. A `SECURITY.md` threat model; SPDX license headers; `npm audit` fixes under the MCP SDK; an opt-in fuzz script (`npm run fuzz`). Fixed, each with a test: a `Content-Encoding` Tiresias does not decode now fails the request (a compressed robots.txt used to read as "no rules"); a robots.txt that is not text, any 2xx answer, a 206, and rules that write `*` or `$` as `%2A` / `%24` are all honored (unreadable means the site says no); the daily log is bounded; CLI `receipts` and `history` put their records inside `untrusted_content`, as MCP does; one MCP server runs at most 8 tool calls, 4 parser workers and 2 renders at once, with fixed-text refusals past their queues; the MCP message boundary caps arguments (1,000 elements, 8 MB) and open requests (64), pauses reading while output backs up, and never repeats a caller's text outside the envelope; bad dates get fixed texts; the in-process cooldown list is bounded (at most 10,000 waits by name, the rest in a fixed 8 MiB keyed store that never forgets or shortens a wait). README and SECURITY.md wording that overclaimed was corrected. Full record: `docs/v0.9/hardening.md`.
81
+
82
+ The MCP client is trusted, the server has a memory ceiling, and Docker is optional (W-33). The AI app that starts `tiresias mcp` is now trusted not to flood its own server (SECURITY.md says so; every limit stays). `tiresias mcp` runs the server under its own memory ceiling, `TIRESIAS_MEMORY_MB` (default 1024 MB, `off` to skip): at the ceiling it stops with a clear line on standard error and a "Server stopped" line in the daily log, and the app can start it again. A `Dockerfile` (official Node 22 slim image pinned by digest, non-root, rendering off) and `docs/docker.md` give advanced owners a hard box with memory, CPU and process limits and a read-only file system; same commands, same store. v1.0 will support macOS and Linux; Windows is not yet supported. Full record: `docs/v0.9/client-trust.md`.
83
+
84
+ A site's echo of a secret is withheld, not masked. When the answer to a sent form repeats a hidden value or a cookie the site set for that call (6 characters or more), Tiresias now withholds the whole answer (`form.reason: "secret_withheld"`, the HTTP status and size only) instead of masking it as `[hidden value]`. The answer is checked once as received, with HTML, percent and JSON escapes decoded and case, quotes and spaces ignored, and again before anything from it is recorded. `counts.receipts_withheld` is gone. Best effort, stated in SECURITY.md: an echo in an encoding Tiresias does not decode is not caught.
85
+
86
+ Private launch rehearsal: a newcomer's first hour, end to end (an npm tarball installed into a temporary prefix, a fresh home, the README followed literally, MCP through Claude Code and a small client, rendering, saving, anchoring against a local calendar, slips and forms, the upgrade from a v0.8 store, and the unhappy paths). Fixed what tripped them up: a label Tiresias makes, `(top of page)` and `(structured data)`, is no longer quoted in a receipt as if the page said it; `receipts --url https://example.com` finds what was recorded as `https://example.com/`; an older Node, a mistyped option, an address with no `https://`, a store folder that cannot be written and being offline each get a plain message saying what to do (the store one used to say "internal error"); `--text` prints a page read's receipt count instead of every receipt as JSON, and says so when a listing is empty; hints that name a risky setting now carry its phrase; `--help` starts with the first command to try. The README's Claude Code line uses `--scope user`, and states that an npm install puts pdf.js's optional native add-on on disk, where the PDF child's permission model keeps it from loading. New `docs/QUICKSTART.md` (shipped in the package). The recurring render flake under full-suite load (the first render of a test file starting Chrome cold under a shortened limit) is fixed in the tests. The parser version is 13, so a page cached by the older reader is parsed again. Full record: `docs/v0.9/rehearsal.md`.
87
+
88
+ ## v0.8.0 (2026-10-31)
89
+
90
+ Save page (W-28) and tamper-proof receipts (W-08). With the owner's new `saving` setting at `ask` (default `off`; owner only, never through MCP or an environment variable), `read --save` and the MCP read's `save` keep a snapshot in the profile's `saved/` folder: the bytes the read received (after transfer framing and Content-Encoding, before charset decoding) and, for a rendered read, the rendered HTML, with a `manifest.json` signed by the profile's slip key. The snapshot's id is the SHA-256 of the manifest's facts; every receipt of that read carries it as `snapshot`. `tiresias saved` lists, shows, verifies (re-hash, signature, and each receipt's quote found in the saved bytes by the same reader steps) and deletes snapshots. A save always fetches; private mode, `no-store` pages, walls and errors save nothing; a file warning saves only its fingerprint. Caps: `max_saved_mb` (100) and 1,000 snapshots, a save over them is refused, never evicting; `keep_saved_days` (90) through `clear` and the daily trim. Full record: `docs/v0.8/save.md`. And with the owner's new `anchoring` setting `on` (default `off`; owner only, never through MCP or an environment variable), `tiresias anchor` timestamps the fingerprints of new receipt records and saved snapshots through OpenTimestamps (free, Bitcoin-backed, no wallet or account): one Merkle tree per batch over salted leaves in a random order, and only its root sent to the calendars (`anchor_calendars`, default the four public ones), through Tiresias's own fetcher. `anchor status`, `anchor upgrade`; `tiresias receipts` shows each record's `record_fingerprint` (a file warning keeps its own `fingerprint`, the file's SHA-256); `receipts prove <id> [--out]` exports the record, salt, Merkle path and a standard `.ots` file; `receipts verify <bundle>` checks it offline and names the step that needs a Bitcoin block header. The OpenTimestamps format is implemented in `src/ots.ts` (no new package). Private mode is never anchored; each profile anchors separately; anchoring is never on the read path. Full record: `docs/v0.8/anchor.md`. At integration the receipts listing's new field was named `record_fingerprint`, so a file-warning record keeps its own `fingerprint`. Integration review also closed five private-mode gaps, three of them older than v0.8: a private session (`--private` or `TIRESIAS_PRIVATE=1`) now refuses `receipts prove --out`, `clear` (except `--dry-run`), `log --to`, `keys init` and `keys sign-directory --out`, and `tiresias mcp --private` now serves a private session (the flag used to be ignored). 16 MEDIUM and LOW findings are deferred to the compute-day sweep (docs/SPEC.md section 10).
91
+
92
+ ## v0.7.0 (2026-10-31)
93
+
94
+ JavaScript-only pages (W-10), translate (W-26), and walls and hidden text revisited with rendering (N-02, N-05). Optional rendering in the Google Chrome already installed, driven over a pipe with no new package, off by default and turned on only by the owner (`rendering` off|ask|auto); every request the page makes goes through Tiresias's own rules, under caps on time, requests and bytes. On a rendered page the browser decides walls and hidden text. Every read reports its language (13 languages, no dependency), the shield matches phrases aimed at an AI in all 13, and an optional owner-set translation hook (`tiresias translator`) answers `read` with `translate`. Every request is now counted and capped by the bytes its connection receives, for plain reads too. Built as three tracks, reviewed by Codex, then together; deferred MEDIUM and LOW findings are listed in SPEC.md, section 10, v0.7.0.
95
+
96
+ ## v0.6.1 (2026-10-30)
97
+
98
+ The compute-day sweep: the 10 MEDIUM and LOW findings deferred from v0.6.0, each fixed with a test (written by Codex, reviewed by Claude). Office parts with repeated or out-of-order containers are refused; long Office lines page in bounded blocks and the whole reply stays in its limit; receipts from a file a person saved keep their source; resume honors section and question for a saved Office file; Office answers to a sent form withhold receipts touched by hidden values before anything is logged; tabs honor current site settings for cached copies; the downloads shelf keeps the macros flag; `tiresias settings` lists `TIRESIAS_MAX_INSPECT_BYTES`. See SPEC.md, "Deferred to compute-day sweep".
99
+
100
+ ## v0.6.0 (2026-10-26)
101
+
102
+ Inside files, plus browser basics. Zip listing without unpacking, Office text without macros (with receipts, section paging and question search), and calendar events from `.ics` files; tabs and back/forward (W-24), the downloads shelf (W-27), site settings per host (W-25), the `tiresias settings` command (W-29) and a fast first read with the door checks in the background (W-30). Closes W-24, W-25, W-27, W-29, W-30 and the "Not in v0.5" items of section 4.4.
103
+
104
+ ## v0.5.0 (2026-10-20)
105
+
106
+ Released: dangerous files (W-20) and the owner's own scanner (W-21), built to the coordinator's scope in section 4.4, after Codex round 68, the final, severity-rated round, found nothing HIGH (`reviews/2026-10-20-codex-round-68.md`: 0 high, 1 medium, 1 low). The entries below are the work log kept while v0.5 was built.
107
+
108
+ ## v0.4.0 (2026-10-18)
109
+
110
+ Released: profiles (W-18), permission slips (W-05) and permission-gated forms (W-04), after Codex round 57 found nothing blocking. The entries below are the work log kept while v0.4 was built.
111
+
112
+ ## v0.3.0 (2026-10-17)
113
+
114
+ Released: needs report (W-12), diffs (W-02), bookmarks and watchlist (W-16), handoff and resume (W-06), shared cache (W-03), Safe Browsing (W-19), after Codex round 45 found nothing blocking. The entries below are the work log kept while v0.3 was built.
115
+
116
+ ## v0.2.0 (2026-10-17)
117
+
118
+ Released: signing (N-07), the daily log (W-13), private mode (W-14), storage limits (W-15) and PDFs (W-17), merged to main after Codex round 35 found nothing. **W-09:** submitted to Cloudflare's Bots and Agents Directory (BotBase) on 2026-10-17 as Tiresias, operator Raibit, Direct, category Agent, content use immediate, Web Bot Auth with the raibit.ai key directory. Status: waiting for review. The entries below are the work log kept while v0.2 was built.
119
+
120
+ ## v0.1.8 (2026-10-16)
121
+
122
+ Software release. Closes the six non-blocking items from Codex review round 8 (`reviews/2026-10-16-codex-round-8.md`: no blocking findings, ready for a private push).
123
+
124
+ ## v0.1.7 (2026-10-16)
125
+
126
+ Software release. Fixes Codex review round 7 (`reviews/2026-10-16-codex-round-7.md`: three blocking, two non-blocking).
127
+
128
+ ## v0.1.6 (2026-10-16)
129
+
130
+ Software release. Fixes Codex review round 6 (`reviews/2026-10-16-codex-round-6.md`: round 5 verified fixed; four blocking items from a whole-repo pass, one small regression).
131
+
132
+ ## v0.1.5 (2026-10-16)
133
+
134
+ Software release. Fixes Codex review round 5 (`reviews/2026-10-16-codex-round-5.md`: three blocking items in rotation, two small).
135
+
136
+ ## v0.1.4 (2026-10-16)
137
+
138
+ Software release. Fixes Codex review round 4 (`reviews/2026-10-16-codex-round-4.md`: one blocking item, B1, plus small ones). Numbers: A = round-3 item status, B-1 and B-2 = new in round 4.
139
+
140
+ ## v0.1.3 (2026-10-16)
141
+
142
+ Software release. Fixes Codex review round 3 (`reviews/2026-10-16-codex-round-3.md`: no High; B1 to B5 new, plus residual partials).
143
+
144
+ ## v0.1.2 (2026-10-16)
145
+
146
+ Software release. Fixes Codex review round 2 (`reviews/2026-10-16-codex-round-2.md`: 13 fixed, 13 partial, 13 new). Numbers below: P = round-1 finding that was partial, N = new finding in round 2.
147
+
148
+ ## v0.1.1 (2026-10-16)
149
+
150
+ Software release. Spec stays at 0.5 apart from section 6.1. Fixes Codex review round 1 (`reviews/2026-10-16-codex-round-1.md`, 26 findings, verdict "not ready").
151
+
152
+ ## v0.1.0 (2026-10-16)
153
+
154
+ Software release. Spec stays at 0.5.
package/README.md CHANGED
@@ -329,7 +329,7 @@ tiresias slip revoke <id> # take one back; tiresias slip revoke --al
329
329
 
330
330
  - Only you grant, and only from the command line, as a person at a terminal: `slip grant` and `slip revoke` ask you to type the slip's site back (`revoke all` for `--all`) and refuse when their input is not a terminal, so an agent running the command from its shell cannot grant itself a slip. No MCP tool or parameter can make, extend or change a slip. Agents get a read-only `slips` tool that lists the active ones. An agent that can run commands on your machine is otherwise trusted like you (it could, for example, build its own terminal); the gate is against an agent granting itself permission in passing, not against a program determined to impersonate you.
331
331
  - Each use is a signed file, counted up to the highest use number, so deleting a use file gives nothing back. Every revoked slip goes on `slips/revoked.jsonl` until it would have expired, so a copy of a revoked slip put back is never used.
332
- - A slip allows one action (`submit_form` is the only one) on one exact site (`https://host` or `https://host:port`; plain `http` only for localhost, for tests), and optionally only under one path, matched on whole path segments (`/contact` covers `/contact` and `/contact/send`, never `/contactus`). `pay` is refused: this version of Tiresias never pays (paying, with guardrails, is planned for v1.1).
332
+ - A slip allows one action (`submit_form` is the only one) on one exact site (`https://host` or `https://host:port`; plain `http` only for localhost, for tests), and optionally only under one path, matched on whole path segments (`/contact` covers `/contact` and `/contact/send`, never `/contactus`). `pay` is refused: this version of Tiresias never pays (paying, with guardrails, is planned for v1.3).
333
333
  - It lasts 24 hours by default (`--for 12h`, `--for 7d`, or `--until 2026-10-24`, which means through the end of that day), at most 30 days, and allows 1 use by default (`--uses`, at most 100). At most 200 slips per profile.
334
334
  - Each slip is signed with a separate key (`keys/slips.private.jwk`, made on your first grant or first saved snapshot, never printed, never used to sign web requests; it signs slips and snapshot manifests only). A slip file that was edited by hand, damaged, renamed, or copied from another profile does not verify and is never used; `slip list` counts it as damaged. Signing does not protect against a program running as you, which could read the key, the same as everything else in the store.
335
335
  - Slips belong to one profile (`--profile`). `clear` and the daily trim remove expired and used-up slips; `clear --all` never removes an active one, and slips do not count toward the disk cap. Private mode refuses `slip grant` and `slip revoke`, and a private session never uses a slip.
@@ -15,6 +15,7 @@
15
15
  import { callContext } from "./context.js";
16
16
  import { probeOrigin } from "./doors.js";
17
17
  import * as safebrowsing from "./safebrowsing.js";
18
+ import { snapshotConfig } from "./config.js";
18
19
  import { withSignal } from "./safety.js";
19
20
  import { canonOrigin } from "./sitekey.js";
20
21
  import { MAX_SPACING_S, loadSites } from "./sites.js";
@@ -43,8 +44,10 @@ export function startDoorProbe(origin) {
43
44
  if (jobs.size >= MAX_JOBS)
44
45
  return false;
45
46
  let gate;
47
+ let config;
46
48
  try {
47
- gate = safebrowsing.makeGate({ allowFlagged: false, priv: false });
49
+ config = snapshotConfig();
50
+ gate = callContext.run({ config }, () => safebrowsing.makeGate({ allowFlagged: false, priv: false }));
48
51
  }
49
52
  catch {
50
53
  return false;
@@ -52,10 +55,10 @@ export function startDoorProbe(origin) {
52
55
  const ctl = new AbortController();
53
56
  const signal = AbortSignal.any([AbortSignal.timeout(JOB_MS), ctl.signal, shutdownSignal]);
54
57
  const sent = { requests: 0, signed: 0 };
55
- // Outside the asking call's context (callContext.exit): not its deadline, its cancellation, its
58
+ // Outside the asking call's context: not its deadline, its cancellation, its
56
59
  // per-call robots cache or its private flag; withSignal then gives the job a context of its own.
57
60
  const done = callContext
58
- .exit(() => withSignal(signal, () => {
61
+ .run({ config }, () => withSignal(signal, () => {
59
62
  // A job is its own call: it reads the site settings once, when it starts (sites.ts).
60
63
  const store = callContext.getStore();
61
64
  if (store)
package/dist/config.js CHANGED
@@ -6,7 +6,8 @@ import path from "node:path";
6
6
  import { releaseLock, stillOwned, takeLock } from "./ownedlock.js";
7
7
  import { NOFOLLOW, NONBLOCK, home, ownFile, randomTmp, readOwnedText } from "./store.js";
8
8
  // W-32 (v0.9): readers of a risky setting use the value in force (an unconfirmed loosening is held back).
9
- import { inForceConfig } from "./confirm.js";
9
+ import { clamp, inForceConfig, loadRecord } from "./confirm.js";
10
+ import { callContext } from "./context.js";
10
11
  export const DEFAULT_LIMITS = { keep_history_days: 90, keep_receipts_days: 90, keep_cache_days: 30, max_disk_mb: 200, keep_handoff_days: 7, keep_saved_days: 90, max_saved_mb: 100 };
11
12
  export { ConfigError } from "./errors.js";
12
13
  import { ConfigError } from "./errors.js";
@@ -64,7 +65,7 @@ export function loadTranslator() {
64
65
  return parseTranslator(o.translator);
65
66
  }
66
67
  // The whole config.json as an object (null when there is none). A bad file is a ConfigError.
67
- export function readConfigObject() {
68
+ function readConfigFile() {
68
69
  const raw = readOwnedText("config.json");
69
70
  if (raw === null)
70
71
  return null;
@@ -79,6 +80,15 @@ export function readConfigObject() {
79
80
  throw new ConfigError("config.json must be a JSON object.");
80
81
  return obj;
81
82
  }
83
+ export function readConfigObject() {
84
+ const snapshot = callContext.getStore()?.config;
85
+ return snapshot ? snapshot.raw : readConfigFile();
86
+ }
87
+ export function snapshotConfig() {
88
+ const raw = readConfigFile();
89
+ const record = callContext.exit(() => loadRecord());
90
+ return { raw, effective: clamp(raw, record.rec).obj, record };
91
+ }
82
92
  // W-21: the scanner hook of this profile, or null when none is set. A bad setting is a ConfigError.
83
93
  export function loadScanner() {
84
94
  const o = inForceConfig();
@@ -141,36 +151,17 @@ export function parseAgentStyle(v) {
141
151
  // stops a read. Returns null when signing is not configured. A config file that exists
142
152
  // but cannot be read at all is an error (the caller decides whether that matters).
143
153
  export function loadSigningConfig() {
144
- const raw = readOwnedText("config.json");
145
- if (raw === null)
154
+ const o = readConfigObject();
155
+ if (o === null)
146
156
  return null;
147
- let obj;
148
- try {
149
- obj = JSON.parse(raw);
150
- }
151
- catch {
152
- throw new ConfigError("config.json in the Tiresias folder is not valid JSON.");
153
- }
154
- if (typeof obj !== "object" || obj === null || Array.isArray(obj))
155
- throw new ConfigError("config.json must be a JSON object.");
156
- const o = obj;
157
157
  if (!Object.hasOwn(o, "signature_agent"))
158
158
  return null;
159
159
  return { signature_agent: parseSignatureAgent(o.signature_agent), style: Object.hasOwn(o, "signature_agent_style") ? parseAgentStyle(o.signature_agent_style) : "quoted" };
160
160
  }
161
161
  export function loadLimits() {
162
- const raw = readOwnedText("config.json");
163
- if (raw === null)
162
+ const obj = readConfigObject();
163
+ if (obj === null)
164
164
  return { ...DEFAULT_LIMITS };
165
- let obj;
166
- try {
167
- obj = JSON.parse(raw);
168
- }
169
- catch {
170
- throw new ConfigError("config.json in the Tiresias folder is not valid JSON. Fix it or delete it to use the defaults.");
171
- }
172
- if (typeof obj !== "object" || obj === null || Array.isArray(obj))
173
- throw new ConfigError("config.json must be a JSON object, like {\"keep_history_days\": 90}.");
174
165
  const out = { ...DEFAULT_LIMITS };
175
166
  // Own properties only: "constructor", "toString" and "__proto__" are not settings.
176
167
  // (JSON.parse makes "__proto__" an ordinary own key, so entries() lists it and it is rejected.)
package/dist/confirm.js CHANGED
@@ -12,12 +12,13 @@
12
12
  // Nothing reachable over MCP reads or changes a setting or a confirmation; MCP replies only carry the notice.
13
13
  import fs from "node:fs";
14
14
  import path from "node:path";
15
+ import { callContext } from "./context.js";
15
16
  import { DEFAULT_CALENDARS, parseAnchoring, parseCalendars, parseSafeBrowsing, parseSaving, parseScanner, parseTranslator, readConfigObject, writeConfig } from "./config.js";
16
17
  import { parseRender } from "./render.js";
17
18
  import { ConfigError } from "./errors.js";
18
19
  import * as history from "./history.js";
19
- import { candidates, DEFAULT_PORTS, MIN_SPACING_S, normalizeSite, parsePortList, parseSiteRule, parseSites } from "./sites.js";
20
- import { NOFOLLOW, NONBLOCK, home, ownFile, randomTmp, readOwnedText } from "./store.js";
20
+ import { candidates, DEFAULT_PORTS, MAX_CONFIRMED_SITES, MIN_SPACING_S, normalizeSite, parsePortList, parseSiteRule, parseSites } from "./sites.js";
21
+ import { NOFOLLOW, NONBLOCK, home, homeDir, ownFile, randomTmp, readOwnedText } from "./store.js";
21
22
  export const RECORD = "confirmations.json";
22
23
  export const NEVER_ASK = "YES, NEVER ASK AGAIN FOR RISKY SETTINGS";
23
24
  export const ASK_AGAIN = "YES, ASK AGAIN FOR RISKY SETTINGS";
@@ -176,6 +177,9 @@ function parseRecord(raw) {
176
177
  }
177
178
  // The record and how it was found. Missing or damaged: the defaults, asking on (fail closed).
178
179
  export function loadRecord() {
180
+ const snapshot = callContext.getStore()?.config?.record;
181
+ if (snapshot)
182
+ return snapshot;
179
183
  let raw;
180
184
  try {
181
185
  raw = readOwnedText(RECORD, RECORD_MAX);
@@ -222,7 +226,7 @@ const sitesOf = (o) => o && isObj(o.sites) && Object.values(o.sites).every(isObj
222
226
  // The record's own confirmed sites: each entry as the readers accept it, without the 500-site cap of config.json
223
227
  // (the record can also hold confirmed entries config.json no longer has), at most 5,000.
224
228
  const recordSitesValid = (v) => {
225
- if (!isObj(v) || Object.keys(v).length > 5000)
229
+ if (!isObj(v) || Object.keys(v).length > MAX_CONFIRMED_SITES)
226
230
  return false;
227
231
  try {
228
232
  for (const [k, r] of Object.entries(v)) {
@@ -330,7 +334,8 @@ function confirmedSites(rec) {
330
334
  }
331
335
  // The config.json object every reader of a setting uses (null when there is none). A bad file is a ConfigError.
332
336
  export function inForceConfig() {
333
- return clamp(readConfigObject(), loadRecord().rec).obj;
337
+ const snapshot = callContext.getStore()?.config;
338
+ return snapshot ? snapshot.effective : clamp(readConfigObject(), loadRecord().rec).obj;
334
339
  }
335
340
  export const asking = () => loadRecord().rec.ask;
336
341
  // ---- environment variables ----
@@ -663,7 +668,7 @@ export function stateForView(raw) {
663
668
  const { rec, state } = loadRecord();
664
669
  let file = RECORD;
665
670
  try {
666
- file = path.join(home(), RECORD);
671
+ file = path.join(homeDir(), RECORD);
667
672
  }
668
673
  catch {
669
674
  // no profile folder yet
package/dist/cooldown.js CHANGED
@@ -19,6 +19,12 @@ import { appendLine, cachePath, readJson, readLogSync, rootDir } from "./store.j
19
19
  // full, never shortened.
20
20
  const ABSURD_SECONDS = 10 * 365 * 86_400;
21
21
  export const ABSURD_FALLBACK_SECONDS = 86_400;
22
+ // The three HTTP-date spellings, excluding Date.parse's permissive numeric and ISO inputs.
23
+ const WEEKDAY = "(?:Mon|Tue|Wed|Thu|Fri|Sat|Sun)";
24
+ const MONTH = "(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)";
25
+ const CLOCK = "[0-2][0-9]:[0-5][0-9]:[0-5][0-9]";
26
+ const HTTP_DATE = new RegExp(`^(?:${WEEKDAY}, [0-3][0-9] ${MONTH} [0-9]{4} ${CLOCK} GMT|(?:Monday|Tuesday|Wednesday|Thursday|Friday|Saturday|Sunday), [0-3][0-9]-${MONTH}-[0-9]{2} ${CLOCK} GMT|${WEEKDAY} ${MONTH} (?: [1-9]|[0-3][0-9]) ${CLOCK} [0-9]{4})$`);
27
+ const httpDate = (s) => HTTP_DATE.test(s) ? Date.parse(s) : NaN;
22
28
  // Retry-After is either whole seconds or an HTTP date. Returns null when it
23
29
  // is missing, unreadable or absurd.
24
30
  export function parseRetryAfter(v) {
@@ -29,7 +35,7 @@ export function parseRetryAfter(v) {
29
35
  if (/^\d+$/.test(t))
30
36
  secs = Number(t);
31
37
  else {
32
- const when = Date.parse(t);
38
+ const when = httpDate(t);
33
39
  if (Number.isNaN(when))
34
40
  return null;
35
41
  secs = Math.max(0, Math.ceil((when - Date.now()) / 1000));
@@ -43,7 +49,7 @@ export function cooldownSeconds(status, header) {
43
49
  return parsed;
44
50
  if (header && header.trim()) {
45
51
  const t = header.trim();
46
- const absurd = /^\d+$/.test(t) || !Number.isNaN(Date.parse(t));
52
+ const absurd = /^\d+$/.test(t) || !Number.isNaN(httpDate(t));
47
53
  if (absurd)
48
54
  return ABSURD_FALLBACK_SECONDS;
49
55
  }
package/dist/engine.js CHANGED
@@ -16,11 +16,12 @@ export * from "./watchflow.js";
16
16
  export * from "./resume.js";
17
17
  export * from "./formflow.js";
18
18
  export const read = async (url, opts = {}, ctx = DEFAULT) => {
19
+ const saveFallback = opts.save ? saveRefusal() : undefined;
19
20
  const out = await guarded("read", url, ctx, opts.question, (m) => run("read", url, opts, m, ctx.agent), {}, { create: { question: opts.question, section: opts.section, render: opts.render, translate: opts.translate } });
20
21
  // W-28 (v0.8): a read that asked to save always says what happened to the save (the pipeline sets it when it got
21
22
  // as far as a page or a file).
22
23
  if (opts.save && !out.saved)
23
- addSaved(out, out.private ? { status: "private" } : saveRefusal());
24
+ addSaved(out, out.private ? { status: "private" } : saveFallback);
24
25
  return out;
25
26
  };
26
27
  function saveRefusal() {
package/dist/envelope.js CHANGED
@@ -1,3 +1,8 @@
1
+ // SPDX-License-Identifier: Apache-2.0
2
+ // Reply building: the envelope, its budget, the error map, the handoff ticket for a wall, and guarded(),
3
+ // the wrapper every tool call runs in (deadline, Safe Browsing, private mode, history).
4
+ // (Split out of engine.ts in v0.5.1, behavior unchanged.)
5
+ import { snapshotConfig } from "./config.js";
1
6
  import { TiresiasError } from "./errors.js";
2
7
  import * as handoffs from "./handoffs.js";
3
8
  import * as history from "./history.js";
@@ -202,7 +207,18 @@ export function errorEnv(url, e) {
202
207
  return env(url, "error", { message: e.message });
203
208
  }
204
209
  }
205
- export async function guarded(tool, url, ctx, purpose, fn, meta = {}, hand, safetyCheck = true, // W-19: false only for a page a person provided (nothing is requested)
210
+ export async function guarded(...args) {
211
+ let config;
212
+ let configError;
213
+ try {
214
+ config = snapshotConfig();
215
+ }
216
+ catch (e) {
217
+ configError = e;
218
+ }
219
+ return callContext.run({ ...callContext.getStore(), config, configError }, () => guardedCall(...args));
220
+ }
221
+ async function guardedCall(tool, url, ctx, purpose, fn, meta = {}, hand, safetyCheck = true, // W-19: false only for a page a person provided (nothing is requested)
206
222
  record = true) {
207
223
  const priv = !!ctx.private || sessionPrivate();
208
224
  // W-15: the once-a-day trim. Not in private mode (a private call leaves the store as it found it).
@@ -218,6 +234,8 @@ record = true) {
218
234
  try {
219
235
  // W-19: the asked URL is checked first, before robots.txt or anything else, and before a copy is
220
236
  // served from the cache. Every later hop on a new host is checked by the fetcher (safety.ts).
237
+ if (callContext.getStore()?.configError)
238
+ throw callContext.getStore().configError;
221
239
  assertProfile(); // W-18: a profile that does not exist fails here, by name, before anything else
222
240
  gate = safetyCheck ? safebrowsing.makeGate({ allowFlagged: !!ctx.allow_flagged, priv, readOnly: !record }) : undefined;
223
241
  const g = gate;
@@ -265,7 +283,7 @@ record = true) {
265
283
  return out;
266
284
  try {
267
285
  history.record({
268
- time: new Date().toISOString(), tool, url, agent: ctx.agent, purpose: purpose?.slice(0, 500), status: out.status, door: out.door,
286
+ time: new Date().toISOString(), tool, url, agent: ctx.agent, purpose, status: out.status, door: out.door,
269
287
  reason: out.handoff?.reason, matched: meta.matched, best_door: meta.best_door, llms_txt: meta.llms_txt,
270
288
  ticket: meta.ticket, source: meta.source, cache: meta.cache, flag: meta.flag, slip: meta.slip, file_kind: meta.file_kind, scan: meta.scan,
271
289
  lang: out.language?.code, // W-26 (v0.7)
package/dist/files.js CHANGED
@@ -75,7 +75,7 @@ export function signature(b, tail) {
75
75
  return { family: "archive", kind: "addon" }; // Chrome extension
76
76
  // Archives and disk images
77
77
  if (at(b, 0, "PK\x03\x04") || at(b, 0, "PK\x05\x06") || at(b, 0, "PK\x07\x08"))
78
- return { family: "archive", kind: "archive" }; // zip and its family
78
+ return { family: "archive", kind: "archive", zip: true }; // zip and its family
79
79
  if (at(b, 0, "Rar!\x1a\x07"))
80
80
  return { family: "archive", kind: "archive" };
81
81
  if (at(b, 0, [0x37, 0x7a, 0xbc, 0xaf, 0x27, 0x1c]))
@@ -350,11 +350,11 @@ function doubleExtension(name) {
350
350
  }
351
351
  // A zip, OLE or Chrome add-on is one family with several kinds. The kind inside the family is taken from
352
352
  // what the response says it is, since nothing inside the file is read.
353
- function refine(kind, sigFamily, ct, names) {
353
+ function refine(kind, sigFamily, ct, names, zip = false) {
354
354
  const exts = [names.disposition, ...(names.others ?? []), names.url].map(lastExt).filter((x) => !!x);
355
355
  if (sigFamily === "ole")
356
356
  return exts.includes("msi") || ct === "application/x-msi" ? "program" : "office";
357
- if (kind !== "archive" || sigFamily !== "archive")
357
+ if (kind !== "archive" || !zip)
358
358
  return kind;
359
359
  if (exts.some((e) => ADDON_EXT.has(e)) || ct === "application/x-xpinstall" || ct === "application/x-chrome-extension")
360
360
  return "addon";
@@ -438,7 +438,7 @@ function decide(head, m, names, sig) {
438
438
  return { file: "unknown", flags: [] };
439
439
  }
440
440
  if (sig && sig.kind !== "script")
441
- return { file: refine(sig.kind, sig.family, m, names), flags: [] };
441
+ return { file: refine(sig.kind, sig.family, m, names, sig.zip), flags: [] };
442
442
  if (!sig && m === "application/pdf")
443
443
  return { read: "pdf", flags: [] }; // W-17: the declared type alone (a broken PDF fails to read)
444
444
  if (looksBinary(head))
package/dist/hosts.js CHANGED
@@ -60,8 +60,8 @@ function readNext(file, spacing) {
60
60
  }
61
61
  // Reserve the next slot for `origin` with `spacing` ms between starts and return when it begins (epoch
62
62
  // ms). `earliest` is this process's own earliest time. Store trouble never fails a request: the slot is
63
- // then this process's own.
64
- export function reserve(origin, spacing, earliest) {
63
+ // then this process's own. `whenReady` leaves a future slot unclaimed so a cancelled wait moves nothing.
64
+ export function reserve(origin, spacing, earliest, whenReady = false) {
65
65
  const now = Date.now();
66
66
  let dir;
67
67
  try {
@@ -89,6 +89,8 @@ export function reserve(origin, spacing, earliest) {
89
89
  return Math.max(now, earliest, readNext(file, spacing)) + spacing;
90
90
  try {
91
91
  const slot = Math.max(now, earliest, readNext(file, spacing));
92
+ if (whenReady && slot > now)
93
+ return slot;
92
94
  const tmp = `${file}.${process.pid}.${Math.random().toString(36).slice(2)}.tmp`;
93
95
  try {
94
96
  const fd = fs.openSync(tmp, fs.constants.O_WRONLY | fs.constants.O_CREAT | fs.constants.O_EXCL | NOFOLLOW, 0o600);
package/dist/inside.js CHANGED
@@ -231,8 +231,8 @@ export function fitInside(e, hardLimit) {
231
231
  uc.dropped_for_size = dropped;
232
232
  const steps = [
233
233
  [ins, "entries", "inside.entries"], [ins, "skipped_parts", "inside.skipped_parts"], [ins, "embedded", "inside.embedded"], [ins, "external_links", "inside.external_links"],
234
- [uc, "receipts", "receipts"], [ins, "sections", "inside.sections"], [ins, "section", "inside.section"],
235
- [ins, "passages", "inside.passages"], [ins, "outline", "inside.outline"], [uc, "flag_details", "flag_details"], [ins, "note", "inside.note"],
234
+ [ins, "sections", "inside.sections"], [ins, "section", "inside.section"],
235
+ [ins, "passages", "inside.passages"], [ins, "outline", "inside.outline"], [uc, "receipts", "receipts"], [uc, "flag_details", "flag_details"], [ins, "note", "inside.note"],
236
236
  ];
237
237
  for (const [obj, key, label] of steps) {
238
238
  if (jsonBytes(e) <= hardLimit)
package/dist/inspect.js CHANGED
@@ -364,7 +364,10 @@ export function inspectScript(mark) {
364
364
  // Judge that control, and mark the element so its raw-text serialization keeps the mark.
365
365
  const all = tag(el) === 'TEXTAREA' ? [rect(el)] : range.getClientRects();
366
366
  const rects = [];
367
- for (let k = 0; k < all.length && rects.length < 50; k++) if (all[k].width >= 2 && all[k].height >= 2) rects.push(all[k]);
367
+ for (let k = 0; k < all.length; k++) {
368
+ if (timeUp()) { out.complete = false; return 'unmeasured'; }
369
+ if (all[k].width >= 2 && all[k].height >= 2) rects.push(all[k]);
370
+ }
368
371
  if (!rects.length) return 'zero_size';
369
372
  const sx = window.scrollX, sy = window.scrollY;
370
373
  const page = rects.map((r) => ({ l: r.left + sx, t: r.top + sy, r: r.right + sx, b: r.bottom + sy }));
package/dist/pipeline.js CHANGED
@@ -99,9 +99,16 @@ export function pickSource(u, od) {
99
99
  // built from that name too (Codex round 6).
100
100
  const cu = new URL(u.href);
101
101
  cu.hostname = hostKey(cu.hostname);
102
- const base = cu.href.split("#")[0];
102
+ cu.hash = "";
103
+ const base = cu.pathname;
103
104
  const stripped = base.replace(/\/$/, "");
104
- const candidates = [base + ".md", stripped + ".md", base.endsWith("/") ? base + "index.html.md" : ""];
105
+ const candidates = [base + ".md", stripped + ".md", base.endsWith("/") ? base + "index.html.md" : ""].map((p) => {
106
+ if (!p)
107
+ return "";
108
+ const twin = new URL(cu.href);
109
+ twin.pathname = p;
110
+ return twin.href;
111
+ });
105
112
  const hit = candidates.find((c) => c && od.llms_md.includes(c));
106
113
  if (hit)
107
114
  return { source: hit, detail: "markdown" };
@@ -278,7 +285,8 @@ export async function run(tool, rawUrl, opts, meta = {}, agent) {
278
285
  const renderMode = tool === "read" || tool === "check" ? loadRenderMode() : "off";
279
286
  // Explicit renders and checks of rendered copies need a body and a new browser run, even after a 304.
280
287
  // The one shortcut is an already rendered copy still fresh under the site's cache rules (below).
281
- const rerender = renderMode !== "off" && (!!opts.render || (tool === "check" && !!cache?.rendered));
288
+ const rerender = renderMode !== "off" && (!!opts.render || (tool === "check" && !!cache?.rendered) || (renderMode === "auto" && !!cache?.shell && !cache.rendered));
289
+ const needsBody = rerender || (renderMode === "off" && !!cache?.rendered);
282
290
  // W-28 (v0.8): a read that asks to save, when the owner allows it (a bad setting fails the read, by name). A save
283
291
  // keeps what THIS read received, so it never uses a fresh copy, a 304 or a cached rendering: it always fetches.
284
292
  const sv = tool === "read" && opts.save && savePolicy() === "ask" ? {} : undefined;
@@ -287,7 +295,7 @@ export async function run(tool, rawUrl, opts, meta = {}, agent) {
287
295
  // Only a copy whose fetch had no redirect at all (recorded by the fetch itself; a missing field
288
296
  // means no) and that was read from the page itself, not llms.txt or a twin: the rules just
289
297
  // checked are then the rules of the page being served (Codex rounds 38 and 39).
290
- if (cache && !staleParse && (!rerender || cache.rendered) && !sv && !opts.fresh && cache.redirected === false && cache.source_url === url && isFresh(cache)) {
298
+ if (cache && !staleParse && (!needsBody || (renderMode !== "off" && cache.rendered)) && !sv && !opts.fresh && cache.redirected === false && cache.source_url === url && isFresh(cache)) {
291
299
  meta.cache = "fresh";
292
300
  const od = tool === "read" ? cachedDoors(u.origin) : null;
293
301
  if (od)
@@ -295,7 +303,7 @@ export async function run(tool, rawUrl, opts, meta = {}, agent) {
295
303
  // W-30 (Codex round 1): a copy served from the cache with no door answers starts (or reports) the
296
304
  // background checks too, so a cancelled or refused job is retried and a running one is reported.
297
305
  const later = tool === "read" && !od && !priv && doorsMode === "background";
298
- const out = await finishPage(tool, url, cache, true, null, opts, later ? pageOnly(u.origin) : od, new Date().toISOString(), () => { }, meta, { fresh: false }, false, { cache: "fresh", agent });
306
+ const out = await finishPage(tool, url, cache, true, null, opts, later ? pageOnly(u.origin) : od, new Date().toISOString(), () => { }, meta, { fresh: false }, false, { cache: "fresh", agent, ...(opts.render && renderMode === "off" ? { render: { status: "not_allowed" } } : {}) });
299
307
  return later ? afterDeferred(out, u.origin) : out;
300
308
  }
301
309
  // check never probes; it just revisits where it read from last time.
@@ -313,7 +321,7 @@ export async function run(tool, rawUrl, opts, meta = {}, agent) {
313
321
  // While the checks are pending, the reply still lists the doors the page itself shows (feeds, an API
314
322
  // description, structured data), from an empty origin answer (Codex round 1).
315
323
  const shown = deferred ? pageOnly(u.origin) : od;
316
- let { res, started, validatorsSent } = await fetchSource(source, cache, staleParse || rerender || !!sv, !!sv);
324
+ let { res, started, validatorsSent } = await fetchSource(source, cache, staleParse || needsBody || !!sv, !!sv);
317
325
  // An unsolicited 304 proves nothing about this parse or representation. Ask once for a full body.
318
326
  if (res.status === 304 && (!validatorsSent || staleParse)) {
319
327
  throwIfCancelled();
package/dist/queue.js CHANGED
@@ -142,17 +142,23 @@ export async function throttle(origin, delay, outer, url, hold = false) {
142
142
  await Promise.race([prev, cancelled]);
143
143
  // Holding the turn: first the cross-process lease (Codex round 76: waiting for it AFTER reserving a
144
144
  // slot let the slot go stale), then the shared slot and its spacing.
145
- if (hold)
146
- activeRelease = await Promise.race([acquireActive(origin, mine.signal), cancelled]);
145
+ activeRelease = await Promise.race([acquireActive(origin, mine.signal), cancelled]);
147
146
  // W-25 (v0.6): a site's own spacing (sites.ts) raises the spacing. Like every spacing since v0.5.1 it is per
148
147
  // ORIGIN (scheme, host and port). Settings are the call's snapshot, read once when the call started
149
148
  // (coordinator's rule after Codex round 11): a change saved meanwhile applies to calls that start after it.
150
149
  const site = target ? siteSpacing(target) : 0;
151
150
  const eff = Math.max(delay, site);
152
- const slot = reserve(origin, eff * 1000, Math.max((lastHit.get(origin) ?? 0) + eff * 1000, nextOk.get(origin) ?? 0));
153
- const wait = slot - Date.now();
154
- if (wait > 0)
151
+ for (;;) {
152
+ if (mine.signal.aborted)
153
+ throw new Error("queue-cancelled");
154
+ if (cooldownLeft(origin) > 0)
155
+ throw cooldownRefusal(origin, url);
156
+ const slot = reserve(origin, eff * 1000, Math.max((lastHit.get(origin) ?? 0) + eff * 1000, nextOk.get(origin) ?? 0), true);
157
+ const wait = slot - Date.now();
158
+ if (wait <= 0)
159
+ break;
155
160
  await Promise.race([sleep(wait, mine.signal), cancelled]);
161
+ }
156
162
  // The checks that matter: right before sending, not before queueing.
157
163
  if (mine.signal.aborted)
158
164
  throw new Error("queue-cancelled");
package/dist/reader.js CHANGED
@@ -72,6 +72,11 @@ function collapse(parts) {
72
72
  }
73
73
  return { text, marks };
74
74
  }
75
+ // Keep a UTF-16 boundary before an entire code point.
76
+ function splitEnd(text, end) {
77
+ const hi = text.charCodeAt(end - 1), lo = text.charCodeAt(end);
78
+ return hi >= 0xd800 && hi <= 0xdbff && lo >= 0xdc00 && lo <= 0xdfff ? end - 1 : end;
79
+ }
75
80
  // Split an overlong paragraph at spaces. Every piece is an exact substring.
76
81
  function splitBlock(b) {
77
82
  if (b.text.length <= MAX_BLOCK)
@@ -85,6 +90,7 @@ function splitBlock(b) {
85
90
  if (sp > start + MAX_BLOCK / 2)
86
91
  end = sp;
87
92
  }
93
+ end = splitEnd(b.text, end);
88
94
  const lo = start;
89
95
  const marks = b.marks.filter(([pos]) => (pos > lo || lo === 0) && pos <= end).map(([pos, n]) => [pos - lo, n]);
90
96
  out.push({ text: b.text.slice(start, end), marks });
@@ -143,14 +149,14 @@ class Builder {
143
149
  pre(raw, marks = [], lead = "", indent = "") {
144
150
  if (!this.cur)
145
151
  this.startTop();
146
- for (let i = 0; i < raw.length; i += MAX_BLOCK) {
147
- const piece = raw.slice(i, i + MAX_BLOCK);
152
+ for (let i = 0; i < raw.length;) {
153
+ const end = splitEnd(raw, Math.min(i + MAX_BLOCK, raw.length));
154
+ const piece = raw.slice(i, end);
148
155
  // A fence longer than any run of backticks inside, so the text cannot close it (round 78).
149
156
  let run = 0;
150
157
  for (const m of piece.match(/`+/g) ?? [])
151
158
  run = Math.max(run, m.length);
152
159
  const fence = "`".repeat(Math.max(3, run + 1));
153
- const end = i + piece.length;
154
160
  const head = i === 0 && lead ? lead : indent;
155
161
  const body = indent ? piece.split("\n").map((l) => indent + l).join("\n") : piece;
156
162
  const m = marks
@@ -162,6 +168,7 @@ class Builder {
162
168
  });
163
169
  this.cur.blocks.push({ text: `${head}${fence}\n${body}\n${indent}${fence}`, marks: m });
164
170
  this.shield.scan(piece, "visible text");
171
+ i = end;
165
172
  }
166
173
  }
167
174
  // Text before the first heading goes under a label Tiresias makes, not text from the page: it gets no
@@ -646,6 +653,8 @@ function scanHead(root) {
646
653
  found.title = n;
647
654
  else if (n.name === "script" && /ld\+json/i.test(n.attribs?.type ?? ""))
648
655
  found.ld.push(n);
656
+ else if (n.name === "base" && !found.base && n.attribs?.href !== undefined)
657
+ found.base = n;
649
658
  else if (n.name === "link")
650
659
  found.links.push(n);
651
660
  else if (n.name === "meta" && found.metas.length < 200)
@@ -725,10 +734,11 @@ export function lineBlocks(text) {
725
734
  let j = text.indexOf("\n", i);
726
735
  if (j === -1)
727
736
  j = text.length;
728
- for (let k = i; k < j; k += MAX_BLOCK) {
729
- const e = Math.min(j, k + MAX_BLOCK);
737
+ for (let k = i; k < j;) {
738
+ const e = splitEnd(text, Math.min(j, k + MAX_BLOCK));
730
739
  if (text.slice(k, e).trim())
731
740
  lines.push([k, e]);
741
+ k = e;
732
742
  }
733
743
  i = j + 1;
734
744
  }
@@ -762,8 +772,11 @@ export function lineBlocks(text) {
762
772
  }
763
773
  function rawBlocks(text) {
764
774
  const out = [];
765
- for (let i = 0; i < text.length; i += MAX_BLOCK)
766
- out.push({ text: text.slice(i, i + MAX_BLOCK), marks: [] });
775
+ for (let i = 0; i < text.length;) {
776
+ const end = splitEnd(text, Math.min(i + MAX_BLOCK, text.length));
777
+ out.push({ text: text.slice(i, end), marks: [] });
778
+ i = end;
779
+ }
767
780
  return out;
768
781
  }
769
782
  // `keep`, when given, receives the forms WITH their hidden values (W-04 preview and submit only).
@@ -787,11 +800,20 @@ export function parseHtml(html, baseUrl, keep, rendered) {
787
800
  shield.sheet.inherited.set(attrs, why);
788
801
  }
789
802
  }
790
- const b = new Builder(shield, baseUrl);
803
+ const found = scanHead(doc);
804
+ let linkBase = baseUrl;
805
+ try {
806
+ const base = new URL(found.base?.attribs?.href ?? "", baseUrl);
807
+ if (base.protocol === "http:" || base.protocol === "https:")
808
+ linkBase = base.href;
809
+ }
810
+ catch {
811
+ // An unusable first base leaves links relative to the fetched page.
812
+ }
813
+ const b = new Builder(shield, linkBase);
791
814
  chooseMain(doc, b);
792
815
  walk(doc, b);
793
816
  b.flush();
794
- const found = scanHead(doc);
795
817
  let title = found.title ? norm(rawText(found.title, MAX_TITLE * 2)).slice(0, MAX_TITLE) : "";
796
818
  if (!title)
797
819
  title = (b.sections.find((s) => s.level === 1)?.heading ?? "").slice(0, MAX_TITLE);
@@ -799,7 +821,7 @@ export function parseHtml(html, baseUrl, keep, rendered) {
799
821
  // Features of the page itself.
800
822
  const resolve = (h) => {
801
823
  try {
802
- return h ? new URL(h, baseUrl).href : null;
824
+ return h ? new URL(h, linkBase).href : null;
803
825
  }
804
826
  catch {
805
827
  return null;
@@ -819,7 +841,7 @@ export function parseHtml(html, baseUrl, keep, rendered) {
819
841
  service_desc.push(href);
820
842
  }
821
843
  const page = finish(b, { title, feeds, service_desc });
822
- page.meta = pageMeta(found, baseUrl);
844
+ page.meta = pageMeta(found, linkBase);
823
845
  for (const v of [page.meta?.byline, page.meta?.description, page.meta?.og?.title, page.meta?.og?.description])
824
846
  if (v)
825
847
  shield.scan(v, "page metadata");
@@ -945,8 +967,9 @@ export function parseSvg(body, baseUrl) {
945
967
  }
946
968
  const PLAIN_HEADING = { script: "(script)", csv: "(CSV)", calendar: "(calendar)", contact: "(contact)" };
947
969
  export function parsePlain(text, baseUrl, mode) {
948
- if (text.length > MAX_EXPANDED)
949
- text = text.slice(0, MAX_EXPANDED);
970
+ const omitted = text.length > MAX_EXPANDED ? text.length - splitEnd(text, MAX_EXPANDED) : 0;
971
+ if (omitted)
972
+ text = text.slice(0, text.length - omitted);
950
973
  // Calendar and contact lines may be folded (a line break followed by a space or tab continues the line).
951
974
  if (mode === "calendar" || mode === "contact")
952
975
  text = text.replace(/\r?\n[ \t]/g, "");
@@ -959,7 +982,7 @@ export function parsePlain(text, baseUrl, mode) {
959
982
  if (mode === "calendar" || mode === "contact")
960
983
  for (const u of bareUrls(text))
961
984
  b.link(u, u);
962
- const page = finish(b, {});
985
+ const page = finish(b, { text_omitted: omitted || undefined });
963
986
  if (mode === "calendar")
964
987
  calendarEvents(text, page, shield); // v0.6: the invite's events, structured (ics.ts)
965
988
  const own = [];
@@ -1312,8 +1335,9 @@ export function parseJsonText(text, baseUrl) {
1312
1335
  }
1313
1336
  if (pretty.length > MAX_EXPANDED) {
1314
1337
  // raw text that is not JSON: keep the first MAX_EXPANDED characters and count the rest
1315
- omitted = pretty.length - MAX_EXPANDED;
1316
- pretty = pretty.slice(0, MAX_EXPANDED);
1338
+ const end = splitEnd(pretty, MAX_EXPANDED);
1339
+ omitted = pretty.length - end;
1340
+ pretty = pretty.slice(0, end);
1317
1341
  }
1318
1342
  const shield = new Shield();
1319
1343
  shield.scan(pretty, "JSON body");
@@ -1367,8 +1391,9 @@ function pdfBlocks(text) {
1367
1391
  let l = line;
1368
1392
  while (l.length > MAX_BLOCK) {
1369
1393
  flush();
1370
- out.push({ text: l.slice(0, MAX_BLOCK), marks: [] });
1371
- l = l.slice(MAX_BLOCK);
1394
+ const end = splitEnd(l, MAX_BLOCK);
1395
+ out.push({ text: l.slice(0, end), marks: [] });
1396
+ l = l.slice(end);
1372
1397
  }
1373
1398
  if (cur && cur.length + 1 + l.length > MAX_BLOCK)
1374
1399
  flush();
package/dist/resume.js CHANGED
@@ -2,7 +2,7 @@
2
2
  // W-06: handoff tickets and resume, including a page a person provided.
3
3
  // (Split out of engine.ts in v0.5.1, behavior unchanged.)
4
4
  import path from "node:path";
5
- import { ConfigError, loadLimits } from "./config.js";
5
+ import { ConfigError, DEFAULT_LIMITS, readConfigObject } from "./config.js";
6
6
  import { classify } from "./files.js";
7
7
  import * as handoffs from "./handoffs.js";
8
8
  import { decode, fetchCap, throwIfCancelled } from "./safety.js";
@@ -90,7 +90,14 @@ export async function fromPerson(t, p, opts, meta, agent) {
90
90
  out.message = `${PROVIDED}${out.message ? " " + out.message : ""}`;
91
91
  return out;
92
92
  }
93
- export const keepDays = () => loadLimits().keep_handoff_days;
93
+ export const keepDays = () => {
94
+ const value = readConfigObject()?.keep_handoff_days;
95
+ if (value === undefined)
96
+ return DEFAULT_LIMITS.keep_handoff_days;
97
+ if (typeof value !== "number" || !Number.isSafeInteger(value) || value < 1)
98
+ throw new ConfigError('config.json: "keep_handoff_days" must be a whole number of 1 or more.');
99
+ return value;
100
+ };
94
101
  // W-06: pick up a read that stopped at a wall. Without `provided`, the original read runs again
95
102
  // (same URL, question, section, render and translate options) with every rule in force: robots.txt, waits, signing and wall
96
103
  // detection; it always asks the site (a cached copy is never taken as proof the wall is gone).
package/dist/rules.js CHANGED
@@ -250,11 +250,12 @@ async function fetchRobots(origin) {
250
250
  const ROBOTS_COST = ROBOTS_MAX + WIRE_EXTRA; // what its connections may receive (safety.ts Wire), plus one read of slack
251
251
  if (budget && !budget.take(ROBOTS_COST))
252
252
  throw new FetchError("The render's byte budget is spent.", "budget");
253
- let charged = budget ? ROBOTS_COST : 0; // a failure is charged in full (bytes may have arrived)
253
+ let reserved = budget ? ROBOTS_COST : 0;
254
+ let charged = reserved; // a failure is charged in full (bytes may have arrived)
254
255
  const turns = new Turns(); // robots.txt holds its site's turn until read AND parsed, its delay noted (rounds 74, 86)
255
256
  try {
256
257
  try {
257
- const res = await safeFetch(origin + "/robots.txt", {
258
+ const fetchRobots = () => safeFetch(origin + "/robots.txt", {
258
259
  maxBytes: ROBOTS_MAX,
259
260
  wireBytes: ROBOTS_COST - READ_SLACK,
260
261
  truncate: true,
@@ -277,6 +278,18 @@ async function fetchRobots(origin) {
277
278
  await turns.take(u, spacingMs() / 1000, signal); // v0.5.1: robots.txt waits its turn too (v0.6: and a site's own spacing, in the queue)
278
279
  },
279
280
  });
281
+ let res = await fetchRobots();
282
+ if (res.status === 304) {
283
+ turns.giveBack();
284
+ if (budget) {
285
+ budget.release(ROBOTS_COST, res.size ?? ROBOTS_COST);
286
+ reserved = charged = 0;
287
+ if (!budget.take(ROBOTS_COST))
288
+ throw new FetchError("The render's byte budget is spent.", "budget");
289
+ reserved = charged = ROBOTS_COST;
290
+ }
291
+ res = await fetchRobots();
292
+ }
280
293
  if (budget)
281
294
  charged = res.size ?? ROBOTS_COST; // what its connections received
282
295
  keep = !priv && !/(^|[\s,])no-store\b/i.test(res.headers.get("cache-control") ?? "");
@@ -296,6 +309,8 @@ async function fetchRobots(origin) {
296
309
  robots = { mode: "wall", rules: [], crawlDelay: 0, sitemaps: [], wall: w, wallUrl: res.url, expires: Date.now() + 60_000 };
297
310
  // v0.9 hardening (Codex round 3b, HIGH): every success is a robots.txt to follow (RFC 9309 2.3.1.1), not
298
311
  // only a 200; a partial one (206) cannot be followed faithfully, so it means "the site says no".
312
+ else if (res.status === 304)
313
+ robots = closed("robots.txt returned an unexpected HTTP 304, so Tiresias assumes the site says no.", 60_000);
299
314
  else if (res.status === 206)
300
315
  robots = closed("robots.txt came back partial (HTTP 206), so Tiresias assumes the site says no.", 60_000);
301
316
  else if (res.status >= 200 && res.status < 300 && (res.bytes !== undefined || res.file !== undefined))
@@ -341,8 +356,8 @@ async function fetchRobots(origin) {
341
356
  }
342
357
  finally {
343
358
  turns.giveBack();
344
- if (budget)
345
- budget.release(ROBOTS_COST, charged);
359
+ if (budget && reserved)
360
+ budget.release(reserved, charged);
346
361
  }
347
362
  // Cache-Control: no-store: keep it for this one call only, never across calls.
348
363
  if (keep)
package/dist/saved.js CHANGED
@@ -16,6 +16,7 @@ import fs from "node:fs";
16
16
  import path from "node:path";
17
17
  import { randomBytes, sign as edSign, verify as edVerify } from "node:crypto";
18
18
  import { ensureSlipKey, readSlipKey } from "./keys.js";
19
+ import { releaseLock, stillOwned, takeLock } from "./savedlock.js";
19
20
  import { bound as boundReceipt } from "./receipts.js";
20
21
  import { DEFAULT_LIMITS, loadLimits, loadSavingMode } from "./config.js";
21
22
  import { NOFOLLOW, NONBLOCK, checkSub, rmRegular, isPrivate, ownFile, profileName, readBounded, readHome, sha256, subDir, PRIVATE_WRITE, PrivateWriteError } from "./store.js";
@@ -244,53 +245,67 @@ export function saveSnapshot(input, receipts, limits) {
244
245
  return { status: "failed", reason: "store" };
245
246
  // Everything this snapshot will take on disk, the manifest included (Codex v0.8 round 1).
246
247
  const newBytes = parts.reduce((n, p) => n + p.data.length, 0);
247
- let d;
248
- let dirId;
248
+ // Capacity and publication share one lock with cleanup; only a dead owner's lock can be recovered.
249
+ let held;
249
250
  try {
250
- const lim = limits ?? loadLimits();
251
- d = savedDir(true);
252
- dirId = identOf(d);
253
- const { snaps, tmps, complete } = entries(d);
254
- if (!complete)
255
- return { status: "failed", reason: "store" }; // the caps cannot be checked: fail closed
256
- // Temp folders take disk too (another save in progress, or one left behind): they count (Codex v0.8 round 3).
257
- const used = [...snaps, ...tmps].reduce((n, e) => n + e.bytes, 0);
258
- // A save never evicts: a prompt-injected agent saving again and again must not push out older evidence.
259
- if (snaps.length >= MAX_SNAPSHOTS || used + newBytes + manifestBytes.length > (lim.max_saved_mb ?? DEFAULT_LIMITS.max_saved_mb) * 1024 * 1024)
260
- return { status: "over_cap" };
261
- }
262
- catch (e) {
263
- if (e instanceof PrivateWriteError)
264
- return { status: "private" };
265
- return { status: "failed", reason: "store" };
266
- }
267
- const tmpName = `.tmp-${randomBytes(8).toString("hex")}`;
268
- const tmp = path.join(d, tmpName);
269
- let made = false;
270
- try {
271
- sameDir(d, dirId);
272
- fs.mkdirSync(tmp, { mode: 0o700 });
273
- made = true;
274
- const tmpId = identOf(tmp);
275
- for (const p of parts) {
251
+ let d;
252
+ let dirId;
253
+ try {
254
+ const lim = limits ?? loadLimits();
255
+ d = savedDir(true);
256
+ dirId = identOf(d);
257
+ const token = takeLock(d);
258
+ if (!token)
259
+ return { status: "failed", reason: "store" };
260
+ held = { file: d, token };
261
+ const { snaps, tmps, complete } = entries(d);
262
+ if (!complete)
263
+ return { status: "failed", reason: "store" }; // the caps cannot be checked: fail closed
264
+ // Temp folders take disk too (another save in progress, or one left behind): they count (Codex v0.8 round 3).
265
+ const used = [...snaps, ...tmps].reduce((n, e) => n + e.bytes, 0);
266
+ // A save never evicts: a prompt-injected agent saving again and again must not push out older evidence.
267
+ if (snaps.length >= MAX_SNAPSHOTS || used + newBytes + manifestBytes.length > (lim.max_saved_mb ?? DEFAULT_LIMITS.max_saved_mb) * 1024 * 1024)
268
+ return { status: "over_cap" };
269
+ }
270
+ catch (e) {
271
+ if (e instanceof PrivateWriteError)
272
+ return { status: "private" };
273
+ return { status: "failed", reason: "store" };
274
+ }
275
+ const tmpName = `.tmp-${randomBytes(8).toString("hex")}`;
276
+ const tmp = path.join(d, tmpName);
277
+ let made = false;
278
+ try {
279
+ sameDir(d, dirId);
280
+ fs.mkdirSync(tmp, { mode: 0o700 });
281
+ made = true;
282
+ const tmpId = identOf(tmp);
283
+ for (const p of parts) {
284
+ sameDir(d, dirId), sameDir(tmp, tmpId);
285
+ writeNew(path.join(tmp, p.name), p.data);
286
+ }
287
+ sameDir(d, dirId), sameDir(tmp, tmpId);
288
+ writeNew(path.join(tmp, "manifest.json"), manifestBytes);
276
289
  sameDir(d, dirId), sameDir(tmp, tmpId);
277
- writeNew(path.join(tmp, p.name), p.data);
290
+ if (!stillOwned(d, held.token))
291
+ throw new SavedError("The snapshot lock changed.");
292
+ fs.renameSync(tmp, path.join(d, id)); // one step: a reader never sees half a snapshot
293
+ made = false;
294
+ sameDir(path.join(d, id), tmpId); // the folder now named by the id is the one just written
295
+ }
296
+ catch {
297
+ if (made)
298
+ removeDir(tmp, true);
299
+ return { status: "failed", reason: "store" };
278
300
  }
279
- sameDir(d, dirId), sameDir(tmp, tmpId);
280
- writeNew(path.join(tmp, "manifest.json"), manifestBytes);
281
- sameDir(d, dirId), sameDir(tmp, tmpId);
282
- fs.renameSync(tmp, path.join(d, id)); // one step: a reader never sees half a snapshot
283
- made = false;
284
- sameDir(path.join(d, id), tmpId); // the folder now named by the id is the one just written
301
+ for (const r of receipts.slice(0, MAX_RECEIPTS))
302
+ r.snapshot = id;
303
+ return { status: "saved", id, files: parts.length, bytes: newBytes, ...(input.rendered ? { rendered: true } : {}), ...(input.file && !parts.length ? { fingerprint_only: true } : {}) };
285
304
  }
286
- catch {
287
- if (made)
288
- removeDir(tmp, true);
289
- return { status: "failed", reason: "store" };
305
+ finally {
306
+ if (held)
307
+ releaseLock(held.file, held.token);
290
308
  }
291
- for (const r of receipts.slice(0, MAX_RECEIPTS))
292
- r.snapshot = id;
293
- return { status: "saved", id, files: parts.length, bytes: newBytes, ...(input.rendered ? { rendered: true } : {}), ...(input.file && !parts.length ? { fingerprint_only: true } : {}) };
294
309
  }
295
310
  // ---- reading back ----
296
311
  export class SavedError extends Error {
@@ -615,15 +630,33 @@ export function planCleanup(now, limits) {
615
630
  }
616
631
  export function applyCleanup(plan) {
617
632
  const result = { removed: 0, kept: plan.kept, bytes_freed: 0 };
618
- for (const r of plan.remove) {
619
- try {
620
- removeDir(r.dir);
621
- result.removed++;
622
- result.bytes_freed += r.bytes;
623
- }
624
- catch {
625
- result.kept++; // left for the next run
633
+ if (!plan.remove.length)
634
+ return result;
635
+ let held;
636
+ try {
637
+ const d = savedDir(false);
638
+ if (!d || isPrivate())
639
+ return { ...result, kept: result.kept + plan.remove.length };
640
+ const token = takeLock(d);
641
+ if (!token)
642
+ return { ...result, kept: result.kept + plan.remove.length };
643
+ held = { file: d, token };
644
+ for (const r of plan.remove) {
645
+ try {
646
+ if (!stillOwned(d, token))
647
+ throw new SavedError("The snapshot lock changed.");
648
+ removeDir(r.dir);
649
+ result.removed++;
650
+ result.bytes_freed += r.bytes;
651
+ }
652
+ catch {
653
+ result.kept++; // left for the next run
654
+ }
626
655
  }
656
+ return result;
657
+ }
658
+ finally {
659
+ if (held)
660
+ releaseLock(held.file, held.token);
627
661
  }
628
- return result;
629
662
  }
@@ -0,0 +1,114 @@
1
+ // SPDX-License-Identifier: Apache-2.0
2
+ // Saves run in killable MCP servers. Publish a nonempty lock directory atomically, so even a kill during
3
+ // acquisition leaves a named owner. Only ESRCH permits recovery; a live owner is never timed out.
4
+ // Recovery removes the dead owner's unique entry, then rmdir: a delayed recoverer cannot remove a new,
5
+ // nonempty lock. Config commands keep ownedlock.ts's deliberately manual recovery policy.
6
+ import { randomBytes } from "node:crypto";
7
+ import fs from "node:fs";
8
+ import path from "node:path";
9
+ import { NOFOLLOW, NONBLOCK } from "./store.js";
10
+ const TRY_MS = 1_000;
11
+ const OWNER_RE = /^[1-9][0-9]*-[0-9a-f]{24}$/;
12
+ const sleepSync = (ms) => Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms);
13
+ function dead(pid) {
14
+ if (!Number.isSafeInteger(pid) || pid <= 0)
15
+ return false;
16
+ try {
17
+ process.kill(pid, 0);
18
+ return false;
19
+ }
20
+ catch (e) {
21
+ return e.code === "ESRCH";
22
+ }
23
+ }
24
+ function recover(file) {
25
+ const l = file + ".lock";
26
+ try {
27
+ const st = fs.lstatSync(l);
28
+ if (st.isDirectory()) {
29
+ const d = fs.opendirSync(l);
30
+ let owner;
31
+ try {
32
+ const entry = d.readSync();
33
+ if (entry?.isFile() && OWNER_RE.test(entry.name) && d.readSync() === null)
34
+ owner = entry.name;
35
+ }
36
+ finally {
37
+ d.closeSync();
38
+ }
39
+ if (!owner || !dead(Number(owner.split("-")[0])))
40
+ return;
41
+ releaseLock(file, owner);
42
+ }
43
+ else if (st.isFile()) {
44
+ // Recover a lock left by the previous saved.ts implementation. unlink cannot remove a new directory.
45
+ const fd = fs.openSync(l, fs.constants.O_RDONLY | NOFOLLOW | NONBLOCK);
46
+ try {
47
+ if (fs.fstatSync(fd).size > 256)
48
+ return;
49
+ const owner = JSON.parse(fs.readFileSync(fd, "utf8"));
50
+ if (dead(owner?.pid))
51
+ fs.unlinkSync(l);
52
+ }
53
+ finally {
54
+ fs.closeSync(fd);
55
+ }
56
+ }
57
+ }
58
+ catch {
59
+ // Gone meanwhile, malformed, or not ours: retry acquisition without removing anything else.
60
+ }
61
+ }
62
+ export function takeLock(file, tryMs = TRY_MS) {
63
+ const l = file + ".lock";
64
+ const token = `${process.pid}-${randomBytes(12).toString("hex")}`;
65
+ const tmp = l + "-" + token;
66
+ const until = Date.now() + tryMs;
67
+ try {
68
+ fs.mkdirSync(tmp, { mode: 0o700 });
69
+ fs.closeSync(fs.openSync(path.join(tmp, token), fs.constants.O_WRONLY | fs.constants.O_CREAT | fs.constants.O_EXCL | NOFOLLOW, 0o600));
70
+ for (;;) {
71
+ try {
72
+ fs.renameSync(tmp, l); // replaces an empty recovered directory, never a live nonempty one
73
+ return token;
74
+ }
75
+ catch (e) {
76
+ if (!["EEXIST", "ENOTEMPTY", "ENOTDIR", "EISDIR"].includes(e.code ?? ""))
77
+ return null;
78
+ }
79
+ recover(file);
80
+ if (Date.now() >= until)
81
+ return null;
82
+ sleepSync(5);
83
+ }
84
+ }
85
+ catch {
86
+ return null;
87
+ }
88
+ finally {
89
+ try {
90
+ fs.unlinkSync(path.join(tmp, token));
91
+ fs.rmdirSync(tmp);
92
+ }
93
+ catch {
94
+ // Renamed into place, or acquisition failed before the owner entry was written.
95
+ }
96
+ }
97
+ }
98
+ export function stillOwned(file, token) {
99
+ try {
100
+ return fs.lstatSync(file + ".lock").isDirectory() && fs.lstatSync(path.join(file + ".lock", token)).isFile();
101
+ }
102
+ catch {
103
+ return false;
104
+ }
105
+ }
106
+ export function releaseLock(file, token) {
107
+ try {
108
+ fs.unlinkSync(path.join(file + ".lock", token));
109
+ fs.rmdirSync(file + ".lock");
110
+ }
111
+ catch {
112
+ // Gone meanwhile, or a new nonempty lock has already replaced ours.
113
+ }
114
+ }
package/dist/sites.js CHANGED
@@ -24,12 +24,14 @@
24
24
  // A bad "sites" value fails every call that sends a request, with an error naming the site and the setting
25
25
  // (fail closed: a broken entry might be a "never_visit"). clear and the daily trim never read it.
26
26
  import net from "node:net";
27
+ import { readConfigObject } from "./config.js";
27
28
  import { inForceConfig } from "./confirm.js";
28
29
  import { callContext } from "./context.js";
29
30
  import { ConfigError, Refused, SafetyError } from "./errors.js";
30
31
  import { hostKey } from "./sitekey.js";
31
32
  export const SITE_SETTINGS = ["spacing_seconds", "ports", "hand_off", "never_visit"];
32
33
  export const MAX_SITES = 500;
34
+ export const MAX_CONFIRMED_SITES = 5000;
33
35
  export const MIN_SPACING_S = 1;
34
36
  export const MAX_SPACING_S = 30;
35
37
  export const DEFAULT_PORTS = [[80, 80], [443, 443], [8000, 8999]];
@@ -121,12 +123,12 @@ export function parseSiteRule(site, v) {
121
123
  }
122
124
  return out;
123
125
  }
124
- export function parseSites(v) {
126
+ export function parseSites(v, max = MAX_SITES) {
125
127
  if (typeof v !== "object" || v === null || Array.isArray(v))
126
128
  throw new ConfigError('config.json: "sites" must be an object, like {"example.com": {"spacing_seconds": 5}}.');
127
129
  const keys = Object.keys(v);
128
- if (keys.length > MAX_SITES)
129
- throw new ConfigError(`config.json: "sites" has more than ${MAX_SITES} sites.`);
130
+ if (keys.length > max)
131
+ throw new ConfigError(`config.json: "sites" has more than ${max} sites.`);
130
132
  const out = Object.create(null);
131
133
  for (const k of keys) {
132
134
  normalizeSite(k); // strict: the written form
@@ -135,8 +137,11 @@ export function parseSites(v) {
135
137
  return out;
136
138
  }
137
139
  export function loadSites() {
140
+ const raw = readConfigObject();
141
+ if (raw && Object.hasOwn(raw, "sites"))
142
+ parseSites(raw.sites);
138
143
  const o = inForceConfig(); // W-32: an unconfirmed loosening is held back
139
- return o && Object.hasOwn(o, "sites") ? parseSites(o.sites) : Object.create(null);
144
+ return o && Object.hasOwn(o, "sites") ? parseSites(o.sites, MAX_SITES + MAX_CONFIRMED_SITES) : Object.create(null);
140
145
  }
141
146
  // The entries that can apply to a host, most specific first: the host itself, then each wildcard above it.
142
147
  // A host name longer than DNS allows (253 characters) names no site anyone can reach: it is refused before any
package/dist/types.js CHANGED
@@ -1,8 +1,8 @@
1
1
  // SPDX-License-Identifier: Apache-2.0
2
2
  // Shared shapes. The envelope matches SPEC section 6.1.
3
- export const VERSION = "1.0.0";
3
+ export const VERSION = "1.0.1";
4
4
  // Bump when the reader changes what it extracts, so older cached pages are re-read.
5
- export const PARSER_VERSION = 14; // 14: v0.9.1 sweep ("(JSON)" heading generated, list indentation capped at 8 levels, markdown code kept as preformatted, nesting past 256 flattened). 13: v0.9 labels Tiresias makes ("(top of page)", "(structured data)") are marked generated, never quoted. 12: v0.7 language (W-26). 11: v0.6 calendar events (ics.ts). 10: v0.5.1 JSON as line blocks, W-22 reader. 9: W-20 file kinds (scripts, CSV, calendars and contacts read as plain text; SVG; file flags)
5
+ export const PARSER_VERSION = 15; // 15: v1.0.1 reader fixes. 14: v0.9.1 sweep ("(JSON)" heading generated, list indentation capped at 8 levels, markdown code kept as preformatted, nesting past 256 flattened). 13: v0.9 labels Tiresias makes ("(top of page)", "(structured data)") are marked generated, never quoted. 12: v0.7 language (W-26). 11: v0.6 calendar events (ics.ts). 10: v0.5.1 JSON as line blocks, W-22 reader. 9: W-20 file kinds (scripts, CSV, calendars and contacts read as plain text; SVG; file flags)
6
6
  export const MAX_URL = 2048; // longer URLs are omitted (and counted), never shortened;
7
7
  // Points at Raibit's page on raibit.ai, which links to the GitHub repo once it is public; it stays (Travis,
8
8
  // SPEC section 9, 2026-11-01).
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@raibitsi/tiresias",
3
- "version": "1.0.0",
3
+ "version": "1.0.1",
4
4
  "description": "The honest browser for AI agents. The browser you never open.",
5
5
  "license": "Apache-2.0",
6
6
  "author": "Raibit (https://raibit.ai/)",
@@ -33,6 +33,7 @@
33
33
  "!dist/test",
34
34
  "stubs",
35
35
  "README.md",
36
+ "CHANGELOG.md",
36
37
  "docs/QUICKSTART.md",
37
38
  "docs/DEMO.md",
38
39
  "LICENSE",