@better-fs-tools/read 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -18,7 +18,7 @@ Most hosts also install one adapter package. Each adapter lists its own peers:
18
18
  | -------------------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------- |
19
19
  | `@better-fs-tools/read` | The core, the helpers, and the `./signature` and `./formats` subpaths | `npm install @better-fs-tools/read` |
20
20
  | `@better-fs-tools/fs` | The `FileSystem` contract, `memoryFileSystem`, and the conformance suite | `npm install @better-fs-tools/fs` |
21
- | `@better-fs-tools/node` | Local reads on Node 24 or Bun, and `createNodeReadTool()` | `npm install @better-fs-tools/node @better-fs-tools/read` |
21
+ | `@better-fs-tools/node` | The local filesystem on Node 24 or Bun, and `nodeDigest()` | `npm install @better-fs-tools/node @better-fs-tools/read` |
22
22
  | `@better-fs-tools/ai-sdk` | An AI SDK 7 tool | `npm install @better-fs-tools/ai-sdk @better-fs-tools/read ai` |
23
23
  | `@better-fs-tools/pi` | A Pi tool and a Pi extension | `npm install @better-fs-tools/pi @better-fs-tools/read @earendil-works/pi-coding-agent typebox` |
24
24
  | `@better-fs-tools/cloudflare-shell` | A filesystem over a Cloudflare Shell Workspace | `npm install @better-fs-tools/cloudflare-shell @better-fs-tools/read` |
@@ -29,14 +29,14 @@ All packages have the same version. Install matching versions.
29
29
 
30
30
  ## Quick start
31
31
 
32
- On Node or Bun, `createNodeReadTool()` gives a working local reader with no options:
32
+ On Node or Bun, give `createReadTool()` a `nodeFileSystem()`:
33
33
 
34
34
  ```ts
35
- import { createNodeReadTool } from "@better-fs-tools/node";
36
- import { textOf } from "@better-fs-tools/read";
35
+ import { nodeFileSystem } from "@better-fs-tools/node";
36
+ import { createReadTool, textOf } from "@better-fs-tools/read";
37
37
 
38
38
  // Rooted at process.cwd(), with SHA-256 observations.
39
- const read = createNodeReadTool();
39
+ const read = createReadTool({ fs: nodeFileSystem({ allowedRoots: [process.cwd()] }) });
40
40
 
41
41
  const result = await read({ path: "package.json", limit: 2 });
42
42
  console.log(result.status); // "ok"
@@ -47,8 +47,6 @@ console.log(textOf(result));
47
47
  // [read:continue] Output stopped at the line limit. Continue with {"path":"package.json","offset":3,"limit":2}.
48
48
  ```
49
49
 
50
- `createNodeReadTool()` uses `nodeFileSystem` rooted at `process.cwd()`, and `nodeDigest()` for content hashes. It passes every other option to `createReadTool()`.
51
-
52
50
  `createReadTool()` is the core. It needs a filesystem, and it works with any `FileSystem`:
53
51
 
54
52
  ```ts
@@ -118,38 +116,39 @@ read({ path, offset, limit }, ctx)
118
116
  ReadResult
119
117
  ```
120
118
 
121
- You cannot remove the input validation, the scan, the limits, the type check, or change detection. Every other step is a dependency that you can replace. [docs/architecture.md](docs/architecture.md) explains each stage.
119
+ You cannot remove the input validation, the scan, the limits, the type check, or change detection. Every other step is an option that you can replace. [docs/architecture.md](docs/architecture.md) explains each stage.
122
120
 
123
121
  ## What you can change
124
122
 
125
- Pass dependencies to `createReadTool()`, `createNodeReadTool()`, or an adapter factory (`createAiSdkReadTool()`, `createPiReadTool()`). The adapter passes them to the core.
126
-
127
- | Dimension | Dependency | Default |
128
- | ------------------------------------------------------ | --------------------------- | ------------------------------------------------------------------------------------------------- |
129
- | Tool name, parameters, range model, descriptions | `signature` (adapters only) | `defaultReadSignature()`: `read` with `path`, `offset`, `limit` |
130
- | Filesystem, allowed roots, deny roots, symlinks | `fs` | Required. `createNodeReadTool()` uses `nodeFileSystem` over `process.cwd()` |
131
- | Path rewrites (`~`, `file://`, `@`, Unicode repair) | `resolve` | `null` (the path is used as given) |
132
- | Names to suggest when a file is missing | `suggest` | `defaultSuggest()` |
133
- | Per-path policy, size ceiling, user approval | `authorize` | `null` (allow) |
134
- | Binary and format detection | `classifiers` | `defaultClassifiers()` |
135
- | File conversion, images, directories | `converters` | `[]` |
136
- | Line, byte, scan, conversion, and media limits | `limits` | `defaultReadLimits` |
137
- | Token limit | `budget` | `null` |
138
- | Redaction, repeat-read guard, your own after-read code | `hooks` | `[]` |
139
- | Output text or content parts | `formatter` | `lineNumberFormatter()` |
140
- | Note and message wording | `messages` | `defaultReadMessages` |
141
- | Read records for later tools | `state` | `null` |
142
- | Content hashes and observations | `digest` | `null`. `createNodeReadTool()` and the Pi tool use `nodeDigest()`. `sha256Digest()` runs anywhere |
143
- | Timestamps | `clock` | `() => new Date()` |
144
-
145
- `limits` and `messages` merge over their defaults key by key. Every other dependency replaces its default. To add to a list, include the default yourself:
123
+ Pass options to `createReadTool()`, `readToolDefinition()`, or an integration factory (`createAiSdkReadTool()`, `createPiReadTool()`, `createAgentCoreReadTool()`). Every integration takes the same options with the same defaults, and passes them to the core.
124
+
125
+ | Dimension | Option | Default |
126
+ | ------------------------------------------------------ | --------------------------- | -------------------------------------------------------------------------------------------- |
127
+ | Tool name, parameters, range model, descriptions | `signature` (adapters only) | `defaultReadSignature()`: `read` with `path`, `offset`, `limit` |
128
+ | Filesystem, allowed roots, deny roots, symlinks | `fs` | Required. `nodeFileSystem()` from `@better-fs-tools/node` reads the local disk |
129
+ | Path rewrites (`~`, `file://`, `@`, Unicode repair) | `resolve` | `null` (the path is used as given) |
130
+ | Names to suggest when a file is missing | `suggest` | `defaultSuggest()` |
131
+ | Per-path policy, size ceiling, user approval | `authorize` | `null` (allow) |
132
+ | Binary and format detection | `classifiers` | `defaultClassifiers()` |
133
+ | File conversion, images, directories | `converters` | `[imageConverter()]`: an image becomes a media part. `[]` turns images off |
134
+ | Line, byte, scan, conversion, and media limits | `limits` | `defaultReadLimits` |
135
+ | Token limit | `budget` | `null` |
136
+ | Redaction, repeat-read guard, your own after-read code | `hooks` | `[]` |
137
+ | Output text or content parts | `formatter` | `lineNumberFormatter()` |
138
+ | Note and message wording | `messages` | `defaultReadMessages` |
139
+ | Read records for later tools | `state` | `null` |
140
+ | Content hashes and observations | `digest` | `sha256Digest()`, which runs anywhere. `nodeDigest()` is faster on Node. `null` turns it off |
141
+ | Timestamps | `clock` | `() => new Date()` |
142
+
143
+ `limits` and `messages` merge over their defaults key by key. Every other option replaces its default. To add to a list, include the default yourself:
146
144
 
147
145
  ```ts
148
- import { createNodeReadTool } from "@better-fs-tools/node";
149
- import { defaultClassifiers, extensionClassifier } from "@better-fs-tools/read";
146
+ import { nodeFileSystem } from "@better-fs-tools/node";
147
+ import { createReadTool, defaultClassifiers, extensionClassifier } from "@better-fs-tools/read";
150
148
 
151
- export const read = createNodeReadTool({
152
- // A list dependency replaces the default. Include the default to add to it.
149
+ export const read = createReadTool({
150
+ fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
151
+ // A list option replaces the default. Include the default to add to it.
153
152
  classifiers: [
154
153
  extensionClassifier({ unsupported: { ".parquet": { code: "BINARY" } } }),
155
154
  ...defaultClassifiers(),
@@ -160,7 +159,7 @@ export const read = createNodeReadTool({
160
159
  });
161
160
  ```
162
161
 
163
- `createReadTool()` checks the dependencies once, when you call it. An unknown key, an empty classifier list, a limit that is not a positive safe integer, or a message that is not a function throws `TypeError`.
162
+ `createReadTool()` checks the options once, when you call it. An unknown key, an empty classifier list, a limit that is not a positive safe integer, or a message that is not a function throws `TypeError`.
164
163
 
165
164
  ## Tool signature and docs
166
165
 
@@ -172,7 +171,7 @@ The model sees a tool name, a description, a parameter schema, and parameter des
172
171
  - `param(name)` gives the host name of a canonical parameter, or `""` when the signature has none.
173
172
  - `fromInput(retry)` maps a canonical retry back to the model's names. Notes use it when they tell the model how to continue. Only the read signature has it.
174
173
 
175
- Two builders cover the common cases. Both take the shared `SignatureDocs` options: `name`, `description`, `describe`, and `names`. `describe` and `names` use the builder's own parameter names as keys.
174
+ Two builders make the signature. `defaultReadSignature()` gives `read({ path, offset?, limit? })`. `lineRangeSignature()` gives `read({ path, start?, end? })`, with one-based and inclusive lines. Both take the shared `SignatureDocs` options: `name`, `description`, `describe`, and `names`. `describe` and `names` use the builder's own parameter names as keys.
176
175
 
177
176
  ```ts
178
177
  import { defaultReadSignature, lineRangeSignature } from "@better-fs-tools/read/signature";
@@ -196,7 +195,9 @@ export const lineRange = lineRangeSignature({
196
195
  });
197
196
  ```
198
197
 
199
- With `lineRangeSignature`, a truncated read tells the model to continue with `{"file_path":"src/a.ts","start_line":101,"end_line":200}`. The structured result stays canonical: `result.continuation.next` is still `{ path, offset, limit }`.
198
+ With `renamed`, a truncated read tells the model to continue with `{"file_path":"src/a.ts","start":101,"max_lines":100}`. With `lineRange`, it says `{"file_path":"src/a.ts","start_line":101,"end_line":200}`. The structured result stays canonical: `result.continuation.next` is still `{ path, offset, limit }`.
199
+
200
+ For another input shape, write your own `ReadSignature`. It needs `name`, `description`, `schema`, `toInput`, `param`, and `fromInput`.
200
201
 
201
202
  `createAiSdkReadTool()` and `createPiReadTool()` wire the retry wording for you with `readSignatureMessages(signature)`. If you call the core yourself behind a signature, pass `messages: readSignatureMessages(signature)`. Otherwise the retry text uses `path`, `offset`, and `limit`.
202
203
 
@@ -210,13 +211,19 @@ The tool reads one path for each call. For multi-file reads or globs, call the c
210
211
 
211
212
  ```ts
212
213
  import { homedir } from "node:os";
213
- import { createNodeReadTool, nodeFileSystem } from "@better-fs-tools/node";
214
- import { expandHome, pathResolvers, stripPrefixes, unicodeRepair } from "@better-fs-tools/read";
214
+ import { nodeFileSystem } from "@better-fs-tools/node";
215
+ import {
216
+ createReadTool,
217
+ expandHome,
218
+ pathResolvers,
219
+ stripPrefixes,
220
+ unicodeRepair,
221
+ } from "@better-fs-tools/read";
215
222
 
216
223
  const cwd = process.cwd();
217
224
  const home = homedir();
218
225
 
219
- export const read = createNodeReadTool({
226
+ export const read = createReadTool({
220
227
  fs: nodeFileSystem({ cwd, allowedRoots: [cwd, `${home}/.config/app`] }),
221
228
  resolve: pathResolvers(
222
229
  stripPrefixes(), // "file:///x" and "@src/x"
@@ -285,13 +292,20 @@ The filesystem owns the root policy: allowed roots, deny roots, and symlinks. `a
285
292
  What `open()` has fetched before `authorize` runs depends on the backend. `@better-fs-tools/cloudflare-shell` and `@better-fs-tools/just-bash` buffer the whole file inside `open()`, so those bytes have left the backend. `@better-fs-tools/cloudflare-computer` does not buffer: its `open()` starts a `readFile` stream, so the request reaches the backend, but it reads no chunk before `authorize`. `nodeFileSystem` streams, and `memoryFileSystem` already holds the bytes. In every case the core passes no byte to a classifier, a converter, or the model until `authorize` allows it. If a denied read must not reach the backend, refuse the path in the filesystem with a deny root, or leave it out of the allowed roots.
286
293
 
287
294
  ```ts
288
- import { createNodeReadTool } from "@better-fs-tools/node";
289
- import { askUser, readAuthorizers, denyPaths, sizeCeiling } from "@better-fs-tools/read";
295
+ import { nodeFileSystem } from "@better-fs-tools/node";
296
+ import {
297
+ askUser,
298
+ createReadTool,
299
+ denyPaths,
300
+ readAuthorizers,
301
+ sizeCeiling,
302
+ } from "@better-fs-tools/read";
290
303
 
291
304
  // Your own prompt, for example a dialog in your UI.
292
305
  declare function confirmInUi(question: string): Promise<boolean>;
293
306
 
294
- export const read = createNodeReadTool({
307
+ export const read = createReadTool({
308
+ fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
295
309
  authorize: readAuthorizers(
296
310
  denyPaths(["**/.env", "**/.env.*", "**/*.pem"]),
297
311
  sizeCeiling({ maxBytes: 256 * 1024, unrangedOnly: true }),
@@ -375,8 +389,9 @@ A classifier only decides what a file is. A converter changes the content.
375
389
  `converters` is a list. It runs after classification. The first file converter whose `accepts` returns `true` handles the file. The first directory converter handles directories. With no converters, text is read, other formats are `unsupported`, and a directory gives `NOT_A_FILE`. The core has no converters by default. The Pi and AI SDK hosts add `imageConverter()` when you pass no `converters`.
376
390
 
377
391
  ```ts
378
- import { createNodeReadTool } from "@better-fs-tools/node";
392
+ import { nodeFileSystem } from "@better-fs-tools/node";
379
393
  import {
394
+ createReadTool,
380
395
  defaultClassifiers,
381
396
  directoryListing,
382
397
  imageConverter,
@@ -388,7 +403,8 @@ import {
388
403
  // Your own PDF extraction, for example a pdftotext process that reads stdin.
389
404
  declare function pdfToText(source: AsyncIterable<Uint8Array>): AsyncIterable<string>;
390
405
 
391
- export const read = createNodeReadTool({
406
+ export const read = createReadTool({
407
+ fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
392
408
  // The default chain reads a notebook as JSON text. notebookClassifier()
393
409
  // marks it NOTEBOOK, so notebookConverter() can take it.
394
410
  classifiers: [notebookClassifier(), ...defaultClassifiers()],
@@ -453,10 +469,11 @@ Every tool has one parse helper with the same pattern: `parseReadInput`, `parseE
453
469
  For a token limit, set `budget`. The scanner stops at the last whole line that fits, and the result has a normal continuation. The truncation reason is `"budget"`.
454
470
 
455
471
  ```ts
456
- import { createNodeReadTool } from "@better-fs-tools/node";
457
- import { charsPerToken } from "@better-fs-tools/read";
472
+ import { nodeFileSystem } from "@better-fs-tools/node";
473
+ import { charsPerToken, createReadTool } from "@better-fs-tools/read";
458
474
 
459
- export const read = createNodeReadTool({
475
+ export const read = createReadTool({
476
+ fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
460
477
  limits: { maxLines: 1_000, maxViewBytes: 64 * 1024 },
461
478
  budget: charsPerToken({ ratio: 4, max: 25_000 }),
462
479
  });
@@ -471,11 +488,12 @@ For a size ceiling that refuses a whole file, use the `sizeCeiling` authorizer.
471
488
  `hooks` is an ordered list of `afterRead` functions. They run after the scan and change detection, and before the result is recorded and formatted.
472
489
 
473
490
  ```ts
474
- import { createNodeReadTool } from "@better-fs-tools/node";
475
- import { memoryStore, redact, repeatReadGuard } from "@better-fs-tools/read";
491
+ import { nodeFileSystem } from "@better-fs-tools/node";
492
+ import { createReadTool, memoryStore, redact, repeatReadGuard } from "@better-fs-tools/read";
476
493
 
477
- // createNodeReadTool sets digest to nodeDigest(), which repeatReadGuard needs.
478
- export const read = createNodeReadTool({
494
+ // repeatReadGuard needs a digest: the default sha256Digest() is one.
495
+ export const read = createReadTool({
496
+ fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
479
497
  state: memoryStore(),
480
498
  hooks: [repeatReadGuard(), redact({ patterns: [/AKIA[0-9A-Z]{16}/g] })],
481
499
  });
@@ -513,7 +531,7 @@ export const agentsNote: ReadHook<unknown> = {
513
531
  };
514
532
  ```
515
533
 
516
- A repeat read that the guard empties is recorded as it was shown, with `wholeFileVisible: false`. Redaction works line by line, so a secret split across two lines is not matched.
534
+ A repeat read that the guard empties has `observation.wholeFileVisible: false`. The stored record keeps the earlier read's `wholeFileVisible` and line ranges when the content is the same. Redaction works line by line, so a secret split across two lines is not matched.
517
535
 
518
536
  ## Output format
519
537
 
@@ -542,15 +560,17 @@ Its options:
542
560
  Helpers for common formats:
543
561
 
544
562
  ```ts
545
- import { createNodeReadTool } from "@better-fs-tools/node";
563
+ import { nodeFileSystem } from "@better-fs-tools/node";
546
564
  import {
565
+ createReadTool,
547
566
  eofFooter,
548
567
  fileHashHeader,
549
568
  hashlineGutter,
550
569
  lineNumberFormatter,
551
570
  } from "@better-fs-tools/read";
552
571
 
553
- export const read = createNodeReadTool({
572
+ export const read = createReadTool({
573
+ fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
554
574
  formatter: lineNumberFormatter({
555
575
  gutter: hashlineGutter({ width: 2 }), // "12:a3|text"
556
576
  header: fileHashHeader(), // "file-hash: sha256:..."
@@ -569,10 +589,14 @@ export const read = createNodeReadTool({
569
589
  Presets that copy the read output of other harnesses are in `@better-fs-tools/read/formats`: `opencodeFormat()`, `deepAgentsFormat()`, `hashlineFormat()`, and `hermesFormat()`. They take no options.
570
590
 
571
591
  ```ts
572
- import { createNodeReadTool } from "@better-fs-tools/node";
592
+ import { nodeFileSystem } from "@better-fs-tools/node";
593
+ import { createReadTool } from "@better-fs-tools/read";
573
594
  import { opencodeFormat } from "@better-fs-tools/read/formats";
574
595
 
575
- export const read = createNodeReadTool({ formatter: opencodeFormat() });
596
+ export const read = createReadTool({
597
+ fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
598
+ formatter: opencodeFormat(),
599
+ });
576
600
  ```
577
601
 
578
602
  A custom formatter implements `format(outcome, ctx)`. It is synchronous and pure over the outcome. `ctx` has the `digest`, the `limits`, the `call`, and a `mode`. The mode is `"model"` for model output and `"view"` for the body only, with no notes, header, or footer. The Pi adapter uses `"view"` for its truncation details. If your formatter returns parts in `"view"` mode, Pi leaves those details out. A formatter sees only the finished outcome. It cannot read the file. If the formatter throws or returns neither a string nor an array, the call still succeeds: the core formats the result with `lineNumberFormatter()` and adds no note.
@@ -609,7 +633,7 @@ No default message names `offset` or `limit`. Every message that suggests a retr
609
633
  | `unsupported` | A classifier or converter refused the format. `code` is open, for example `PDF` or `TOO_LARGE`. |
610
634
  | `error` | `error.code` is one of thirteen `ReadErrorCode` values. |
611
635
 
612
- The error codes are `INVALID_INPUT`, `NOT_FOUND`, `NOT_A_FILE`, `DANGEROUS_PATH`, `OUTSIDE_ALLOWED_ROOTS`, `PERMISSION_DENIED`, `DENIED`, `TOO_LARGE`, `CHANGED_DURING_READ`, `ABORTED`, `UNSUPPORTED_BACKEND`, `EXTENSION_FAILED`, and `IO_ERROR`. `TOO_LARGE` is an `error` when a backend byte ceiling refuses the file, and an `unsupported` code when a converter or media limit stops it. `EXTENSION_FAILED` means that host code threw or broke a rule. Its note data names the dependency and the stage, for example `{ extension: "hooks", phase: "hooks", id: "redact" }`.
636
+ The error codes are `INVALID_INPUT`, `NOT_FOUND`, `NOT_A_FILE`, `DANGEROUS_PATH`, `OUTSIDE_ALLOWED_ROOTS`, `PERMISSION_DENIED`, `DENIED`, `TOO_LARGE`, `CHANGED_DURING_READ`, `ABORTED`, `UNSUPPORTED_BACKEND`, `EXTENSION_FAILED`, and `IO_ERROR`. `TOO_LARGE` is an `error` when a backend byte ceiling refuses the file, and an `unsupported` code when a converter or media limit stops it. `EXTENSION_FAILED` means that host code threw or broke a rule. Its note data names the option and the stage, for example `{ extension: "hooks", phase: "hooks", id: "redact" }`.
613
637
 
614
638
  An `ok` result has structured lines, and an `error` result has its error:
615
639
 
@@ -676,16 +700,16 @@ await read({ path: "/a.txt" }, { host: session, callId: "call-1" });
676
700
 
677
701
  - `fs` can be a function of the call. It runs once for each read, before `resolve`. Use it to choose a backend for each call.
678
702
  - `sha256Digest()` is SHA-256 in plain JavaScript. It is synchronous and needs no Node module, so it works in a Cloudflare Worker. Its id and its values (`sha256:<hex>`) are the same as `nodeDigest()`, so records from either one match.
679
- - `state` needs a `digest`, because a record names the digest that made it. `createReadTool` throws `TypeError` for a `state` without a `digest`, and the dependency type (`StateNeedsDigest`) refuses it at compile time. `StateNeedsDigestOrDefault` is the same rule for a factory with its own default digest, such as the Node and Pi factories: leave `digest` out for the default, or pass `digest: null` with no `state`. The bundles (`createFsTools()` and the host bundles that wrap it) always have a digest, so they need neither.
703
+ - `state` needs a `digest`, because a record names the digest that made it. `digest` defaults to `sha256Digest()`, so a `state` alone works. `createReadTool` throws `TypeError` for a `state` with `digest: null`, and the options type (`StateNeedsDigest`) refuses it at compile time.
680
704
  - `state` can be a function of the call. It runs at most once for each read, and only when the core needs the store. Return `null` for no store. A factory that throws gives `EXTENSION_FAILED`. A store whose `get` or `put` fails never fails the read.
681
- - The record key is `file.resolvedPath`. A read stores a `ReadRecord` with `schema: 2` and `origin: "read"`. It holds the backend `version`, the `digest` id, the content and view ids, and the read range. The write tools store records with `origin: "write"` and `request: null` after a commit. A record of another schema counts as absent.
705
+ - The record key is `file.resolvedPath`. A read stores a `ReadRecord` with `schema: 2` and `origin: "read"`. It holds the backend `version`, the `digest` id, the content and view ids, the read range, and `seenLines`. `seenLines` joins the ranges that reads of the same content showed in full. When they run from line 1 to the last line, the record is `wholeFileVisible`, so reads in ranges count as a read of the whole file. The write tools store records with `origin: "write"` and `request: null` after a commit. A record of another schema counts as absent.
682
706
  - `result.file.version` is the backend's change token from `open()`, or `null`. It is kept when the backend has no identity capability.
683
707
  - `memoryStore()` from `@better-fs-tools/read` keeps records in memory, with a size cap and a TTL. Its `clock` option is the same `Clock` (`() => Date`) that the tools take, so one fake clock fits both. It sits on the root of the package that owns `ReadStateStore`, as `memoryLocks()` sits on the root of `@better-fs-tools/write`, which owns `LockManager`.
684
708
  - The helpers in this package are typed with `unknown` for the host, so they fit a tool with any host type. `askUser`, `imageConverter`, and `textConverter` can take your host type, as `askUser<Session>` does above.
685
709
 
686
710
  ## Security guarantees
687
711
 
688
- Your dependencies can narrow access. They cannot widen it.
712
+ Your options can narrow access. They cannot widen it.
689
713
 
690
714
  - The filesystem is the only place that grants access. It checks roots, deny roots, the realpath, and the type in one `open()` call. `resolve` runs before it and `authorize` runs after it.
691
715
  - Each read opens at most one file. A resolver only changes the path that goes into that open.
@@ -697,15 +721,18 @@ Your dependencies can narrow access. They cannot widen it.
697
721
  - A hook cannot turn a failure into a success. A hook that changes the view marks the observation as partial.
698
722
  - A `DENIED` result does not show the file behind the path.
699
723
 
700
- ## Host adapters
724
+ ## Integrations
725
+
726
+ Each integration takes the options of `readToolDefinition()`: every core option, with the same default, plus `signature`. `fs` is required in every one. The integration owns abort forwarding and the mapping from `result.content` to the framework's content types.
701
727
 
702
- Each adapter owns the signature, abort forwarding, and the mapping from `result.content` to the framework's content types. The adapters accept every core dependency and pass it through.
728
+ | Package | Factory | Notes |
729
+ | ------------------------------------------------------------------------------------------ | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------ |
730
+ | [`@better-fs-tools/ai-sdk`](https://www.npmjs.com/package/@better-fs-tools/ai-sdk) | `createAiSdkReadTool(options)` | `execute` returns the whole `ReadResult`. `toModelOutput` maps media parts to AI SDK `file` parts. |
731
+ | [`@better-fs-tools/pi`](https://www.npmjs.com/package/@better-fs-tools/pi) | `createPiReadTool(options)` | Adds the Pi prompt options. `piCwdFileSystem()` from `@better-fs-tools/pi/node` roots each call at Pi's `ctx.cwd`. |
732
+ | [`@better-fs-tools/pi`](https://www.npmjs.com/package/@better-fs-tools/pi) | `createAgentCoreReadTool(options)` from `./agent-core` | An `AgentTool` for `@earendil-works/pi-agent-core`. |
733
+ | [`@better-fs-tools/pi-durable`](https://www.npmjs.com/package/@better-fs-tools/pi-durable) | `createDurableFsTools(options)` | A bundle only: the tools register inside a pi-durable extension. |
703
734
 
704
- | Package | Factory | Notes |
705
- | ---------------------------------------------------------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
706
- | [`@better-fs-tools/ai-sdk`](https://www.npmjs.com/package/@better-fs-tools/ai-sdk) | `createAiSdkReadTool(options)` | `execute` returns the whole `ReadResult`. `toModelOutput` maps media parts to AI SDK `file` parts. You pass `fs`. |
707
- | [`@better-fs-tools/pi`](https://www.npmjs.com/package/@better-fs-tools/pi) | `createPiReadTool(options)` | Each call is confined to Pi's `ctx.cwd`. `fs`, `cwd`, and `allowedRoots` are refused. The `pi.extensions` entry registers `read`, `edit`, and `write`. |
708
- | [`@better-fs-tools/node`](https://www.npmjs.com/package/@better-fs-tools/node) | `createNodeReadTool(options?)` | The core with Node defaults, for your own tool. |
735
+ For your own framework, call `readToolDefinition()` from `./signature`. It returns the name, description, and JSON Schema to declare, a `validate` for model input, and an `execute` that maps model input and runs the tool.
709
736
 
710
737
  Cloudflare Agents hosts use `@better-fs-tools/ai-sdk` with `cloudflareShellFileSystem` from [`@better-fs-tools/cloudflare-shell`](https://www.npmjs.com/package/@better-fs-tools/cloudflare-shell). [`@better-fs-tools/cloudflare-computer`](https://www.npmjs.com/package/@better-fs-tools/cloudflare-computer) and [`@better-fs-tools/just-bash`](https://www.npmjs.com/package/@better-fs-tools/just-bash) are filesystems. Use them with `createReadTool()` or with an adapter.
711
738
 
@@ -730,7 +757,8 @@ This Pi host sets most dimensions. You will usually need only a few of these opt
730
757
  ```ts
731
758
  import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
732
759
  import { createPiReadTool } from "@better-fs-tools/pi";
733
- import { lineRangeSignature } from "@better-fs-tools/read/signature";
760
+ import { piCwdFileSystem } from "@better-fs-tools/pi/node";
761
+ import { defaultReadSignature } from "@better-fs-tools/read/signature";
734
762
  import {
735
763
  askUser,
736
764
  denyPaths,
@@ -751,9 +779,9 @@ import {
751
779
  const stores = new Map<string, ReturnType<typeof memoryStore>>();
752
780
 
753
781
  export const read = createPiReadTool({
754
- signature: lineRangeSignature({
755
- name: "read",
756
- names: { path: "file_path", start: "start_line", end: "end_line" },
782
+ fs: piCwdFileSystem(),
783
+ signature: defaultReadSignature({
784
+ names: { path: "file_path", offset: "start_line", limit: "max_lines" },
757
785
  }),
758
786
  resolve: pathResolvers(stripPrefixes(), unicodeRepair({ note: false })),
759
787
  authorize: readAuthorizers(
@@ -107,15 +107,43 @@ export interface ToolSignature<TInput, TParam extends string = string> {
107
107
  * that names it.
108
108
  */
109
109
  param(name: TParam): string;
110
- /** Set on freeform signatures. Hosts that support grammar tools use it. */
110
+ /** A Lark grammar for hosts with grammar tools. freeformPatchSignature() sets CODEX_PATCH_GRAMMAR. */
111
111
  readonly grammar?: {
112
112
  readonly lark: string;
113
113
  };
114
114
  }
115
115
  /**
116
- * The options every signature preset takes. `TParam` is the preset's own
116
+ * A core tool bound to a signature: what a framework adapter needs to declare
117
+ * the tool and run it. The xxxToolDefinition() helpers in each `./signature`
118
+ * subpath build one, and every integration uses them.
119
+ */
120
+ export interface ToolDefinition<TInput, TResult, THost = undefined, TParam extends string = string> {
121
+ /** The tool name the model sees: `signature.name`. */
122
+ readonly name: string;
123
+ readonly description: string;
124
+ /** Plain JSON Schema of the model input: `signature.schema`. */
125
+ readonly inputSchema: JsonObject;
126
+ /** Set when the signature has a grammar. */
127
+ readonly grammar?: {
128
+ readonly lark: string;
129
+ };
130
+ readonly signature: ToolSignature<TInput, TParam>;
131
+ /**
132
+ * Checks model input with the signature and with the core input check at the
133
+ * tool's limits, and returns the canonical input. Throws TypeError that names
134
+ * the signature's parameters.
135
+ */
136
+ validate(modelInput: unknown): TInput;
137
+ /**
138
+ * Maps model input with `signature.toInput` and runs the core tool. Bad input
139
+ * rejects with TypeError. A tool error is a result, not a rejection.
140
+ */
141
+ execute(modelInput: unknown, ctx: ToolCallContext<THost>): Promise<TResult>;
142
+ }
143
+ /**
144
+ * The options every default signature takes. `TParam` is the signature's own
117
145
  * parameter names. `names` gives a parameter another name in the schema, and
118
- * `describe` replaces its description. Both are keyed by the preset's names.
146
+ * `describe` replaces its description. Both are keyed by the signature's names.
119
147
  */
120
148
  export interface SignatureDocs<TParam extends string> {
121
149
  /** The tool name. */
@@ -20,7 +20,10 @@ export interface ReadDependencies<THost = undefined> {
20
20
  readonly suggest: Suggest<THost> | null;
21
21
  /** Host access policy for the opened file ("read") and before every listing ("list"). null allows everything. */
22
22
  readonly authorize: ReadAuthorizer<THost> | null;
23
- /** File converters in order: the first that accepts runs. The first directory converter lists directories. */
23
+ /**
24
+ * File converters in order: the first that accepts runs. The first directory
25
+ * converter lists directories. Default [imageConverter()]. [] turns images off.
26
+ */
24
27
  readonly converters: readonly Converter<THost>[];
25
28
  /** Stops the view early, at a line boundary, when the next line would pass `max`. null has no budget. */
26
29
  readonly budget: ViewBudget | null;
@@ -28,32 +31,24 @@ export interface ReadDependencies<THost = undefined> {
28
31
  readonly hooks: readonly ReadHook<THost>[];
29
32
  /** A store, or a factory called at most once for each read, and only when needed. Needs `digest`. */
30
33
  readonly state: ReadStateStore | ((call: ReadContext<THost>) => ReadStateStore | null) | null;
34
+ /** Names content for the read state and the observations. Default sha256Digest(). null turns both off. */
31
35
  readonly digest: Digest | null;
32
36
  readonly clock: Clock;
33
37
  readonly formatter: ReadFormatter<THost>;
34
38
  }
35
39
  /**
36
40
  * `state` needs a `digest`, because a stored record names the digest that made
37
- * it. `state` without `digest` does not type-check, and throws TypeError at run
38
- * time.
41
+ * it. Leave `digest` out to get sha256Digest(). `digest: null` turns it off,
42
+ * and then `state` must be left out or null. A `state` with `digest: null`
43
+ * does not type-check, and throws TypeError at run time.
39
44
  */
40
45
  export type StateNeedsDigest = {
41
46
  readonly state?: null;
42
- } | {
43
- readonly digest: Digest;
44
- };
45
- /**
46
- * The same pairing for a factory with its own default digest, such as the
47
- * Node and Pi factories. Leave `digest` out to get the default. `digest: null`
48
- * turns it off, and then `state` must be left out or null.
49
- */
50
- export type StateNeedsDigestOrDefault = {
51
- readonly state?: null;
52
47
  readonly digest: null;
53
48
  } | {
54
49
  readonly digest?: Digest;
55
50
  };
56
- export type ReadToolDeps<THost = undefined> = {
51
+ export type ReadToolOptions<THost = undefined> = {
57
52
  readonly fs: ReadDependencies<THost>["fs"];
58
53
  readonly limits?: Partial<ReadLimits>;
59
54
  readonly messages?: Partial<ReadMessageCatalog>;
@@ -150,4 +150,14 @@ export interface ReadObservation {
150
150
  readonly observedAt: string;
151
151
  /** True when offset is 1, nothing was truncated or clamped, and no hook edited the view. */
152
152
  readonly wholeFileVisible: boolean;
153
+ /**
154
+ * The source lines that the view showed in full. null when the view showed no
155
+ * line, clamped a line, came from a converter, or was edited by a hook.
156
+ */
157
+ readonly visibleLines: ReadLineRange | null;
158
+ }
159
+ /** An inclusive range of 1-based source line numbers. */
160
+ export interface ReadLineRange {
161
+ readonly start: number;
162
+ readonly end: number;
153
163
  }
@@ -1,3 +1,4 @@
1
+ import type { ReadLineRange } from "./result.ts";
1
2
  /**
2
3
  * Session state for read observations. A store is a cache: a failed store call
3
4
  * never fails a read.
@@ -26,7 +27,16 @@ export interface ReadRecord {
26
27
  /** For a write record: equal to contentId. */
27
28
  readonly viewId: string;
28
29
  readonly observedAt: string;
30
+ /**
31
+ * True when one read showed the whole file, or when the reads of this
32
+ * contentId together showed every line.
33
+ */
29
34
  readonly wholeFileVisible: boolean;
35
+ /**
36
+ * The lines of this contentId that reads showed in full, sorted and joined.
37
+ * Empty for a write record.
38
+ */
39
+ readonly seenLines: readonly ReadLineRange[];
30
40
  readonly totalsExact: boolean;
31
41
  /** The read range. null for a write record. */
32
42
  readonly request: {
@@ -126,6 +126,7 @@ function mediaOutcome(input, classification, parts, notes, source) {
126
126
  observedAt: deps.clock().toISOString(),
127
127
  // Media is not source text, so it cannot back a text edit.
128
128
  wholeFileVisible: false,
129
+ visibleLines: null,
129
130
  }),
130
131
  notes,
131
132
  };
@@ -1,7 +1,7 @@
1
1
  import type { ReadTool } from "../contract/context.ts";
2
- import type { ReadToolDeps, StateNeedsDigest } from "../contract/deps.ts";
2
+ import type { ReadToolOptions, StateNeedsDigest } from "../contract/deps.ts";
3
3
  /**
4
- * Creates the read tool. Throws TypeError on an unknown or malformed dependency,
4
+ * Creates the read tool. Throws TypeError on an unknown or malformed option,
5
5
  * a missing `fs`, an empty `classifiers` list, or a `state` without a `digest`.
6
6
  */
7
- export declare function createReadTool<THost = undefined>(deps: ReadToolDeps<THost> & StateNeedsDigest): ReadTool<THost>;
7
+ export declare function createReadTool<THost = undefined>(options: ReadToolOptions<THost> & StateNeedsDigest): ReadTool<THost>;
@@ -2,11 +2,11 @@ import { resolveDependencies } from "./deps.js";
2
2
  import { isRecord } from "./input.js";
3
3
  import { runRead } from "./pipeline.js";
4
4
  /**
5
- * Creates the read tool. Throws TypeError on an unknown or malformed dependency,
5
+ * Creates the read tool. Throws TypeError on an unknown or malformed option,
6
6
  * a missing `fs`, an empty `classifiers` list, or a `state` without a `digest`.
7
7
  */
8
- export function createReadTool(deps) {
9
- const resolved = resolveDependencies(deps);
8
+ export function createReadTool(options) {
9
+ const resolved = resolveDependencies(options);
10
10
  const read = async (input, ctx) => {
11
11
  if (ctx !== undefined && !isRecord(ctx)) {
12
12
  throw new TypeError("read context must be an object");
@@ -1,6 +1,6 @@
1
1
  import type { FileSystem } from "@better-fs-tools/fs";
2
- import type { ReadDependencies, ReadToolDeps } from "../contract/deps.ts";
2
+ import type { ReadDependencies, ReadToolOptions } from "../contract/deps.ts";
3
3
  import type { ReadStateStore } from "../contract/state.ts";
4
- export declare function resolveDependencies<THost>(deps: ReadToolDeps<THost>): ReadDependencies<THost>;
4
+ export declare function resolveDependencies<THost>(deps: ReadToolOptions<THost>): ReadDependencies<THost>;
5
5
  export declare function isFileSystem(value: unknown): value is FileSystem;
6
6
  export declare function isStateStore(value: unknown): value is ReadStateStore;
package/dist/core/deps.js CHANGED
@@ -1,10 +1,12 @@
1
1
  import { defaultClassifiers } from "../classifiers/index.js";
2
+ import { imageConverter } from "../converters/index.js";
3
+ import { sha256Digest } from "../digest/sha256.js";
2
4
  import { lineNumberFormatter } from "../formatters/index.js";
3
5
  import { defaultSuggest } from "../suggest/index.js";
4
6
  import { isRecord } from "./input.js";
5
7
  import { resolveReadLimits } from "./limits.js";
6
8
  import { resolveReadMessages } from "./messages.js";
7
- const KNOWN_DEPENDENCY_KEYS = new Set([
9
+ const KNOWN_OPTION_KEYS = new Set([
8
10
  "fs",
9
11
  "limits",
10
12
  "messages",
@@ -22,10 +24,10 @@ const KNOWN_DEPENDENCY_KEYS = new Set([
22
24
  ]);
23
25
  export function resolveDependencies(deps) {
24
26
  if (!isRecord(deps))
25
- throw new TypeError("read tool dependencies must be an object");
27
+ throw new TypeError("read tool options must be an object");
26
28
  for (const key of Object.keys(deps)) {
27
- if (!KNOWN_DEPENDENCY_KEYS.has(key))
28
- throw new TypeError(`Unknown read tool dependency: ${key}`);
29
+ if (!KNOWN_OPTION_KEYS.has(key))
30
+ throw new TypeError(`Unknown read tool option: ${key}`);
29
31
  }
30
32
  const { fs } = deps;
31
33
  if (typeof fs !== "function" && !isFileSystem(fs)) {
@@ -51,7 +53,7 @@ export function resolveDependencies(deps) {
51
53
  if (authorize !== null && (!isRecord(authorize) || typeof authorize.authorize !== "function")) {
52
54
  throw new TypeError("authorize must be an authorizer or null");
53
55
  }
54
- const converters = deps.converters === undefined ? [] : deps.converters;
56
+ const converters = deps.converters === undefined ? [imageConverter()] : deps.converters;
55
57
  if (!Array.isArray(converters) || !converters.every(isConverter)) {
56
58
  throw new TypeError("converters must be an array of file and directory converters");
57
59
  }
@@ -70,7 +72,10 @@ export function resolveDependencies(deps) {
70
72
  if (state !== null && typeof state !== "function" && !isStateStore(state)) {
71
73
  throw new TypeError("state must be a store, a function that returns one, or null");
72
74
  }
73
- const digest = deps.digest ?? null;
75
+ const digest = deps.digest === undefined ? sha256Digest() : deps.digest;
76
+ if (digest !== null && !isDigest(digest)) {
77
+ throw new TypeError("digest must have an id, create, and hash, or be null");
78
+ }
74
79
  if (state !== null && digest === null) {
75
80
  throw new TypeError("state needs a digest: records name the digest that made them");
76
81
  }
@@ -113,6 +118,12 @@ function isViewBudget(value) {
113
118
  function isHook(value) {
114
119
  return isRecord(value) && typeof value.id === "string" && typeof value.afterRead === "function";
115
120
  }
121
+ function isDigest(value) {
122
+ return (isRecord(value) &&
123
+ typeof value.id === "string" &&
124
+ typeof value.create === "function" &&
125
+ typeof value.hash === "function");
126
+ }
116
127
  export function isStateStore(value) {
117
128
  return (isRecord(value) &&
118
129
  typeof value.get === "function" &&
@@ -1,11 +1,12 @@
1
1
  import type { Digest } from "../contract/digest.ts";
2
- import type { ContentPart, FileInfo, ReadObservation } from "../contract/result.ts";
2
+ import type { ContentPart, FileInfo, ReadLineRange, ReadObservation } from "../contract/result.ts";
3
3
  export interface ObservationInput {
4
4
  readonly file: FileInfo;
5
5
  readonly contentId: string | null;
6
6
  readonly view: string | readonly ContentPart[];
7
7
  readonly observedAt: string;
8
8
  readonly wholeFileVisible: boolean;
9
+ readonly visibleLines: ReadLineRange | null;
9
10
  }
10
11
  export declare function buildObservation(digest: Digest | null, input: ObservationInput): ReadObservation | null;
11
12
  export declare function withEditedView(digest: Digest, observation: ReadObservation, view: string | readonly ContentPart[]): ReadObservation;
@@ -16,6 +16,7 @@ export function buildObservation(digest, input) {
16
16
  viewId,
17
17
  observedAt: input.observedAt,
18
18
  wholeFileVisible: input.wholeFileVisible,
19
+ visibleLines: input.visibleLines,
19
20
  };
20
21
  }
21
22
  export function withEditedView(digest, observation, view) {
@@ -25,6 +26,7 @@ export function withEditedView(digest, observation, view) {
25
26
  id: observationId(digest, observation.statId, observation.contentId, viewId),
26
27
  viewId,
27
28
  wholeFileVisible: false,
29
+ visibleLines: null,
28
30
  };
29
31
  }
30
32
  function viewIdOf(digest, view) {
@@ -1,3 +1,10 @@
1
+ import { isRecord } from "./input.js";
2
+ import { coversLines, joinLineRanges, storedLineRanges } from "./seen-lines.js";
3
+ /**
4
+ * The record step of each read waits for the step before it on the same store
5
+ * and key, so parallel ranged reads of one file all join the stored ranges.
6
+ */
7
+ const pending = new WeakMap();
1
8
  export async function recordOutcome(scope, outcome) {
2
9
  if (outcome.status !== "ok" && outcome.status !== "media")
3
10
  return;
@@ -8,16 +15,62 @@ export async function recordOutcome(scope, outcome) {
8
15
  const store = scope.stateStore();
9
16
  if (store === null)
10
17
  return;
18
+ const key = outcome.file.resolvedPath;
19
+ await inTurn(store, key, async () => {
20
+ try {
21
+ const previous = await sameContentRecord(store, key, digest.id, observation.contentId);
22
+ await store.put(key, toRecord(outcome, observation, digest.id, previous));
23
+ }
24
+ catch {
25
+ // Session state is a cache: a failing store never fails the read.
26
+ }
27
+ });
28
+ }
29
+ async function inTurn(store, key, step) {
30
+ let keys = pending.get(store);
31
+ if (keys === undefined) {
32
+ keys = new Map();
33
+ pending.set(store, keys);
34
+ }
35
+ const before = keys.get(key) ?? Promise.resolve();
36
+ const current = before.then(step);
37
+ keys.set(key, current);
38
+ try {
39
+ await current;
40
+ }
41
+ finally {
42
+ if (keys.get(key) === current)
43
+ keys.delete(key);
44
+ }
45
+ }
46
+ /** The stored record, when it describes the same bytes with the same digest. */
47
+ async function sameContentRecord(store, key, digest, contentId) {
48
+ if (contentId === null)
49
+ return null;
50
+ let record;
11
51
  try {
12
- await store.put(outcome.file.resolvedPath, toRecord(outcome, observation, digest.id));
52
+ record = await store.get(key);
13
53
  }
14
54
  catch {
15
- // Session state is a cache: a failing store never fails the read.
55
+ return null;
16
56
  }
57
+ if (!isRecord(record) || record.schema !== 2 || record.digest !== digest)
58
+ return null;
59
+ return record.contentId === contentId ? record : null;
17
60
  }
18
61
  /** Copies fields one by one so nothing from the host `call` reaches the store. */
19
- function toRecord(outcome, observation, digest) {
62
+ function toRecord(outcome, observation, digest, previous) {
20
63
  const { file, request } = outcome;
64
+ const seenLines = joinLineRanges([
65
+ ...storedLineRanges(previous?.seenLines),
66
+ ...(observation.visibleLines === null ? [] : [observation.visibleLines]),
67
+ ]);
68
+ // Media is only returned from a whole source, so its source totals are exact.
69
+ const totalsExact = outcome.status === "ok" ? outcome.totals.exact : true;
70
+ const totalLines = outcome.status === "ok" ? outcome.totals.lines : null;
71
+ const wholeFileVisible = observation.wholeFileVisible ||
72
+ previous?.wholeFileVisible === true ||
73
+ (totalsExact && totalLines !== null && coversLines(seenLines, totalLines));
21
74
  return {
22
75
  schema: 2,
23
76
  origin: "read",
@@ -29,9 +82,9 @@ function toRecord(outcome, observation, digest) {
29
82
  contentId: observation.contentId,
30
83
  viewId: observation.viewId,
31
84
  observedAt: observation.observedAt,
32
- wholeFileVisible: observation.wholeFileVisible,
33
- // Media is only returned from a whole source, so its source totals are exact.
34
- totalsExact: outcome.status === "ok" ? outcome.totals.exact : true,
85
+ wholeFileVisible,
86
+ seenLines,
87
+ totalsExact,
35
88
  request: { offset: request.offset, limit: request.limit },
36
89
  };
37
90
  }
@@ -0,0 +1,7 @@
1
+ import type { ReadLineRange } from "../contract/result.ts";
2
+ /** Sorts the ranges and joins ranges that overlap or touch. */
3
+ export declare function joinLineRanges(ranges: readonly ReadLineRange[]): ReadLineRange[];
4
+ /** True when joined ranges show every line from 1 to totalLines. */
5
+ export declare function coversLines(joined: readonly ReadLineRange[], totalLines: number): boolean;
6
+ /** Reads ranges from a stored record. Anything malformed counts as no range. */
7
+ export declare function storedLineRanges(value: unknown): ReadLineRange[];
@@ -0,0 +1,39 @@
1
+ /** Sorts the ranges and joins ranges that overlap or touch. */
2
+ export function joinLineRanges(ranges) {
3
+ const sorted = [...ranges].sort((a, b) => a.start - b.start || a.end - b.end);
4
+ const joined = [];
5
+ for (const range of sorted) {
6
+ const last = joined.at(-1);
7
+ if (last !== undefined && range.start <= last.end + 1) {
8
+ joined[joined.length - 1] = { start: last.start, end: Math.max(last.end, range.end) };
9
+ }
10
+ else {
11
+ joined.push({ start: range.start, end: range.end });
12
+ }
13
+ }
14
+ return joined;
15
+ }
16
+ /** True when joined ranges show every line from 1 to totalLines. */
17
+ export function coversLines(joined, totalLines) {
18
+ if (totalLines === 0)
19
+ return true;
20
+ const first = joined[0];
21
+ return first !== undefined && first.start <= 1 && first.end >= totalLines;
22
+ }
23
+ /** Reads ranges from a stored record. Anything malformed counts as no range. */
24
+ export function storedLineRanges(value) {
25
+ if (!Array.isArray(value))
26
+ return [];
27
+ const ranges = [];
28
+ for (const item of value) {
29
+ if (item === null || typeof item !== "object")
30
+ return [];
31
+ const { start, end } = item;
32
+ if (!Number.isSafeInteger(start) || !Number.isSafeInteger(end))
33
+ return [];
34
+ if (start < 1 || end < start)
35
+ return [];
36
+ ranges.push({ start: start, end: end });
37
+ }
38
+ return ranges;
39
+ }
@@ -19,6 +19,11 @@ export function textOutcome(input) {
19
19
  !continuationAvailable &&
20
20
  !scanCapped &&
21
21
  scanner.clampedLines.length === 0;
22
+ const endLine = scanner.lines.length === 0 ? request.offset - 1 : request.offset + scanner.lines.length - 1;
23
+ // Converted lines are not source lines, so they cannot join a range of the source.
24
+ const visibleLines = conversion === null && scanner.lines.length > 0 && scanner.clampedLines.length === 0
25
+ ? { start: request.offset, end: endLine }
26
+ : null;
22
27
  return {
23
28
  tool: "read",
24
29
  status: "ok",
@@ -29,7 +34,7 @@ export function textOutcome(input) {
29
34
  view: {
30
35
  lines: scanner.lines,
31
36
  startLine: request.offset,
32
- endLine: scanner.lines.length === 0 ? request.offset - 1 : request.offset + scanner.lines.length - 1,
37
+ endLine,
33
38
  bytes: scanner.viewBytes,
34
39
  partial: request.offset !== 1 || continuationAvailable || reasons.length > 0,
35
40
  },
@@ -54,6 +59,7 @@ export function textOutcome(input) {
54
59
  view: scanner.viewText,
55
60
  observedAt: deps.clock().toISOString(),
56
61
  wholeFileVisible,
62
+ visibleLines,
57
63
  }),
58
64
  notes: [
59
65
  ...buildViewNotes({
package/dist/index.d.ts CHANGED
@@ -1,8 +1,8 @@
1
1
  export type { Classification, ClassificationSample, Classifier } from "./contract/classify.ts";
2
2
  export type { BuiltInUnsupportedCode, DefaultClassifierOptions, NoteContext, NoteOverride, TextClassification, UnsupportedClassification, } from "./contract/classify.ts";
3
- export type { AccessDecision, AccessTarget, Note, SignatureDocs, ToolAuthorizer, ToolCallContext, ToolError, ToolHookContext, ToolMessages, ToolName, ToolResolveContext, ToolSignature, } from "./contract/base.ts";
3
+ export type { AccessDecision, AccessTarget, Note, SignatureDocs, ToolAuthorizer, ToolCallContext, ToolDefinition, ToolError, ToolHookContext, ToolMessages, ToolName, ToolResolveContext, ToolSignature, } from "./contract/base.ts";
4
4
  export type { ReadContext, ReadTool } from "./contract/context.ts";
5
- export type { ReadDependencies, ReadToolDeps, StateNeedsDigest, StateNeedsDigestOrDefault, } from "./contract/deps.ts";
5
+ export type { ReadToolOptions, StateNeedsDigest } from "./contract/deps.ts";
6
6
  export type { Clock, Digest, DigestStream } from "./contract/digest.ts";
7
7
  export type { AfterReadContext, Converter, ConverterMatch, ConvertOutcome, DirectoryConverter, DirectoryConvertInput, FileConverter, FileConvertInput, PathResolver, ReadAuthorizeDecision, ReadAuthorizer, ReadAuthorizeTarget, ReadHook, ReadHookContext, ReadResolveContext, ResolveOutcome, Suggest, SuggestContext, ViewBudget, } from "./contract/extensions.ts";
8
8
  export type { ReadFormatContext, ReadFormatter } from "./contract/format.ts";
@@ -10,7 +10,7 @@ export type { ReadInput, ReadRequest } from "./contract/input.ts";
10
10
  export type { JsonObject, JsonValue } from "./contract/json.ts";
11
11
  export type { ReadLimits } from "./contract/limits.ts";
12
12
  export type { ReadMessageCatalog, ReadPhase } from "./contract/messages.ts";
13
- export type { ClassificationInfo, Confidence, ContentPart, ConversionInfo, FileInfo, MediaPart, ReadContinuation, ReadError, ReadErrorCode, ReadFailure, ReadLine, ReadMedia, ReadNote, ReadObservation, ReadOk, ReadReport, ReadResult, ReadTotals, ReadTruncation, ReadUnsupported, ReadView, TextPart, TruncationReason, } from "./contract/result.ts";
13
+ export type { ClassificationInfo, Confidence, ContentPart, ConversionInfo, FileInfo, MediaPart, ReadContinuation, ReadError, ReadErrorCode, ReadFailure, ReadLine, ReadLineRange, ReadMedia, ReadNote, ReadObservation, ReadOk, ReadReport, ReadResult, ReadTotals, ReadTruncation, ReadUnsupported, ReadView, TextPart, TruncationReason, } from "./contract/result.ts";
14
14
  export type { ReadRecord, ReadStateStore } from "./contract/state.ts";
15
15
  export { createReadTool } from "./core/create-read-tool.ts";
16
16
  export { parseReadInput } from "./core/input.ts";
@@ -0,0 +1,30 @@
1
+ import type { ToolDefinition } from "../contract/base.ts";
2
+ import type { ReadToolOptions, StateNeedsDigest } from "../contract/deps.ts";
3
+ import type { Digest } from "../contract/digest.ts";
4
+ import type { ReadFormatter } from "../contract/format.ts";
5
+ import type { ReadInput } from "../contract/input.ts";
6
+ import type { ReadLimits } from "../contract/limits.ts";
7
+ import type { ReadResult } from "../contract/result.ts";
8
+ import type { ReadParam, ReadSignature } from "./contract.ts";
9
+ /** The read tool's options, plus the signature the model sees. */
10
+ export type ReadToolDefinitionOptions<THost = undefined> = ReadToolOptions<THost> & StateNeedsDigest & {
11
+ /** Default defaultReadSignature(). */
12
+ readonly signature?: ReadSignature;
13
+ };
14
+ /**
15
+ * The read tool bound to its signature. `limits`, `formatter`, and `digest`
16
+ * are the ones the tool uses, for a host that formats the result again.
17
+ */
18
+ export interface ReadToolDefinition<THost = undefined> extends ToolDefinition<ReadInput, ReadResult, THost, ReadParam> {
19
+ readonly signature: ReadSignature;
20
+ readonly limits: Readonly<ReadLimits>;
21
+ readonly formatter: ReadFormatter<THost>;
22
+ readonly digest: Digest | null;
23
+ }
24
+ /**
25
+ * Creates the read tool and binds it to a signature, default
26
+ * defaultReadSignature(). The signature's retry text goes under
27
+ * `options.messages`, so a host message still wins. Throws TypeError where
28
+ * createReadTool() does.
29
+ */
30
+ export declare function readToolDefinition<THost = undefined>(options: ReadToolDefinitionOptions<THost>): ReadToolDefinition<THost>;
@@ -0,0 +1,48 @@
1
+ import { createReadTool } from "../core/create-read-tool.js";
2
+ import { parseReadInput } from "../core/input.js";
3
+ import { resolveReadLimits } from "../core/limits.js";
4
+ import { sha256Digest } from "../digest/sha256.js";
5
+ import { lineNumberFormatter } from "../formatters/index.js";
6
+ import { readSignatureMessages } from "./messages.js";
7
+ import { defaultReadSignature } from "./offset-limit.js";
8
+ /**
9
+ * Creates the read tool and binds it to a signature, default
10
+ * defaultReadSignature(). The signature's retry text goes under
11
+ * `options.messages`, so a host message still wins. Throws TypeError where
12
+ * createReadTool() does.
13
+ */
14
+ export function readToolDefinition(options) {
15
+ if (options === null || typeof options !== "object" || Array.isArray(options)) {
16
+ throw new TypeError("readToolDefinition options must be an object");
17
+ }
18
+ const { signature = defaultReadSignature(), ...rest } = options;
19
+ const limits = resolveReadLimits(rest.limits);
20
+ const formatter = rest.formatter ?? lineNumberFormatter();
21
+ const digest = rest.digest === undefined ? sha256Digest() : rest.digest;
22
+ const read = createReadTool({
23
+ ...rest,
24
+ limits,
25
+ formatter,
26
+ digest,
27
+ messages: { ...readSignatureMessages(signature), ...rest.messages },
28
+ });
29
+ return bindReadTool(signature, read, { limits, formatter, digest });
30
+ }
31
+ function bindReadTool(signature, read, resolved) {
32
+ return Object.freeze({
33
+ name: signature.name,
34
+ description: signature.description,
35
+ inputSchema: signature.schema,
36
+ ...(signature.grammar === undefined ? {} : { grammar: signature.grammar }),
37
+ signature,
38
+ limits: resolved.limits,
39
+ formatter: resolved.formatter,
40
+ digest: resolved.digest,
41
+ validate(modelInput) {
42
+ const input = signature.toInput(modelInput);
43
+ parseReadInput(input, resolved.limits);
44
+ return input;
45
+ },
46
+ execute: async (modelInput, ctx) => read(signature.toInput(modelInput), ctx),
47
+ });
48
+ }
@@ -1,4 +1,6 @@
1
1
  export type { DefaultReadModelInput, LineRangeParam, ReadParam, ReadSignature, } from "./contract.ts";
2
+ export { readToolDefinition } from "./definition.ts";
3
+ export type { ReadToolDefinition, ReadToolDefinitionOptions } from "./definition.ts";
2
4
  export { lineRangeSignature } from "./line-range.ts";
3
5
  export { readSignatureMessages } from "./messages.ts";
4
6
  export { defaultReadSignature } from "./offset-limit.ts";
@@ -1,3 +1,4 @@
1
+ export { readToolDefinition } from "./definition.js";
1
2
  export { lineRangeSignature } from "./line-range.js";
2
3
  export { readSignatureMessages } from "./messages.js";
3
4
  export { defaultReadSignature } from "./offset-limit.js";
@@ -5,7 +5,7 @@ export interface MemoryStoreOptions {
5
5
  readonly maxEntries?: number;
6
6
  /** Default 30 minutes. */
7
7
  readonly ttlMs?: number;
8
- /** The same clock type as the tools' `clock` dependency. Default the time now. */
8
+ /** The same clock type as the tools' `clock` option. Default the time now. */
9
9
  readonly clock?: Clock;
10
10
  }
11
11
  /**
@@ -1,2 +1,6 @@
1
- /** Up to `maximum` names close to `requested`: same stem, a shared prefix, or a small edit distance. */
1
+ /**
2
+ * Up to `maximum` names close to `requested`, closest first. The match ignores case.
3
+ * The rank is: the same name in another case, then the same stem, then a shared prefix,
4
+ * then an edit distance of 3 or less. Names with the same rank sort by name.
5
+ */
2
6
  export declare function suggestFileNames(requested: string, names: readonly string[], maximum: number): readonly string[];
@@ -1,4 +1,8 @@
1
- /** Up to `maximum` names close to `requested`: same stem, a shared prefix, or a small edit distance. */
1
+ /**
2
+ * Up to `maximum` names close to `requested`, closest first. The match ignores case.
3
+ * The rank is: the same name in another case, then the same stem, then a shared prefix,
4
+ * then an edit distance of 3 or less. Names with the same rank sort by name.
5
+ */
2
6
  export function suggestFileNames(requested, names, maximum) {
3
7
  const requestedLower = requested.toLowerCase();
4
8
  const requestedStem = stem(requestedLower);
@@ -39,7 +39,7 @@ formatter.format(outcome, { mode: "model", call, ... }) ─▶ content
39
39
  ReadResult
40
40
  ```
41
41
 
42
- | Stage | Phase | Dependency | What happens |
42
+ | Stage | Phase | Option | What happens |
43
43
  | ------------------- | -------------- | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
44
44
  | Input | `input` | `limits` | `parseReadInput` accepts only `path`, `offset`, and `limit`. The path must be a non-blank string with no NUL. `offset` and `limit` must be positive safe integers. `limit` is clamped to `limits.maxLines`, and the tool adds a `clamped` info note. |
45
45
  | Filesystem | `open` | `fs` | A `FileSystem`, or a factory of the call context. The factory runs once for each read. |
@@ -54,7 +54,7 @@ ReadResult
54
54
  | Record | none | `state`, `digest` | Stores the observation. A store failure does not fail the read. |
55
55
  | Format | none | `formatter`, `messages` | Turns the outcome into content parts. |
56
56
 
57
- The core owns line scanning, view selection, clamping, the byte limit, continuation arithmetic, change detection, and note assembly. That list is the reason the package exists. Everything else is a dependency.
57
+ The core owns line scanning, view selection, clamping, the byte limit, continuation arithmetic, change detection, and note assembly. That list is the reason the package exists. Everything else is an option.
58
58
 
59
59
  ### What open() fetches before authorize
60
60
 
@@ -65,7 +65,7 @@ The core owns line scanning, view selection, clamping, the byte limit, continuat
65
65
  | Path | Role |
66
66
  | ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
67
67
  | `src/contract/` | The public types, one file for each area. No runtime code. `base.ts` holds the tool-neutral types |
68
- | `src/core/create-read-tool.ts`, `core/deps.ts` | Checks and resolves the dependencies once, synchronously |
68
+ | `src/core/create-read-tool.ts`, `core/deps.ts` | Checks and resolves the options once, synchronously |
69
69
  | `src/core/pipeline.ts` | The stage order, and nothing else |
70
70
  | `src/core/call-scope.ts` | Per-read state: the call object, the phase, `fs(call)`, `state(call)`, the listing budget, and abort checks |
71
71
  | `src/core/resolve.ts`, `open.ts`, `suggest.ts`, `authorize.ts` | Resolve, open, suggestions on a miss, and the authorize checks |
@@ -179,15 +179,16 @@ Hooks are host code. The model cannot choose which files a hook reads, so a hook
179
179
 
180
180
  An observation exists when there is a `digest`:
181
181
 
182
- | Field | Value |
183
- | ------------------ | ------------------------------------------------------------------------------------------------ |
184
- | `statId` | Hash of the resolved path, identity, size, and modification time |
185
- | `contentId` | Hash of the source bytes. `null` when the scan stopped before the end. |
186
- | `viewId` | Hash of what the model saw: the view text, or the media parts |
187
- | `id` | Hash of `statId`, `contentId`, and `viewId` |
188
- | `wholeFileVisible` | `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view |
182
+ | Field | Value |
183
+ | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
184
+ | `statId` | Hash of the resolved path, identity, size, and modification time |
185
+ | `contentId` | Hash of the source bytes. `null` when the scan stopped before the end. |
186
+ | `viewId` | Hash of what the model saw: the view text, or the media parts |
187
+ | `id` | Hash of `statId`, `contentId`, and `viewId` |
188
+ | `wholeFileVisible` | `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view |
189
+ | `visibleLines` | The source lines that the view showed in full. `null` when no line was shown, a line was clamped, a converter made the text, or a hook changed the view |
189
190
 
190
- The record stores these fields with the request range, under the key `file.resolvedPath`. It is schema 2: it also has `origin: "read"`, the backend `version`, and the `digest` id. The write tools store records with `origin: "write"` and `request: null` in the same store, so a later edit knows what the model has seen. The hook runner treats a record of another schema as absent, and `repeatReadGuard` ignores a write record. `record` builds it field by field from the outcome, so nothing from `call` reaches the store. A store whose `get` or `put` fails does not fail the read. A `state` factory that throws does.
191
+ The record stores these fields with the request range, under the key `file.resolvedPath`. Before it stores a record, the core reads the stored record. When that record has the same `digest` and the same non-null `contentId`, the core joins its `seenLines` with this view's `visibleLines`. The record's `wholeFileVisible` is then `true` when this read showed the whole file, when the stored record was whole, or when the joined ranges run from line 1 to the last line and the totals are exact. So a model that reads a file in ranges can still `write` it. Records for one store and key are written one at a time inside the process, so parallel ranged reads all join. Two processes that share a store can still lose a range. It is schema 2: it also has `origin: "read"`, the backend `version`, and the `digest` id. The write tools store records with `origin: "write"` and `request: null` in the same store, so a later edit knows what the model has seen. The hook runner treats a record of another schema as absent, and `repeatReadGuard` ignores a write record. `record` builds it field by field from the outcome, so nothing from `call` reaches the store. A store whose `get` or `put` fails does not fail the read. A `state` factory that throws does.
191
192
 
192
193
  ## Errors
193
194
 
@@ -213,7 +214,7 @@ The core makes the other failures itself:
213
214
  - `CHANGED_DURING_READ` from the size check or `verify()`.
214
215
  - `UNSUPPORTED_BACKEND` when no classifier had an opinion.
215
216
  - `IO_ERROR` when a filesystem method or the byte stream throws.
216
- - `EXTENSION_FAILED` when host code throws or breaks a rule. `data.extension` names the dependency (`fs`, `state`, `resolve`, `suggest`, `authorize`, `converters`, `budget`, or `hooks`), `data.phase` names the stage, and `data.id` names the extension object when it has one.
217
+ - `EXTENSION_FAILED` when host code throws or breaks a rule. `data.extension` names the option (`fs`, `state`, `resolve`, `suggest`, `authorize`, `converters`, `budget`, or `hooks`), `data.phase` names the stage, and `data.id` names the extension object when it has one.
217
218
 
218
219
  A formatter that throws, or returns neither a string nor an array, does not reject the call. The core formats the result with `lineNumberFormatter()` and adds no note. The status stays as it was.
219
220
 
@@ -242,7 +243,7 @@ The AI SDK adapter passes `ToolExecutionOptions` as `host`. The Pi adapter passe
242
243
  ## Deliberate omissions
243
244
 
244
245
  - **No alias repair.** The core accepts `path`, `offset`, and `limit` only. Other names belong to a signature.
245
- - **No Node defaults in the core.** `fs` is required and `digest` defaults to `null`. `createNodeReadTool()` in `@better-fs-tools/node` gives the local defaults. A Workers build cannot pull in `node:fs` by mistake.
246
+ - **No Node defaults in the core.** `fs` is required, and `digest` defaults to `sha256Digest()`, which is plain JavaScript. The integrations keep the same defaults. A Workers build cannot pull in `node:fs` by mistake.
246
247
  - **No converter path access.** A converter gets the capped stream, never the path.
247
248
  - **No nested reads from hooks.** A hook that needs another file reads it with host code.
248
249
  - **No write guard.** The precondition check belongs with the write tool that uses the record.
@@ -50,7 +50,7 @@ The outcome is plain data. Only media parts hold bytes, as `Uint8Array`. Nothing
50
50
  truncation: { truncated, reasons, primary },
51
51
  continuation: { available, next },
52
52
  totals: { lines, exact, bytes },
53
- observation: { id, statId, contentId, viewId, observedAt, wholeFileVisible } | null,
53
+ observation: { id, statId, contentId, viewId, observedAt, wholeFileVisible, visibleLines } | null,
54
54
  notes: [...],
55
55
  content: [{ type: "text", text: "1|..." }]
56
56
  }
@@ -70,7 +70,7 @@ Field notes:
70
70
  - `truncation.reasons` can hold `lines`, `bytes`, `budget`, `scan-limit`, and `line-length`, in that order. `primary` is the first, or `null`.
71
71
  - `continuation.next` is a complete canonical input, `{ path, offset, limit }`. Send it again to continue. It uses canonical names even when the tool uses a renamed signature.
72
72
  - `totals.lines` is `null` and `totals.exact` is `false` when the scan limit stopped counting. `totals.bytes` is the scanned byte count when the scan reached the end, and the file size otherwise. For converted text, it is the size of the converted text that was scanned.
73
- - `observation` is `null` with no `digest`, and for directories. `contentId` is `null` when the scan stopped before the end. For converted text it is still the hash of the source bytes. `wholeFileVisible` is `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view.
73
+ - `observation` is `null` with no `digest`, and for directories. `contentId` is `null` when the scan stopped before the end. For converted text it is still the hash of the source bytes. `wholeFileVisible` is `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view. `visibleLines` is `{ start, end }`, the source lines that the view showed in full. It is `null` when no line was shown, a line was clamped, a converter made the text, or a hook changed the view.
74
74
 
75
75
  ## `media`
76
76
 
@@ -211,11 +211,14 @@ With a `state` store and a `digest`, an `ok` or `media` result is stored under `
211
211
  schema: 2,
212
212
  origin: "read",
213
213
  observationId, resolvedPath, identity, version, digest,
214
- contentId, viewId, observedAt, wholeFileVisible, totalsExact,
214
+ contentId, viewId, observedAt, wholeFileVisible, seenLines, totalsExact,
215
215
  request: { offset, limit }
216
216
  }
217
217
  ```
218
218
 
219
+ - `seenLines` is a sorted list of `{ start, end }` ranges that reads of this `contentId` showed in full. A read joins its `visibleLines` to the ranges of a stored record with the same `digest` and `contentId`. A record for other content starts again from this read.
220
+ - `wholeFileVisible` is `true` when this read showed the whole file, when the stored record for the same content was whole, or when `seenLines` runs from line 1 to the last line and the totals are exact.
221
+
219
222
  - `version` is `file.version`. `digest` is the `Digest.id` that made `contentId` and `viewId`.
220
223
  - A write tool stores a record with `origin: "write"` after a commit. Its `viewId` equals its `contentId`, and its `request` is `null`.
221
224
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@better-fs-tools/read",
3
- "version": "0.2.0",
3
+ "version": "0.3.0",
4
4
  "description": "A bounded file reader for TypeScript agent tools.",
5
5
  "license": "MIT",
6
6
  "repository": {
@@ -46,7 +46,7 @@
46
46
  "access": "public"
47
47
  },
48
48
  "dependencies": {
49
- "@better-fs-tools/fs": "^0.2.0"
49
+ "@better-fs-tools/fs": "^0.3.0"
50
50
  },
51
51
  "devDependencies": {
52
52
  "typebox": "^1.3.16"