@better-fs-tools/read 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +99 -71
- package/dist/contract/base.d.ts +31 -3
- package/dist/contract/deps.d.ts +9 -14
- package/dist/contract/result.d.ts +10 -0
- package/dist/contract/state.d.ts +10 -0
- package/dist/core/convert.js +1 -0
- package/dist/core/create-read-tool.d.ts +3 -3
- package/dist/core/create-read-tool.js +3 -3
- package/dist/core/deps.d.ts +2 -2
- package/dist/core/deps.js +17 -6
- package/dist/core/observation.d.ts +2 -1
- package/dist/core/observation.js +2 -0
- package/dist/core/record.js +59 -6
- package/dist/core/seen-lines.d.ts +7 -0
- package/dist/core/seen-lines.js +39 -0
- package/dist/core/text-outcome.js +7 -1
- package/dist/index.d.ts +3 -3
- package/dist/signature/definition.d.ts +30 -0
- package/dist/signature/definition.js +48 -0
- package/dist/signature/index.d.ts +2 -0
- package/dist/signature/index.js +1 -0
- package/dist/state/index.d.ts +1 -1
- package/dist/suggest/suggest-file-names.d.ts +5 -1
- package/dist/suggest/suggest-file-names.js +5 -1
- package/docs/architecture.md +14 -13
- package/docs/result-schema.md +6 -3
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -18,7 +18,7 @@ Most hosts also install one adapter package. Each adapter lists its own peers:
|
|
|
18
18
|
| -------------------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------- |
|
|
19
19
|
| `@better-fs-tools/read` | The core, the helpers, and the `./signature` and `./formats` subpaths | `npm install @better-fs-tools/read` |
|
|
20
20
|
| `@better-fs-tools/fs` | The `FileSystem` contract, `memoryFileSystem`, and the conformance suite | `npm install @better-fs-tools/fs` |
|
|
21
|
-
| `@better-fs-tools/node` |
|
|
21
|
+
| `@better-fs-tools/node` | The local filesystem on Node 24 or Bun, and `nodeDigest()` | `npm install @better-fs-tools/node @better-fs-tools/read` |
|
|
22
22
|
| `@better-fs-tools/ai-sdk` | An AI SDK 7 tool | `npm install @better-fs-tools/ai-sdk @better-fs-tools/read ai` |
|
|
23
23
|
| `@better-fs-tools/pi` | A Pi tool and a Pi extension | `npm install @better-fs-tools/pi @better-fs-tools/read @earendil-works/pi-coding-agent typebox` |
|
|
24
24
|
| `@better-fs-tools/cloudflare-shell` | A filesystem over a Cloudflare Shell Workspace | `npm install @better-fs-tools/cloudflare-shell @better-fs-tools/read` |
|
|
@@ -29,14 +29,14 @@ All packages have the same version. Install matching versions.
|
|
|
29
29
|
|
|
30
30
|
## Quick start
|
|
31
31
|
|
|
32
|
-
On Node or Bun, `
|
|
32
|
+
On Node or Bun, give `createReadTool()` a `nodeFileSystem()`:
|
|
33
33
|
|
|
34
34
|
```ts
|
|
35
|
-
import {
|
|
36
|
-
import { textOf } from "@better-fs-tools/read";
|
|
35
|
+
import { nodeFileSystem } from "@better-fs-tools/node";
|
|
36
|
+
import { createReadTool, textOf } from "@better-fs-tools/read";
|
|
37
37
|
|
|
38
38
|
// Rooted at process.cwd(), with SHA-256 observations.
|
|
39
|
-
const read =
|
|
39
|
+
const read = createReadTool({ fs: nodeFileSystem({ allowedRoots: [process.cwd()] }) });
|
|
40
40
|
|
|
41
41
|
const result = await read({ path: "package.json", limit: 2 });
|
|
42
42
|
console.log(result.status); // "ok"
|
|
@@ -47,8 +47,6 @@ console.log(textOf(result));
|
|
|
47
47
|
// [read:continue] Output stopped at the line limit. Continue with {"path":"package.json","offset":3,"limit":2}.
|
|
48
48
|
```
|
|
49
49
|
|
|
50
|
-
`createNodeReadTool()` uses `nodeFileSystem` rooted at `process.cwd()`, and `nodeDigest()` for content hashes. It passes every other option to `createReadTool()`.
|
|
51
|
-
|
|
52
50
|
`createReadTool()` is the core. It needs a filesystem, and it works with any `FileSystem`:
|
|
53
51
|
|
|
54
52
|
```ts
|
|
@@ -118,38 +116,39 @@ read({ path, offset, limit }, ctx)
|
|
|
118
116
|
ReadResult
|
|
119
117
|
```
|
|
120
118
|
|
|
121
|
-
You cannot remove the input validation, the scan, the limits, the type check, or change detection. Every other step is
|
|
119
|
+
You cannot remove the input validation, the scan, the limits, the type check, or change detection. Every other step is an option that you can replace. [docs/architecture.md](docs/architecture.md) explains each stage.
|
|
122
120
|
|
|
123
121
|
## What you can change
|
|
124
122
|
|
|
125
|
-
Pass
|
|
126
|
-
|
|
127
|
-
| Dimension |
|
|
128
|
-
| ------------------------------------------------------ | --------------------------- |
|
|
129
|
-
| Tool name, parameters, range model, descriptions | `signature` (adapters only) | `defaultReadSignature()`: `read` with `path`, `offset`, `limit`
|
|
130
|
-
| Filesystem, allowed roots, deny roots, symlinks | `fs` | Required. `
|
|
131
|
-
| Path rewrites (`~`, `file://`, `@`, Unicode repair) | `resolve` | `null` (the path is used as given)
|
|
132
|
-
| Names to suggest when a file is missing | `suggest` | `defaultSuggest()`
|
|
133
|
-
| Per-path policy, size ceiling, user approval | `authorize` | `null` (allow)
|
|
134
|
-
| Binary and format detection | `classifiers` | `defaultClassifiers()`
|
|
135
|
-
| File conversion, images, directories | `converters` | `[]`
|
|
136
|
-
| Line, byte, scan, conversion, and media limits | `limits` | `defaultReadLimits`
|
|
137
|
-
| Token limit | `budget` | `null`
|
|
138
|
-
| Redaction, repeat-read guard, your own after-read code | `hooks` | `[]`
|
|
139
|
-
| Output text or content parts | `formatter` | `lineNumberFormatter()`
|
|
140
|
-
| Note and message wording | `messages` | `defaultReadMessages`
|
|
141
|
-
| Read records for later tools | `state` | `null`
|
|
142
|
-
| Content hashes and observations | `digest` | `
|
|
143
|
-
| Timestamps | `clock` | `() => new Date()`
|
|
144
|
-
|
|
145
|
-
`limits` and `messages` merge over their defaults key by key. Every other
|
|
123
|
+
Pass options to `createReadTool()`, `readToolDefinition()`, or an integration factory (`createAiSdkReadTool()`, `createPiReadTool()`, `createAgentCoreReadTool()`). Every integration takes the same options with the same defaults, and passes them to the core.
|
|
124
|
+
|
|
125
|
+
| Dimension | Option | Default |
|
|
126
|
+
| ------------------------------------------------------ | --------------------------- | -------------------------------------------------------------------------------------------- |
|
|
127
|
+
| Tool name, parameters, range model, descriptions | `signature` (adapters only) | `defaultReadSignature()`: `read` with `path`, `offset`, `limit` |
|
|
128
|
+
| Filesystem, allowed roots, deny roots, symlinks | `fs` | Required. `nodeFileSystem()` from `@better-fs-tools/node` reads the local disk |
|
|
129
|
+
| Path rewrites (`~`, `file://`, `@`, Unicode repair) | `resolve` | `null` (the path is used as given) |
|
|
130
|
+
| Names to suggest when a file is missing | `suggest` | `defaultSuggest()` |
|
|
131
|
+
| Per-path policy, size ceiling, user approval | `authorize` | `null` (allow) |
|
|
132
|
+
| Binary and format detection | `classifiers` | `defaultClassifiers()` |
|
|
133
|
+
| File conversion, images, directories | `converters` | `[imageConverter()]`: an image becomes a media part. `[]` turns images off |
|
|
134
|
+
| Line, byte, scan, conversion, and media limits | `limits` | `defaultReadLimits` |
|
|
135
|
+
| Token limit | `budget` | `null` |
|
|
136
|
+
| Redaction, repeat-read guard, your own after-read code | `hooks` | `[]` |
|
|
137
|
+
| Output text or content parts | `formatter` | `lineNumberFormatter()` |
|
|
138
|
+
| Note and message wording | `messages` | `defaultReadMessages` |
|
|
139
|
+
| Read records for later tools | `state` | `null` |
|
|
140
|
+
| Content hashes and observations | `digest` | `sha256Digest()`, which runs anywhere. `nodeDigest()` is faster on Node. `null` turns it off |
|
|
141
|
+
| Timestamps | `clock` | `() => new Date()` |
|
|
142
|
+
|
|
143
|
+
`limits` and `messages` merge over their defaults key by key. Every other option replaces its default. To add to a list, include the default yourself:
|
|
146
144
|
|
|
147
145
|
```ts
|
|
148
|
-
import {
|
|
149
|
-
import { defaultClassifiers, extensionClassifier } from "@better-fs-tools/read";
|
|
146
|
+
import { nodeFileSystem } from "@better-fs-tools/node";
|
|
147
|
+
import { createReadTool, defaultClassifiers, extensionClassifier } from "@better-fs-tools/read";
|
|
150
148
|
|
|
151
|
-
export const read =
|
|
152
|
-
|
|
149
|
+
export const read = createReadTool({
|
|
150
|
+
fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
|
|
151
|
+
// A list option replaces the default. Include the default to add to it.
|
|
153
152
|
classifiers: [
|
|
154
153
|
extensionClassifier({ unsupported: { ".parquet": { code: "BINARY" } } }),
|
|
155
154
|
...defaultClassifiers(),
|
|
@@ -160,7 +159,7 @@ export const read = createNodeReadTool({
|
|
|
160
159
|
});
|
|
161
160
|
```
|
|
162
161
|
|
|
163
|
-
`createReadTool()` checks the
|
|
162
|
+
`createReadTool()` checks the options once, when you call it. An unknown key, an empty classifier list, a limit that is not a positive safe integer, or a message that is not a function throws `TypeError`.
|
|
164
163
|
|
|
165
164
|
## Tool signature and docs
|
|
166
165
|
|
|
@@ -172,7 +171,7 @@ The model sees a tool name, a description, a parameter schema, and parameter des
|
|
|
172
171
|
- `param(name)` gives the host name of a canonical parameter, or `""` when the signature has none.
|
|
173
172
|
- `fromInput(retry)` maps a canonical retry back to the model's names. Notes use it when they tell the model how to continue. Only the read signature has it.
|
|
174
173
|
|
|
175
|
-
Two builders
|
|
174
|
+
Two builders make the signature. `defaultReadSignature()` gives `read({ path, offset?, limit? })`. `lineRangeSignature()` gives `read({ path, start?, end? })`, with one-based and inclusive lines. Both take the shared `SignatureDocs` options: `name`, `description`, `describe`, and `names`. `describe` and `names` use the builder's own parameter names as keys.
|
|
176
175
|
|
|
177
176
|
```ts
|
|
178
177
|
import { defaultReadSignature, lineRangeSignature } from "@better-fs-tools/read/signature";
|
|
@@ -196,7 +195,9 @@ export const lineRange = lineRangeSignature({
|
|
|
196
195
|
});
|
|
197
196
|
```
|
|
198
197
|
|
|
199
|
-
With `
|
|
198
|
+
With `renamed`, a truncated read tells the model to continue with `{"file_path":"src/a.ts","start":101,"max_lines":100}`. With `lineRange`, it says `{"file_path":"src/a.ts","start_line":101,"end_line":200}`. The structured result stays canonical: `result.continuation.next` is still `{ path, offset, limit }`.
|
|
199
|
+
|
|
200
|
+
For another input shape, write your own `ReadSignature`. It needs `name`, `description`, `schema`, `toInput`, `param`, and `fromInput`.
|
|
200
201
|
|
|
201
202
|
`createAiSdkReadTool()` and `createPiReadTool()` wire the retry wording for you with `readSignatureMessages(signature)`. If you call the core yourself behind a signature, pass `messages: readSignatureMessages(signature)`. Otherwise the retry text uses `path`, `offset`, and `limit`.
|
|
202
203
|
|
|
@@ -210,13 +211,19 @@ The tool reads one path for each call. For multi-file reads or globs, call the c
|
|
|
210
211
|
|
|
211
212
|
```ts
|
|
212
213
|
import { homedir } from "node:os";
|
|
213
|
-
import {
|
|
214
|
-
import {
|
|
214
|
+
import { nodeFileSystem } from "@better-fs-tools/node";
|
|
215
|
+
import {
|
|
216
|
+
createReadTool,
|
|
217
|
+
expandHome,
|
|
218
|
+
pathResolvers,
|
|
219
|
+
stripPrefixes,
|
|
220
|
+
unicodeRepair,
|
|
221
|
+
} from "@better-fs-tools/read";
|
|
215
222
|
|
|
216
223
|
const cwd = process.cwd();
|
|
217
224
|
const home = homedir();
|
|
218
225
|
|
|
219
|
-
export const read =
|
|
226
|
+
export const read = createReadTool({
|
|
220
227
|
fs: nodeFileSystem({ cwd, allowedRoots: [cwd, `${home}/.config/app`] }),
|
|
221
228
|
resolve: pathResolvers(
|
|
222
229
|
stripPrefixes(), // "file:///x" and "@src/x"
|
|
@@ -285,13 +292,20 @@ The filesystem owns the root policy: allowed roots, deny roots, and symlinks. `a
|
|
|
285
292
|
What `open()` has fetched before `authorize` runs depends on the backend. `@better-fs-tools/cloudflare-shell` and `@better-fs-tools/just-bash` buffer the whole file inside `open()`, so those bytes have left the backend. `@better-fs-tools/cloudflare-computer` does not buffer: its `open()` starts a `readFile` stream, so the request reaches the backend, but it reads no chunk before `authorize`. `nodeFileSystem` streams, and `memoryFileSystem` already holds the bytes. In every case the core passes no byte to a classifier, a converter, or the model until `authorize` allows it. If a denied read must not reach the backend, refuse the path in the filesystem with a deny root, or leave it out of the allowed roots.
|
|
286
293
|
|
|
287
294
|
```ts
|
|
288
|
-
import {
|
|
289
|
-
import {
|
|
295
|
+
import { nodeFileSystem } from "@better-fs-tools/node";
|
|
296
|
+
import {
|
|
297
|
+
askUser,
|
|
298
|
+
createReadTool,
|
|
299
|
+
denyPaths,
|
|
300
|
+
readAuthorizers,
|
|
301
|
+
sizeCeiling,
|
|
302
|
+
} from "@better-fs-tools/read";
|
|
290
303
|
|
|
291
304
|
// Your own prompt, for example a dialog in your UI.
|
|
292
305
|
declare function confirmInUi(question: string): Promise<boolean>;
|
|
293
306
|
|
|
294
|
-
export const read =
|
|
307
|
+
export const read = createReadTool({
|
|
308
|
+
fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
|
|
295
309
|
authorize: readAuthorizers(
|
|
296
310
|
denyPaths(["**/.env", "**/.env.*", "**/*.pem"]),
|
|
297
311
|
sizeCeiling({ maxBytes: 256 * 1024, unrangedOnly: true }),
|
|
@@ -375,8 +389,9 @@ A classifier only decides what a file is. A converter changes the content.
|
|
|
375
389
|
`converters` is a list. It runs after classification. The first file converter whose `accepts` returns `true` handles the file. The first directory converter handles directories. With no converters, text is read, other formats are `unsupported`, and a directory gives `NOT_A_FILE`. The core has no converters by default. The Pi and AI SDK hosts add `imageConverter()` when you pass no `converters`.
|
|
376
390
|
|
|
377
391
|
```ts
|
|
378
|
-
import {
|
|
392
|
+
import { nodeFileSystem } from "@better-fs-tools/node";
|
|
379
393
|
import {
|
|
394
|
+
createReadTool,
|
|
380
395
|
defaultClassifiers,
|
|
381
396
|
directoryListing,
|
|
382
397
|
imageConverter,
|
|
@@ -388,7 +403,8 @@ import {
|
|
|
388
403
|
// Your own PDF extraction, for example a pdftotext process that reads stdin.
|
|
389
404
|
declare function pdfToText(source: AsyncIterable<Uint8Array>): AsyncIterable<string>;
|
|
390
405
|
|
|
391
|
-
export const read =
|
|
406
|
+
export const read = createReadTool({
|
|
407
|
+
fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
|
|
392
408
|
// The default chain reads a notebook as JSON text. notebookClassifier()
|
|
393
409
|
// marks it NOTEBOOK, so notebookConverter() can take it.
|
|
394
410
|
classifiers: [notebookClassifier(), ...defaultClassifiers()],
|
|
@@ -453,10 +469,11 @@ Every tool has one parse helper with the same pattern: `parseReadInput`, `parseE
|
|
|
453
469
|
For a token limit, set `budget`. The scanner stops at the last whole line that fits, and the result has a normal continuation. The truncation reason is `"budget"`.
|
|
454
470
|
|
|
455
471
|
```ts
|
|
456
|
-
import {
|
|
457
|
-
import { charsPerToken } from "@better-fs-tools/read";
|
|
472
|
+
import { nodeFileSystem } from "@better-fs-tools/node";
|
|
473
|
+
import { charsPerToken, createReadTool } from "@better-fs-tools/read";
|
|
458
474
|
|
|
459
|
-
export const read =
|
|
475
|
+
export const read = createReadTool({
|
|
476
|
+
fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
|
|
460
477
|
limits: { maxLines: 1_000, maxViewBytes: 64 * 1024 },
|
|
461
478
|
budget: charsPerToken({ ratio: 4, max: 25_000 }),
|
|
462
479
|
});
|
|
@@ -471,11 +488,12 @@ For a size ceiling that refuses a whole file, use the `sizeCeiling` authorizer.
|
|
|
471
488
|
`hooks` is an ordered list of `afterRead` functions. They run after the scan and change detection, and before the result is recorded and formatted.
|
|
472
489
|
|
|
473
490
|
```ts
|
|
474
|
-
import {
|
|
475
|
-
import { memoryStore, redact, repeatReadGuard } from "@better-fs-tools/read";
|
|
491
|
+
import { nodeFileSystem } from "@better-fs-tools/node";
|
|
492
|
+
import { createReadTool, memoryStore, redact, repeatReadGuard } from "@better-fs-tools/read";
|
|
476
493
|
|
|
477
|
-
//
|
|
478
|
-
export const read =
|
|
494
|
+
// repeatReadGuard needs a digest: the default sha256Digest() is one.
|
|
495
|
+
export const read = createReadTool({
|
|
496
|
+
fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
|
|
479
497
|
state: memoryStore(),
|
|
480
498
|
hooks: [repeatReadGuard(), redact({ patterns: [/AKIA[0-9A-Z]{16}/g] })],
|
|
481
499
|
});
|
|
@@ -513,7 +531,7 @@ export const agentsNote: ReadHook<unknown> = {
|
|
|
513
531
|
};
|
|
514
532
|
```
|
|
515
533
|
|
|
516
|
-
A repeat read that the guard empties
|
|
534
|
+
A repeat read that the guard empties has `observation.wholeFileVisible: false`. The stored record keeps the earlier read's `wholeFileVisible` and line ranges when the content is the same. Redaction works line by line, so a secret split across two lines is not matched.
|
|
517
535
|
|
|
518
536
|
## Output format
|
|
519
537
|
|
|
@@ -542,15 +560,17 @@ Its options:
|
|
|
542
560
|
Helpers for common formats:
|
|
543
561
|
|
|
544
562
|
```ts
|
|
545
|
-
import {
|
|
563
|
+
import { nodeFileSystem } from "@better-fs-tools/node";
|
|
546
564
|
import {
|
|
565
|
+
createReadTool,
|
|
547
566
|
eofFooter,
|
|
548
567
|
fileHashHeader,
|
|
549
568
|
hashlineGutter,
|
|
550
569
|
lineNumberFormatter,
|
|
551
570
|
} from "@better-fs-tools/read";
|
|
552
571
|
|
|
553
|
-
export const read =
|
|
572
|
+
export const read = createReadTool({
|
|
573
|
+
fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
|
|
554
574
|
formatter: lineNumberFormatter({
|
|
555
575
|
gutter: hashlineGutter({ width: 2 }), // "12:a3|text"
|
|
556
576
|
header: fileHashHeader(), // "file-hash: sha256:..."
|
|
@@ -569,10 +589,14 @@ export const read = createNodeReadTool({
|
|
|
569
589
|
Presets that copy the read output of other harnesses are in `@better-fs-tools/read/formats`: `opencodeFormat()`, `deepAgentsFormat()`, `hashlineFormat()`, and `hermesFormat()`. They take no options.
|
|
570
590
|
|
|
571
591
|
```ts
|
|
572
|
-
import {
|
|
592
|
+
import { nodeFileSystem } from "@better-fs-tools/node";
|
|
593
|
+
import { createReadTool } from "@better-fs-tools/read";
|
|
573
594
|
import { opencodeFormat } from "@better-fs-tools/read/formats";
|
|
574
595
|
|
|
575
|
-
export const read =
|
|
596
|
+
export const read = createReadTool({
|
|
597
|
+
fs: nodeFileSystem({ allowedRoots: [process.cwd()] }),
|
|
598
|
+
formatter: opencodeFormat(),
|
|
599
|
+
});
|
|
576
600
|
```
|
|
577
601
|
|
|
578
602
|
A custom formatter implements `format(outcome, ctx)`. It is synchronous and pure over the outcome. `ctx` has the `digest`, the `limits`, the `call`, and a `mode`. The mode is `"model"` for model output and `"view"` for the body only, with no notes, header, or footer. The Pi adapter uses `"view"` for its truncation details. If your formatter returns parts in `"view"` mode, Pi leaves those details out. A formatter sees only the finished outcome. It cannot read the file. If the formatter throws or returns neither a string nor an array, the call still succeeds: the core formats the result with `lineNumberFormatter()` and adds no note.
|
|
@@ -609,7 +633,7 @@ No default message names `offset` or `limit`. Every message that suggests a retr
|
|
|
609
633
|
| `unsupported` | A classifier or converter refused the format. `code` is open, for example `PDF` or `TOO_LARGE`. |
|
|
610
634
|
| `error` | `error.code` is one of thirteen `ReadErrorCode` values. |
|
|
611
635
|
|
|
612
|
-
The error codes are `INVALID_INPUT`, `NOT_FOUND`, `NOT_A_FILE`, `DANGEROUS_PATH`, `OUTSIDE_ALLOWED_ROOTS`, `PERMISSION_DENIED`, `DENIED`, `TOO_LARGE`, `CHANGED_DURING_READ`, `ABORTED`, `UNSUPPORTED_BACKEND`, `EXTENSION_FAILED`, and `IO_ERROR`. `TOO_LARGE` is an `error` when a backend byte ceiling refuses the file, and an `unsupported` code when a converter or media limit stops it. `EXTENSION_FAILED` means that host code threw or broke a rule. Its note data names the
|
|
636
|
+
The error codes are `INVALID_INPUT`, `NOT_FOUND`, `NOT_A_FILE`, `DANGEROUS_PATH`, `OUTSIDE_ALLOWED_ROOTS`, `PERMISSION_DENIED`, `DENIED`, `TOO_LARGE`, `CHANGED_DURING_READ`, `ABORTED`, `UNSUPPORTED_BACKEND`, `EXTENSION_FAILED`, and `IO_ERROR`. `TOO_LARGE` is an `error` when a backend byte ceiling refuses the file, and an `unsupported` code when a converter or media limit stops it. `EXTENSION_FAILED` means that host code threw or broke a rule. Its note data names the option and the stage, for example `{ extension: "hooks", phase: "hooks", id: "redact" }`.
|
|
613
637
|
|
|
614
638
|
An `ok` result has structured lines, and an `error` result has its error:
|
|
615
639
|
|
|
@@ -676,16 +700,16 @@ await read({ path: "/a.txt" }, { host: session, callId: "call-1" });
|
|
|
676
700
|
|
|
677
701
|
- `fs` can be a function of the call. It runs once for each read, before `resolve`. Use it to choose a backend for each call.
|
|
678
702
|
- `sha256Digest()` is SHA-256 in plain JavaScript. It is synchronous and needs no Node module, so it works in a Cloudflare Worker. Its id and its values (`sha256:<hex>`) are the same as `nodeDigest()`, so records from either one match.
|
|
679
|
-
- `state` needs a `digest`, because a record names the digest that made it. `
|
|
703
|
+
- `state` needs a `digest`, because a record names the digest that made it. `digest` defaults to `sha256Digest()`, so a `state` alone works. `createReadTool` throws `TypeError` for a `state` with `digest: null`, and the options type (`StateNeedsDigest`) refuses it at compile time.
|
|
680
704
|
- `state` can be a function of the call. It runs at most once for each read, and only when the core needs the store. Return `null` for no store. A factory that throws gives `EXTENSION_FAILED`. A store whose `get` or `put` fails never fails the read.
|
|
681
|
-
- The record key is `file.resolvedPath`. A read stores a `ReadRecord` with `schema: 2` and `origin: "read"`. It holds the backend `version`, the `digest` id, the content and view ids, and the read
|
|
705
|
+
- The record key is `file.resolvedPath`. A read stores a `ReadRecord` with `schema: 2` and `origin: "read"`. It holds the backend `version`, the `digest` id, the content and view ids, the read range, and `seenLines`. `seenLines` joins the ranges that reads of the same content showed in full. When they run from line 1 to the last line, the record is `wholeFileVisible`, so reads in ranges count as a read of the whole file. The write tools store records with `origin: "write"` and `request: null` after a commit. A record of another schema counts as absent.
|
|
682
706
|
- `result.file.version` is the backend's change token from `open()`, or `null`. It is kept when the backend has no identity capability.
|
|
683
707
|
- `memoryStore()` from `@better-fs-tools/read` keeps records in memory, with a size cap and a TTL. Its `clock` option is the same `Clock` (`() => Date`) that the tools take, so one fake clock fits both. It sits on the root of the package that owns `ReadStateStore`, as `memoryLocks()` sits on the root of `@better-fs-tools/write`, which owns `LockManager`.
|
|
684
708
|
- The helpers in this package are typed with `unknown` for the host, so they fit a tool with any host type. `askUser`, `imageConverter`, and `textConverter` can take your host type, as `askUser<Session>` does above.
|
|
685
709
|
|
|
686
710
|
## Security guarantees
|
|
687
711
|
|
|
688
|
-
Your
|
|
712
|
+
Your options can narrow access. They cannot widen it.
|
|
689
713
|
|
|
690
714
|
- The filesystem is the only place that grants access. It checks roots, deny roots, the realpath, and the type in one `open()` call. `resolve` runs before it and `authorize` runs after it.
|
|
691
715
|
- Each read opens at most one file. A resolver only changes the path that goes into that open.
|
|
@@ -697,15 +721,18 @@ Your dependencies can narrow access. They cannot widen it.
|
|
|
697
721
|
- A hook cannot turn a failure into a success. A hook that changes the view marks the observation as partial.
|
|
698
722
|
- A `DENIED` result does not show the file behind the path.
|
|
699
723
|
|
|
700
|
-
##
|
|
724
|
+
## Integrations
|
|
725
|
+
|
|
726
|
+
Each integration takes the options of `readToolDefinition()`: every core option, with the same default, plus `signature`. `fs` is required in every one. The integration owns abort forwarding and the mapping from `result.content` to the framework's content types.
|
|
701
727
|
|
|
702
|
-
|
|
728
|
+
| Package | Factory | Notes |
|
|
729
|
+
| ------------------------------------------------------------------------------------------ | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------ |
|
|
730
|
+
| [`@better-fs-tools/ai-sdk`](https://www.npmjs.com/package/@better-fs-tools/ai-sdk) | `createAiSdkReadTool(options)` | `execute` returns the whole `ReadResult`. `toModelOutput` maps media parts to AI SDK `file` parts. |
|
|
731
|
+
| [`@better-fs-tools/pi`](https://www.npmjs.com/package/@better-fs-tools/pi) | `createPiReadTool(options)` | Adds the Pi prompt options. `piCwdFileSystem()` from `@better-fs-tools/pi/node` roots each call at Pi's `ctx.cwd`. |
|
|
732
|
+
| [`@better-fs-tools/pi`](https://www.npmjs.com/package/@better-fs-tools/pi) | `createAgentCoreReadTool(options)` from `./agent-core` | An `AgentTool` for `@earendil-works/pi-agent-core`. |
|
|
733
|
+
| [`@better-fs-tools/pi-durable`](https://www.npmjs.com/package/@better-fs-tools/pi-durable) | `createDurableFsTools(options)` | A bundle only: the tools register inside a pi-durable extension. |
|
|
703
734
|
|
|
704
|
-
|
|
705
|
-
| ---------------------------------------------------------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
706
|
-
| [`@better-fs-tools/ai-sdk`](https://www.npmjs.com/package/@better-fs-tools/ai-sdk) | `createAiSdkReadTool(options)` | `execute` returns the whole `ReadResult`. `toModelOutput` maps media parts to AI SDK `file` parts. You pass `fs`. |
|
|
707
|
-
| [`@better-fs-tools/pi`](https://www.npmjs.com/package/@better-fs-tools/pi) | `createPiReadTool(options)` | Each call is confined to Pi's `ctx.cwd`. `fs`, `cwd`, and `allowedRoots` are refused. The `pi.extensions` entry registers `read`, `edit`, and `write`. |
|
|
708
|
-
| [`@better-fs-tools/node`](https://www.npmjs.com/package/@better-fs-tools/node) | `createNodeReadTool(options?)` | The core with Node defaults, for your own tool. |
|
|
735
|
+
For your own framework, call `readToolDefinition()` from `./signature`. It returns the name, description, and JSON Schema to declare, a `validate` for model input, and an `execute` that maps model input and runs the tool.
|
|
709
736
|
|
|
710
737
|
Cloudflare Agents hosts use `@better-fs-tools/ai-sdk` with `cloudflareShellFileSystem` from [`@better-fs-tools/cloudflare-shell`](https://www.npmjs.com/package/@better-fs-tools/cloudflare-shell). [`@better-fs-tools/cloudflare-computer`](https://www.npmjs.com/package/@better-fs-tools/cloudflare-computer) and [`@better-fs-tools/just-bash`](https://www.npmjs.com/package/@better-fs-tools/just-bash) are filesystems. Use them with `createReadTool()` or with an adapter.
|
|
711
738
|
|
|
@@ -730,7 +757,8 @@ This Pi host sets most dimensions. You will usually need only a few of these opt
|
|
|
730
757
|
```ts
|
|
731
758
|
import type { ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
732
759
|
import { createPiReadTool } from "@better-fs-tools/pi";
|
|
733
|
-
import {
|
|
760
|
+
import { piCwdFileSystem } from "@better-fs-tools/pi/node";
|
|
761
|
+
import { defaultReadSignature } from "@better-fs-tools/read/signature";
|
|
734
762
|
import {
|
|
735
763
|
askUser,
|
|
736
764
|
denyPaths,
|
|
@@ -751,9 +779,9 @@ import {
|
|
|
751
779
|
const stores = new Map<string, ReturnType<typeof memoryStore>>();
|
|
752
780
|
|
|
753
781
|
export const read = createPiReadTool({
|
|
754
|
-
|
|
755
|
-
|
|
756
|
-
names: { path: "file_path",
|
|
782
|
+
fs: piCwdFileSystem(),
|
|
783
|
+
signature: defaultReadSignature({
|
|
784
|
+
names: { path: "file_path", offset: "start_line", limit: "max_lines" },
|
|
757
785
|
}),
|
|
758
786
|
resolve: pathResolvers(stripPrefixes(), unicodeRepair({ note: false })),
|
|
759
787
|
authorize: readAuthorizers(
|
package/dist/contract/base.d.ts
CHANGED
|
@@ -107,15 +107,43 @@ export interface ToolSignature<TInput, TParam extends string = string> {
|
|
|
107
107
|
* that names it.
|
|
108
108
|
*/
|
|
109
109
|
param(name: TParam): string;
|
|
110
|
-
/**
|
|
110
|
+
/** A Lark grammar for hosts with grammar tools. freeformPatchSignature() sets CODEX_PATCH_GRAMMAR. */
|
|
111
111
|
readonly grammar?: {
|
|
112
112
|
readonly lark: string;
|
|
113
113
|
};
|
|
114
114
|
}
|
|
115
115
|
/**
|
|
116
|
-
*
|
|
116
|
+
* A core tool bound to a signature: what a framework adapter needs to declare
|
|
117
|
+
* the tool and run it. The xxxToolDefinition() helpers in each `./signature`
|
|
118
|
+
* subpath build one, and every integration uses them.
|
|
119
|
+
*/
|
|
120
|
+
export interface ToolDefinition<TInput, TResult, THost = undefined, TParam extends string = string> {
|
|
121
|
+
/** The tool name the model sees: `signature.name`. */
|
|
122
|
+
readonly name: string;
|
|
123
|
+
readonly description: string;
|
|
124
|
+
/** Plain JSON Schema of the model input: `signature.schema`. */
|
|
125
|
+
readonly inputSchema: JsonObject;
|
|
126
|
+
/** Set when the signature has a grammar. */
|
|
127
|
+
readonly grammar?: {
|
|
128
|
+
readonly lark: string;
|
|
129
|
+
};
|
|
130
|
+
readonly signature: ToolSignature<TInput, TParam>;
|
|
131
|
+
/**
|
|
132
|
+
* Checks model input with the signature and with the core input check at the
|
|
133
|
+
* tool's limits, and returns the canonical input. Throws TypeError that names
|
|
134
|
+
* the signature's parameters.
|
|
135
|
+
*/
|
|
136
|
+
validate(modelInput: unknown): TInput;
|
|
137
|
+
/**
|
|
138
|
+
* Maps model input with `signature.toInput` and runs the core tool. Bad input
|
|
139
|
+
* rejects with TypeError. A tool error is a result, not a rejection.
|
|
140
|
+
*/
|
|
141
|
+
execute(modelInput: unknown, ctx: ToolCallContext<THost>): Promise<TResult>;
|
|
142
|
+
}
|
|
143
|
+
/**
|
|
144
|
+
* The options every default signature takes. `TParam` is the signature's own
|
|
117
145
|
* parameter names. `names` gives a parameter another name in the schema, and
|
|
118
|
-
* `describe` replaces its description. Both are keyed by the
|
|
146
|
+
* `describe` replaces its description. Both are keyed by the signature's names.
|
|
119
147
|
*/
|
|
120
148
|
export interface SignatureDocs<TParam extends string> {
|
|
121
149
|
/** The tool name. */
|
package/dist/contract/deps.d.ts
CHANGED
|
@@ -20,7 +20,10 @@ export interface ReadDependencies<THost = undefined> {
|
|
|
20
20
|
readonly suggest: Suggest<THost> | null;
|
|
21
21
|
/** Host access policy for the opened file ("read") and before every listing ("list"). null allows everything. */
|
|
22
22
|
readonly authorize: ReadAuthorizer<THost> | null;
|
|
23
|
-
/**
|
|
23
|
+
/**
|
|
24
|
+
* File converters in order: the first that accepts runs. The first directory
|
|
25
|
+
* converter lists directories. Default [imageConverter()]. [] turns images off.
|
|
26
|
+
*/
|
|
24
27
|
readonly converters: readonly Converter<THost>[];
|
|
25
28
|
/** Stops the view early, at a line boundary, when the next line would pass `max`. null has no budget. */
|
|
26
29
|
readonly budget: ViewBudget | null;
|
|
@@ -28,32 +31,24 @@ export interface ReadDependencies<THost = undefined> {
|
|
|
28
31
|
readonly hooks: readonly ReadHook<THost>[];
|
|
29
32
|
/** A store, or a factory called at most once for each read, and only when needed. Needs `digest`. */
|
|
30
33
|
readonly state: ReadStateStore | ((call: ReadContext<THost>) => ReadStateStore | null) | null;
|
|
34
|
+
/** Names content for the read state and the observations. Default sha256Digest(). null turns both off. */
|
|
31
35
|
readonly digest: Digest | null;
|
|
32
36
|
readonly clock: Clock;
|
|
33
37
|
readonly formatter: ReadFormatter<THost>;
|
|
34
38
|
}
|
|
35
39
|
/**
|
|
36
40
|
* `state` needs a `digest`, because a stored record names the digest that made
|
|
37
|
-
* it.
|
|
38
|
-
*
|
|
41
|
+
* it. Leave `digest` out to get sha256Digest(). `digest: null` turns it off,
|
|
42
|
+
* and then `state` must be left out or null. A `state` with `digest: null`
|
|
43
|
+
* does not type-check, and throws TypeError at run time.
|
|
39
44
|
*/
|
|
40
45
|
export type StateNeedsDigest = {
|
|
41
46
|
readonly state?: null;
|
|
42
|
-
} | {
|
|
43
|
-
readonly digest: Digest;
|
|
44
|
-
};
|
|
45
|
-
/**
|
|
46
|
-
* The same pairing for a factory with its own default digest, such as the
|
|
47
|
-
* Node and Pi factories. Leave `digest` out to get the default. `digest: null`
|
|
48
|
-
* turns it off, and then `state` must be left out or null.
|
|
49
|
-
*/
|
|
50
|
-
export type StateNeedsDigestOrDefault = {
|
|
51
|
-
readonly state?: null;
|
|
52
47
|
readonly digest: null;
|
|
53
48
|
} | {
|
|
54
49
|
readonly digest?: Digest;
|
|
55
50
|
};
|
|
56
|
-
export type
|
|
51
|
+
export type ReadToolOptions<THost = undefined> = {
|
|
57
52
|
readonly fs: ReadDependencies<THost>["fs"];
|
|
58
53
|
readonly limits?: Partial<ReadLimits>;
|
|
59
54
|
readonly messages?: Partial<ReadMessageCatalog>;
|
|
@@ -150,4 +150,14 @@ export interface ReadObservation {
|
|
|
150
150
|
readonly observedAt: string;
|
|
151
151
|
/** True when offset is 1, nothing was truncated or clamped, and no hook edited the view. */
|
|
152
152
|
readonly wholeFileVisible: boolean;
|
|
153
|
+
/**
|
|
154
|
+
* The source lines that the view showed in full. null when the view showed no
|
|
155
|
+
* line, clamped a line, came from a converter, or was edited by a hook.
|
|
156
|
+
*/
|
|
157
|
+
readonly visibleLines: ReadLineRange | null;
|
|
158
|
+
}
|
|
159
|
+
/** An inclusive range of 1-based source line numbers. */
|
|
160
|
+
export interface ReadLineRange {
|
|
161
|
+
readonly start: number;
|
|
162
|
+
readonly end: number;
|
|
153
163
|
}
|
package/dist/contract/state.d.ts
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import type { ReadLineRange } from "./result.ts";
|
|
1
2
|
/**
|
|
2
3
|
* Session state for read observations. A store is a cache: a failed store call
|
|
3
4
|
* never fails a read.
|
|
@@ -26,7 +27,16 @@ export interface ReadRecord {
|
|
|
26
27
|
/** For a write record: equal to contentId. */
|
|
27
28
|
readonly viewId: string;
|
|
28
29
|
readonly observedAt: string;
|
|
30
|
+
/**
|
|
31
|
+
* True when one read showed the whole file, or when the reads of this
|
|
32
|
+
* contentId together showed every line.
|
|
33
|
+
*/
|
|
29
34
|
readonly wholeFileVisible: boolean;
|
|
35
|
+
/**
|
|
36
|
+
* The lines of this contentId that reads showed in full, sorted and joined.
|
|
37
|
+
* Empty for a write record.
|
|
38
|
+
*/
|
|
39
|
+
readonly seenLines: readonly ReadLineRange[];
|
|
30
40
|
readonly totalsExact: boolean;
|
|
31
41
|
/** The read range. null for a write record. */
|
|
32
42
|
readonly request: {
|
package/dist/core/convert.js
CHANGED
|
@@ -126,6 +126,7 @@ function mediaOutcome(input, classification, parts, notes, source) {
|
|
|
126
126
|
observedAt: deps.clock().toISOString(),
|
|
127
127
|
// Media is not source text, so it cannot back a text edit.
|
|
128
128
|
wholeFileVisible: false,
|
|
129
|
+
visibleLines: null,
|
|
129
130
|
}),
|
|
130
131
|
notes,
|
|
131
132
|
};
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import type { ReadTool } from "../contract/context.ts";
|
|
2
|
-
import type {
|
|
2
|
+
import type { ReadToolOptions, StateNeedsDigest } from "../contract/deps.ts";
|
|
3
3
|
/**
|
|
4
|
-
* Creates the read tool. Throws TypeError on an unknown or malformed
|
|
4
|
+
* Creates the read tool. Throws TypeError on an unknown or malformed option,
|
|
5
5
|
* a missing `fs`, an empty `classifiers` list, or a `state` without a `digest`.
|
|
6
6
|
*/
|
|
7
|
-
export declare function createReadTool<THost = undefined>(
|
|
7
|
+
export declare function createReadTool<THost = undefined>(options: ReadToolOptions<THost> & StateNeedsDigest): ReadTool<THost>;
|
|
@@ -2,11 +2,11 @@ import { resolveDependencies } from "./deps.js";
|
|
|
2
2
|
import { isRecord } from "./input.js";
|
|
3
3
|
import { runRead } from "./pipeline.js";
|
|
4
4
|
/**
|
|
5
|
-
* Creates the read tool. Throws TypeError on an unknown or malformed
|
|
5
|
+
* Creates the read tool. Throws TypeError on an unknown or malformed option,
|
|
6
6
|
* a missing `fs`, an empty `classifiers` list, or a `state` without a `digest`.
|
|
7
7
|
*/
|
|
8
|
-
export function createReadTool(
|
|
9
|
-
const resolved = resolveDependencies(
|
|
8
|
+
export function createReadTool(options) {
|
|
9
|
+
const resolved = resolveDependencies(options);
|
|
10
10
|
const read = async (input, ctx) => {
|
|
11
11
|
if (ctx !== undefined && !isRecord(ctx)) {
|
|
12
12
|
throw new TypeError("read context must be an object");
|
package/dist/core/deps.d.ts
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import type { FileSystem } from "@better-fs-tools/fs";
|
|
2
|
-
import type { ReadDependencies,
|
|
2
|
+
import type { ReadDependencies, ReadToolOptions } from "../contract/deps.ts";
|
|
3
3
|
import type { ReadStateStore } from "../contract/state.ts";
|
|
4
|
-
export declare function resolveDependencies<THost>(deps:
|
|
4
|
+
export declare function resolveDependencies<THost>(deps: ReadToolOptions<THost>): ReadDependencies<THost>;
|
|
5
5
|
export declare function isFileSystem(value: unknown): value is FileSystem;
|
|
6
6
|
export declare function isStateStore(value: unknown): value is ReadStateStore;
|
package/dist/core/deps.js
CHANGED
|
@@ -1,10 +1,12 @@
|
|
|
1
1
|
import { defaultClassifiers } from "../classifiers/index.js";
|
|
2
|
+
import { imageConverter } from "../converters/index.js";
|
|
3
|
+
import { sha256Digest } from "../digest/sha256.js";
|
|
2
4
|
import { lineNumberFormatter } from "../formatters/index.js";
|
|
3
5
|
import { defaultSuggest } from "../suggest/index.js";
|
|
4
6
|
import { isRecord } from "./input.js";
|
|
5
7
|
import { resolveReadLimits } from "./limits.js";
|
|
6
8
|
import { resolveReadMessages } from "./messages.js";
|
|
7
|
-
const
|
|
9
|
+
const KNOWN_OPTION_KEYS = new Set([
|
|
8
10
|
"fs",
|
|
9
11
|
"limits",
|
|
10
12
|
"messages",
|
|
@@ -22,10 +24,10 @@ const KNOWN_DEPENDENCY_KEYS = new Set([
|
|
|
22
24
|
]);
|
|
23
25
|
export function resolveDependencies(deps) {
|
|
24
26
|
if (!isRecord(deps))
|
|
25
|
-
throw new TypeError("read tool
|
|
27
|
+
throw new TypeError("read tool options must be an object");
|
|
26
28
|
for (const key of Object.keys(deps)) {
|
|
27
|
-
if (!
|
|
28
|
-
throw new TypeError(`Unknown read tool
|
|
29
|
+
if (!KNOWN_OPTION_KEYS.has(key))
|
|
30
|
+
throw new TypeError(`Unknown read tool option: ${key}`);
|
|
29
31
|
}
|
|
30
32
|
const { fs } = deps;
|
|
31
33
|
if (typeof fs !== "function" && !isFileSystem(fs)) {
|
|
@@ -51,7 +53,7 @@ export function resolveDependencies(deps) {
|
|
|
51
53
|
if (authorize !== null && (!isRecord(authorize) || typeof authorize.authorize !== "function")) {
|
|
52
54
|
throw new TypeError("authorize must be an authorizer or null");
|
|
53
55
|
}
|
|
54
|
-
const converters = deps.converters === undefined ? [] : deps.converters;
|
|
56
|
+
const converters = deps.converters === undefined ? [imageConverter()] : deps.converters;
|
|
55
57
|
if (!Array.isArray(converters) || !converters.every(isConverter)) {
|
|
56
58
|
throw new TypeError("converters must be an array of file and directory converters");
|
|
57
59
|
}
|
|
@@ -70,7 +72,10 @@ export function resolveDependencies(deps) {
|
|
|
70
72
|
if (state !== null && typeof state !== "function" && !isStateStore(state)) {
|
|
71
73
|
throw new TypeError("state must be a store, a function that returns one, or null");
|
|
72
74
|
}
|
|
73
|
-
const digest = deps.digest
|
|
75
|
+
const digest = deps.digest === undefined ? sha256Digest() : deps.digest;
|
|
76
|
+
if (digest !== null && !isDigest(digest)) {
|
|
77
|
+
throw new TypeError("digest must have an id, create, and hash, or be null");
|
|
78
|
+
}
|
|
74
79
|
if (state !== null && digest === null) {
|
|
75
80
|
throw new TypeError("state needs a digest: records name the digest that made them");
|
|
76
81
|
}
|
|
@@ -113,6 +118,12 @@ function isViewBudget(value) {
|
|
|
113
118
|
function isHook(value) {
|
|
114
119
|
return isRecord(value) && typeof value.id === "string" && typeof value.afterRead === "function";
|
|
115
120
|
}
|
|
121
|
+
function isDigest(value) {
|
|
122
|
+
return (isRecord(value) &&
|
|
123
|
+
typeof value.id === "string" &&
|
|
124
|
+
typeof value.create === "function" &&
|
|
125
|
+
typeof value.hash === "function");
|
|
126
|
+
}
|
|
116
127
|
export function isStateStore(value) {
|
|
117
128
|
return (isRecord(value) &&
|
|
118
129
|
typeof value.get === "function" &&
|
|
@@ -1,11 +1,12 @@
|
|
|
1
1
|
import type { Digest } from "../contract/digest.ts";
|
|
2
|
-
import type { ContentPart, FileInfo, ReadObservation } from "../contract/result.ts";
|
|
2
|
+
import type { ContentPart, FileInfo, ReadLineRange, ReadObservation } from "../contract/result.ts";
|
|
3
3
|
export interface ObservationInput {
|
|
4
4
|
readonly file: FileInfo;
|
|
5
5
|
readonly contentId: string | null;
|
|
6
6
|
readonly view: string | readonly ContentPart[];
|
|
7
7
|
readonly observedAt: string;
|
|
8
8
|
readonly wholeFileVisible: boolean;
|
|
9
|
+
readonly visibleLines: ReadLineRange | null;
|
|
9
10
|
}
|
|
10
11
|
export declare function buildObservation(digest: Digest | null, input: ObservationInput): ReadObservation | null;
|
|
11
12
|
export declare function withEditedView(digest: Digest, observation: ReadObservation, view: string | readonly ContentPart[]): ReadObservation;
|
package/dist/core/observation.js
CHANGED
|
@@ -16,6 +16,7 @@ export function buildObservation(digest, input) {
|
|
|
16
16
|
viewId,
|
|
17
17
|
observedAt: input.observedAt,
|
|
18
18
|
wholeFileVisible: input.wholeFileVisible,
|
|
19
|
+
visibleLines: input.visibleLines,
|
|
19
20
|
};
|
|
20
21
|
}
|
|
21
22
|
export function withEditedView(digest, observation, view) {
|
|
@@ -25,6 +26,7 @@ export function withEditedView(digest, observation, view) {
|
|
|
25
26
|
id: observationId(digest, observation.statId, observation.contentId, viewId),
|
|
26
27
|
viewId,
|
|
27
28
|
wholeFileVisible: false,
|
|
29
|
+
visibleLines: null,
|
|
28
30
|
};
|
|
29
31
|
}
|
|
30
32
|
function viewIdOf(digest, view) {
|
package/dist/core/record.js
CHANGED
|
@@ -1,3 +1,10 @@
|
|
|
1
|
+
import { isRecord } from "./input.js";
|
|
2
|
+
import { coversLines, joinLineRanges, storedLineRanges } from "./seen-lines.js";
|
|
3
|
+
/**
|
|
4
|
+
* The record step of each read waits for the step before it on the same store
|
|
5
|
+
* and key, so parallel ranged reads of one file all join the stored ranges.
|
|
6
|
+
*/
|
|
7
|
+
const pending = new WeakMap();
|
|
1
8
|
export async function recordOutcome(scope, outcome) {
|
|
2
9
|
if (outcome.status !== "ok" && outcome.status !== "media")
|
|
3
10
|
return;
|
|
@@ -8,16 +15,62 @@ export async function recordOutcome(scope, outcome) {
|
|
|
8
15
|
const store = scope.stateStore();
|
|
9
16
|
if (store === null)
|
|
10
17
|
return;
|
|
18
|
+
const key = outcome.file.resolvedPath;
|
|
19
|
+
await inTurn(store, key, async () => {
|
|
20
|
+
try {
|
|
21
|
+
const previous = await sameContentRecord(store, key, digest.id, observation.contentId);
|
|
22
|
+
await store.put(key, toRecord(outcome, observation, digest.id, previous));
|
|
23
|
+
}
|
|
24
|
+
catch {
|
|
25
|
+
// Session state is a cache: a failing store never fails the read.
|
|
26
|
+
}
|
|
27
|
+
});
|
|
28
|
+
}
|
|
29
|
+
async function inTurn(store, key, step) {
|
|
30
|
+
let keys = pending.get(store);
|
|
31
|
+
if (keys === undefined) {
|
|
32
|
+
keys = new Map();
|
|
33
|
+
pending.set(store, keys);
|
|
34
|
+
}
|
|
35
|
+
const before = keys.get(key) ?? Promise.resolve();
|
|
36
|
+
const current = before.then(step);
|
|
37
|
+
keys.set(key, current);
|
|
38
|
+
try {
|
|
39
|
+
await current;
|
|
40
|
+
}
|
|
41
|
+
finally {
|
|
42
|
+
if (keys.get(key) === current)
|
|
43
|
+
keys.delete(key);
|
|
44
|
+
}
|
|
45
|
+
}
|
|
46
|
+
/** The stored record, when it describes the same bytes with the same digest. */
|
|
47
|
+
async function sameContentRecord(store, key, digest, contentId) {
|
|
48
|
+
if (contentId === null)
|
|
49
|
+
return null;
|
|
50
|
+
let record;
|
|
11
51
|
try {
|
|
12
|
-
await store.
|
|
52
|
+
record = await store.get(key);
|
|
13
53
|
}
|
|
14
54
|
catch {
|
|
15
|
-
|
|
55
|
+
return null;
|
|
16
56
|
}
|
|
57
|
+
if (!isRecord(record) || record.schema !== 2 || record.digest !== digest)
|
|
58
|
+
return null;
|
|
59
|
+
return record.contentId === contentId ? record : null;
|
|
17
60
|
}
|
|
18
61
|
/** Copies fields one by one so nothing from the host `call` reaches the store. */
|
|
19
|
-
function toRecord(outcome, observation, digest) {
|
|
62
|
+
function toRecord(outcome, observation, digest, previous) {
|
|
20
63
|
const { file, request } = outcome;
|
|
64
|
+
const seenLines = joinLineRanges([
|
|
65
|
+
...storedLineRanges(previous?.seenLines),
|
|
66
|
+
...(observation.visibleLines === null ? [] : [observation.visibleLines]),
|
|
67
|
+
]);
|
|
68
|
+
// Media is only returned from a whole source, so its source totals are exact.
|
|
69
|
+
const totalsExact = outcome.status === "ok" ? outcome.totals.exact : true;
|
|
70
|
+
const totalLines = outcome.status === "ok" ? outcome.totals.lines : null;
|
|
71
|
+
const wholeFileVisible = observation.wholeFileVisible ||
|
|
72
|
+
previous?.wholeFileVisible === true ||
|
|
73
|
+
(totalsExact && totalLines !== null && coversLines(seenLines, totalLines));
|
|
21
74
|
return {
|
|
22
75
|
schema: 2,
|
|
23
76
|
origin: "read",
|
|
@@ -29,9 +82,9 @@ function toRecord(outcome, observation, digest) {
|
|
|
29
82
|
contentId: observation.contentId,
|
|
30
83
|
viewId: observation.viewId,
|
|
31
84
|
observedAt: observation.observedAt,
|
|
32
|
-
wholeFileVisible
|
|
33
|
-
|
|
34
|
-
totalsExact
|
|
85
|
+
wholeFileVisible,
|
|
86
|
+
seenLines,
|
|
87
|
+
totalsExact,
|
|
35
88
|
request: { offset: request.offset, limit: request.limit },
|
|
36
89
|
};
|
|
37
90
|
}
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
import type { ReadLineRange } from "../contract/result.ts";
|
|
2
|
+
/** Sorts the ranges and joins ranges that overlap or touch. */
|
|
3
|
+
export declare function joinLineRanges(ranges: readonly ReadLineRange[]): ReadLineRange[];
|
|
4
|
+
/** True when joined ranges show every line from 1 to totalLines. */
|
|
5
|
+
export declare function coversLines(joined: readonly ReadLineRange[], totalLines: number): boolean;
|
|
6
|
+
/** Reads ranges from a stored record. Anything malformed counts as no range. */
|
|
7
|
+
export declare function storedLineRanges(value: unknown): ReadLineRange[];
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
/** Sorts the ranges and joins ranges that overlap or touch. */
|
|
2
|
+
export function joinLineRanges(ranges) {
|
|
3
|
+
const sorted = [...ranges].sort((a, b) => a.start - b.start || a.end - b.end);
|
|
4
|
+
const joined = [];
|
|
5
|
+
for (const range of sorted) {
|
|
6
|
+
const last = joined.at(-1);
|
|
7
|
+
if (last !== undefined && range.start <= last.end + 1) {
|
|
8
|
+
joined[joined.length - 1] = { start: last.start, end: Math.max(last.end, range.end) };
|
|
9
|
+
}
|
|
10
|
+
else {
|
|
11
|
+
joined.push({ start: range.start, end: range.end });
|
|
12
|
+
}
|
|
13
|
+
}
|
|
14
|
+
return joined;
|
|
15
|
+
}
|
|
16
|
+
/** True when joined ranges show every line from 1 to totalLines. */
|
|
17
|
+
export function coversLines(joined, totalLines) {
|
|
18
|
+
if (totalLines === 0)
|
|
19
|
+
return true;
|
|
20
|
+
const first = joined[0];
|
|
21
|
+
return first !== undefined && first.start <= 1 && first.end >= totalLines;
|
|
22
|
+
}
|
|
23
|
+
/** Reads ranges from a stored record. Anything malformed counts as no range. */
|
|
24
|
+
export function storedLineRanges(value) {
|
|
25
|
+
if (!Array.isArray(value))
|
|
26
|
+
return [];
|
|
27
|
+
const ranges = [];
|
|
28
|
+
for (const item of value) {
|
|
29
|
+
if (item === null || typeof item !== "object")
|
|
30
|
+
return [];
|
|
31
|
+
const { start, end } = item;
|
|
32
|
+
if (!Number.isSafeInteger(start) || !Number.isSafeInteger(end))
|
|
33
|
+
return [];
|
|
34
|
+
if (start < 1 || end < start)
|
|
35
|
+
return [];
|
|
36
|
+
ranges.push({ start: start, end: end });
|
|
37
|
+
}
|
|
38
|
+
return ranges;
|
|
39
|
+
}
|
|
@@ -19,6 +19,11 @@ export function textOutcome(input) {
|
|
|
19
19
|
!continuationAvailable &&
|
|
20
20
|
!scanCapped &&
|
|
21
21
|
scanner.clampedLines.length === 0;
|
|
22
|
+
const endLine = scanner.lines.length === 0 ? request.offset - 1 : request.offset + scanner.lines.length - 1;
|
|
23
|
+
// Converted lines are not source lines, so they cannot join a range of the source.
|
|
24
|
+
const visibleLines = conversion === null && scanner.lines.length > 0 && scanner.clampedLines.length === 0
|
|
25
|
+
? { start: request.offset, end: endLine }
|
|
26
|
+
: null;
|
|
22
27
|
return {
|
|
23
28
|
tool: "read",
|
|
24
29
|
status: "ok",
|
|
@@ -29,7 +34,7 @@ export function textOutcome(input) {
|
|
|
29
34
|
view: {
|
|
30
35
|
lines: scanner.lines,
|
|
31
36
|
startLine: request.offset,
|
|
32
|
-
endLine
|
|
37
|
+
endLine,
|
|
33
38
|
bytes: scanner.viewBytes,
|
|
34
39
|
partial: request.offset !== 1 || continuationAvailable || reasons.length > 0,
|
|
35
40
|
},
|
|
@@ -54,6 +59,7 @@ export function textOutcome(input) {
|
|
|
54
59
|
view: scanner.viewText,
|
|
55
60
|
observedAt: deps.clock().toISOString(),
|
|
56
61
|
wholeFileVisible,
|
|
62
|
+
visibleLines,
|
|
57
63
|
}),
|
|
58
64
|
notes: [
|
|
59
65
|
...buildViewNotes({
|
package/dist/index.d.ts
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
export type { Classification, ClassificationSample, Classifier } from "./contract/classify.ts";
|
|
2
2
|
export type { BuiltInUnsupportedCode, DefaultClassifierOptions, NoteContext, NoteOverride, TextClassification, UnsupportedClassification, } from "./contract/classify.ts";
|
|
3
|
-
export type { AccessDecision, AccessTarget, Note, SignatureDocs, ToolAuthorizer, ToolCallContext, ToolError, ToolHookContext, ToolMessages, ToolName, ToolResolveContext, ToolSignature, } from "./contract/base.ts";
|
|
3
|
+
export type { AccessDecision, AccessTarget, Note, SignatureDocs, ToolAuthorizer, ToolCallContext, ToolDefinition, ToolError, ToolHookContext, ToolMessages, ToolName, ToolResolveContext, ToolSignature, } from "./contract/base.ts";
|
|
4
4
|
export type { ReadContext, ReadTool } from "./contract/context.ts";
|
|
5
|
-
export type {
|
|
5
|
+
export type { ReadToolOptions, StateNeedsDigest } from "./contract/deps.ts";
|
|
6
6
|
export type { Clock, Digest, DigestStream } from "./contract/digest.ts";
|
|
7
7
|
export type { AfterReadContext, Converter, ConverterMatch, ConvertOutcome, DirectoryConverter, DirectoryConvertInput, FileConverter, FileConvertInput, PathResolver, ReadAuthorizeDecision, ReadAuthorizer, ReadAuthorizeTarget, ReadHook, ReadHookContext, ReadResolveContext, ResolveOutcome, Suggest, SuggestContext, ViewBudget, } from "./contract/extensions.ts";
|
|
8
8
|
export type { ReadFormatContext, ReadFormatter } from "./contract/format.ts";
|
|
@@ -10,7 +10,7 @@ export type { ReadInput, ReadRequest } from "./contract/input.ts";
|
|
|
10
10
|
export type { JsonObject, JsonValue } from "./contract/json.ts";
|
|
11
11
|
export type { ReadLimits } from "./contract/limits.ts";
|
|
12
12
|
export type { ReadMessageCatalog, ReadPhase } from "./contract/messages.ts";
|
|
13
|
-
export type { ClassificationInfo, Confidence, ContentPart, ConversionInfo, FileInfo, MediaPart, ReadContinuation, ReadError, ReadErrorCode, ReadFailure, ReadLine, ReadMedia, ReadNote, ReadObservation, ReadOk, ReadReport, ReadResult, ReadTotals, ReadTruncation, ReadUnsupported, ReadView, TextPart, TruncationReason, } from "./contract/result.ts";
|
|
13
|
+
export type { ClassificationInfo, Confidence, ContentPart, ConversionInfo, FileInfo, MediaPart, ReadContinuation, ReadError, ReadErrorCode, ReadFailure, ReadLine, ReadLineRange, ReadMedia, ReadNote, ReadObservation, ReadOk, ReadReport, ReadResult, ReadTotals, ReadTruncation, ReadUnsupported, ReadView, TextPart, TruncationReason, } from "./contract/result.ts";
|
|
14
14
|
export type { ReadRecord, ReadStateStore } from "./contract/state.ts";
|
|
15
15
|
export { createReadTool } from "./core/create-read-tool.ts";
|
|
16
16
|
export { parseReadInput } from "./core/input.ts";
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
import type { ToolDefinition } from "../contract/base.ts";
|
|
2
|
+
import type { ReadToolOptions, StateNeedsDigest } from "../contract/deps.ts";
|
|
3
|
+
import type { Digest } from "../contract/digest.ts";
|
|
4
|
+
import type { ReadFormatter } from "../contract/format.ts";
|
|
5
|
+
import type { ReadInput } from "../contract/input.ts";
|
|
6
|
+
import type { ReadLimits } from "../contract/limits.ts";
|
|
7
|
+
import type { ReadResult } from "../contract/result.ts";
|
|
8
|
+
import type { ReadParam, ReadSignature } from "./contract.ts";
|
|
9
|
+
/** The read tool's options, plus the signature the model sees. */
|
|
10
|
+
export type ReadToolDefinitionOptions<THost = undefined> = ReadToolOptions<THost> & StateNeedsDigest & {
|
|
11
|
+
/** Default defaultReadSignature(). */
|
|
12
|
+
readonly signature?: ReadSignature;
|
|
13
|
+
};
|
|
14
|
+
/**
|
|
15
|
+
* The read tool bound to its signature. `limits`, `formatter`, and `digest`
|
|
16
|
+
* are the ones the tool uses, for a host that formats the result again.
|
|
17
|
+
*/
|
|
18
|
+
export interface ReadToolDefinition<THost = undefined> extends ToolDefinition<ReadInput, ReadResult, THost, ReadParam> {
|
|
19
|
+
readonly signature: ReadSignature;
|
|
20
|
+
readonly limits: Readonly<ReadLimits>;
|
|
21
|
+
readonly formatter: ReadFormatter<THost>;
|
|
22
|
+
readonly digest: Digest | null;
|
|
23
|
+
}
|
|
24
|
+
/**
|
|
25
|
+
* Creates the read tool and binds it to a signature, default
|
|
26
|
+
* defaultReadSignature(). The signature's retry text goes under
|
|
27
|
+
* `options.messages`, so a host message still wins. Throws TypeError where
|
|
28
|
+
* createReadTool() does.
|
|
29
|
+
*/
|
|
30
|
+
export declare function readToolDefinition<THost = undefined>(options: ReadToolDefinitionOptions<THost>): ReadToolDefinition<THost>;
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
import { createReadTool } from "../core/create-read-tool.js";
|
|
2
|
+
import { parseReadInput } from "../core/input.js";
|
|
3
|
+
import { resolveReadLimits } from "../core/limits.js";
|
|
4
|
+
import { sha256Digest } from "../digest/sha256.js";
|
|
5
|
+
import { lineNumberFormatter } from "../formatters/index.js";
|
|
6
|
+
import { readSignatureMessages } from "./messages.js";
|
|
7
|
+
import { defaultReadSignature } from "./offset-limit.js";
|
|
8
|
+
/**
|
|
9
|
+
* Creates the read tool and binds it to a signature, default
|
|
10
|
+
* defaultReadSignature(). The signature's retry text goes under
|
|
11
|
+
* `options.messages`, so a host message still wins. Throws TypeError where
|
|
12
|
+
* createReadTool() does.
|
|
13
|
+
*/
|
|
14
|
+
export function readToolDefinition(options) {
|
|
15
|
+
if (options === null || typeof options !== "object" || Array.isArray(options)) {
|
|
16
|
+
throw new TypeError("readToolDefinition options must be an object");
|
|
17
|
+
}
|
|
18
|
+
const { signature = defaultReadSignature(), ...rest } = options;
|
|
19
|
+
const limits = resolveReadLimits(rest.limits);
|
|
20
|
+
const formatter = rest.formatter ?? lineNumberFormatter();
|
|
21
|
+
const digest = rest.digest === undefined ? sha256Digest() : rest.digest;
|
|
22
|
+
const read = createReadTool({
|
|
23
|
+
...rest,
|
|
24
|
+
limits,
|
|
25
|
+
formatter,
|
|
26
|
+
digest,
|
|
27
|
+
messages: { ...readSignatureMessages(signature), ...rest.messages },
|
|
28
|
+
});
|
|
29
|
+
return bindReadTool(signature, read, { limits, formatter, digest });
|
|
30
|
+
}
|
|
31
|
+
function bindReadTool(signature, read, resolved) {
|
|
32
|
+
return Object.freeze({
|
|
33
|
+
name: signature.name,
|
|
34
|
+
description: signature.description,
|
|
35
|
+
inputSchema: signature.schema,
|
|
36
|
+
...(signature.grammar === undefined ? {} : { grammar: signature.grammar }),
|
|
37
|
+
signature,
|
|
38
|
+
limits: resolved.limits,
|
|
39
|
+
formatter: resolved.formatter,
|
|
40
|
+
digest: resolved.digest,
|
|
41
|
+
validate(modelInput) {
|
|
42
|
+
const input = signature.toInput(modelInput);
|
|
43
|
+
parseReadInput(input, resolved.limits);
|
|
44
|
+
return input;
|
|
45
|
+
},
|
|
46
|
+
execute: async (modelInput, ctx) => read(signature.toInput(modelInput), ctx),
|
|
47
|
+
});
|
|
48
|
+
}
|
|
@@ -1,4 +1,6 @@
|
|
|
1
1
|
export type { DefaultReadModelInput, LineRangeParam, ReadParam, ReadSignature, } from "./contract.ts";
|
|
2
|
+
export { readToolDefinition } from "./definition.ts";
|
|
3
|
+
export type { ReadToolDefinition, ReadToolDefinitionOptions } from "./definition.ts";
|
|
2
4
|
export { lineRangeSignature } from "./line-range.ts";
|
|
3
5
|
export { readSignatureMessages } from "./messages.ts";
|
|
4
6
|
export { defaultReadSignature } from "./offset-limit.ts";
|
package/dist/signature/index.js
CHANGED
package/dist/state/index.d.ts
CHANGED
|
@@ -5,7 +5,7 @@ export interface MemoryStoreOptions {
|
|
|
5
5
|
readonly maxEntries?: number;
|
|
6
6
|
/** Default 30 minutes. */
|
|
7
7
|
readonly ttlMs?: number;
|
|
8
|
-
/** The same clock type as the tools' `clock`
|
|
8
|
+
/** The same clock type as the tools' `clock` option. Default the time now. */
|
|
9
9
|
readonly clock?: Clock;
|
|
10
10
|
}
|
|
11
11
|
/**
|
|
@@ -1,2 +1,6 @@
|
|
|
1
|
-
/**
|
|
1
|
+
/**
|
|
2
|
+
* Up to `maximum` names close to `requested`, closest first. The match ignores case.
|
|
3
|
+
* The rank is: the same name in another case, then the same stem, then a shared prefix,
|
|
4
|
+
* then an edit distance of 3 or less. Names with the same rank sort by name.
|
|
5
|
+
*/
|
|
2
6
|
export declare function suggestFileNames(requested: string, names: readonly string[], maximum: number): readonly string[];
|
|
@@ -1,4 +1,8 @@
|
|
|
1
|
-
/**
|
|
1
|
+
/**
|
|
2
|
+
* Up to `maximum` names close to `requested`, closest first. The match ignores case.
|
|
3
|
+
* The rank is: the same name in another case, then the same stem, then a shared prefix,
|
|
4
|
+
* then an edit distance of 3 or less. Names with the same rank sort by name.
|
|
5
|
+
*/
|
|
2
6
|
export function suggestFileNames(requested, names, maximum) {
|
|
3
7
|
const requestedLower = requested.toLowerCase();
|
|
4
8
|
const requestedStem = stem(requestedLower);
|
package/docs/architecture.md
CHANGED
|
@@ -39,7 +39,7 @@ formatter.format(outcome, { mode: "model", call, ... }) ─▶ content
|
|
|
39
39
|
ReadResult
|
|
40
40
|
```
|
|
41
41
|
|
|
42
|
-
| Stage | Phase |
|
|
42
|
+
| Stage | Phase | Option | What happens |
|
|
43
43
|
| ------------------- | -------------- | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
44
44
|
| Input | `input` | `limits` | `parseReadInput` accepts only `path`, `offset`, and `limit`. The path must be a non-blank string with no NUL. `offset` and `limit` must be positive safe integers. `limit` is clamped to `limits.maxLines`, and the tool adds a `clamped` info note. |
|
|
45
45
|
| Filesystem | `open` | `fs` | A `FileSystem`, or a factory of the call context. The factory runs once for each read. |
|
|
@@ -54,7 +54,7 @@ ReadResult
|
|
|
54
54
|
| Record | none | `state`, `digest` | Stores the observation. A store failure does not fail the read. |
|
|
55
55
|
| Format | none | `formatter`, `messages` | Turns the outcome into content parts. |
|
|
56
56
|
|
|
57
|
-
The core owns line scanning, view selection, clamping, the byte limit, continuation arithmetic, change detection, and note assembly. That list is the reason the package exists. Everything else is
|
|
57
|
+
The core owns line scanning, view selection, clamping, the byte limit, continuation arithmetic, change detection, and note assembly. That list is the reason the package exists. Everything else is an option.
|
|
58
58
|
|
|
59
59
|
### What open() fetches before authorize
|
|
60
60
|
|
|
@@ -65,7 +65,7 @@ The core owns line scanning, view selection, clamping, the byte limit, continuat
|
|
|
65
65
|
| Path | Role |
|
|
66
66
|
| ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
|
|
67
67
|
| `src/contract/` | The public types, one file for each area. No runtime code. `base.ts` holds the tool-neutral types |
|
|
68
|
-
| `src/core/create-read-tool.ts`, `core/deps.ts` | Checks and resolves the
|
|
68
|
+
| `src/core/create-read-tool.ts`, `core/deps.ts` | Checks and resolves the options once, synchronously |
|
|
69
69
|
| `src/core/pipeline.ts` | The stage order, and nothing else |
|
|
70
70
|
| `src/core/call-scope.ts` | Per-read state: the call object, the phase, `fs(call)`, `state(call)`, the listing budget, and abort checks |
|
|
71
71
|
| `src/core/resolve.ts`, `open.ts`, `suggest.ts`, `authorize.ts` | Resolve, open, suggestions on a miss, and the authorize checks |
|
|
@@ -179,15 +179,16 @@ Hooks are host code. The model cannot choose which files a hook reads, so a hook
|
|
|
179
179
|
|
|
180
180
|
An observation exists when there is a `digest`:
|
|
181
181
|
|
|
182
|
-
| Field | Value
|
|
183
|
-
| ------------------ |
|
|
184
|
-
| `statId` | Hash of the resolved path, identity, size, and modification time
|
|
185
|
-
| `contentId` | Hash of the source bytes. `null` when the scan stopped before the end.
|
|
186
|
-
| `viewId` | Hash of what the model saw: the view text, or the media parts
|
|
187
|
-
| `id` | Hash of `statId`, `contentId`, and `viewId`
|
|
188
|
-
| `wholeFileVisible` | `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view
|
|
182
|
+
| Field | Value |
|
|
183
|
+
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
184
|
+
| `statId` | Hash of the resolved path, identity, size, and modification time |
|
|
185
|
+
| `contentId` | Hash of the source bytes. `null` when the scan stopped before the end. |
|
|
186
|
+
| `viewId` | Hash of what the model saw: the view text, or the media parts |
|
|
187
|
+
| `id` | Hash of `statId`, `contentId`, and `viewId` |
|
|
188
|
+
| `wholeFileVisible` | `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view |
|
|
189
|
+
| `visibleLines` | The source lines that the view showed in full. `null` when no line was shown, a line was clamped, a converter made the text, or a hook changed the view |
|
|
189
190
|
|
|
190
|
-
The record stores these fields with the request range, under the key `file.resolvedPath`. It is schema 2: it also has `origin: "read"`, the backend `version`, and the `digest` id. The write tools store records with `origin: "write"` and `request: null` in the same store, so a later edit knows what the model has seen. The hook runner treats a record of another schema as absent, and `repeatReadGuard` ignores a write record. `record` builds it field by field from the outcome, so nothing from `call` reaches the store. A store whose `get` or `put` fails does not fail the read. A `state` factory that throws does.
|
|
191
|
+
The record stores these fields with the request range, under the key `file.resolvedPath`. Before it stores a record, the core reads the stored record. When that record has the same `digest` and the same non-null `contentId`, the core joins its `seenLines` with this view's `visibleLines`. The record's `wholeFileVisible` is then `true` when this read showed the whole file, when the stored record was whole, or when the joined ranges run from line 1 to the last line and the totals are exact. So a model that reads a file in ranges can still `write` it. Records for one store and key are written one at a time inside the process, so parallel ranged reads all join. Two processes that share a store can still lose a range. It is schema 2: it also has `origin: "read"`, the backend `version`, and the `digest` id. The write tools store records with `origin: "write"` and `request: null` in the same store, so a later edit knows what the model has seen. The hook runner treats a record of another schema as absent, and `repeatReadGuard` ignores a write record. `record` builds it field by field from the outcome, so nothing from `call` reaches the store. A store whose `get` or `put` fails does not fail the read. A `state` factory that throws does.
|
|
191
192
|
|
|
192
193
|
## Errors
|
|
193
194
|
|
|
@@ -213,7 +214,7 @@ The core makes the other failures itself:
|
|
|
213
214
|
- `CHANGED_DURING_READ` from the size check or `verify()`.
|
|
214
215
|
- `UNSUPPORTED_BACKEND` when no classifier had an opinion.
|
|
215
216
|
- `IO_ERROR` when a filesystem method or the byte stream throws.
|
|
216
|
-
- `EXTENSION_FAILED` when host code throws or breaks a rule. `data.extension` names the
|
|
217
|
+
- `EXTENSION_FAILED` when host code throws or breaks a rule. `data.extension` names the option (`fs`, `state`, `resolve`, `suggest`, `authorize`, `converters`, `budget`, or `hooks`), `data.phase` names the stage, and `data.id` names the extension object when it has one.
|
|
217
218
|
|
|
218
219
|
A formatter that throws, or returns neither a string nor an array, does not reject the call. The core formats the result with `lineNumberFormatter()` and adds no note. The status stays as it was.
|
|
219
220
|
|
|
@@ -242,7 +243,7 @@ The AI SDK adapter passes `ToolExecutionOptions` as `host`. The Pi adapter passe
|
|
|
242
243
|
## Deliberate omissions
|
|
243
244
|
|
|
244
245
|
- **No alias repair.** The core accepts `path`, `offset`, and `limit` only. Other names belong to a signature.
|
|
245
|
-
- **No Node defaults in the core.** `fs` is required and `digest` defaults to `
|
|
246
|
+
- **No Node defaults in the core.** `fs` is required, and `digest` defaults to `sha256Digest()`, which is plain JavaScript. The integrations keep the same defaults. A Workers build cannot pull in `node:fs` by mistake.
|
|
246
247
|
- **No converter path access.** A converter gets the capped stream, never the path.
|
|
247
248
|
- **No nested reads from hooks.** A hook that needs another file reads it with host code.
|
|
248
249
|
- **No write guard.** The precondition check belongs with the write tool that uses the record.
|
package/docs/result-schema.md
CHANGED
|
@@ -50,7 +50,7 @@ The outcome is plain data. Only media parts hold bytes, as `Uint8Array`. Nothing
|
|
|
50
50
|
truncation: { truncated, reasons, primary },
|
|
51
51
|
continuation: { available, next },
|
|
52
52
|
totals: { lines, exact, bytes },
|
|
53
|
-
observation: { id, statId, contentId, viewId, observedAt, wholeFileVisible } | null,
|
|
53
|
+
observation: { id, statId, contentId, viewId, observedAt, wholeFileVisible, visibleLines } | null,
|
|
54
54
|
notes: [...],
|
|
55
55
|
content: [{ type: "text", text: "1|..." }]
|
|
56
56
|
}
|
|
@@ -70,7 +70,7 @@ Field notes:
|
|
|
70
70
|
- `truncation.reasons` can hold `lines`, `bytes`, `budget`, `scan-limit`, and `line-length`, in that order. `primary` is the first, or `null`.
|
|
71
71
|
- `continuation.next` is a complete canonical input, `{ path, offset, limit }`. Send it again to continue. It uses canonical names even when the tool uses a renamed signature.
|
|
72
72
|
- `totals.lines` is `null` and `totals.exact` is `false` when the scan limit stopped counting. `totals.bytes` is the scanned byte count when the scan reached the end, and the file size otherwise. For converted text, it is the size of the converted text that was scanned.
|
|
73
|
-
- `observation` is `null` with no `digest`, and for directories. `contentId` is `null` when the scan stopped before the end. For converted text it is still the hash of the source bytes. `wholeFileVisible` is `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view.
|
|
73
|
+
- `observation` is `null` with no `digest`, and for directories. `contentId` is `null` when the scan stopped before the end. For converted text it is still the hash of the source bytes. `wholeFileVisible` is `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view. `visibleLines` is `{ start, end }`, the source lines that the view showed in full. It is `null` when no line was shown, a line was clamped, a converter made the text, or a hook changed the view.
|
|
74
74
|
|
|
75
75
|
## `media`
|
|
76
76
|
|
|
@@ -211,11 +211,14 @@ With a `state` store and a `digest`, an `ok` or `media` result is stored under `
|
|
|
211
211
|
schema: 2,
|
|
212
212
|
origin: "read",
|
|
213
213
|
observationId, resolvedPath, identity, version, digest,
|
|
214
|
-
contentId, viewId, observedAt, wholeFileVisible, totalsExact,
|
|
214
|
+
contentId, viewId, observedAt, wholeFileVisible, seenLines, totalsExact,
|
|
215
215
|
request: { offset, limit }
|
|
216
216
|
}
|
|
217
217
|
```
|
|
218
218
|
|
|
219
|
+
- `seenLines` is a sorted list of `{ start, end }` ranges that reads of this `contentId` showed in full. A read joins its `visibleLines` to the ranges of a stored record with the same `digest` and `contentId`. A record for other content starts again from this read.
|
|
220
|
+
- `wholeFileVisible` is `true` when this read showed the whole file, when the stored record for the same content was whole, or when `seenLines` runs from line 1 to the last line and the totals are exact.
|
|
221
|
+
|
|
219
222
|
- `version` is `file.version`. `digest` is the `Digest.id` that made `contentId` and `viewId`.
|
|
220
223
|
- A write tool stores a record with `origin: "write"` after a commit. Its `viewId` equals its `contentId`, and its `request` is `null`.
|
|
221
224
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@better-fs-tools/read",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "A bounded file reader for TypeScript agent tools.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -46,7 +46,7 @@
|
|
|
46
46
|
"access": "public"
|
|
47
47
|
},
|
|
48
48
|
"dependencies": {
|
|
49
|
-
"@better-fs-tools/fs": "^0.
|
|
49
|
+
"@better-fs-tools/fs": "^0.3.0"
|
|
50
50
|
},
|
|
51
51
|
"devDependencies": {
|
|
52
52
|
"typebox": "^1.3.16"
|