@better-fs-tools/read 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +119 -84
- package/dist/classifiers/index.d.ts +4 -3
- package/dist/classifiers/index.js +4 -5
- package/dist/contract/base.d.ts +31 -3
- package/dist/contract/deps.d.ts +9 -14
- package/dist/contract/limits.d.ts +1 -1
- package/dist/contract/messages.d.ts +0 -10
- package/dist/contract/result.d.ts +10 -0
- package/dist/contract/state.d.ts +10 -0
- package/dist/core/convert.js +3 -3
- package/dist/core/create-read-tool.d.ts +3 -3
- package/dist/core/create-read-tool.js +3 -3
- package/dist/core/deps.d.ts +2 -2
- package/dist/core/deps.js +17 -6
- package/dist/core/format.js +1 -15
- package/dist/core/limits.js +1 -1
- package/dist/core/messages.js +5 -8
- package/dist/core/notes.d.ts +0 -2
- package/dist/core/notes.js +0 -20
- package/dist/core/observation.d.ts +2 -1
- package/dist/core/observation.js +2 -0
- package/dist/core/record.js +59 -6
- package/dist/core/seen-lines.d.ts +7 -0
- package/dist/core/seen-lines.js +39 -0
- package/dist/core/text-outcome.js +8 -3
- package/dist/index.d.ts +3 -3
- package/dist/signature/definition.d.ts +30 -0
- package/dist/signature/definition.js +48 -0
- package/dist/signature/index.d.ts +2 -0
- package/dist/signature/index.js +1 -0
- package/dist/state/index.d.ts +1 -1
- package/dist/suggest/suggest-file-names.d.ts +5 -1
- package/dist/suggest/suggest-file-names.js +5 -1
- package/docs/architecture.md +19 -17
- package/docs/result-schema.md +24 -21
- package/package.json +2 -2
package/docs/architecture.md
CHANGED
|
@@ -39,7 +39,7 @@ formatter.format(outcome, { mode: "model", call, ... }) ─▶ content
|
|
|
39
39
|
ReadResult
|
|
40
40
|
```
|
|
41
41
|
|
|
42
|
-
| Stage | Phase |
|
|
42
|
+
| Stage | Phase | Option | What happens |
|
|
43
43
|
| ------------------- | -------------- | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
44
44
|
| Input | `input` | `limits` | `parseReadInput` accepts only `path`, `offset`, and `limit`. The path must be a non-blank string with no NUL. `offset` and `limit` must be positive safe integers. `limit` is clamped to `limits.maxLines`, and the tool adds a `clamped` info note. |
|
|
45
45
|
| Filesystem | `open` | `fs` | A `FileSystem`, or a factory of the call context. The factory runs once for each read. |
|
|
@@ -54,7 +54,7 @@ ReadResult
|
|
|
54
54
|
| Record | none | `state`, `digest` | Stores the observation. A store failure does not fail the read. |
|
|
55
55
|
| Format | none | `formatter`, `messages` | Turns the outcome into content parts. |
|
|
56
56
|
|
|
57
|
-
The core owns line scanning, view selection, clamping, the byte limit, continuation arithmetic, change detection, and note assembly. That list is the reason the package exists. Everything else is
|
|
57
|
+
The core owns line scanning, view selection, clamping, the byte limit, continuation arithmetic, change detection, and note assembly. That list is the reason the package exists. Everything else is an option.
|
|
58
58
|
|
|
59
59
|
### What open() fetches before authorize
|
|
60
60
|
|
|
@@ -65,7 +65,7 @@ The core owns line scanning, view selection, clamping, the byte limit, continuat
|
|
|
65
65
|
| Path | Role |
|
|
66
66
|
| ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
|
|
67
67
|
| `src/contract/` | The public types, one file for each area. No runtime code. `base.ts` holds the tool-neutral types |
|
|
68
|
-
| `src/core/create-read-tool.ts`, `core/deps.ts` | Checks and resolves the
|
|
68
|
+
| `src/core/create-read-tool.ts`, `core/deps.ts` | Checks and resolves the options once, synchronously |
|
|
69
69
|
| `src/core/pipeline.ts` | The stage order, and nothing else |
|
|
70
70
|
| `src/core/call-scope.ts` | Per-read state: the call object, the phase, `fs(call)`, `state(call)`, the listing budget, and abort checks |
|
|
71
71
|
| `src/core/resolve.ts`, `open.ts`, `suggest.ts`, `authorize.ts` | Resolve, open, suggestions on a miss, and the authorize checks |
|
|
@@ -113,10 +113,11 @@ handle.verify() did the opened object change?
|
|
|
113
113
|
handle.close() always, from a finally block
|
|
114
114
|
```
|
|
115
115
|
|
|
116
|
-
|
|
116
|
+
One capability remains. It adds no note to the result, because the model cannot act on it:
|
|
117
117
|
|
|
118
|
-
- `
|
|
119
|
-
|
|
118
|
+
- `identity: false` sets `file.identity` to `null`. The write tools then judge freshness by the content hash only.
|
|
119
|
+
|
|
120
|
+
The read core reads `bytes()` the same way whether it yields one chunk or many, so a backend does not report which.
|
|
120
121
|
|
|
121
122
|
An optional `list()` is the list capability. Without it, there are no suggestions and no directory listings.
|
|
122
123
|
|
|
@@ -178,15 +179,16 @@ Hooks are host code. The model cannot choose which files a hook reads, so a hook
|
|
|
178
179
|
|
|
179
180
|
An observation exists when there is a `digest`:
|
|
180
181
|
|
|
181
|
-
| Field | Value
|
|
182
|
-
| ------------------ |
|
|
183
|
-
| `statId` | Hash of the resolved path, identity, size, and modification time
|
|
184
|
-
| `contentId` | Hash of the source bytes. `null` when the scan stopped before the end.
|
|
185
|
-
| `viewId` | Hash of what the model saw: the view text, or the media parts
|
|
186
|
-
| `id` | Hash of `statId`, `contentId`, and `viewId`
|
|
187
|
-
| `wholeFileVisible` | `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view
|
|
182
|
+
| Field | Value |
|
|
183
|
+
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
184
|
+
| `statId` | Hash of the resolved path, identity, size, and modification time |
|
|
185
|
+
| `contentId` | Hash of the source bytes. `null` when the scan stopped before the end. |
|
|
186
|
+
| `viewId` | Hash of what the model saw: the view text, or the media parts |
|
|
187
|
+
| `id` | Hash of `statId`, `contentId`, and `viewId` |
|
|
188
|
+
| `wholeFileVisible` | `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view |
|
|
189
|
+
| `visibleLines` | The source lines that the view showed in full. `null` when no line was shown, a line was clamped, a converter made the text, or a hook changed the view |
|
|
188
190
|
|
|
189
|
-
The record stores these fields with the request range, under the key `file.resolvedPath`. It is schema 2: it also has `origin: "read"`, the backend `version`, and the `digest` id. The write tools store records with `origin: "write"` and `request: null` in the same store, so a later edit knows what the model has seen. The hook runner treats a record of another schema as absent, and `repeatReadGuard` ignores a write record. `record` builds it field by field from the outcome, so nothing from `call` reaches the store. A store whose `get` or `put` fails does not fail the read. A `state` factory that throws does.
|
|
191
|
+
The record stores these fields with the request range, under the key `file.resolvedPath`. Before it stores a record, the core reads the stored record. When that record has the same `digest` and the same non-null `contentId`, the core joins its `seenLines` with this view's `visibleLines`. The record's `wholeFileVisible` is then `true` when this read showed the whole file, when the stored record was whole, or when the joined ranges run from line 1 to the last line and the totals are exact. So a model that reads a file in ranges can still `write` it. Records for one store and key are written one at a time inside the process, so parallel ranged reads all join. Two processes that share a store can still lose a range. It is schema 2: it also has `origin: "read"`, the backend `version`, and the `digest` id. The write tools store records with `origin: "write"` and `request: null` in the same store, so a later edit knows what the model has seen. The hook runner treats a record of another schema as absent, and `repeatReadGuard` ignores a write record. `record` builds it field by field from the outcome, so nothing from `call` reaches the store. A store whose `get` or `put` fails does not fail the read. A `state` factory that throws does.
|
|
190
192
|
|
|
191
193
|
## Errors
|
|
192
194
|
|
|
@@ -212,9 +214,9 @@ The core makes the other failures itself:
|
|
|
212
214
|
- `CHANGED_DURING_READ` from the size check or `verify()`.
|
|
213
215
|
- `UNSUPPORTED_BACKEND` when no classifier had an opinion.
|
|
214
216
|
- `IO_ERROR` when a filesystem method or the byte stream throws.
|
|
215
|
-
- `EXTENSION_FAILED` when host code throws or breaks a rule. `data.extension` names the
|
|
217
|
+
- `EXTENSION_FAILED` when host code throws or breaks a rule. `data.extension` names the option (`fs`, `state`, `resolve`, `suggest`, `authorize`, `converters`, `budget`, or `hooks`), `data.phase` names the stage, and `data.id` names the extension object when it has one.
|
|
216
218
|
|
|
217
|
-
A formatter that throws, or returns neither a string nor an array, does not reject the call. The core
|
|
219
|
+
A formatter that throws, or returns neither a string nor an array, does not reject the call. The core formats the result with `lineNumberFormatter()` and adds no note. The status stays as it was.
|
|
218
220
|
|
|
219
221
|
## Adapters
|
|
220
222
|
|
|
@@ -241,7 +243,7 @@ The AI SDK adapter passes `ToolExecutionOptions` as `host`. The Pi adapter passe
|
|
|
241
243
|
## Deliberate omissions
|
|
242
244
|
|
|
243
245
|
- **No alias repair.** The core accepts `path`, `offset`, and `limit` only. Other names belong to a signature.
|
|
244
|
-
- **No Node defaults in the core.** `fs` is required and `digest` defaults to `
|
|
246
|
+
- **No Node defaults in the core.** `fs` is required, and `digest` defaults to `sha256Digest()`, which is plain JavaScript. The integrations keep the same defaults. A Workers build cannot pull in `node:fs` by mistake.
|
|
245
247
|
- **No converter path access.** A converter gets the capped stream, never the path.
|
|
246
248
|
- **No nested reads from hooks.** A hook that needs another file reads it with host code.
|
|
247
249
|
- **No write guard.** The precondition check belongs with the write tool that uses the record.
|
package/docs/result-schema.md
CHANGED
|
@@ -50,7 +50,7 @@ The outcome is plain data. Only media parts hold bytes, as `Uint8Array`. Nothing
|
|
|
50
50
|
truncation: { truncated, reasons, primary },
|
|
51
51
|
continuation: { available, next },
|
|
52
52
|
totals: { lines, exact, bytes },
|
|
53
|
-
observation: { id, statId, contentId, viewId, observedAt, wholeFileVisible } | null,
|
|
53
|
+
observation: { id, statId, contentId, viewId, observedAt, wholeFileVisible, visibleLines } | null,
|
|
54
54
|
notes: [...],
|
|
55
55
|
content: [{ type: "text", text: "1|..." }]
|
|
56
56
|
}
|
|
@@ -70,7 +70,7 @@ Field notes:
|
|
|
70
70
|
- `truncation.reasons` can hold `lines`, `bytes`, `budget`, `scan-limit`, and `line-length`, in that order. `primary` is the first, or `null`.
|
|
71
71
|
- `continuation.next` is a complete canonical input, `{ path, offset, limit }`. Send it again to continue. It uses canonical names even when the tool uses a renamed signature.
|
|
72
72
|
- `totals.lines` is `null` and `totals.exact` is `false` when the scan limit stopped counting. `totals.bytes` is the scanned byte count when the scan reached the end, and the file size otherwise. For converted text, it is the size of the converted text that was scanned.
|
|
73
|
-
- `observation` is `null` with no `digest`, and for directories. `contentId` is `null` when the scan stopped before the end. For converted text it is still the hash of the source bytes. `wholeFileVisible` is `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view.
|
|
73
|
+
- `observation` is `null` with no `digest`, and for directories. `contentId` is `null` when the scan stopped before the end. For converted text it is still the hash of the source bytes. `wholeFileVisible` is `true` only when the offset is 1, nothing was truncated or clamped, and no hook changed the view. `visibleLines` is `{ start, end }`, the source lines that the view showed in full. It is `null` when no line was shown, a line was clamped, a converter made the text, or a hook changed the view.
|
|
74
74
|
|
|
75
75
|
## `media`
|
|
76
76
|
|
|
@@ -97,12 +97,13 @@ Field notes:
|
|
|
97
97
|
|
|
98
98
|
`code` comes from the classifier, a converter refusal, or the core:
|
|
99
99
|
|
|
100
|
-
| Code
|
|
101
|
-
|
|
|
102
|
-
| `IMAGE`, `PDF`, `OFFICE_DOCUMENT`, `
|
|
103
|
-
| `
|
|
104
|
-
| `
|
|
105
|
-
|
|
|
100
|
+
| Code | Source |
|
|
101
|
+
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
102
|
+
| `IMAGE`, `PDF`, `OFFICE_DOCUMENT`, `BINARY`, `UNKNOWN_ENCODING` | The default classifiers |
|
|
103
|
+
| `NOTEBOOK` | `notebookClassifier()`, which is not in the default chain |
|
|
104
|
+
| `INVALID_NOTEBOOK` | `notebookConverter()` on broken notebook JSON |
|
|
105
|
+
| `TOO_LARGE` | The core: converter input over `maxConvertBytes`, or media over `maxMediaBytes`. A backend ceiling gives `status: "error"` with `TOO_LARGE` instead |
|
|
106
|
+
| any other string | Your own classifier or converter |
|
|
106
107
|
|
|
107
108
|
## `error`
|
|
108
109
|
|
|
@@ -162,8 +163,6 @@ Notes from the core:
|
|
|
162
163
|
| `scan-limit` | warning | none | `{ maxScanBytes }` |
|
|
163
164
|
| `offset-past-eof` | info | the last line | `{ totalLines }` |
|
|
164
165
|
| `empty` | info | none | none |
|
|
165
|
-
| `weak-identity` | info | none | `{ backend }` |
|
|
166
|
-
| `buffered-backend` | info | none | `{ backend }` |
|
|
167
166
|
| `not-found` | warning | none | `{ suggestions?, entriesTruncated?, detail?, cause? }` |
|
|
168
167
|
| `not-a-file` | warning | none | `{ kind, detail?, cause? }` |
|
|
169
168
|
| `dangerous-path`, `outside-allowed-roots`, `permission-denied`, `denied`, `unsupported-backend`, `io-error` | warning | none | `{ detail?, cause? }` from the filesystem. A thrown error gives `io-error` with `{ detail }`. |
|
|
@@ -172,20 +171,21 @@ Notes from the core:
|
|
|
172
171
|
| `changed-during-read` | warning | the same range | none |
|
|
173
172
|
| `aborted` | warning | none | `{ phase }` |
|
|
174
173
|
| `invalid-input` | warning | none | `{ path }` when the input had a string path |
|
|
175
|
-
| `extension-failed` | warning | none | `{ extension, phase, id? }
|
|
174
|
+
| `extension-failed` | warning | none | `{ extension, phase, id? }` |
|
|
176
175
|
| `view-modified` | info | none | `{ hook }` |
|
|
177
176
|
|
|
178
177
|
Notes from the built-in helpers:
|
|
179
178
|
|
|
180
|
-
| `code`
|
|
181
|
-
|
|
|
182
|
-
| `utf8-bom`
|
|
183
|
-
| `unsupported-image`, `unsupported-pdf`, `unsupported-office-document`, `unsupported-
|
|
184
|
-
| `
|
|
185
|
-
| `
|
|
186
|
-
| `
|
|
187
|
-
| `
|
|
188
|
-
| `
|
|
179
|
+
| `code` | Severity | From | `data` |
|
|
180
|
+
| --------------------------------------------------------------------------------------------------------------------------- | -------- | ----------------------- | ------------------------------- |
|
|
181
|
+
| `utf8-bom` | info | `utf8Classifier` | none |
|
|
182
|
+
| `unsupported-image`, `unsupported-pdf`, `unsupported-office-document`, `unsupported-binary`, `unsupported-unknown-encoding` | warning | the default classifiers | as set by a `NoteOverride` |
|
|
183
|
+
| `unsupported-notebook` | warning | `notebookClassifier()` | as set by a `NoteOverride` |
|
|
184
|
+
| `path-repaired` | warning | `unicodeRepair()` | `{ from, to }` |
|
|
185
|
+
| `invalid-notebook` | warning | `notebookConverter()` | none |
|
|
186
|
+
| `directory-truncated` | warning | `directoryListing()` | `{ maxDirectoryEntries }` |
|
|
187
|
+
| `empty-directory` | info | `directoryListing()` | none |
|
|
188
|
+
| `repeat-read` | info | `repeatReadGuard()` | `{ observationId, observedAt }` |
|
|
189
189
|
|
|
190
190
|
`directoryListing()` also has a `list-failed` refusal for a listing that failed. The core reports the listing's own error first, so a read with the built-in converter gives that error, for example `DENIED` or `IO_ERROR`.
|
|
191
191
|
|
|
@@ -211,11 +211,14 @@ With a `state` store and a `digest`, an `ok` or `media` result is stored under `
|
|
|
211
211
|
schema: 2,
|
|
212
212
|
origin: "read",
|
|
213
213
|
observationId, resolvedPath, identity, version, digest,
|
|
214
|
-
contentId, viewId, observedAt, wholeFileVisible, totalsExact,
|
|
214
|
+
contentId, viewId, observedAt, wholeFileVisible, seenLines, totalsExact,
|
|
215
215
|
request: { offset, limit }
|
|
216
216
|
}
|
|
217
217
|
```
|
|
218
218
|
|
|
219
|
+
- `seenLines` is a sorted list of `{ start, end }` ranges that reads of this `contentId` showed in full. A read joins its `visibleLines` to the ranges of a stored record with the same `digest` and `contentId`. A record for other content starts again from this read.
|
|
220
|
+
- `wholeFileVisible` is `true` when this read showed the whole file, when the stored record for the same content was whole, or when `seenLines` runs from line 1 to the last line and the totals are exact.
|
|
221
|
+
|
|
219
222
|
- `version` is `file.version`. `digest` is the `Digest.id` that made `contentId` and `viewId`.
|
|
220
223
|
- A write tool stores a record with `origin: "write"` after a commit. Its `viewId` equals its `contentId`, and its `request` is `null`.
|
|
221
224
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@better-fs-tools/read",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "A bounded file reader for TypeScript agent tools.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -46,7 +46,7 @@
|
|
|
46
46
|
"access": "public"
|
|
47
47
|
},
|
|
48
48
|
"dependencies": {
|
|
49
|
-
"@better-fs-tools/fs": "^0.
|
|
49
|
+
"@better-fs-tools/fs": "^0.3.0"
|
|
50
50
|
},
|
|
51
51
|
"devDependencies": {
|
|
52
52
|
"typebox": "^1.3.16"
|