@theokit/sdk-tools 0.27.5 → 0.28.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +59 -0
- package/dist/extract-document.cjs +175 -0
- package/dist/extract-document.cjs.map +1 -0
- package/dist/extract-document.d.cts +59 -0
- package/dist/extract-document.d.ts +59 -0
- package/dist/extract-document.js +173 -0
- package/dist/extract-document.js.map +1 -0
- package/dist/github-link-preview.cjs +227 -0
- package/dist/github-link-preview.cjs.map +1 -0
- package/dist/github-link-preview.d.cts +69 -0
- package/dist/github-link-preview.d.ts +69 -0
- package/dist/github-link-preview.js +225 -0
- package/dist/github-link-preview.js.map +1 -0
- package/dist/index.cjs +127 -7
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +108 -1
- package/dist/index.d.ts +108 -1
- package/dist/index.js +128 -11
- package/dist/index.js.map +1 -1
- package/dist/read-pdf.cjs +154 -0
- package/dist/read-pdf.cjs.map +1 -0
- package/dist/read-pdf.d.cts +58 -0
- package/dist/read-pdf.d.ts +58 -0
- package/dist/read-pdf.js +152 -0
- package/dist/read-pdf.js.map +1 -0
- package/dist/screenshot-logbook.cjs +245 -0
- package/dist/screenshot-logbook.cjs.map +1 -0
- package/dist/screenshot-logbook.d.cts +124 -0
- package/dist/screenshot-logbook.d.ts +124 -0
- package/dist/screenshot-logbook.js +241 -0
- package/dist/screenshot-logbook.js.map +1 -0
- package/package.json +55 -5
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,64 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.28.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- [#794](https://github.com/usetheokit/theokit-sdk/pull/794) [`1c7ec5b`](https://github.com/usetheokit/theokit-sdk/commit/1c7ec5ba2c717b3a8375e09d16614f084f5c68ef) Thanks [@usetheodev](https://github.com/usetheodev)! - Add `createGithubLinkPreviewTool`, imported from `@theokit/sdk-tools/github-link-preview`. It gives an
|
|
8
|
+
agent a `github_link_preview` tool that turns a github.com repository, issue or pull request URL into
|
|
9
|
+
`{ ok, kind, url, title, state, summary }`: `open`, `closed` or `merged` for issues and pull requests,
|
|
10
|
+
`active` or `archived` for repositories. The tool never requests the model's URL. It validates owner,
|
|
11
|
+
repository and number and calls one fixed `https://api.github.com` endpoint built from them, with
|
|
12
|
+
redirects refused. Defaults: a 10-second timeout covering the body read, a 1 MiB body cap and a
|
|
13
|
+
500-character summary; the timeout and summary length are options. The `token` option is optional,
|
|
14
|
+
reaches only that request and never appears in a result. Every failure is a typed result
|
|
15
|
+
(`unsupported_url`, `redirect_blocked`, `not_found`, `rate_limited` with `retry_after_seconds`,
|
|
16
|
+
`upstream_error`, `timeout`, `fetch_failed`, `invalid_response`), never a throw. The tool ships from its
|
|
17
|
+
own subpath because the root bundle has no room in its size budget.
|
|
18
|
+
|
|
19
|
+
- [#796](https://github.com/usetheokit/theokit-sdk/pull/796) [`4264514`](https://github.com/usetheokit/theokit-sdk/commit/4264514840471fcb836964b6dbbff9440fb0cbe5) Thanks [@usetheodev](https://github.com/usetheodev)! - Add `screenshot_logbook`, a built-in tool that keeps a captioned log of images already inside the project root, imported from the subpath `@theokit/sdk-tools/screenshot-logbook` (`createScreenshotLogbookTool`). Action `record` appends an entry (root-relative path, media type, size, caption, time) for a png, jpeg, gif or webp file; action `list` returns the latest entries and the total. The log is a JSON Lines file under the project root (`logbook/screenshots.jsonl` by default) that holds references, never image bytes, and is rewritten through a temp file and a rename. Recording the same image twice gives two entries. Captions default to at most 500 characters and a log to at most 1000 entries (`maxCaptionChars`, `maxEntries`). Records on one tool instance run one at a time; two processes or two instances writing one log are not serialised and can lose an entry. Paths outside the root or under a secret segment are refused, and every failure is a typed result (`invalid_input`, `path_traversal`, `caption_too_long`, `unsupported_image_type`, `image_not_found`, `empty_image`, `log_full`, `log_corrupt`, `write_failed`) that leaves the log unchanged.
|
|
20
|
+
|
|
21
|
+
- [#787](https://github.com/usetheokit/theokit-sdk/pull/787) [`e681e9d`](https://github.com/usetheokit/theokit-sdk/commit/e681e9d98a8ff56d9bd92cf49f117b867311896c) Thanks [@usetheodev](https://github.com/usetheodev)! - `createTextToSpeechTool` adds a `text_to_speech` tool. The speech provider is the consumer's: the
|
|
22
|
+
factory takes a `synthesize(text)` callback that returns the encoded clip and its mime type, and
|
|
23
|
+
the tool holds no key and opens no socket. The clip is written atomically to a file under the
|
|
24
|
+
project root (`audio/` by default, created when absent), with the same path confinement and
|
|
25
|
+
secret-segment refusal as `view_image` and `read_file`. The model receives the relative path, the
|
|
26
|
+
mime type and the byte count, never the audio. Text over 4096 characters and clips over 10 MiB are
|
|
27
|
+
refused by default (`maxTextChars`, `maxAudioBytes`), as are unlisted mime types and empty audio;
|
|
28
|
+
every failure is a typed result and leaves no file behind.
|
|
29
|
+
|
|
30
|
+
### Patch Changes
|
|
31
|
+
|
|
32
|
+
- [#789](https://github.com/usetheokit/theokit-sdk/pull/789) [`87c5e88`](https://github.com/usetheokit/theokit-sdk/commit/87c5e8832e37ab0cbbc33a4615096d1f3ec2dc6d) Thanks [@usetheodev](https://github.com/usetheodev)! - Add `extract_document`, a built-in tool that returns the text of a `.docx` inside the project root, imported from the subpath `@theokit/sdk-tools/extract-document` (`createExtractDocumentTool`). Paths outside the root, absolute paths and secret paths are refused before any file is opened, and failures are typed results (`unsupported_document_type`, `not_found`, `not_a_document`, `document_too_large`, `document_unreadable`, `document_engine_missing`). Text is cut at `maxChars` with `truncated` set. The engine is the optional peer `mammoth` (`npm install mammoth`), loaded on first call; pass `extractText` to use another engine.
|
|
33
|
+
|
|
34
|
+
- [#793](https://github.com/usetheokit/theokit-sdk/pull/793) [`8635229`](https://github.com/usetheokit/theokit-sdk/commit/8635229e593b4eb6769d038934f918377b1c03e9) Thanks [@usetheodev](https://github.com/usetheodev)! - New `createReadPdfTool`, imported from the subpath `@theokit/sdk-tools/read-pdf`: a `read_pdf` built-in that gives an agent the text of a PDF inside the
|
|
35
|
+
project root, with its page count and character count. It refuses traversal, absolute paths outside
|
|
36
|
+
the root, secret paths and symlinks that escape the root before opening a file, and returns typed
|
|
37
|
+
`not_found`, `not_a_pdf`, `pdf_too_large`, `pdf_unreadable` and `pdf_engine_missing` results
|
|
38
|
+
instead of throwing. The text is cut at `maxChars` (default 100000) with `truncated` set; files over
|
|
39
|
+
`maxBytes` (default 10 MB) are refused unread.
|
|
40
|
+
|
|
41
|
+
Text extraction uses `unpdf`, a new optional peer loaded on the first call: install it with
|
|
42
|
+
`npm install unpdf` to use the default engine. Without it the tool returns `pdf_engine_missing`
|
|
43
|
+
with that command and the load error. Pass `extractText` to use another engine. `unpdf` 2.x
|
|
44
|
+
declares Node >=24 in its `engines`; the tool's tests pass with it on Node 22.12.
|
|
45
|
+
|
|
46
|
+
The tool ships from its own entry, not from the package root, so `@theokit/sdk-tools` itself does
|
|
47
|
+
not grow: `import { createReadPdfTool } from "@theokit/sdk-tools/read-pdf"`.
|
|
48
|
+
|
|
49
|
+
## 0.27.6
|
|
50
|
+
|
|
51
|
+
### Patch Changes
|
|
52
|
+
|
|
53
|
+
- [#724](https://github.com/usetheokit/theokit-sdk/pull/724) [`d58f5ef`](https://github.com/usetheokit/theokit-sdk/commit/d58f5efbc911ffa429082688a53b36fad659e305) Thanks [@usetheodev](https://github.com/usetheodev)! - The optional `simple-git` peer range moves from `^3.0.0` to `^4.0.1`. Every 3.x release up to
|
|
54
|
+
3.36.0 is covered by published advisories (GHSA-x6jw-m9v5-85vh, GHSA-g4wm-2vf7-vfgr,
|
|
55
|
+
GHSA-858h-whjf-mvg5): Git configuration includes and abbreviated long options can lead to command
|
|
56
|
+
execution. 4.0.1 is the first release that fixes all of them.
|
|
57
|
+
|
|
58
|
+
No tool in this package imports `simple-git`. `createGitDiffTool` and `createGitStatusTool` run
|
|
59
|
+
`git` as a child process, so their behaviour does not change. A project that still has
|
|
60
|
+
`simple-git` 3.x installed gets an unmet-peer warning and nothing else.
|
|
61
|
+
|
|
3
62
|
## 0.27.5
|
|
4
63
|
|
|
5
64
|
### Patch Changes
|
|
@@ -0,0 +1,175 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
var fs = require('fs');
|
|
4
|
+
var path = require('path');
|
|
5
|
+
var sdk = require('@theokit/sdk');
|
|
6
|
+
var zod = require('zod');
|
|
7
|
+
var pathSafety = require('@theokit/sdk/path-safety');
|
|
8
|
+
|
|
9
|
+
// src/extract-document.ts
|
|
10
|
+
|
|
11
|
+
// src/path-scope.ts
|
|
12
|
+
function checkPathScope(path, projectRoot) {
|
|
13
|
+
if (path === void 0 || path === "") return null;
|
|
14
|
+
try {
|
|
15
|
+
const abs = pathSafety.safePathJoin(projectRoot, path);
|
|
16
|
+
pathSafety.assertNoSymlinkEscape(abs, projectRoot);
|
|
17
|
+
return null;
|
|
18
|
+
} catch (err) {
|
|
19
|
+
if (err instanceof pathSafety.PathTraversalError || err instanceof pathSafety.ForbiddenPathError) {
|
|
20
|
+
return JSON.stringify({ ok: false, error: "path_traversal", path });
|
|
21
|
+
}
|
|
22
|
+
throw err;
|
|
23
|
+
}
|
|
24
|
+
}
|
|
25
|
+
var SENSITIVE_SEGMENTS = /* @__PURE__ */ new Set([".env", ".git", "node_modules", ".theo"]);
|
|
26
|
+
function isForbiddenAtAnyDepth(path) {
|
|
27
|
+
const segs = path.replace(/\\/g, "/").split("/").filter(Boolean);
|
|
28
|
+
return segs.some((s) => {
|
|
29
|
+
if (s === ".env.example") return false;
|
|
30
|
+
return SENSITIVE_SEGMENTS.has(s) || /^\.env\./.test(s);
|
|
31
|
+
});
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
// src/extract-document.ts
|
|
35
|
+
var DEFAULT_MAX_DOCUMENT_BYTES = 10 * 1024 * 1024;
|
|
36
|
+
var DEFAULT_MAX_DOCUMENT_CHARS = 1e5;
|
|
37
|
+
var SUPPORTED_EXTENSIONS = [".docx"];
|
|
38
|
+
var ZIP_SIGNATURE = [80, 75, 3, 4];
|
|
39
|
+
var INSTALL_COMMAND = "npm install mammoth";
|
|
40
|
+
var limitsSchema = zod.z.object({
|
|
41
|
+
maxBytes: zod.z.number().int().positive(),
|
|
42
|
+
maxChars: zod.z.number().int().positive()
|
|
43
|
+
});
|
|
44
|
+
var json = (result) => JSON.stringify(result);
|
|
45
|
+
function createExtractDocumentTool(options) {
|
|
46
|
+
const { projectRoot } = options;
|
|
47
|
+
const limits = parseLimits(options);
|
|
48
|
+
const extract = options.extractText;
|
|
49
|
+
return sdk.Tool.create({
|
|
50
|
+
name: options.name ?? "extract_document",
|
|
51
|
+
description: options.description ?? "Extract the text of an office document (.docx) in the project. Returns the text, cut at a character limit.",
|
|
52
|
+
inputSchema: zod.z.object({
|
|
53
|
+
path: zod.z.string().describe("Path to the document, relative to the project root.")
|
|
54
|
+
}),
|
|
55
|
+
handler: async (input) => {
|
|
56
|
+
const refused = checkPathScope(input.path, projectRoot);
|
|
57
|
+
if (refused !== null) return refused;
|
|
58
|
+
if (isForbiddenAtAnyDepth(input.path)) {
|
|
59
|
+
return json({ ok: false, error: "path_traversal", path: input.path });
|
|
60
|
+
}
|
|
61
|
+
if (!isSupported(input.path)) {
|
|
62
|
+
return json({
|
|
63
|
+
ok: false,
|
|
64
|
+
error: "unsupported_document_type",
|
|
65
|
+
path: input.path,
|
|
66
|
+
supported: [...SUPPORTED_EXTENSIONS]
|
|
67
|
+
});
|
|
68
|
+
}
|
|
69
|
+
const outcome = readWithinBudget(pathSafety.safePathJoin(projectRoot, input.path), limits.maxBytes);
|
|
70
|
+
if (!outcome.ok) return json({ ...outcome.failure, path: input.path });
|
|
71
|
+
return json(await extractWithin(outcome.bytes, input.path, limits.maxChars, extract));
|
|
72
|
+
}
|
|
73
|
+
});
|
|
74
|
+
}
|
|
75
|
+
var isSupported = (path$1) => SUPPORTED_EXTENSIONS.includes(path.extname(path$1).toLowerCase());
|
|
76
|
+
function parseLimits(options) {
|
|
77
|
+
const parsed = limitsSchema.safeParse({
|
|
78
|
+
maxBytes: options.maxBytes ?? DEFAULT_MAX_DOCUMENT_BYTES,
|
|
79
|
+
maxChars: options.maxChars ?? DEFAULT_MAX_DOCUMENT_CHARS
|
|
80
|
+
});
|
|
81
|
+
if (!parsed.success) {
|
|
82
|
+
const detail = parsed.error.issues.map((issue) => `${issue.path.join(".")}: ${issue.message}`).join("; ");
|
|
83
|
+
throw new RangeError(`createExtractDocumentTool: invalid options (${detail})`);
|
|
84
|
+
}
|
|
85
|
+
return parsed.data;
|
|
86
|
+
}
|
|
87
|
+
var NOT_A_DOCUMENT = {
|
|
88
|
+
ok: false,
|
|
89
|
+
failure: { ok: false, error: "not_a_document" }
|
|
90
|
+
};
|
|
91
|
+
var hasZipSignature = (bytes) => ZIP_SIGNATURE.every((byte, index) => bytes[index] === byte);
|
|
92
|
+
function readWithinBudget(absolute, maxBytes) {
|
|
93
|
+
const opened = openForRead(absolute);
|
|
94
|
+
if (!opened.ok) return opened;
|
|
95
|
+
const { fd } = opened;
|
|
96
|
+
try {
|
|
97
|
+
const stat = fs.fstatSync(fd);
|
|
98
|
+
if (!stat.isFile()) return NOT_A_DOCUMENT;
|
|
99
|
+
if (stat.size > maxBytes) {
|
|
100
|
+
return {
|
|
101
|
+
ok: false,
|
|
102
|
+
failure: {
|
|
103
|
+
ok: false,
|
|
104
|
+
error: "document_too_large",
|
|
105
|
+
bytes: stat.size,
|
|
106
|
+
limit_bytes: maxBytes
|
|
107
|
+
}
|
|
108
|
+
};
|
|
109
|
+
}
|
|
110
|
+
const buffer = Buffer.alloc(stat.size);
|
|
111
|
+
const bytes = new Uint8Array(buffer.subarray(0, fs.readSync(fd, buffer, 0, stat.size, 0)));
|
|
112
|
+
return hasZipSignature(bytes) ? { ok: true, bytes } : NOT_A_DOCUMENT;
|
|
113
|
+
} finally {
|
|
114
|
+
fs.closeSync(fd);
|
|
115
|
+
}
|
|
116
|
+
}
|
|
117
|
+
function openForRead(absolute) {
|
|
118
|
+
try {
|
|
119
|
+
return { ok: true, fd: fs.openSync(absolute, "r") };
|
|
120
|
+
} catch (err) {
|
|
121
|
+
if (err.code === "ENOENT") {
|
|
122
|
+
return { ok: false, failure: { ok: false, error: "not_found" } };
|
|
123
|
+
}
|
|
124
|
+
return {
|
|
125
|
+
ok: false,
|
|
126
|
+
failure: { ok: false, error: "document_unreadable", message: messageOf(err) }
|
|
127
|
+
};
|
|
128
|
+
}
|
|
129
|
+
}
|
|
130
|
+
var messageOf = (err) => err instanceof Error ? err.message : String(err);
|
|
131
|
+
async function extractWithin(bytes, path, maxChars, injected) {
|
|
132
|
+
const engine = injected ? { ok: true, extract: injected } : await loadMammoth();
|
|
133
|
+
if (!engine.ok) {
|
|
134
|
+
return {
|
|
135
|
+
ok: false,
|
|
136
|
+
error: "document_engine_missing",
|
|
137
|
+
path,
|
|
138
|
+
install: INSTALL_COMMAND,
|
|
139
|
+
message: engine.message
|
|
140
|
+
};
|
|
141
|
+
}
|
|
142
|
+
let text;
|
|
143
|
+
try {
|
|
144
|
+
text = (await engine.extract(bytes)).trim();
|
|
145
|
+
} catch (err) {
|
|
146
|
+
return { ok: false, error: "document_unreadable", path, message: messageOf(err) };
|
|
147
|
+
}
|
|
148
|
+
return {
|
|
149
|
+
ok: true,
|
|
150
|
+
path,
|
|
151
|
+
format: "docx",
|
|
152
|
+
chars: text.length,
|
|
153
|
+
truncated: text.length > maxChars,
|
|
154
|
+
text: text.slice(0, maxChars)
|
|
155
|
+
};
|
|
156
|
+
}
|
|
157
|
+
async function loadMammoth() {
|
|
158
|
+
try {
|
|
159
|
+
const loaded = await import('mammoth');
|
|
160
|
+
const mammoth = loaded.default ?? loaded;
|
|
161
|
+
return {
|
|
162
|
+
ok: true,
|
|
163
|
+
extract: async (bytes) => {
|
|
164
|
+
const buffer = Buffer.from(bytes.buffer, bytes.byteOffset, bytes.byteLength);
|
|
165
|
+
return (await mammoth.extractRawText({ buffer })).value;
|
|
166
|
+
}
|
|
167
|
+
};
|
|
168
|
+
} catch (err) {
|
|
169
|
+
return { ok: false, message: messageOf(err) };
|
|
170
|
+
}
|
|
171
|
+
}
|
|
172
|
+
|
|
173
|
+
exports.createExtractDocumentTool = createExtractDocumentTool;
|
|
174
|
+
//# sourceMappingURL=extract-document.cjs.map
|
|
175
|
+
//# sourceMappingURL=extract-document.cjs.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"sources":["../src/path-scope.ts","../src/extract-document.ts"],"names":["safePathJoin","assertNoSymlinkEscape","PathTraversalError","ForbiddenPathError","z","Tool","path","extname","fstatSync","readSync","closeSync","openSync"],"mappings":";;;;;;;;;;;AAeO,SAAS,cAAA,CAAe,MAA0B,WAAA,EAAoC;AAC3F,EAAA,IAAI,IAAA,KAAS,MAAA,IAAa,IAAA,KAAS,EAAA,EAAI,OAAO,IAAA;AAC9C,EAAA,IAAI;AACF,IAAA,MAAM,GAAA,GAAMA,uBAAA,CAAa,WAAA,EAAa,IAAI,CAAA;AAC1C,IAAAC,gCAAA,CAAsB,KAAK,WAAW,CAAA;AACtC,IAAA,OAAO,IAAA;AAAA,EACT,SAAS,GAAA,EAAK;AACZ,IAAA,IAAI,GAAA,YAAeC,6BAAA,IAAsB,GAAA,YAAeC,6BAAA,EAAoB;AAC1E,MAAA,OAAO,IAAA,CAAK,UAAU,EAAE,EAAA,EAAI,OAAO,KAAA,EAAO,gBAAA,EAAkB,MAAM,CAAA;AAAA,IACpE;AACA,IAAA,MAAM,GAAA;AAAA,EACR;AACF;AASA,IAAM,kBAAA,uBAAyB,GAAA,CAAI,CAAC,QAAQ,MAAA,EAAQ,cAAA,EAAgB,OAAO,CAAC,CAAA;AASrE,SAAS,sBAAsB,IAAA,EAAuB;AAC3D,EAAA,MAAM,IAAA,GAAO,IAAA,CAAK,OAAA,CAAQ,KAAA,EAAO,GAAG,EAAE,KAAA,CAAM,GAAG,CAAA,CAAE,MAAA,CAAO,OAAO,CAAA;AAC/D,EAAA,OAAO,IAAA,CAAK,IAAA,CAAK,CAAC,CAAA,KAAM;AACtB,IAAA,IAAI,CAAA,KAAM,gBAAgB,OAAO,KAAA;AACjC,IAAA,OAAO,mBAAmB,GAAA,CAAI,CAAC,CAAA,IAAK,UAAA,CAAW,KAAK,CAAC,CAAA;AAAA,EACvD,CAAC,CAAA;AACH;;;ACZA,IAAM,0BAAA,GAA6B,KAAK,IAAA,GAAO,IAAA;AAG/C,IAAM,0BAAA,GAA6B,GAAA;AAGnC,IAAM,oBAAA,GAAuB,CAAC,OAAO,CAAA;AAGrC,IAAM,aAAA,GAAgB,CAAC,EAAA,EAAM,EAAA,EAAM,GAAM,CAAI,CAAA;AAE7C,IAAM,eAAA,GAAkB,qBAAA;AAyBxB,IAAM,YAAA,GAAeC,MAAE,MAAA,CAAO;AAAA,EAC5B,UAAUA,KAAA,CAAE,MAAA,EAAO,CAAE,GAAA,GAAM,QAAA,EAAS;AAAA,EACpC,UAAUA,KAAA,CAAE,MAAA,EAAO,CAAE,GAAA,GAAM,QAAA;AAC7B,CAAC,CAAA;AA6BD,IAAM,IAAA,GAAO,CAAC,MAAA,KAA+D,IAAA,CAAK,UAAU,MAAM,CAAA;AAO3F,SAAS,0BAA0B,OAAA,EAAuD;AAC/F,EAAA,MAAM,EAAE,aAAY,GAAI,OAAA;AACxB,EAAA,MAAM,MAAA,GAAS,YAAY,OAAO,CAAA;AAClC,EAAA,MAAM,UAAU,OAAA,CAAQ,WAAA;AAExB,EAAA,OAAOC,SAAK,MAAA,CAAO;AAAA,IACjB,IAAA,EAAM,QAAQ,IAAA,IAAQ,kBAAA;AAAA,IACtB,WAAA,EACE,QAAQ,WAAA,IACR,4GAAA;AAAA,IACF,WAAA,EAAaD,MAAE,MAAA,CAAO;AAAA,MACpB,IAAA,EAAMA,KAAA,CAAE,MAAA,EAAO,CAAE,SAAS,qDAAqD;AAAA,KAChF,CAAA;AAAA,IACD,OAAA,EAAS,OAAO,KAAA,KAA6C;AAG3D,MAAA,MAAM,OAAA,GAAU,cAAA,CAAe,KAAA,CAAM,IAAA,EAAM,WAAW,CAAA;AACtD,MAAA,IAAI,OAAA,KAAY,MAAM,OAAO,OAAA;AAC7B,MAAA,IAAI,qBAAA,CAAsB,KAAA,CAAM,IAAI,CAAA,EAAG;AACrC,QAAA,OAAO,IAAA,CAAK,EAAE,EAAA,EAAI,KAAA,EAAO,OAAO,gBAAA,EAAkB,IAAA,EAAM,KAAA,CAAM,IAAA,EAAM,CAAA;AAAA,MACtE;AACA,MAAA,IAAI,CAAC,WAAA,CAAY,KAAA,CAAM,IAAI,CAAA,EAAG;AAC5B,QAAA,OAAO,IAAA,CAAK;AAAA,UACV,EAAA,EAAI,KAAA;AAAA,UACJ,KAAA,EAAO,2BAAA;AAAA,UACP,MAAM,KAAA,CAAM,IAAA;AAAA,UACZ,SAAA,EAAW,CAAC,GAAG,oBAAoB;AAAA,SACpC,CAAA;AAAA,MACH;AAEA,MAAA,MAAM,OAAA,GAAU,iBAAiBJ,uBAAA,CAAa,WAAA,EAAa,MAAM,IAAI,CAAA,EAAG,OAAO,QAAQ,CAAA;AACvF,MAAA,IAAI,CAAC,OAAA,CAAQ,EAAA,EAAI,OAAO,IAAA,CAAK,EAAE,GAAG,OAAA,CAAQ,OAAA,EAAS,IAAA,EAAM,KAAA,CAAM,IAAA,EAAM,CAAA;AAErE,MAAA,OAAO,IAAA,CAAK,MAAM,aAAA,CAAc,OAAA,CAAQ,KAAA,EAAO,MAAM,IAAA,EAAM,MAAA,CAAO,QAAA,EAAU,OAAO,CAAC,CAAA;AAAA,IACtF;AAAA,GACD,CAAA;AACH;AAEA,IAAM,WAAA,GAAc,CAACM,MAAA,KAClB,oBAAA,CAA2C,SAASC,YAAA,CAAQD,MAAI,CAAA,CAAE,WAAA,EAAa,CAAA;AAGlF,SAAS,YAAY,OAAA,EAAyE;AAC5F,EAAA,MAAM,MAAA,GAAS,aAAa,SAAA,CAAU;AAAA,IACpC,QAAA,EAAU,QAAQ,QAAA,IAAY,0BAAA;AAAA,IAC9B,QAAA,EAAU,QAAQ,QAAA,IAAY;AAAA,GAC/B,CAAA;AACD,EAAA,IAAI,CAAC,OAAO,OAAA,EAAS;AACnB,IAAA,MAAM,SAAS,MAAA,CAAO,KAAA,CAAM,OACzB,GAAA,CAAI,CAAC,UAAU,CAAA,EAAG,KAAA,CAAM,KAAK,IAAA,CAAK,GAAG,CAAC,CAAA,EAAA,EAAK,KAAA,CAAM,OAAO,CAAA,CAAE,CAAA,CAC1D,KAAK,IAAI,CAAA;AACZ,IAAA,MAAM,IAAI,UAAA,CAAW,CAAA,4CAAA,EAA+C,MAAM,CAAA,CAAA,CAAG,CAAA;AAAA,EAC/E;AACA,EAAA,OAAO,MAAA,CAAO,IAAA;AAChB;AAMA,IAAM,cAAA,GAA8B;AAAA,EAClC,EAAA,EAAI,KAAA;AAAA,EACJ,OAAA,EAAS,EAAE,EAAA,EAAI,KAAA,EAAO,OAAO,gBAAA;AAC/B,CAAA;AAEA,IAAM,eAAA,GAAkB,CAAC,KAAA,KACvB,aAAA,CAAc,KAAA,CAAM,CAAC,IAAA,EAAM,KAAA,KAAU,KAAA,CAAM,KAAK,CAAA,KAAM,IAAI,CAAA;AAO5D,SAAS,gBAAA,CAAiB,UAAkB,QAAA,EAA+B;AACzE,EAAA,MAAM,MAAA,GAAS,YAAY,QAAQ,CAAA;AACnC,EAAA,IAAI,CAAC,MAAA,CAAO,EAAA,EAAI,OAAO,MAAA;AACvB,EAAA,MAAM,EAAE,IAAG,GAAI,MAAA;AACf,EAAA,IAAI;AACF,IAAA,MAAM,IAAA,GAAOE,aAAU,EAAE,CAAA;AAEzB,IAAA,IAAI,CAAC,IAAA,CAAK,MAAA,EAAO,EAAG,OAAO,cAAA;AAE3B,IAAA,IAAI,IAAA,CAAK,OAAO,QAAA,EAAU;AACxB,MAAA,OAAO;AAAA,QACL,EAAA,EAAI,KAAA;AAAA,QACJ,OAAA,EAAS;AAAA,UACP,EAAA,EAAI,KAAA;AAAA,UACJ,KAAA,EAAO,oBAAA;AAAA,UACP,OAAO,IAAA,CAAK,IAAA;AAAA,UACZ,WAAA,EAAa;AAAA;AACf,OACF;AAAA,IACF;AACA,IAAA,MAAM,MAAA,GAAS,MAAA,CAAO,KAAA,CAAM,IAAA,CAAK,IAAI,CAAA;AACrC,IAAA,MAAM,KAAA,GAAQ,IAAI,UAAA,CAAW,MAAA,CAAO,SAAS,CAAA,EAAGC,WAAA,CAAS,EAAA,EAAI,MAAA,EAAQ,CAAA,EAAG,IAAA,CAAK,IAAA,EAAM,CAAC,CAAC,CAAC,CAAA;AACtF,IAAA,OAAO,gBAAgB,KAAK,CAAA,GAAI,EAAE,EAAA,EAAI,IAAA,EAAM,OAAM,GAAI,cAAA;AAAA,EACxD,CAAA,SAAE;AACA,IAAAC,YAAA,CAAU,EAAE,CAAA;AAAA,EACd;AACF;AAMA,SAAS,YACP,QAAA,EACkF;AAClF,EAAA,IAAI;AACF,IAAA,OAAO,EAAE,EAAA,EAAI,IAAA,EAAM,IAAIC,WAAA,CAAS,QAAA,EAAU,GAAG,CAAA,EAAE;AAAA,EACjD,SAAS,GAAA,EAAK;AACZ,IAAA,IAAK,GAAA,CAA8B,SAAS,QAAA,EAAU;AACpD,MAAA,OAAO,EAAE,IAAI,KAAA,EAAO,OAAA,EAAS,EAAE,EAAA,EAAI,KAAA,EAAO,KAAA,EAAO,WAAA,EAAY,EAAE;AAAA,IACjE;AACA,IAAA,OAAO;AAAA,MACL,EAAA,EAAI,KAAA;AAAA,MACJ,OAAA,EAAS,EAAE,EAAA,EAAI,KAAA,EAAO,OAAO,qBAAA,EAAuB,OAAA,EAAS,SAAA,CAAU,GAAG,CAAA;AAAE,KAC9E;AAAA,EACF;AACF;AAEA,IAAM,SAAA,GAAY,CAAC,GAAA,KAA0B,GAAA,YAAe,QAAQ,GAAA,CAAI,OAAA,GAAU,OAAO,GAAG,CAAA;AAG5F,eAAe,aAAA,CACb,KAAA,EACA,IAAA,EACA,QAAA,EACA,QAAA,EACqD;AACrD,EAAA,MAAM,MAAA,GAAS,WAAW,EAAE,EAAA,EAAI,MAAe,OAAA,EAAS,QAAA,EAAS,GAAI,MAAM,WAAA,EAAY;AACvF,EAAA,IAAI,CAAC,OAAO,EAAA,EAAI;AACd,IAAA,OAAO;AAAA,MACL,EAAA,EAAI,KAAA;AAAA,MACJ,KAAA,EAAO,yBAAA;AAAA,MACP,IAAA;AAAA,MACA,OAAA,EAAS,eAAA;AAAA,MACT,SAAS,MAAA,CAAO;AAAA,KAClB;AAAA,EACF;AACA,EAAA,IAAI,IAAA;AACJ,EAAA,IAAI;AACF,IAAA,IAAA,GAAA,CAAQ,MAAM,MAAA,CAAO,OAAA,CAAQ,KAAK,GAAG,IAAA,EAAK;AAAA,EAC5C,SAAS,GAAA,EAAK;AAEZ,IAAA,OAAO,EAAE,IAAI,KAAA,EAAO,KAAA,EAAO,uBAAuB,IAAA,EAAM,OAAA,EAAS,SAAA,CAAU,GAAG,CAAA,EAAE;AAAA,EAClF;AACA,EAAA,OAAO;AAAA,IACL,EAAA,EAAI,IAAA;AAAA,IACJ,IAAA;AAAA,IACA,MAAA,EAAQ,MAAA;AAAA,IACR,OAAO,IAAA,CAAK,MAAA;AAAA,IACZ,SAAA,EAAW,KAAK,MAAA,GAAS,QAAA;AAAA,IACzB,IAAA,EAAM,IAAA,CAAK,KAAA,CAAM,CAAA,EAAG,QAAQ;AAAA,GAC9B;AACF;AAeA,eAAe,WAAA,GAGb;AACA,EAAA,IAAI;AACF,IAAA,MAAM,MAAA,GAAU,MAAM,OAAO,SAAS,CAAA;AACtC,IAAA,MAAM,OAAA,GAAU,OAAO,OAAA,IAAW,MAAA;AAClC,IAAA,OAAO;AAAA,MACL,EAAA,EAAI,IAAA;AAAA,MACJ,OAAA,EAAS,OAAO,KAAA,KAAU;AACxB,QAAA,MAAM,MAAA,GAAS,OAAO,IAAA,CAAK,KAAA,CAAM,QAAQ,KAAA,CAAM,UAAA,EAAY,MAAM,UAAU,CAAA;AAC3E,QAAA,OAAA,CAAQ,MAAM,OAAA,CAAQ,cAAA,CAAe,EAAE,MAAA,EAAQ,CAAA,EAAG,KAAA;AAAA,MACpD;AAAA,KACF;AAAA,EACF,SAAS,GAAA,EAAK;AACZ,IAAA,OAAO,EAAE,EAAA,EAAI,KAAA,EAAO,OAAA,EAAS,SAAA,CAAU,GAAG,CAAA,EAAE;AAAA,EAC9C;AACF","file":"extract-document.cjs","sourcesContent":["/**\n * Shared `safePathJoin + assertNoSymlinkEscape` helper for built-in coding\n * tools (git-diff, run-vitest, list-dir, read-file, search-text). Returns a\n * pre-formatted `path_traversal` JSON string on failure or `null` on pass.\n *\n * @internal\n */\n\nimport {\n assertNoSymlinkEscape,\n ForbiddenPathError,\n PathTraversalError,\n safePathJoin,\n} from \"./internal/path-guard.js\";\n\nexport function checkPathScope(path: string | undefined, projectRoot: string): string | null {\n if (path === undefined || path === \"\") return null;\n try {\n const abs = safePathJoin(projectRoot, path);\n assertNoSymlinkEscape(abs, projectRoot);\n return null;\n } catch (err) {\n if (err instanceof PathTraversalError || err instanceof ForbiddenPathError) {\n return JSON.stringify({ ok: false, error: \"path_traversal\", path });\n }\n throw err;\n }\n}\n\n/**\n * Segments that may never appear in a path honored by `allowAbsolute`.\n *\n * M76 — promoted from `read-file.ts`, where it was private. Duplicating it in `list-dir` would duplicate\n * security KNOWLEDGE: the copies would have to agree on what counts as a secret, and one fixed\n * without the other reopens the hole in the forgotten tool.\n */\nconst SENSITIVE_SEGMENTS = new Set([\".env\", \".git\", \"node_modules\", \".theo\"]);\n\n/**\n * ANY-segment secret guard — the half of `allowAbsolute` that cannot be separated.\n *\n * `isForbiddenPath` only blocks the sensitive item when it is the FIRST segment (relative to the\n * project). An absolute path (`/home/u/proj/.env/sub`) puts it deeper, and it would pass. This one\n * checks every segment, closing the \"reads-anywhere\" exfiltration.\n */\nexport function isForbiddenAtAnyDepth(path: string): boolean {\n const segs = path.replace(/\\\\/g, \"/\").split(\"/\").filter(Boolean);\n return segs.some((s) => {\n if (s === \".env.example\") return false; // a template — safe\n return SENSITIVE_SEGMENTS.has(s) || /^\\.env\\./.test(s);\n });\n}\n","/**\n * `extract_document` — let the agent read the text of an office document in the project.\n *\n * ## Why this is a built-in\n *\n * A `.docx` is a zip of XML parts: `read_file` returns compressed bytes, not the words in it. The\n * confinement is the same as `view_image`'s and is the part worth reviewing once: a document\n * reader that honours any path is a file exfiltration primitive.\n *\n * ## The engine\n *\n * Text extraction is delegated to `mammoth`, an OPTIONAL peer loaded lazily on the first call, so\n * a consumer of the other tools never pays for it. Its absence is a typed `document_engine_missing`\n * result naming the install command, never a throw. A consumer can pass `extractText` to use any\n * other engine; the envelope stays the same.\n *\n * ## Result shape\n *\n * - `{ ok: true, path, format: \"docx\", chars, truncated, text }` — `text` is the trimmed\n * extracted text cut at `maxChars`; `chars` is the length before the cut.\n * - `{ ok: false, error: \"path_traversal\" | \"unsupported_document_type\" | \"not_found\" |\n * \"not_a_document\" | \"document_too_large\" | \"document_unreadable\" |\n * \"document_engine_missing\", path, … }`\n *\n * `maxBytes` bounds the file read, not what the engine inflates it to; `maxChars` bounds what\n * reaches the model.\n */\n\nimport { closeSync, fstatSync, openSync, readSync } from \"node:fs\";\nimport { extname } from \"node:path\";\n\nimport type { CustomTool } from \"@theokit/sdk\";\nimport { Tool } from \"@theokit/sdk\";\nimport { z } from \"zod\";\n\nimport { safePathJoin } from \"./internal/path-guard.js\";\nimport { checkPathScope, isForbiddenAtAnyDepth } from \"./path-scope.js\";\n\n/** Default ceiling on the file read, in bytes on disk: 10 MB. */\nconst DEFAULT_MAX_DOCUMENT_BYTES = 10 * 1024 * 1024;\n\n/** Default ceiling on the text returned to the model, in characters. */\nconst DEFAULT_MAX_DOCUMENT_CHARS = 100_000;\n\n/** The extensions the tool reads, lower-cased. A new format is one entry plus its engine. */\nconst SUPPORTED_EXTENSIONS = [\".docx\"] as const;\n\n/** A `.docx` is a zip archive; every zip starts with this local-file-header signature. */\nconst ZIP_SIGNATURE = [0x50, 0x4b, 0x03, 0x04] as const;\n\nconst INSTALL_COMMAND = \"npm install mammoth\";\n\n/** The engine contract: document bytes in, raw text out. */\ntype DocumentTextExtractor = (bytes: Uint8Array) => Promise<string>;\n\n/**\n * Options for {@link createExtractDocumentTool}.\n *\n * @public\n */\nexport interface CreateExtractDocumentToolOptions {\n /** Root the tool reads from. Every path is resolved inside it. */\n projectRoot: string;\n /** Name exposed to the model. Omitted ⇒ `extract_document`. */\n name?: string;\n /** Description exposed to the model. Omitted ⇒ the literal below. */\n description?: string;\n /** Ceiling on the file size in bytes. Omitted ⇒ {@link DEFAULT_MAX_DOCUMENT_BYTES}. */\n maxBytes?: number;\n /** Ceiling on the returned text in characters. Omitted ⇒ {@link DEFAULT_MAX_DOCUMENT_CHARS}. */\n maxChars?: number;\n /** Engine that turns the bytes into text. Omitted ⇒ the optional peer `mammoth`, loaded lazily. */\n extractText?: DocumentTextExtractor;\n}\n\nconst limitsSchema = z.object({\n maxBytes: z.number().int().positive(),\n maxChars: z.number().int().positive(),\n});\n\ntype DocumentErrorCode =\n | \"path_traversal\"\n | \"unsupported_document_type\"\n | \"not_found\"\n | \"not_a_document\"\n | \"document_too_large\"\n | \"document_unreadable\"\n | \"document_engine_missing\";\n\ninterface ExtractDocumentOk {\n ok: true;\n path: string;\n format: \"docx\";\n chars: number;\n truncated: boolean;\n text: string;\n}\n\n/** A failure before the path is attached: the read helper does not know the input path. */\ninterface DocumentFailureBody {\n ok: false;\n error: DocumentErrorCode;\n [extra: string]: unknown;\n}\n\ntype ExtractDocumentFailure = DocumentFailureBody & { path: string };\n\nconst json = (result: ExtractDocumentOk | ExtractDocumentFailure): string => JSON.stringify(result);\n\n/**\n * Extract the text of a `.docx` inside the project root.\n *\n * @public\n */\nexport function createExtractDocumentTool(options: CreateExtractDocumentToolOptions): CustomTool {\n const { projectRoot } = options;\n const limits = parseLimits(options);\n const extract = options.extractText;\n\n return Tool.create({\n name: options.name ?? \"extract_document\",\n description:\n options.description ??\n \"Extract the text of an office document (.docx) in the project. Returns the text, cut at a character limit.\",\n inputSchema: z.object({\n path: z.string().describe(\"Path to the document, relative to the project root.\"),\n }),\n handler: async (input: { path: string }): Promise<string> => {\n // Same order as `view_image`: the shared scope guard (lexical and symlink escapes), then the\n // any-depth secret guard, and only then a join that can no longer throw outside the envelope.\n const refused = checkPathScope(input.path, projectRoot);\n if (refused !== null) return refused;\n if (isForbiddenAtAnyDepth(input.path)) {\n return json({ ok: false, error: \"path_traversal\", path: input.path });\n }\n if (!isSupported(input.path)) {\n return json({\n ok: false,\n error: \"unsupported_document_type\",\n path: input.path,\n supported: [...SUPPORTED_EXTENSIONS],\n });\n }\n\n const outcome = readWithinBudget(safePathJoin(projectRoot, input.path), limits.maxBytes);\n if (!outcome.ok) return json({ ...outcome.failure, path: input.path });\n\n return json(await extractWithin(outcome.bytes, input.path, limits.maxChars, extract));\n },\n }) as unknown as CustomTool;\n}\n\nconst isSupported = (path: string): boolean =>\n (SUPPORTED_EXTENSIONS as readonly string[]).includes(extname(path).toLowerCase());\n\n/** Validates the numeric options once, at construction, so a bad limit fails the factory call. */\nfunction parseLimits(options: CreateExtractDocumentToolOptions): z.infer<typeof limitsSchema> {\n const parsed = limitsSchema.safeParse({\n maxBytes: options.maxBytes ?? DEFAULT_MAX_DOCUMENT_BYTES,\n maxChars: options.maxChars ?? DEFAULT_MAX_DOCUMENT_CHARS,\n });\n if (!parsed.success) {\n const detail = parsed.error.issues\n .map((issue) => `${issue.path.join(\".\")}: ${issue.message}`)\n .join(\"; \");\n throw new RangeError(`createExtractDocumentTool: invalid options (${detail})`);\n }\n return parsed.data;\n}\n\ntype ReadOutcome =\n | { readonly ok: true; readonly bytes: Uint8Array }\n | { readonly ok: false; readonly failure: DocumentFailureBody };\n\nconst NOT_A_DOCUMENT: ReadOutcome = {\n ok: false,\n failure: { ok: false, error: \"not_a_document\" },\n};\n\nconst hasZipSignature = (bytes: Uint8Array): boolean =>\n ZIP_SIGNATURE.every((byte, index) => bytes[index] === byte);\n\n/**\n * Sizes, reads and checks one file through a single descriptor (CodeQL js/file-system-race): the\n * size cap must describe the bytes actually returned, so the read is bounded by the size the\n * descriptor reported, and a file that grows after `fstat` cannot push past the cap.\n */\nfunction readWithinBudget(absolute: string, maxBytes: number): ReadOutcome {\n const opened = openForRead(absolute);\n if (!opened.ok) return opened;\n const { fd } = opened;\n try {\n const stat = fstatSync(fd);\n // A directory named `x.docx` opens on Linux; it is still not a document.\n if (!stat.isFile()) return NOT_A_DOCUMENT;\n // `>` and not `>=`: a file of exactly the limit fits.\n if (stat.size > maxBytes) {\n return {\n ok: false,\n failure: {\n ok: false,\n error: \"document_too_large\",\n bytes: stat.size,\n limit_bytes: maxBytes,\n },\n };\n }\n const buffer = Buffer.alloc(stat.size);\n const bytes = new Uint8Array(buffer.subarray(0, readSync(fd, buffer, 0, stat.size, 0)));\n return hasZipSignature(bytes) ? { ok: true, bytes } : NOT_A_DOCUMENT;\n } finally {\n closeSync(fd);\n }\n}\n\n/**\n * Opens the file, telling a missing file (`not_found`, which the model answers with another path)\n * apart from one it may not read (`document_unreadable` with the system's message).\n */\nfunction openForRead(\n absolute: string,\n): { readonly ok: true; readonly fd: number } | Extract<ReadOutcome, { ok: false }> {\n try {\n return { ok: true, fd: openSync(absolute, \"r\") };\n } catch (err) {\n if ((err as NodeJS.ErrnoException).code === \"ENOENT\") {\n return { ok: false, failure: { ok: false, error: \"not_found\" } };\n }\n return {\n ok: false,\n failure: { ok: false, error: \"document_unreadable\", message: messageOf(err) },\n };\n }\n}\n\nconst messageOf = (err: unknown): string => (err instanceof Error ? err.message : String(err));\n\n/** Runs the engine, trims and cuts its text, mapping every engine failure to a typed result. */\nasync function extractWithin(\n bytes: Uint8Array,\n path: string,\n maxChars: number,\n injected: DocumentTextExtractor | undefined,\n): Promise<ExtractDocumentOk | ExtractDocumentFailure> {\n const engine = injected ? { ok: true as const, extract: injected } : await loadMammoth();\n if (!engine.ok) {\n return {\n ok: false,\n error: \"document_engine_missing\",\n path,\n install: INSTALL_COMMAND,\n message: engine.message,\n };\n }\n let text: string;\n try {\n text = (await engine.extract(bytes)).trim();\n } catch (err) {\n // A zip that is not a Word document (an `.xlsx` renamed, a corrupt or encrypted file) lands here.\n return { ok: false, error: \"document_unreadable\", path, message: messageOf(err) };\n }\n return {\n ok: true,\n path,\n format: \"docx\",\n chars: text.length,\n truncated: text.length > maxChars,\n text: text.slice(0, maxChars),\n };\n}\n\n/** The slice of `mammoth`'s API this tool uses; declared here so the build never needs its types. */\ninterface MammothLike {\n extractRawText(input: { buffer: Buffer }): Promise<{ value: string }>;\n}\n\n/**\n * Loads the optional `mammoth` peer, or the reason it could not be loaded.\n *\n * `mammoth` is CommonJS: an ESM `import()` exposes it under `default`, the CJS build's `require`\n * returns it bare, so both shapes are read. Any load failure counts, not only a missing module;\n * the cause is kept in `message` so a broken install is not reported as a merely absent one.\n * Nothing is cached, so installing the peer mid-session takes effect on the next call.\n */\nasync function loadMammoth(): Promise<\n | { readonly ok: true; readonly extract: DocumentTextExtractor }\n | { readonly ok: false; readonly message: string }\n> {\n try {\n const loaded = (await import(\"mammoth\")) as unknown as MammothLike & { default?: MammothLike };\n const mammoth = loaded.default ?? loaded;\n return {\n ok: true,\n extract: async (bytes) => {\n const buffer = Buffer.from(bytes.buffer, bytes.byteOffset, bytes.byteLength);\n return (await mammoth.extractRawText({ buffer })).value;\n },\n };\n } catch (err) {\n return { ok: false, message: messageOf(err) };\n }\n}\n"]}
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
import { CustomTool } from '@theokit/sdk';
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* `extract_document` — let the agent read the text of an office document in the project.
|
|
5
|
+
*
|
|
6
|
+
* ## Why this is a built-in
|
|
7
|
+
*
|
|
8
|
+
* A `.docx` is a zip of XML parts: `read_file` returns compressed bytes, not the words in it. The
|
|
9
|
+
* confinement is the same as `view_image`'s and is the part worth reviewing once: a document
|
|
10
|
+
* reader that honours any path is a file exfiltration primitive.
|
|
11
|
+
*
|
|
12
|
+
* ## The engine
|
|
13
|
+
*
|
|
14
|
+
* Text extraction is delegated to `mammoth`, an OPTIONAL peer loaded lazily on the first call, so
|
|
15
|
+
* a consumer of the other tools never pays for it. Its absence is a typed `document_engine_missing`
|
|
16
|
+
* result naming the install command, never a throw. A consumer can pass `extractText` to use any
|
|
17
|
+
* other engine; the envelope stays the same.
|
|
18
|
+
*
|
|
19
|
+
* ## Result shape
|
|
20
|
+
*
|
|
21
|
+
* - `{ ok: true, path, format: "docx", chars, truncated, text }` — `text` is the trimmed
|
|
22
|
+
* extracted text cut at `maxChars`; `chars` is the length before the cut.
|
|
23
|
+
* - `{ ok: false, error: "path_traversal" | "unsupported_document_type" | "not_found" |
|
|
24
|
+
* "not_a_document" | "document_too_large" | "document_unreadable" |
|
|
25
|
+
* "document_engine_missing", path, … }`
|
|
26
|
+
*
|
|
27
|
+
* `maxBytes` bounds the file read, not what the engine inflates it to; `maxChars` bounds what
|
|
28
|
+
* reaches the model.
|
|
29
|
+
*/
|
|
30
|
+
|
|
31
|
+
/** The engine contract: document bytes in, raw text out. */
|
|
32
|
+
type DocumentTextExtractor = (bytes: Uint8Array) => Promise<string>;
|
|
33
|
+
/**
|
|
34
|
+
* Options for {@link createExtractDocumentTool}.
|
|
35
|
+
*
|
|
36
|
+
* @public
|
|
37
|
+
*/
|
|
38
|
+
interface CreateExtractDocumentToolOptions {
|
|
39
|
+
/** Root the tool reads from. Every path is resolved inside it. */
|
|
40
|
+
projectRoot: string;
|
|
41
|
+
/** Name exposed to the model. Omitted ⇒ `extract_document`. */
|
|
42
|
+
name?: string;
|
|
43
|
+
/** Description exposed to the model. Omitted ⇒ the literal below. */
|
|
44
|
+
description?: string;
|
|
45
|
+
/** Ceiling on the file size in bytes. Omitted ⇒ {@link DEFAULT_MAX_DOCUMENT_BYTES}. */
|
|
46
|
+
maxBytes?: number;
|
|
47
|
+
/** Ceiling on the returned text in characters. Omitted ⇒ {@link DEFAULT_MAX_DOCUMENT_CHARS}. */
|
|
48
|
+
maxChars?: number;
|
|
49
|
+
/** Engine that turns the bytes into text. Omitted ⇒ the optional peer `mammoth`, loaded lazily. */
|
|
50
|
+
extractText?: DocumentTextExtractor;
|
|
51
|
+
}
|
|
52
|
+
/**
|
|
53
|
+
* Extract the text of a `.docx` inside the project root.
|
|
54
|
+
*
|
|
55
|
+
* @public
|
|
56
|
+
*/
|
|
57
|
+
declare function createExtractDocumentTool(options: CreateExtractDocumentToolOptions): CustomTool;
|
|
58
|
+
|
|
59
|
+
export { type CreateExtractDocumentToolOptions, createExtractDocumentTool };
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
import { CustomTool } from '@theokit/sdk';
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* `extract_document` — let the agent read the text of an office document in the project.
|
|
5
|
+
*
|
|
6
|
+
* ## Why this is a built-in
|
|
7
|
+
*
|
|
8
|
+
* A `.docx` is a zip of XML parts: `read_file` returns compressed bytes, not the words in it. The
|
|
9
|
+
* confinement is the same as `view_image`'s and is the part worth reviewing once: a document
|
|
10
|
+
* reader that honours any path is a file exfiltration primitive.
|
|
11
|
+
*
|
|
12
|
+
* ## The engine
|
|
13
|
+
*
|
|
14
|
+
* Text extraction is delegated to `mammoth`, an OPTIONAL peer loaded lazily on the first call, so
|
|
15
|
+
* a consumer of the other tools never pays for it. Its absence is a typed `document_engine_missing`
|
|
16
|
+
* result naming the install command, never a throw. A consumer can pass `extractText` to use any
|
|
17
|
+
* other engine; the envelope stays the same.
|
|
18
|
+
*
|
|
19
|
+
* ## Result shape
|
|
20
|
+
*
|
|
21
|
+
* - `{ ok: true, path, format: "docx", chars, truncated, text }` — `text` is the trimmed
|
|
22
|
+
* extracted text cut at `maxChars`; `chars` is the length before the cut.
|
|
23
|
+
* - `{ ok: false, error: "path_traversal" | "unsupported_document_type" | "not_found" |
|
|
24
|
+
* "not_a_document" | "document_too_large" | "document_unreadable" |
|
|
25
|
+
* "document_engine_missing", path, … }`
|
|
26
|
+
*
|
|
27
|
+
* `maxBytes` bounds the file read, not what the engine inflates it to; `maxChars` bounds what
|
|
28
|
+
* reaches the model.
|
|
29
|
+
*/
|
|
30
|
+
|
|
31
|
+
/** The engine contract: document bytes in, raw text out. */
|
|
32
|
+
type DocumentTextExtractor = (bytes: Uint8Array) => Promise<string>;
|
|
33
|
+
/**
|
|
34
|
+
* Options for {@link createExtractDocumentTool}.
|
|
35
|
+
*
|
|
36
|
+
* @public
|
|
37
|
+
*/
|
|
38
|
+
interface CreateExtractDocumentToolOptions {
|
|
39
|
+
/** Root the tool reads from. Every path is resolved inside it. */
|
|
40
|
+
projectRoot: string;
|
|
41
|
+
/** Name exposed to the model. Omitted ⇒ `extract_document`. */
|
|
42
|
+
name?: string;
|
|
43
|
+
/** Description exposed to the model. Omitted ⇒ the literal below. */
|
|
44
|
+
description?: string;
|
|
45
|
+
/** Ceiling on the file size in bytes. Omitted ⇒ {@link DEFAULT_MAX_DOCUMENT_BYTES}. */
|
|
46
|
+
maxBytes?: number;
|
|
47
|
+
/** Ceiling on the returned text in characters. Omitted ⇒ {@link DEFAULT_MAX_DOCUMENT_CHARS}. */
|
|
48
|
+
maxChars?: number;
|
|
49
|
+
/** Engine that turns the bytes into text. Omitted ⇒ the optional peer `mammoth`, loaded lazily. */
|
|
50
|
+
extractText?: DocumentTextExtractor;
|
|
51
|
+
}
|
|
52
|
+
/**
|
|
53
|
+
* Extract the text of a `.docx` inside the project root.
|
|
54
|
+
*
|
|
55
|
+
* @public
|
|
56
|
+
*/
|
|
57
|
+
declare function createExtractDocumentTool(options: CreateExtractDocumentToolOptions): CustomTool;
|
|
58
|
+
|
|
59
|
+
export { type CreateExtractDocumentToolOptions, createExtractDocumentTool };
|
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
import { fstatSync, readSync, closeSync, openSync } from 'fs';
|
|
2
|
+
import { extname } from 'path';
|
|
3
|
+
import { Tool } from '@theokit/sdk';
|
|
4
|
+
import { z } from 'zod';
|
|
5
|
+
import { safePathJoin, assertNoSymlinkEscape, PathTraversalError, ForbiddenPathError } from '@theokit/sdk/path-safety';
|
|
6
|
+
|
|
7
|
+
// src/extract-document.ts
|
|
8
|
+
|
|
9
|
+
// src/path-scope.ts
|
|
10
|
+
function checkPathScope(path, projectRoot) {
|
|
11
|
+
if (path === void 0 || path === "") return null;
|
|
12
|
+
try {
|
|
13
|
+
const abs = safePathJoin(projectRoot, path);
|
|
14
|
+
assertNoSymlinkEscape(abs, projectRoot);
|
|
15
|
+
return null;
|
|
16
|
+
} catch (err) {
|
|
17
|
+
if (err instanceof PathTraversalError || err instanceof ForbiddenPathError) {
|
|
18
|
+
return JSON.stringify({ ok: false, error: "path_traversal", path });
|
|
19
|
+
}
|
|
20
|
+
throw err;
|
|
21
|
+
}
|
|
22
|
+
}
|
|
23
|
+
var SENSITIVE_SEGMENTS = /* @__PURE__ */ new Set([".env", ".git", "node_modules", ".theo"]);
|
|
24
|
+
function isForbiddenAtAnyDepth(path) {
|
|
25
|
+
const segs = path.replace(/\\/g, "/").split("/").filter(Boolean);
|
|
26
|
+
return segs.some((s) => {
|
|
27
|
+
if (s === ".env.example") return false;
|
|
28
|
+
return SENSITIVE_SEGMENTS.has(s) || /^\.env\./.test(s);
|
|
29
|
+
});
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
// src/extract-document.ts
|
|
33
|
+
var DEFAULT_MAX_DOCUMENT_BYTES = 10 * 1024 * 1024;
|
|
34
|
+
var DEFAULT_MAX_DOCUMENT_CHARS = 1e5;
|
|
35
|
+
var SUPPORTED_EXTENSIONS = [".docx"];
|
|
36
|
+
var ZIP_SIGNATURE = [80, 75, 3, 4];
|
|
37
|
+
var INSTALL_COMMAND = "npm install mammoth";
|
|
38
|
+
var limitsSchema = z.object({
|
|
39
|
+
maxBytes: z.number().int().positive(),
|
|
40
|
+
maxChars: z.number().int().positive()
|
|
41
|
+
});
|
|
42
|
+
var json = (result) => JSON.stringify(result);
|
|
43
|
+
function createExtractDocumentTool(options) {
|
|
44
|
+
const { projectRoot } = options;
|
|
45
|
+
const limits = parseLimits(options);
|
|
46
|
+
const extract = options.extractText;
|
|
47
|
+
return Tool.create({
|
|
48
|
+
name: options.name ?? "extract_document",
|
|
49
|
+
description: options.description ?? "Extract the text of an office document (.docx) in the project. Returns the text, cut at a character limit.",
|
|
50
|
+
inputSchema: z.object({
|
|
51
|
+
path: z.string().describe("Path to the document, relative to the project root.")
|
|
52
|
+
}),
|
|
53
|
+
handler: async (input) => {
|
|
54
|
+
const refused = checkPathScope(input.path, projectRoot);
|
|
55
|
+
if (refused !== null) return refused;
|
|
56
|
+
if (isForbiddenAtAnyDepth(input.path)) {
|
|
57
|
+
return json({ ok: false, error: "path_traversal", path: input.path });
|
|
58
|
+
}
|
|
59
|
+
if (!isSupported(input.path)) {
|
|
60
|
+
return json({
|
|
61
|
+
ok: false,
|
|
62
|
+
error: "unsupported_document_type",
|
|
63
|
+
path: input.path,
|
|
64
|
+
supported: [...SUPPORTED_EXTENSIONS]
|
|
65
|
+
});
|
|
66
|
+
}
|
|
67
|
+
const outcome = readWithinBudget(safePathJoin(projectRoot, input.path), limits.maxBytes);
|
|
68
|
+
if (!outcome.ok) return json({ ...outcome.failure, path: input.path });
|
|
69
|
+
return json(await extractWithin(outcome.bytes, input.path, limits.maxChars, extract));
|
|
70
|
+
}
|
|
71
|
+
});
|
|
72
|
+
}
|
|
73
|
+
var isSupported = (path) => SUPPORTED_EXTENSIONS.includes(extname(path).toLowerCase());
|
|
74
|
+
function parseLimits(options) {
|
|
75
|
+
const parsed = limitsSchema.safeParse({
|
|
76
|
+
maxBytes: options.maxBytes ?? DEFAULT_MAX_DOCUMENT_BYTES,
|
|
77
|
+
maxChars: options.maxChars ?? DEFAULT_MAX_DOCUMENT_CHARS
|
|
78
|
+
});
|
|
79
|
+
if (!parsed.success) {
|
|
80
|
+
const detail = parsed.error.issues.map((issue) => `${issue.path.join(".")}: ${issue.message}`).join("; ");
|
|
81
|
+
throw new RangeError(`createExtractDocumentTool: invalid options (${detail})`);
|
|
82
|
+
}
|
|
83
|
+
return parsed.data;
|
|
84
|
+
}
|
|
85
|
+
var NOT_A_DOCUMENT = {
|
|
86
|
+
ok: false,
|
|
87
|
+
failure: { ok: false, error: "not_a_document" }
|
|
88
|
+
};
|
|
89
|
+
var hasZipSignature = (bytes) => ZIP_SIGNATURE.every((byte, index) => bytes[index] === byte);
|
|
90
|
+
function readWithinBudget(absolute, maxBytes) {
|
|
91
|
+
const opened = openForRead(absolute);
|
|
92
|
+
if (!opened.ok) return opened;
|
|
93
|
+
const { fd } = opened;
|
|
94
|
+
try {
|
|
95
|
+
const stat = fstatSync(fd);
|
|
96
|
+
if (!stat.isFile()) return NOT_A_DOCUMENT;
|
|
97
|
+
if (stat.size > maxBytes) {
|
|
98
|
+
return {
|
|
99
|
+
ok: false,
|
|
100
|
+
failure: {
|
|
101
|
+
ok: false,
|
|
102
|
+
error: "document_too_large",
|
|
103
|
+
bytes: stat.size,
|
|
104
|
+
limit_bytes: maxBytes
|
|
105
|
+
}
|
|
106
|
+
};
|
|
107
|
+
}
|
|
108
|
+
const buffer = Buffer.alloc(stat.size);
|
|
109
|
+
const bytes = new Uint8Array(buffer.subarray(0, readSync(fd, buffer, 0, stat.size, 0)));
|
|
110
|
+
return hasZipSignature(bytes) ? { ok: true, bytes } : NOT_A_DOCUMENT;
|
|
111
|
+
} finally {
|
|
112
|
+
closeSync(fd);
|
|
113
|
+
}
|
|
114
|
+
}
|
|
115
|
+
function openForRead(absolute) {
|
|
116
|
+
try {
|
|
117
|
+
return { ok: true, fd: openSync(absolute, "r") };
|
|
118
|
+
} catch (err) {
|
|
119
|
+
if (err.code === "ENOENT") {
|
|
120
|
+
return { ok: false, failure: { ok: false, error: "not_found" } };
|
|
121
|
+
}
|
|
122
|
+
return {
|
|
123
|
+
ok: false,
|
|
124
|
+
failure: { ok: false, error: "document_unreadable", message: messageOf(err) }
|
|
125
|
+
};
|
|
126
|
+
}
|
|
127
|
+
}
|
|
128
|
+
var messageOf = (err) => err instanceof Error ? err.message : String(err);
|
|
129
|
+
async function extractWithin(bytes, path, maxChars, injected) {
|
|
130
|
+
const engine = injected ? { ok: true, extract: injected } : await loadMammoth();
|
|
131
|
+
if (!engine.ok) {
|
|
132
|
+
return {
|
|
133
|
+
ok: false,
|
|
134
|
+
error: "document_engine_missing",
|
|
135
|
+
path,
|
|
136
|
+
install: INSTALL_COMMAND,
|
|
137
|
+
message: engine.message
|
|
138
|
+
};
|
|
139
|
+
}
|
|
140
|
+
let text;
|
|
141
|
+
try {
|
|
142
|
+
text = (await engine.extract(bytes)).trim();
|
|
143
|
+
} catch (err) {
|
|
144
|
+
return { ok: false, error: "document_unreadable", path, message: messageOf(err) };
|
|
145
|
+
}
|
|
146
|
+
return {
|
|
147
|
+
ok: true,
|
|
148
|
+
path,
|
|
149
|
+
format: "docx",
|
|
150
|
+
chars: text.length,
|
|
151
|
+
truncated: text.length > maxChars,
|
|
152
|
+
text: text.slice(0, maxChars)
|
|
153
|
+
};
|
|
154
|
+
}
|
|
155
|
+
async function loadMammoth() {
|
|
156
|
+
try {
|
|
157
|
+
const loaded = await import('mammoth');
|
|
158
|
+
const mammoth = loaded.default ?? loaded;
|
|
159
|
+
return {
|
|
160
|
+
ok: true,
|
|
161
|
+
extract: async (bytes) => {
|
|
162
|
+
const buffer = Buffer.from(bytes.buffer, bytes.byteOffset, bytes.byteLength);
|
|
163
|
+
return (await mammoth.extractRawText({ buffer })).value;
|
|
164
|
+
}
|
|
165
|
+
};
|
|
166
|
+
} catch (err) {
|
|
167
|
+
return { ok: false, message: messageOf(err) };
|
|
168
|
+
}
|
|
169
|
+
}
|
|
170
|
+
|
|
171
|
+
export { createExtractDocumentTool };
|
|
172
|
+
//# sourceMappingURL=extract-document.js.map
|
|
173
|
+
//# sourceMappingURL=extract-document.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"sources":["../src/path-scope.ts","../src/extract-document.ts"],"names":[],"mappings":";;;;;;;;;AAeO,SAAS,cAAA,CAAe,MAA0B,WAAA,EAAoC;AAC3F,EAAA,IAAI,IAAA,KAAS,MAAA,IAAa,IAAA,KAAS,EAAA,EAAI,OAAO,IAAA;AAC9C,EAAA,IAAI;AACF,IAAA,MAAM,GAAA,GAAM,YAAA,CAAa,WAAA,EAAa,IAAI,CAAA;AAC1C,IAAA,qBAAA,CAAsB,KAAK,WAAW,CAAA;AACtC,IAAA,OAAO,IAAA;AAAA,EACT,SAAS,GAAA,EAAK;AACZ,IAAA,IAAI,GAAA,YAAe,kBAAA,IAAsB,GAAA,YAAe,kBAAA,EAAoB;AAC1E,MAAA,OAAO,IAAA,CAAK,UAAU,EAAE,EAAA,EAAI,OAAO,KAAA,EAAO,gBAAA,EAAkB,MAAM,CAAA;AAAA,IACpE;AACA,IAAA,MAAM,GAAA;AAAA,EACR;AACF;AASA,IAAM,kBAAA,uBAAyB,GAAA,CAAI,CAAC,QAAQ,MAAA,EAAQ,cAAA,EAAgB,OAAO,CAAC,CAAA;AASrE,SAAS,sBAAsB,IAAA,EAAuB;AAC3D,EAAA,MAAM,IAAA,GAAO,IAAA,CAAK,OAAA,CAAQ,KAAA,EAAO,GAAG,EAAE,KAAA,CAAM,GAAG,CAAA,CAAE,MAAA,CAAO,OAAO,CAAA;AAC/D,EAAA,OAAO,IAAA,CAAK,IAAA,CAAK,CAAC,CAAA,KAAM;AACtB,IAAA,IAAI,CAAA,KAAM,gBAAgB,OAAO,KAAA;AACjC,IAAA,OAAO,mBAAmB,GAAA,CAAI,CAAC,CAAA,IAAK,UAAA,CAAW,KAAK,CAAC,CAAA;AAAA,EACvD,CAAC,CAAA;AACH;;;ACZA,IAAM,0BAAA,GAA6B,KAAK,IAAA,GAAO,IAAA;AAG/C,IAAM,0BAAA,GAA6B,GAAA;AAGnC,IAAM,oBAAA,GAAuB,CAAC,OAAO,CAAA;AAGrC,IAAM,aAAA,GAAgB,CAAC,EAAA,EAAM,EAAA,EAAM,GAAM,CAAI,CAAA;AAE7C,IAAM,eAAA,GAAkB,qBAAA;AAyBxB,IAAM,YAAA,GAAe,EAAE,MAAA,CAAO;AAAA,EAC5B,UAAU,CAAA,CAAE,MAAA,EAAO,CAAE,GAAA,GAAM,QAAA,EAAS;AAAA,EACpC,UAAU,CAAA,CAAE,MAAA,EAAO,CAAE,GAAA,GAAM,QAAA;AAC7B,CAAC,CAAA;AA6BD,IAAM,IAAA,GAAO,CAAC,MAAA,KAA+D,IAAA,CAAK,UAAU,MAAM,CAAA;AAO3F,SAAS,0BAA0B,OAAA,EAAuD;AAC/F,EAAA,MAAM,EAAE,aAAY,GAAI,OAAA;AACxB,EAAA,MAAM,MAAA,GAAS,YAAY,OAAO,CAAA;AAClC,EAAA,MAAM,UAAU,OAAA,CAAQ,WAAA;AAExB,EAAA,OAAO,KAAK,MAAA,CAAO;AAAA,IACjB,IAAA,EAAM,QAAQ,IAAA,IAAQ,kBAAA;AAAA,IACtB,WAAA,EACE,QAAQ,WAAA,IACR,4GAAA;AAAA,IACF,WAAA,EAAa,EAAE,MAAA,CAAO;AAAA,MACpB,IAAA,EAAM,CAAA,CAAE,MAAA,EAAO,CAAE,SAAS,qDAAqD;AAAA,KAChF,CAAA;AAAA,IACD,OAAA,EAAS,OAAO,KAAA,KAA6C;AAG3D,MAAA,MAAM,OAAA,GAAU,cAAA,CAAe,KAAA,CAAM,IAAA,EAAM,WAAW,CAAA;AACtD,MAAA,IAAI,OAAA,KAAY,MAAM,OAAO,OAAA;AAC7B,MAAA,IAAI,qBAAA,CAAsB,KAAA,CAAM,IAAI,CAAA,EAAG;AACrC,QAAA,OAAO,IAAA,CAAK,EAAE,EAAA,EAAI,KAAA,EAAO,OAAO,gBAAA,EAAkB,IAAA,EAAM,KAAA,CAAM,IAAA,EAAM,CAAA;AAAA,MACtE;AACA,MAAA,IAAI,CAAC,WAAA,CAAY,KAAA,CAAM,IAAI,CAAA,EAAG;AAC5B,QAAA,OAAO,IAAA,CAAK;AAAA,UACV,EAAA,EAAI,KAAA;AAAA,UACJ,KAAA,EAAO,2BAAA;AAAA,UACP,MAAM,KAAA,CAAM,IAAA;AAAA,UACZ,SAAA,EAAW,CAAC,GAAG,oBAAoB;AAAA,SACpC,CAAA;AAAA,MACH;AAEA,MAAA,MAAM,OAAA,GAAU,iBAAiB,YAAA,CAAa,WAAA,EAAa,MAAM,IAAI,CAAA,EAAG,OAAO,QAAQ,CAAA;AACvF,MAAA,IAAI,CAAC,OAAA,CAAQ,EAAA,EAAI,OAAO,IAAA,CAAK,EAAE,GAAG,OAAA,CAAQ,OAAA,EAAS,IAAA,EAAM,KAAA,CAAM,IAAA,EAAM,CAAA;AAErE,MAAA,OAAO,IAAA,CAAK,MAAM,aAAA,CAAc,OAAA,CAAQ,KAAA,EAAO,MAAM,IAAA,EAAM,MAAA,CAAO,QAAA,EAAU,OAAO,CAAC,CAAA;AAAA,IACtF;AAAA,GACD,CAAA;AACH;AAEA,IAAM,WAAA,GAAc,CAAC,IAAA,KAClB,oBAAA,CAA2C,SAAS,OAAA,CAAQ,IAAI,CAAA,CAAE,WAAA,EAAa,CAAA;AAGlF,SAAS,YAAY,OAAA,EAAyE;AAC5F,EAAA,MAAM,MAAA,GAAS,aAAa,SAAA,CAAU;AAAA,IACpC,QAAA,EAAU,QAAQ,QAAA,IAAY,0BAAA;AAAA,IAC9B,QAAA,EAAU,QAAQ,QAAA,IAAY;AAAA,GAC/B,CAAA;AACD,EAAA,IAAI,CAAC,OAAO,OAAA,EAAS;AACnB,IAAA,MAAM,SAAS,MAAA,CAAO,KAAA,CAAM,OACzB,GAAA,CAAI,CAAC,UAAU,CAAA,EAAG,KAAA,CAAM,KAAK,IAAA,CAAK,GAAG,CAAC,CAAA,EAAA,EAAK,KAAA,CAAM,OAAO,CAAA,CAAE,CAAA,CAC1D,KAAK,IAAI,CAAA;AACZ,IAAA,MAAM,IAAI,UAAA,CAAW,CAAA,4CAAA,EAA+C,MAAM,CAAA,CAAA,CAAG,CAAA;AAAA,EAC/E;AACA,EAAA,OAAO,MAAA,CAAO,IAAA;AAChB;AAMA,IAAM,cAAA,GAA8B;AAAA,EAClC,EAAA,EAAI,KAAA;AAAA,EACJ,OAAA,EAAS,EAAE,EAAA,EAAI,KAAA,EAAO,OAAO,gBAAA;AAC/B,CAAA;AAEA,IAAM,eAAA,GAAkB,CAAC,KAAA,KACvB,aAAA,CAAc,KAAA,CAAM,CAAC,IAAA,EAAM,KAAA,KAAU,KAAA,CAAM,KAAK,CAAA,KAAM,IAAI,CAAA;AAO5D,SAAS,gBAAA,CAAiB,UAAkB,QAAA,EAA+B;AACzE,EAAA,MAAM,MAAA,GAAS,YAAY,QAAQ,CAAA;AACnC,EAAA,IAAI,CAAC,MAAA,CAAO,EAAA,EAAI,OAAO,MAAA;AACvB,EAAA,MAAM,EAAE,IAAG,GAAI,MAAA;AACf,EAAA,IAAI;AACF,IAAA,MAAM,IAAA,GAAO,UAAU,EAAE,CAAA;AAEzB,IAAA,IAAI,CAAC,IAAA,CAAK,MAAA,EAAO,EAAG,OAAO,cAAA;AAE3B,IAAA,IAAI,IAAA,CAAK,OAAO,QAAA,EAAU;AACxB,MAAA,OAAO;AAAA,QACL,EAAA,EAAI,KAAA;AAAA,QACJ,OAAA,EAAS;AAAA,UACP,EAAA,EAAI,KAAA;AAAA,UACJ,KAAA,EAAO,oBAAA;AAAA,UACP,OAAO,IAAA,CAAK,IAAA;AAAA,UACZ,WAAA,EAAa;AAAA;AACf,OACF;AAAA,IACF;AACA,IAAA,MAAM,MAAA,GAAS,MAAA,CAAO,KAAA,CAAM,IAAA,CAAK,IAAI,CAAA;AACrC,IAAA,MAAM,KAAA,GAAQ,IAAI,UAAA,CAAW,MAAA,CAAO,SAAS,CAAA,EAAG,QAAA,CAAS,EAAA,EAAI,MAAA,EAAQ,CAAA,EAAG,IAAA,CAAK,IAAA,EAAM,CAAC,CAAC,CAAC,CAAA;AACtF,IAAA,OAAO,gBAAgB,KAAK,CAAA,GAAI,EAAE,EAAA,EAAI,IAAA,EAAM,OAAM,GAAI,cAAA;AAAA,EACxD,CAAA,SAAE;AACA,IAAA,SAAA,CAAU,EAAE,CAAA;AAAA,EACd;AACF;AAMA,SAAS,YACP,QAAA,EACkF;AAClF,EAAA,IAAI;AACF,IAAA,OAAO,EAAE,EAAA,EAAI,IAAA,EAAM,IAAI,QAAA,CAAS,QAAA,EAAU,GAAG,CAAA,EAAE;AAAA,EACjD,SAAS,GAAA,EAAK;AACZ,IAAA,IAAK,GAAA,CAA8B,SAAS,QAAA,EAAU;AACpD,MAAA,OAAO,EAAE,IAAI,KAAA,EAAO,OAAA,EAAS,EAAE,EAAA,EAAI,KAAA,EAAO,KAAA,EAAO,WAAA,EAAY,EAAE;AAAA,IACjE;AACA,IAAA,OAAO;AAAA,MACL,EAAA,EAAI,KAAA;AAAA,MACJ,OAAA,EAAS,EAAE,EAAA,EAAI,KAAA,EAAO,OAAO,qBAAA,EAAuB,OAAA,EAAS,SAAA,CAAU,GAAG,CAAA;AAAE,KAC9E;AAAA,EACF;AACF;AAEA,IAAM,SAAA,GAAY,CAAC,GAAA,KAA0B,GAAA,YAAe,QAAQ,GAAA,CAAI,OAAA,GAAU,OAAO,GAAG,CAAA;AAG5F,eAAe,aAAA,CACb,KAAA,EACA,IAAA,EACA,QAAA,EACA,QAAA,EACqD;AACrD,EAAA,MAAM,MAAA,GAAS,WAAW,EAAE,EAAA,EAAI,MAAe,OAAA,EAAS,QAAA,EAAS,GAAI,MAAM,WAAA,EAAY;AACvF,EAAA,IAAI,CAAC,OAAO,EAAA,EAAI;AACd,IAAA,OAAO;AAAA,MACL,EAAA,EAAI,KAAA;AAAA,MACJ,KAAA,EAAO,yBAAA;AAAA,MACP,IAAA;AAAA,MACA,OAAA,EAAS,eAAA;AAAA,MACT,SAAS,MAAA,CAAO;AAAA,KAClB;AAAA,EACF;AACA,EAAA,IAAI,IAAA;AACJ,EAAA,IAAI;AACF,IAAA,IAAA,GAAA,CAAQ,MAAM,MAAA,CAAO,OAAA,CAAQ,KAAK,GAAG,IAAA,EAAK;AAAA,EAC5C,SAAS,GAAA,EAAK;AAEZ,IAAA,OAAO,EAAE,IAAI,KAAA,EAAO,KAAA,EAAO,uBAAuB,IAAA,EAAM,OAAA,EAAS,SAAA,CAAU,GAAG,CAAA,EAAE;AAAA,EAClF;AACA,EAAA,OAAO;AAAA,IACL,EAAA,EAAI,IAAA;AAAA,IACJ,IAAA;AAAA,IACA,MAAA,EAAQ,MAAA;AAAA,IACR,OAAO,IAAA,CAAK,MAAA;AAAA,IACZ,SAAA,EAAW,KAAK,MAAA,GAAS,QAAA;AAAA,IACzB,IAAA,EAAM,IAAA,CAAK,KAAA,CAAM,CAAA,EAAG,QAAQ;AAAA,GAC9B;AACF;AAeA,eAAe,WAAA,GAGb;AACA,EAAA,IAAI;AACF,IAAA,MAAM,MAAA,GAAU,MAAM,OAAO,SAAS,CAAA;AACtC,IAAA,MAAM,OAAA,GAAU,OAAO,OAAA,IAAW,MAAA;AAClC,IAAA,OAAO;AAAA,MACL,EAAA,EAAI,IAAA;AAAA,MACJ,OAAA,EAAS,OAAO,KAAA,KAAU;AACxB,QAAA,MAAM,MAAA,GAAS,OAAO,IAAA,CAAK,KAAA,CAAM,QAAQ,KAAA,CAAM,UAAA,EAAY,MAAM,UAAU,CAAA;AAC3E,QAAA,OAAA,CAAQ,MAAM,OAAA,CAAQ,cAAA,CAAe,EAAE,MAAA,EAAQ,CAAA,EAAG,KAAA;AAAA,MACpD;AAAA,KACF;AAAA,EACF,SAAS,GAAA,EAAK;AACZ,IAAA,OAAO,EAAE,EAAA,EAAI,KAAA,EAAO,OAAA,EAAS,SAAA,CAAU,GAAG,CAAA,EAAE;AAAA,EAC9C;AACF","file":"extract-document.js","sourcesContent":["/**\n * Shared `safePathJoin + assertNoSymlinkEscape` helper for built-in coding\n * tools (git-diff, run-vitest, list-dir, read-file, search-text). Returns a\n * pre-formatted `path_traversal` JSON string on failure or `null` on pass.\n *\n * @internal\n */\n\nimport {\n assertNoSymlinkEscape,\n ForbiddenPathError,\n PathTraversalError,\n safePathJoin,\n} from \"./internal/path-guard.js\";\n\nexport function checkPathScope(path: string | undefined, projectRoot: string): string | null {\n if (path === undefined || path === \"\") return null;\n try {\n const abs = safePathJoin(projectRoot, path);\n assertNoSymlinkEscape(abs, projectRoot);\n return null;\n } catch (err) {\n if (err instanceof PathTraversalError || err instanceof ForbiddenPathError) {\n return JSON.stringify({ ok: false, error: \"path_traversal\", path });\n }\n throw err;\n }\n}\n\n/**\n * Segments that may never appear in a path honored by `allowAbsolute`.\n *\n * M76 — promoted from `read-file.ts`, where it was private. Duplicating it in `list-dir` would duplicate\n * security KNOWLEDGE: the copies would have to agree on what counts as a secret, and one fixed\n * without the other reopens the hole in the forgotten tool.\n */\nconst SENSITIVE_SEGMENTS = new Set([\".env\", \".git\", \"node_modules\", \".theo\"]);\n\n/**\n * ANY-segment secret guard — the half of `allowAbsolute` that cannot be separated.\n *\n * `isForbiddenPath` only blocks the sensitive item when it is the FIRST segment (relative to the\n * project). An absolute path (`/home/u/proj/.env/sub`) puts it deeper, and it would pass. This one\n * checks every segment, closing the \"reads-anywhere\" exfiltration.\n */\nexport function isForbiddenAtAnyDepth(path: string): boolean {\n const segs = path.replace(/\\\\/g, \"/\").split(\"/\").filter(Boolean);\n return segs.some((s) => {\n if (s === \".env.example\") return false; // a template — safe\n return SENSITIVE_SEGMENTS.has(s) || /^\\.env\\./.test(s);\n });\n}\n","/**\n * `extract_document` — let the agent read the text of an office document in the project.\n *\n * ## Why this is a built-in\n *\n * A `.docx` is a zip of XML parts: `read_file` returns compressed bytes, not the words in it. The\n * confinement is the same as `view_image`'s and is the part worth reviewing once: a document\n * reader that honours any path is a file exfiltration primitive.\n *\n * ## The engine\n *\n * Text extraction is delegated to `mammoth`, an OPTIONAL peer loaded lazily on the first call, so\n * a consumer of the other tools never pays for it. Its absence is a typed `document_engine_missing`\n * result naming the install command, never a throw. A consumer can pass `extractText` to use any\n * other engine; the envelope stays the same.\n *\n * ## Result shape\n *\n * - `{ ok: true, path, format: \"docx\", chars, truncated, text }` — `text` is the trimmed\n * extracted text cut at `maxChars`; `chars` is the length before the cut.\n * - `{ ok: false, error: \"path_traversal\" | \"unsupported_document_type\" | \"not_found\" |\n * \"not_a_document\" | \"document_too_large\" | \"document_unreadable\" |\n * \"document_engine_missing\", path, … }`\n *\n * `maxBytes` bounds the file read, not what the engine inflates it to; `maxChars` bounds what\n * reaches the model.\n */\n\nimport { closeSync, fstatSync, openSync, readSync } from \"node:fs\";\nimport { extname } from \"node:path\";\n\nimport type { CustomTool } from \"@theokit/sdk\";\nimport { Tool } from \"@theokit/sdk\";\nimport { z } from \"zod\";\n\nimport { safePathJoin } from \"./internal/path-guard.js\";\nimport { checkPathScope, isForbiddenAtAnyDepth } from \"./path-scope.js\";\n\n/** Default ceiling on the file read, in bytes on disk: 10 MB. */\nconst DEFAULT_MAX_DOCUMENT_BYTES = 10 * 1024 * 1024;\n\n/** Default ceiling on the text returned to the model, in characters. */\nconst DEFAULT_MAX_DOCUMENT_CHARS = 100_000;\n\n/** The extensions the tool reads, lower-cased. A new format is one entry plus its engine. */\nconst SUPPORTED_EXTENSIONS = [\".docx\"] as const;\n\n/** A `.docx` is a zip archive; every zip starts with this local-file-header signature. */\nconst ZIP_SIGNATURE = [0x50, 0x4b, 0x03, 0x04] as const;\n\nconst INSTALL_COMMAND = \"npm install mammoth\";\n\n/** The engine contract: document bytes in, raw text out. */\ntype DocumentTextExtractor = (bytes: Uint8Array) => Promise<string>;\n\n/**\n * Options for {@link createExtractDocumentTool}.\n *\n * @public\n */\nexport interface CreateExtractDocumentToolOptions {\n /** Root the tool reads from. Every path is resolved inside it. */\n projectRoot: string;\n /** Name exposed to the model. Omitted ⇒ `extract_document`. */\n name?: string;\n /** Description exposed to the model. Omitted ⇒ the literal below. */\n description?: string;\n /** Ceiling on the file size in bytes. Omitted ⇒ {@link DEFAULT_MAX_DOCUMENT_BYTES}. */\n maxBytes?: number;\n /** Ceiling on the returned text in characters. Omitted ⇒ {@link DEFAULT_MAX_DOCUMENT_CHARS}. */\n maxChars?: number;\n /** Engine that turns the bytes into text. Omitted ⇒ the optional peer `mammoth`, loaded lazily. */\n extractText?: DocumentTextExtractor;\n}\n\nconst limitsSchema = z.object({\n maxBytes: z.number().int().positive(),\n maxChars: z.number().int().positive(),\n});\n\ntype DocumentErrorCode =\n | \"path_traversal\"\n | \"unsupported_document_type\"\n | \"not_found\"\n | \"not_a_document\"\n | \"document_too_large\"\n | \"document_unreadable\"\n | \"document_engine_missing\";\n\ninterface ExtractDocumentOk {\n ok: true;\n path: string;\n format: \"docx\";\n chars: number;\n truncated: boolean;\n text: string;\n}\n\n/** A failure before the path is attached: the read helper does not know the input path. */\ninterface DocumentFailureBody {\n ok: false;\n error: DocumentErrorCode;\n [extra: string]: unknown;\n}\n\ntype ExtractDocumentFailure = DocumentFailureBody & { path: string };\n\nconst json = (result: ExtractDocumentOk | ExtractDocumentFailure): string => JSON.stringify(result);\n\n/**\n * Extract the text of a `.docx` inside the project root.\n *\n * @public\n */\nexport function createExtractDocumentTool(options: CreateExtractDocumentToolOptions): CustomTool {\n const { projectRoot } = options;\n const limits = parseLimits(options);\n const extract = options.extractText;\n\n return Tool.create({\n name: options.name ?? \"extract_document\",\n description:\n options.description ??\n \"Extract the text of an office document (.docx) in the project. Returns the text, cut at a character limit.\",\n inputSchema: z.object({\n path: z.string().describe(\"Path to the document, relative to the project root.\"),\n }),\n handler: async (input: { path: string }): Promise<string> => {\n // Same order as `view_image`: the shared scope guard (lexical and symlink escapes), then the\n // any-depth secret guard, and only then a join that can no longer throw outside the envelope.\n const refused = checkPathScope(input.path, projectRoot);\n if (refused !== null) return refused;\n if (isForbiddenAtAnyDepth(input.path)) {\n return json({ ok: false, error: \"path_traversal\", path: input.path });\n }\n if (!isSupported(input.path)) {\n return json({\n ok: false,\n error: \"unsupported_document_type\",\n path: input.path,\n supported: [...SUPPORTED_EXTENSIONS],\n });\n }\n\n const outcome = readWithinBudget(safePathJoin(projectRoot, input.path), limits.maxBytes);\n if (!outcome.ok) return json({ ...outcome.failure, path: input.path });\n\n return json(await extractWithin(outcome.bytes, input.path, limits.maxChars, extract));\n },\n }) as unknown as CustomTool;\n}\n\nconst isSupported = (path: string): boolean =>\n (SUPPORTED_EXTENSIONS as readonly string[]).includes(extname(path).toLowerCase());\n\n/** Validates the numeric options once, at construction, so a bad limit fails the factory call. */\nfunction parseLimits(options: CreateExtractDocumentToolOptions): z.infer<typeof limitsSchema> {\n const parsed = limitsSchema.safeParse({\n maxBytes: options.maxBytes ?? DEFAULT_MAX_DOCUMENT_BYTES,\n maxChars: options.maxChars ?? DEFAULT_MAX_DOCUMENT_CHARS,\n });\n if (!parsed.success) {\n const detail = parsed.error.issues\n .map((issue) => `${issue.path.join(\".\")}: ${issue.message}`)\n .join(\"; \");\n throw new RangeError(`createExtractDocumentTool: invalid options (${detail})`);\n }\n return parsed.data;\n}\n\ntype ReadOutcome =\n | { readonly ok: true; readonly bytes: Uint8Array }\n | { readonly ok: false; readonly failure: DocumentFailureBody };\n\nconst NOT_A_DOCUMENT: ReadOutcome = {\n ok: false,\n failure: { ok: false, error: \"not_a_document\" },\n};\n\nconst hasZipSignature = (bytes: Uint8Array): boolean =>\n ZIP_SIGNATURE.every((byte, index) => bytes[index] === byte);\n\n/**\n * Sizes, reads and checks one file through a single descriptor (CodeQL js/file-system-race): the\n * size cap must describe the bytes actually returned, so the read is bounded by the size the\n * descriptor reported, and a file that grows after `fstat` cannot push past the cap.\n */\nfunction readWithinBudget(absolute: string, maxBytes: number): ReadOutcome {\n const opened = openForRead(absolute);\n if (!opened.ok) return opened;\n const { fd } = opened;\n try {\n const stat = fstatSync(fd);\n // A directory named `x.docx` opens on Linux; it is still not a document.\n if (!stat.isFile()) return NOT_A_DOCUMENT;\n // `>` and not `>=`: a file of exactly the limit fits.\n if (stat.size > maxBytes) {\n return {\n ok: false,\n failure: {\n ok: false,\n error: \"document_too_large\",\n bytes: stat.size,\n limit_bytes: maxBytes,\n },\n };\n }\n const buffer = Buffer.alloc(stat.size);\n const bytes = new Uint8Array(buffer.subarray(0, readSync(fd, buffer, 0, stat.size, 0)));\n return hasZipSignature(bytes) ? { ok: true, bytes } : NOT_A_DOCUMENT;\n } finally {\n closeSync(fd);\n }\n}\n\n/**\n * Opens the file, telling a missing file (`not_found`, which the model answers with another path)\n * apart from one it may not read (`document_unreadable` with the system's message).\n */\nfunction openForRead(\n absolute: string,\n): { readonly ok: true; readonly fd: number } | Extract<ReadOutcome, { ok: false }> {\n try {\n return { ok: true, fd: openSync(absolute, \"r\") };\n } catch (err) {\n if ((err as NodeJS.ErrnoException).code === \"ENOENT\") {\n return { ok: false, failure: { ok: false, error: \"not_found\" } };\n }\n return {\n ok: false,\n failure: { ok: false, error: \"document_unreadable\", message: messageOf(err) },\n };\n }\n}\n\nconst messageOf = (err: unknown): string => (err instanceof Error ? err.message : String(err));\n\n/** Runs the engine, trims and cuts its text, mapping every engine failure to a typed result. */\nasync function extractWithin(\n bytes: Uint8Array,\n path: string,\n maxChars: number,\n injected: DocumentTextExtractor | undefined,\n): Promise<ExtractDocumentOk | ExtractDocumentFailure> {\n const engine = injected ? { ok: true as const, extract: injected } : await loadMammoth();\n if (!engine.ok) {\n return {\n ok: false,\n error: \"document_engine_missing\",\n path,\n install: INSTALL_COMMAND,\n message: engine.message,\n };\n }\n let text: string;\n try {\n text = (await engine.extract(bytes)).trim();\n } catch (err) {\n // A zip that is not a Word document (an `.xlsx` renamed, a corrupt or encrypted file) lands here.\n return { ok: false, error: \"document_unreadable\", path, message: messageOf(err) };\n }\n return {\n ok: true,\n path,\n format: \"docx\",\n chars: text.length,\n truncated: text.length > maxChars,\n text: text.slice(0, maxChars),\n };\n}\n\n/** The slice of `mammoth`'s API this tool uses; declared here so the build never needs its types. */\ninterface MammothLike {\n extractRawText(input: { buffer: Buffer }): Promise<{ value: string }>;\n}\n\n/**\n * Loads the optional `mammoth` peer, or the reason it could not be loaded.\n *\n * `mammoth` is CommonJS: an ESM `import()` exposes it under `default`, the CJS build's `require`\n * returns it bare, so both shapes are read. Any load failure counts, not only a missing module;\n * the cause is kept in `message` so a broken install is not reported as a merely absent one.\n * Nothing is cached, so installing the peer mid-session takes effect on the next call.\n */\nasync function loadMammoth(): Promise<\n | { readonly ok: true; readonly extract: DocumentTextExtractor }\n | { readonly ok: false; readonly message: string }\n> {\n try {\n const loaded = (await import(\"mammoth\")) as unknown as MammothLike & { default?: MammothLike };\n const mammoth = loaded.default ?? loaded;\n return {\n ok: true,\n extract: async (bytes) => {\n const buffer = Buffer.from(bytes.buffer, bytes.byteOffset, bytes.byteLength);\n return (await mammoth.extractRawText({ buffer })).value;\n },\n };\n } catch (err) {\n return { ok: false, message: messageOf(err) };\n }\n}\n"]}
|