token-goat 2.6.33 → 2.6.35

package/README.md

@@ -532,6 +532,8 @@ token-goat pdf-extract manual.pdf --pages 12-15 --layout --head 120
 | `token-goat outline "file"` | List top-level symbols with line ranges and docstring hints — one-glance file map. `--force-refresh` reparses from disk first, bypassing a stale index. `--stats` adds a per-symbol reference count and doc-coverage flag, computed live from the index. `--grep <pattern>` narrows to symbols whose name matches a regex (a literal substring when the pattern is not valid regex), which is how you skim one area of a large file without dumping its whole symbol list; `--min-lines <n>` drops symbols shorter than N lines. Both compose, and if a filter removes everything the output says so and names the filter, rather than looking like a file with no symbols. Accepts a comma-separated file list (`"a,b,c"`) to cover several files in one call, one clearly-headed block per file; extra space-separated file arguments are reported in a note naming that comma form instead of being silently dropped. With `--json`, a comma-separated list returns one merged document (rows carry their own `filePath`), not one document per file. |
 | `token-goat yaml-outline <file>` | Structural summary of a YAML document (array shape / object key types) instead of a raw Read. Multi-document streams (`---`-separated) outline as an array of documents. |
 | `token-goat yaml-query <file> <path>` | Extract one value or a projected/filtered subset from a YAML document by dot-path instead of a raw Read (same grammar as `json-query`: `[n]` index, `[*]` wildcard, `[field=value]` filter — e.g. `items[status=active].name`). `--head <n>` caps a projected/filtered result. |
+| `token-goat xml-outline <file>` | Structural summary of an XML document (element tag hierarchy, attribute keys, child counts) instead of a raw Read. |
+| `token-goat xml-query <file> <path>` | Extract one value, element text/XML, or a projected/filtered subset from an XML document by XPath-like dot-path instead of a raw Read (same grammar as `json-query`/`yaml-query`: element tags, `@attr`, `[n]` index, `[*]` wildcard, `[attr=value]` filter, `--head <n>`). |
 | `token-goat json-outline <file>` | Structural summary of a JSON document (array shape / object key types) instead of a raw Read. |
 | `token-goat json-query <file> <path>` | Extract one value or a projected/filtered subset from a JSON document by dot-path instead of a raw Read: dot-separated keys with optional bracket segments — `[n]` index, `[*]` wildcard (projects every element/value), `[field=value]` filter. Examples: `data.items[3].name`, `items[*].id`, `items[status=active]`. |
 | `token-goat brief "file::symbol"` | Bundle a symbol's body, resolved callers (grouped by enclosing function), and its containing doc section into one round-trip instead of three separate `read`/`callers`/`section` calls. `--limit <n>` caps the callers shown per symbol (default 20; the true caller count is reported even when truncated). Comma-separated `"file::a,b"` fetches several symbols' bundles from one file in a single call, mirroring `read`'s `file::a,b` multi-symbol grammar. Cross-file `"a.ts::x,b.ts::y"` bundles symbols from several files in one call, mirroring `read`'s cross-file grammar — a bare segment inherits the file to its left, and once more than one file is involved each bundle is keyed by the full `file::symbol` so two files contributing the same symbol name stay distinct. Also accepts `read`'s `symbol@LINE` anchor to pick out an otherwise-ambiguous candidate. `-C, --context <n>` adds N lines of real call-site source around each entry of the caller block. `--json`'s `symbol.filePath` and `callers[].file` render root-relative when a project root resolves, absolute when none does — matching the plain-text block above. `--exclude-tests` hides callers whose call site is in a test file, matching `refs`/`callers`; the caller count and the elided tail both count the filtered set, so they never disagree with the rows shown, and when the filter empties the block it says so instead of reporting a bare zero that would read as "nothing calls this". `--json` adds `hiddenByExcludeTests` only when the filter actually hid something. `--grep <pattern>` narrows the caller block to callers whose enclosing symbol name matches this regex (literal substring if it is not valid regex), the same filter `refs --grep`/`call-chain --grep` apply to their own results — useful for a high-fanout symbol whose default 20-caller window is otherwise mostly noise; composes with `--exclude-tests`, and reports `hiddenByGrep` under `--json` only when it hid something. |
@@ -1242,6 +1244,8 @@ Read is the exception: file content passes through to the model unfiltered, beca
 
 Separately from that pass-through case: when a read hook *denies* a Read and substitutes its own message, any file bytes it embeds in that message (a markdown heading tree, a served compact or notebook sidecar, a re-read diff, a CSV header row, an HTML title) are wrapped in an `<untrusted-file-content>` fence first, so a hostile repo cannot get its own text presented to the model as token-goat speaking. That fencing is unconditional, not gated on the pattern scan.
 
+A third case needs no fence, because the danger is the line break rather than the wording. When token-goat prints its own summary of a file it prints one entry per line and takes the names and values straight out of that file: the column profile behind `csv-profile`, the key listing behind `json-outline` and `yaml-outline`, the entry listing behind `zip-list`, and any hook hint naming the file it is about. Every one of those values may legally contain a newline. A quoted CSV field spans lines by design, a JSON key is an arbitrary string, a zip entry name is whatever whoever built the archive wrote in the header, and a file name may contain a newline on Linux and macOS. So a single cell, key, entry or file name could end token-goat's line and start one of its own that reads exactly like another entry token-goat had written, with nothing but the line break to tell them apart. Control characters, Unicode line separators and format characters in those values are escaped into their visible form, so one entry stays one line and hostile content is shown rather than obeyed. The same rule covers a carriage return that would overwrite the line on screen, an ANSI escape that would recolour it, and a bidi override that would make the rest of it render backwards. Ordinary names and values pass through untouched. This matters most for an archive, since a `.whl`, `.vsix` or `.nupkg` comes from a package registry rather than from you.
+
 The MCP tools (`symbol` when given a `file` filter, `read`, `section`, `skeleton`, `outline`, `refs`, `brief`, `grep`, `imports`, `exports`) are confined to the project root, resolving symlinks before the check. Set `mcp.confine_reads_to_project_root = false` (env `TOKEN_GOAT_MCP_CONFINE_READS`) in your global config if you genuinely need cross-root reads from an MCP client; a per-project file cannot set it. The CLI is deliberately unconfined and unchanged. This is defense in depth for one sink, not a sandbox: an agent that can call these tools can usually call its own read tool too.
 
 Note what that flag does and does not cover. It stops a caller traversing *out of* the root it is given; it does not constrain *which* root the caller supplies. Every MCP tool takes an optional `projectRoot`, and it exists for a reason — the server's cwd is often not the workspace root for MCP clients — but tool arguments are model-generated, so that choice is untrusted input like any other. If your deployment treats MCP as the only path to the filesystem, set `mcp.allowed_roots` (env `TOKEN_GOAT_MCP_ALLOWED_ROOTS`, delimiter-separated like `PATH`) to the roots that may legitimately be named; a resolved root outside every entry is then refused. It is empty by default, which keeps the multi-root behaviour above unchanged.

package/SECURITY.md

@@ -56,9 +56,10 @@ Without the overrides a consumer resolves `protobufjs` at 6.x, which carries mor
 | Package | Advisories it carries | Where it loads | Why the advisory does not reach you through Token-Goat |
 | --- | --- | --- | --- |
 | `@xenova/transformers` | [`protobufjs`](https://github.com/advisories/GHSA-xq3m-2v4x-88gg) (critical), `onnx-proto`, `onnxruntime-web`, and its own pinned `sharp` | optional; loaded only when semantic search builds or queries embeddings | not mitigated, so it is the one to weigh. Skip it with `npm install --omit=optional`, or leave `indexing.embeddings_enabled` off, and the code never loads |
-| `exceljs` | [`uuid`](https://github.com/advisories/GHSA-w5hq-g745-h8pq) | optional; loaded only when an `xlsx-*` command opens a workbook | the advisory is a missing bounds check on a caller-supplied `buf` argument; ExcelJS never passes one |
 
-`npm install --omit=optional` gives you an install without either of these, and `npm audit` reports it clean. Every command still starts; the ones that need a package you skipped say so. The `xlsx-*` commands report that ExcelJS is not installed rather than failing oddly, `zip-list`/`zip-read` do the same for fflate, and `semantic` keeps working on keyword search alone: it is the embedding half that goes away, not the command.
+`npm install --omit=optional` gives you an install without it, and `npm audit` reports it clean. Every command still starts; the ones that need a package you skipped say so. The `xlsx-*`, `docx-*` and `pptx-*` commands report that fflate is not installed rather than failing oddly, `zip-list`/`zip-read` do the same, and `semantic` keeps working on keyword search alone: it is the embedding half that goes away, not the command.
+
+`exceljs` used to appear in that table, carrying [`uuid`](https://github.com/advisories/GHSA-w5hq-g745-h8pq). It is now a development dependency instead. The `xlsx-*` commands read the workbook container directly with `fflate` and `fast-xml-parser`, the same two packages the `.docx` and `.pptx` readers already used, so `exceljs` is only a test fixture writer now. That removes 55 packages from a default install, including every deprecated one in the tree.
 
 `html-to-text` used to appear in that table, carrying [`deepmerge-ts`](https://github.com/advisories/GHSA-ggr8-5vv4-36mx). It is now a development dependency instead. esbuild inlines it into `dist/token-goat.mjs` at build time and nothing in the published bundle imports it, so it was a runtime dependency in name only: moving it removes `html-to-text`, `deepmerge-ts`, `htmlparser2`, `selderee`, and `dom-serializer` from an installed copy while the HTML-to-text output stays byte-for-byte identical. That is what takes the no-optional install to zero, and it is better than the alternative we had considered, rolling `html-to-text` back to 9.x: that version pins `htmlparser2` two majors lower, and `htmlparser2` is what parses fetched pages, so it would have traded an advisory in an options merger for an older parser on the one path that handles untrusted input.
 
@@ -89,8 +90,8 @@ Token-Goat is source-available under the PolyForm Noncommercial License 1.0.0. S
 
 Every production dependency is permissively licensed, but a scan does not read it that way on its
 own. Counted from `package-lock.json`, which lists the packages for every platform rather than only
-the ones this machine installed, 22 entries need a human answer: 7 declare a license a scanner
-cannot resolve, and 15 carry a copyleft term. All 22 arrive through optional dependencies. Install
+the ones this machine installed, 21 entries need a human answer: 7 declare a license a scanner
+cannot resolve, and 14 carry a copyleft term. All 21 arrive through optional dependencies. Install
 with `npm install --omit=optional token-goat` and not one of them is present.
 
 **Declarations a scanner cannot resolve.** Both are upstream mistakes, and both are the same
@@ -110,19 +111,19 @@ an unresolvable one.
 | --- | --- | --- |
 | `@img/sharp-libvips-<platform>` (10 packages) | `LGPL-3.0-or-later` | libvips, shipped as a prebuilt shared library and used unmodified. LGPL asks that the library stay replaceable, and it is: it is a separate package that `sharp` loads at runtime. |
 | `@img/sharp-<platform>` (4 packages) | `Apache-2.0 AND LGPL-3.0-or-later` | the Apache half is `sharp` itself, the LGPL half is the same libvips |
-| `jszip` | `(MIT OR GPL-3.0-or-later)` | a choice between the two, and the MIT half is taken |
 
-`sharp` and `jszip` are both optional: `sharp` powers image shrinking, `jszip` arrives through
-`exceljs` for the `xlsx-*` commands.
+`sharp` is optional: it powers image shrinking. `jszip` used to be listed here too; it arrived
+through `exceljs`, which is no longer a dependency a consumer installs.
 
 **Three packages with no license at all used to be here.** `buffers@0.1.1` and `chainsaw@0.1.0`
 shipped with neither a `license` field nor a license file, and `traverse@0.3.9` had the file but
 not the field. No grant at all is worse for a review than a copyleft grant, because there is
 nothing to apply policy to. They arrived through `exceljs`, which depends on `unzipper`, which
 depended on `binary`, which depended on all three. `unzipper` dropped `binary` in 0.11, so an
-override to `^0.12.5` removes the sub-chain, and the deprecated `fstream` with it. Token-goat reads
-workbooks through `wb.xlsx.readFile()`, which goes through `jszip`; `unzipper` is only reachable
-through the streaming reader in `exceljs`, which this code never calls.
+override to `^0.12.5` removes the sub-chain, and the deprecated `fstream` with it. `exceljs` is now
+a development dependency, so none of that chain reaches an installed copy either way; the override
+stays because it is what keeps this repository's own `npm audit`, which includes development
+dependencies, clean.
 
 Reproduce the whole picture: