@allansantos-dev/smart-tool 0.9.82 → 0.9.83
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +51 -0
- package/README.md +35 -4
- package/affected_tests.py +9 -8
- package/client_hooks.py +1 -1
- package/code_graph.py +10 -6
- package/commit_check.py +184 -0
- package/document_converter.py +4 -4
- package/document_read.py +15 -9
- package/document_url.py +107 -0
- package/hook_call.py +3 -2
- package/hook_decision.py +42 -1
- package/hooks/pre_commit.py +45 -0
- package/package.json +1 -1
- package/release_notes.py +125 -0
- package/rename_preview.py +95 -0
- package/requester.py +1 -1
- package/smart_tool_daemon.py +105 -15
- package/version.py +1 -1
- package/web_fetch_many.py +61 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,56 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.9.83 - beta
|
|
4
|
+
|
|
5
|
+
- Commit check, opt-in: `project_manage action=commit_check` reviews the staged change (`git diff --cached`) with the
|
|
6
|
+
notes the edit hook gives a finished diff, and `action=commit_hook_install` writes a git `pre-commit` hook that prints
|
|
7
|
+
them to stderr at every commit, for edits the agent hook never saw. The hook never blocks (exit 0 always): a daemon
|
|
8
|
+
that is down is one line, `Smart Tool is not running, commit not reviewed`, and a check that fails is printed. It finds
|
|
9
|
+
the daemon through its run file, never through a port. The install refuses and changes nothing when `core.hooksPath`
|
|
10
|
+
is set, husky is used or a `pre-commit` that is not ours exists; `action=commit_hook_uninstall` removes only our own
|
|
11
|
+
hook and puts back a backed-up hook that is not ours. Tests: `tests/test_commit_check.py`.
|
|
12
|
+
- New tool `web_fetch_many`: up to 8 URLs with one prompt in one call. Each URL goes through the same path as
|
|
13
|
+
`web_fetch` (24 h cache, failure memory, browser for JavaScript pages, public addresses only, answer of the
|
|
14
|
+
gateway model), 4 at a time, and the reply holds one block per URL in the order given: the answer, or an `ERROR`
|
|
15
|
+
line with the reason, so a URL that fails (an internal address, a 404, a page without text) never hides the others
|
|
16
|
+
and is released to the native `WebFetch` for 10 minutes like a failed `web_fetch`. A URL repeated (same address
|
|
17
|
+
without the fragment) is read once; more than 8 URLs, a malformed list or an empty prompt fail the whole call before
|
|
18
|
+
anything is read; the call fails as a whole only when every URL failed, with the report as its text. `web_fetch` and
|
|
19
|
+
the redirect of the native `WebFetch` are unchanged; the tool is not a redirect target and loads on demand. Measured
|
|
20
|
+
before the build: in 843 turns of local Claude Code transcripts that fetched a page, 621 fetched two or more and 512 spread
|
|
21
|
+
them over several assistant messages (`docs/research/2026-11-21-web-fetch-many-medicao.md`). Tests:
|
|
22
|
+
`tests/test_web_fetch_many.py`.
|
|
23
|
+
- A native `Read` of a code file of a registered project now names the recorded decisions that mention its functions, as
|
|
24
|
+
extra context before the agent plans an edit, with the same rule as the note after an edit: the first three functions
|
|
25
|
+
of the file in the cached code graph, up to two notes each from `docs/research`, the backlog, `CLAUDE.md` and
|
|
26
|
+
`AGENTS.md`, 300 ms per function, files only (no network, no model, no git history). It reads the graph only when it
|
|
27
|
+
is already cached (no analysis starts and nothing waits for one) and is silent when nothing is cached, for a file that
|
|
28
|
+
is not code, for a file outside a project and when no note names a function; a search that fails or is cut is said.
|
|
29
|
+
It follows `decision_mode` (`warn`, default, or `off`), never denies and never blocks the read, whatever `hook_mode` is.
|
|
30
|
+
The note after an edit is unchanged. A project that installed the hook before `Read` was in its matcher is reported as
|
|
31
|
+
outdated by the same check as any other missing tool and rewritten by the install. About 80 to 115 ms per read on this
|
|
32
|
+
repository with the graph in memory. Tests: `tests/test_read_decision_notes.py`. Measurement:
|
|
33
|
+
`docs/research/2026-11-21-notas-de-decisao-na-leitura.md`.
|
|
34
|
+
- `read_document` also takes an `http://` or `https://` URL as its `path`. The file is downloaded with the `web_fetch`
|
|
35
|
+
download (public addresses only, every redirect hop checked, 5 MB cap), kept for 24 hours in the per-user cache folder
|
|
36
|
+
under a name made of a hash of the URL, and read like a local file: same page, slide and sheet ranges, same token
|
|
37
|
+
estimate. The answer names the URL without its query string and credentials, and the metric keeps only the extension of
|
|
38
|
+
the URL path. A URL of a web page fails with a pointer to `web_fetch`. Any other `path` is read exactly as before.
|
|
39
|
+
- `project_scan.tracked_files` is now the one place that lists the files git tracks: the affected-tests selection and the
|
|
40
|
+
working-tree overlay of the code graph no longer run their own `git ls-files`. Their answers do not change.
|
|
41
|
+
- `project_manage action=rename_preview` with `symbol` lists what a rename of a function touches, before the rename: its
|
|
42
|
+
definitions with file and lines, its callers with line (and the calls the graph left unresolved under that name), the
|
|
43
|
+
tests that reach it, the files importing its module, the anchors in `docs/scope` and the lines of README and `docs` text
|
|
44
|
+
that name it. Each list is capped by `limit` with the totals in `counts`; a symbol that is not found fails with the
|
|
45
|
+
closest names, and a project without `docs/scope` says so instead of returning an empty anchor list. Nothing is
|
|
46
|
+
written. Tests: `tests/test_rename_preview.py`.
|
|
47
|
+
- `project_manage action=release_notes` drafts the notes of the next release in Markdown, without a model, under
|
|
48
|
+
`## Draft since <base>`: the commit subjects since `base` (default the latest tag from `git describe --tags
|
|
49
|
+
--abbrev=0`; no tag and no `base` is an error), the public functions the diff touches with their caller and test counts,
|
|
50
|
+
the decision notes that name them and the documentation entries the diff leaves out of date, all from the builders of
|
|
51
|
+
`pr_impact`. Lists are capped by `limit`; every line is a commit subject, a name or a count read from the project.
|
|
52
|
+
Nothing is written. Tests: `tests/test_release_notes.py`.
|
|
53
|
+
|
|
3
54
|
## 0.9.82 - beta
|
|
4
55
|
|
|
5
56
|
- The edit hook adds two lines to its note, both read from what is already cached and silent otherwise (no analysis is
|
package/README.md
CHANGED
|
@@ -8,7 +8,8 @@ A local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharpe
|
|
|
8
8
|
| `smart_scout` | Answers a question about the code with the lines that answer it: reads the candidate files inside Smart Tool and returns `path:lines` with the literal text, so the files never enter the agent's context; with a `scout_model`, a small model also writes a short answer and keeps only the lines that support it; a question about why something is as it is (why, reason, decided) also lists up to three decision notes from the notes the project keeps about its intent, as ids with one excerpt each |
|
|
9
9
|
| `web_search` / `web_search_result` | Web search across several sources in parallel, in the query language and in English, with a cache shared across sessions; `depth="research"` runs a multi-round research loop |
|
|
10
10
|
| `web_fetch` | Reads a known URL and returns only the answer to your prompt, written by a small model; pages are cached for 24 h. Pages over 60,000 characters are not cut: the model reads the parts closest to the prompt (needs an `embedding_model`) |
|
|
11
|
-
| `
|
|
11
|
+
| `web_fetch_many` | Reads up to 8 known URLs in one call with the same prompt, 4 at a time, each exactly as `web_fetch` reads it (24 h cache, browser for JavaScript pages, public addresses only). One block per URL in the order given: the answer, or an `ERROR` line with the reason, so a URL that fails never hides the others; a repeated URL is read once |
|
|
12
|
+
| `read_document` | Reads a PDF, Word, PowerPoint, Excel, HTML, CSV or EPUB file as compact text, a range at a time: type, size, page/slide/sheet/part/chapter count and estimated tokens first, then only the units asked for (`range="3-5"`), so the file never enters the agent's context whole. The path can also be an `http(s)` URL of a PDF, Word, PowerPoint, Excel, CSV or EPUB file: it is downloaded the way `web_fetch` downloads (public addresses only, 5 MB cap), kept for 24 hours and read with the same ranges and token estimate; a URL of a web page fails and points to `web_fetch`. Opt-in: the native Read is not redirected. Scanned PDFs fail with that reason (no OCR) |
|
|
12
13
|
| `project_manage` | Registers and indexes projects, shows coverage, code maps, duplicated functions and index storage; `graph` with `symbol` tells what changing a function touches: callers, calls, the tests that reach it and the files that import it |
|
|
13
14
|
|
|
14
15
|
Agents that must click, type or log in also get Microsoft's [Playwright MCP](https://github.com/microsoft/playwright-mcp),
|
|
@@ -17,7 +18,7 @@ registered as the `playwright` server where this machine lets a browser be drive
|
|
|
17
18
|
An optional hook routes the agent's native Grep/Glob/Read/Bash/WebSearch/WebFetch calls to these tools when they
|
|
18
19
|
are cheaper, or just tells the agent what Smart Tool would do.
|
|
19
20
|
|
|
20
|
-
**Status:** 0.9.
|
|
21
|
+
**Status:** 0.9.83 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
|
|
21
22
|
|
|
22
23
|
## How it works
|
|
23
24
|
|
|
@@ -84,7 +85,7 @@ gateway such as Ollama, nothing about your code leaves the machine.
|
|
|
84
85
|
## Install
|
|
85
86
|
|
|
86
87
|
```powershell
|
|
87
|
-
npx @allansantos-dev/smart-tool@0.9.
|
|
88
|
+
npx @allansantos-dev/smart-tool@0.9.83 install
|
|
88
89
|
```
|
|
89
90
|
|
|
90
91
|
Updates install themselves, on by default: the tray checks npm when it starts (at logon) and every 24 h, and when there
|
|
@@ -303,6 +304,28 @@ Redirects) and Maintenance (Setup audit, Startup).
|
|
|
303
304
|
value; `links` lists the relative links in tracked HTML and Markdown files that point to a missing file, and the ones
|
|
304
305
|
it could not decide (root-relative, template expressions, pages with a `<base>` element) as unchecked, not broken.
|
|
305
306
|
Each takes `limit` and writes nothing.
|
|
307
|
+
**Rename preview**: `project_manage action=rename_preview symbol=...` answers, before a function is renamed, with its
|
|
308
|
+
definitions (file and lines), its callers with line (including calls the graph left unresolved under that name), the
|
|
309
|
+
tests that reach it, the files importing its module, the `path::name` anchors of `docs/scope` that would go stale and
|
|
310
|
+
the lines of the README and `docs` text that name it (`path::name`, `module.name`, the name in backticks, or a
|
|
311
|
+
compound name; a plain word alone is not counted). A symbol that is not found fails with the closest names. Static
|
|
312
|
+
analysis does not see strings or dynamic access, so search the old name as text too. Takes `limit`, writes nothing.
|
|
313
|
+
**Release notes draft**: `project_manage action=release_notes` (optional `base`, default the latest tag; no tag and no
|
|
314
|
+
`base` is an error) returns a Markdown section headed `## Draft since <base>` built without a model: the commit
|
|
315
|
+
subjects since the base, the public functions the diff touches with their caller and test counts, the decision notes
|
|
316
|
+
that name them and the documentation entries the diff leaves out of date, the same builders `pr_impact` uses. It
|
|
317
|
+
writes no prose of its own, so the agent or the maintainer words the notes from it. Takes `limit`, writes nothing.
|
|
318
|
+
**Commit check** (opt-in): `project_manage action=commit_check` reviews the staged change (`git diff --cached`, not the
|
|
319
|
+
working tree) with the notes the edit hook gives a finished diff (credentials, commented-out code, missing docstrings,
|
|
320
|
+
duplicates, dependency advisories, link and edit notes), so edits the agent hook never saw (an IDE, hand edits,
|
|
321
|
+
another agent) are reviewed before they reach the commit. `action=commit_hook_install` writes a git `pre-commit` hook
|
|
322
|
+
in the project that prints that review to stderr at every commit; it never blocks: the exit status is always 0, a
|
|
323
|
+
daemon that is down is one line (`Smart Tool is not running, commit not reviewed`) and a check that fails is
|
|
324
|
+
printed. The hook finds the daemon through its run file, never through a port. The install refuses, naming what it
|
|
325
|
+
found and changing nothing, when `core.hooksPath` is set, husky is used (a `.husky` folder or `husky` in
|
|
326
|
+
`package.json`) or a `pre-commit` that is not Smart Tool's exists; an older hook of ours is backed up first.
|
|
327
|
+
`action=commit_hook_uninstall` removes only that hook (identified by its marker line) and puts back a backed-up hook
|
|
328
|
+
that is not ours. In a worktree or submodule the hook goes where `git rev-parse --git-path hooks` points.
|
|
306
329
|
|
|
307
330
|
**Layers on edit**: when the project has confirmed layers (`.smart-tool/layers.json`), an edit that adds a call from
|
|
308
331
|
a lower layer to a function of a higher one is flagged to the agent; it never blocks. `layer_mode` in
|
|
@@ -319,6 +342,14 @@ Redirects) and Maintenance (Setup audit, Startup).
|
|
|
319
342
|
network, no model, no git history, 300 ms per function), says when it fails or is cut, and never blocks.
|
|
320
343
|
`decision_mode` in `config.json`: `warn` (default) or `off`, also on the setup screen. The measurement behind the rule is
|
|
321
344
|
in `docs/research/2026-11-18-decision-notes-keyword-quality.md`.
|
|
345
|
+
**Recorded decisions on read**: a native `Read` of a code file of a registered project names the same notes for the
|
|
346
|
+
first three functions the file defines in the cached code graph, as context before the agent plans an edit. It is the
|
|
347
|
+
same rule and the same limits (two notes each, files only, 300 ms per function), and it adds a latency budget: the
|
|
348
|
+
graph is read only when it is already cached, so no analysis starts and nothing waits for one, and a project with
|
|
349
|
+
nothing cached, a file that is not code, a file outside a project and a file no note names get no line at all. A range
|
|
350
|
+
read (`offset`, `limit`) gets the note too. It follows `decision_mode` (`off` says nothing), is context only whatever
|
|
351
|
+
`hook_mode` says, and never blocks the read. About 80 to 115 ms per read on this repository
|
|
352
|
+
(`docs/research/2026-11-21-notas-de-decisao-na-leitura.md`).
|
|
322
353
|
|
|
323
354
|
**Folder and layer in a search**: `smart_search folder="<path>"` (or a list) and `layer="<name>"` restrict the
|
|
324
355
|
search to a folder of the project or to one of its confirmed layers, by their path globs. The filter runs before
|
|
@@ -482,7 +513,7 @@ fails to load is listed in the setup screen with its error and does not affect t
|
|
|
482
513
|
## Uninstall
|
|
483
514
|
|
|
484
515
|
```powershell
|
|
485
|
-
npx @allansantos-dev/smart-tool@0.9.
|
|
516
|
+
npx @allansantos-dev/smart-tool@0.9.83 uninstall [--target DIR] [--keep-data]
|
|
486
517
|
```
|
|
487
518
|
|
|
488
519
|
It asks nothing and opens no window. It stops the tray and the daemon, removes the logon task, the hook entries of Smart
|
package/affected_tests.py
CHANGED
|
@@ -16,6 +16,7 @@ import code_graph
|
|
|
16
16
|
import index_inventory
|
|
17
17
|
import index_profile
|
|
18
18
|
import index_scope
|
|
19
|
+
import project_scan
|
|
19
20
|
|
|
20
21
|
RUN_ALL = re.compile(
|
|
21
22
|
r"(^|/)(package\.json|package-lock\.json|npm-shrinkwrap\.json|yarn\.lock|pnpm-lock\.yaml|pnpm-workspace\.yaml|"
|
|
@@ -284,15 +285,15 @@ def _support(root, path):
|
|
|
284
285
|
def _tracked_tests_outside_graph(root, parsed, profile):
|
|
285
286
|
"""Code test files git tracks that the graph could not analyze (over the size limits, unreadable): their imports
|
|
286
287
|
are unknown, so they are selected instead of silently missing (a 1.4 MB test file covering 1,427 cases did)."""
|
|
287
|
-
|
|
288
|
-
|
|
288
|
+
try:
|
|
289
|
+
tracked = project_scan.tracked_files(root)
|
|
290
|
+
except ValueError as exc:
|
|
291
|
+
if exc.__cause__:
|
|
292
|
+
raise
|
|
289
293
|
return []
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
and index_profile.kind(rel, profile) == "test" and os.path.isfile(os.path.join(root, rel)):
|
|
294
|
-
tests.append(rel)
|
|
295
|
-
return tests
|
|
294
|
+
return [rel for rel in tracked
|
|
295
|
+
if rel not in parsed and rel.lower().endswith(TEST_SOURCES) and os.path.basename(rel) != "conftest.py"
|
|
296
|
+
and index_profile.kind(rel, profile) == "test"]
|
|
296
297
|
|
|
297
298
|
|
|
298
299
|
def _label(symbol, by_id):
|
package/client_hooks.py
CHANGED
|
@@ -36,7 +36,7 @@ HOME = os.path.expanduser("~")
|
|
|
36
36
|
PREVIEW_TTL_S = 600
|
|
37
37
|
CLIENTS = {
|
|
38
38
|
"claude": {"label": "Claude Code", "names": {"claude-code"}, "files": [os.path.join(endpoint_sync.claude_dir(), "settings.json")],
|
|
39
|
-
"docs": "https://code.claude.com/docs/en/hooks", "event": "PreToolUse", "matcher": "Agent|Task|Grep|Glob|Read|Bash|PowerShell|WebSearch|WebFetch|Edit|MultiEdit|Write|mcp__smart-tool__smart_search|mcp__smart-tool__web_search|mcp__smart-tool__web_fetch",
|
|
39
|
+
"docs": "https://code.claude.com/docs/en/hooks", "event": "PreToolUse", "matcher": "Agent|Task|Grep|Glob|Read|Bash|PowerShell|WebSearch|WebFetch|Edit|MultiEdit|Write|mcp__smart-tool__smart_search|mcp__smart-tool__web_search|mcp__smart-tool__web_fetch|mcp__smart-tool__web_fetch_many",
|
|
40
40
|
"post_matcher": "Grep|Bash|PowerShell|WebFetch|WebSearch"},
|
|
41
41
|
"codex": {"label": "Codex", "prefix": "codex", "files": [os.path.join(endpoint_sync.codex_dir(), "hooks.json"), os.path.join(endpoint_sync.codex_dir(), "config.toml")],
|
|
42
42
|
"docs": "https://learn.chatgpt.com/docs/hooks", "event": "PreToolUse", "matcher": "Bash|apply_patch", "post_matcher": "Bash",
|
package/code_graph.py
CHANGED
|
@@ -17,6 +17,7 @@ from pathlib import Path
|
|
|
17
17
|
import indexer
|
|
18
18
|
import index_inventory
|
|
19
19
|
import index_scope
|
|
20
|
+
import project_scan
|
|
20
21
|
import project_identity
|
|
21
22
|
import web_search_adapters
|
|
22
23
|
import web_document_graph
|
|
@@ -634,12 +635,15 @@ def _pending_changes(root,view_path):
|
|
|
634
635
|
changes[rel]='deleted';continue
|
|
635
636
|
if (stat.st_size,stat.st_mtime_ns)!=(size,stamp):changes[rel]='changed'
|
|
636
637
|
scope=index_scope.load_scope(root)
|
|
637
|
-
|
|
638
|
-
|
|
639
|
-
|
|
640
|
-
|
|
641
|
-
|
|
642
|
-
|
|
638
|
+
try:
|
|
639
|
+
tracked=project_scan.tracked_files(root)
|
|
640
|
+
except ValueError as exc:
|
|
641
|
+
if exc.__cause__:raise
|
|
642
|
+
tracked=[]
|
|
643
|
+
if scope:
|
|
644
|
+
for rel in tracked:
|
|
645
|
+
if rel not in indexed and rel not in changes and rel.lower().endswith(SOURCE_EXTENSIONS) \
|
|
646
|
+
and index_scope.in_scope(scope,rel):
|
|
643
647
|
changes[rel]='changed'
|
|
644
648
|
return changes
|
|
645
649
|
|
package/commit_check.py
ADDED
|
@@ -0,0 +1,184 @@
|
|
|
1
|
+
"""Commit-time review of the staged change, and the opt-in git hook that prints it.
|
|
2
|
+
|
|
3
|
+
The review is the one the edit hook gives a finished diff (`hook_decision.review_diff`), over what `git diff --cached`
|
|
4
|
+
holds, so edits the agent hook never saw (an IDE, hand edits, another agent) are reviewed before they reach the commit.
|
|
5
|
+
The `pre-commit` script this module writes only forwards to the daemon (`hooks/pre_commit.py`) and never blocks a commit.
|
|
6
|
+
"""
|
|
7
|
+
import glob
|
|
8
|
+
import json
|
|
9
|
+
import os
|
|
10
|
+
import shlex
|
|
11
|
+
import subprocess
|
|
12
|
+
import sys
|
|
13
|
+
|
|
14
|
+
import affected_tests
|
|
15
|
+
import atomic_io
|
|
16
|
+
import client_hooks
|
|
17
|
+
import config
|
|
18
|
+
import hook_decision
|
|
19
|
+
|
|
20
|
+
MARKER = "# smart-tool-commit-check"
|
|
21
|
+
HOOK_NAME = "pre-commit"
|
|
22
|
+
HOOK_SCRIPT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "hooks", "pre_commit.py")
|
|
23
|
+
HOOK_ORIGIN = client_hooks.HOOK_ORIGIN
|
|
24
|
+
MAX_FILES = 50
|
|
25
|
+
NO_WINDOW = getattr(subprocess, "CREATE_NO_WINDOW", 0)
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
def _git_raw(root, *args):
|
|
29
|
+
"""The finished `git` process (bytes output) run in `root`; the caller reads the exit code."""
|
|
30
|
+
return subprocess.run(["git", *args], cwd=root, capture_output=True, timeout=60, creationflags=NO_WINDOW)
|
|
31
|
+
|
|
32
|
+
|
|
33
|
+
def _staged_text(root, rel):
|
|
34
|
+
"""The text of `rel` as staged, or None when it is binary, not UTF-8 or over the review size."""
|
|
35
|
+
done = _git_raw(root, "show", f":./{rel}")
|
|
36
|
+
if done.returncode or len(done.stdout) > hook_decision.MAX_REVIEW_BYTES:
|
|
37
|
+
return None
|
|
38
|
+
try:
|
|
39
|
+
text = done.stdout.decode("utf-8")
|
|
40
|
+
except UnicodeDecodeError:
|
|
41
|
+
return None
|
|
42
|
+
return None if "\0" in text else text
|
|
43
|
+
|
|
44
|
+
|
|
45
|
+
def _staged_paths(root):
|
|
46
|
+
"""Paths relative to `root` of the files the commit adds or changes (a deletion has nothing to review), sorted."""
|
|
47
|
+
fields = affected_tests._git(root, "diff", "--cached", "--name-status", "-z", "--no-renames", "--relative",
|
|
48
|
+
"--", ".").split("\0")
|
|
49
|
+
return sorted(path for status, path in zip(fields[0::2], fields[1::2]) if status != "D")
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
def review(root):
|
|
53
|
+
"""Reviews the staged change of the repository at `root` with the notes the edit hook gives a finished diff.
|
|
54
|
+
|
|
55
|
+
Parameters: root (str) project folder, inside a git repository.
|
|
56
|
+
Returns: dict with `files` (the staged files that were reviewed) and `notes` (str; empty when every check is quiet,
|
|
57
|
+
and it says how many staged files past MAX_FILES were not reviewed).
|
|
58
|
+
Raises: ValueError when git cannot read the index, RuntimeError or OSError when a check cannot run.
|
|
59
|
+
"""
|
|
60
|
+
paths = _staged_paths(root)
|
|
61
|
+
changes = []
|
|
62
|
+
for rel in paths[:MAX_FILES]:
|
|
63
|
+
after = _staged_text(root, rel)
|
|
64
|
+
if after is None:
|
|
65
|
+
continue
|
|
66
|
+
try:
|
|
67
|
+
before = affected_tests._git(root, "show", f"HEAD:./{rel}")
|
|
68
|
+
except ValueError:
|
|
69
|
+
before = None
|
|
70
|
+
changes.append((rel, os.path.join(root, *rel.split("/")), before, after))
|
|
71
|
+
notes = hook_decision.review_diff([(full, before, after) for _rel, full, before, after in changes],
|
|
72
|
+
config.load_config()) if changes else ""
|
|
73
|
+
left = len(paths) - MAX_FILES
|
|
74
|
+
if left > 0:
|
|
75
|
+
skipped = f"{left} staged file{'s were' if left != 1 else ' was'} not reviewed: the review reads the first {MAX_FILES}."
|
|
76
|
+
notes = f"{notes} {skipped}".strip()
|
|
77
|
+
return {"files": [rel for rel, *_ in changes], "notes": notes}
|
|
78
|
+
|
|
79
|
+
|
|
80
|
+
def hook_python():
|
|
81
|
+
"""The interpreter the hook runs, with forward slashes and no spaces: this process's own, or its console twin when
|
|
82
|
+
that is the windowless `pythonw`, which would drop the notes the hook prints to stderr."""
|
|
83
|
+
path = sys.executable
|
|
84
|
+
if os.path.basename(path).lower() == "pythonw.exe":
|
|
85
|
+
path = os.path.join(os.path.dirname(path), "python.exe")
|
|
86
|
+
return client_hooks._shell_safe(path)
|
|
87
|
+
|
|
88
|
+
|
|
89
|
+
def _hook_text():
|
|
90
|
+
"""The `pre-commit` script: the review client of the product, and a visible line, never a failing exit, when it
|
|
91
|
+
cannot start (a git hook that exits non-zero blocks the commit)."""
|
|
92
|
+
command = f"{shlex.quote(hook_python())} {shlex.quote(client_hooks._shell_safe(HOOK_SCRIPT))} {HOOK_ORIGIN}"
|
|
93
|
+
return (f"#!/bin/sh\n{MARKER}: written by Smart Tool (project_manage action=commit_hook_install); "
|
|
94
|
+
f"commit_hook_uninstall removes it\n"
|
|
95
|
+
f'{command} || echo "Smart Tool commit check could not run (exit $?), commit not reviewed" >&2\n'
|
|
96
|
+
"exit 0\n")
|
|
97
|
+
|
|
98
|
+
|
|
99
|
+
def _hooks_dir(root):
|
|
100
|
+
"""The absolute folder git runs hooks from for the repository at `root` (a worktree or submodule shares the one
|
|
101
|
+
its `.git` file points to)."""
|
|
102
|
+
return os.path.abspath(os.path.join(root, affected_tests._git(root, "rev-parse", "--git-path", "hooks").strip()))
|
|
103
|
+
|
|
104
|
+
|
|
105
|
+
def _read(path):
|
|
106
|
+
with open(path, encoding="utf-8", errors="replace") as stream:
|
|
107
|
+
return stream.read()
|
|
108
|
+
|
|
109
|
+
|
|
110
|
+
def _husky(root):
|
|
111
|
+
"""Sentences naming where husky manages the git hooks of the repository: a `.husky` folder, or `husky` in the
|
|
112
|
+
dependencies or scripts of a package.json, in `root` and in the top of the repository."""
|
|
113
|
+
top = affected_tests._git(root, "rev-parse", "--show-toplevel").strip()
|
|
114
|
+
found = []
|
|
115
|
+
for folder in {os.path.realpath(path): path for path in (root, top)}.values():
|
|
116
|
+
if os.path.isdir(os.path.join(folder, ".husky")):
|
|
117
|
+
found.append(f"husky manages the git hooks here (a .husky folder in {folder})")
|
|
118
|
+
try:
|
|
119
|
+
package = json.loads(_read(os.path.join(folder, "package.json")))
|
|
120
|
+
except (OSError, ValueError):
|
|
121
|
+
continue
|
|
122
|
+
used = isinstance(package, dict) and (
|
|
123
|
+
any("husky" in (package.get(kind) or {}) for kind in ("dependencies", "devDependencies")) or
|
|
124
|
+
any("husky" in str(command) for command in (package.get("scripts") or {}).values()))
|
|
125
|
+
if used:
|
|
126
|
+
found.append(f"husky manages the git hooks here (package.json in {folder} names it)")
|
|
127
|
+
return found
|
|
128
|
+
|
|
129
|
+
|
|
130
|
+
def _blockers(root, hook):
|
|
131
|
+
"""Sentences naming what makes the install refuse: `core.hooksPath` set, husky, or a `pre-commit` that is not ours."""
|
|
132
|
+
found = []
|
|
133
|
+
configured = _git_raw(root, "config", "--show-scope", "--get", "core.hooksPath")
|
|
134
|
+
if configured.returncode == 0:
|
|
135
|
+
scope, _, value = configured.stdout.decode("utf-8", "replace").strip().partition("\t")
|
|
136
|
+
found.append(f"core.hooksPath is set ({scope}) to {value}, so git does not run .git/hooks")
|
|
137
|
+
elif configured.returncode != 1:
|
|
138
|
+
raise ValueError(f"git config core.hooksPath failed: {configured.stderr.decode('utf-8', 'replace').strip()[:300]}")
|
|
139
|
+
found += _husky(root)
|
|
140
|
+
if os.path.isfile(hook) and MARKER not in _read(hook):
|
|
141
|
+
found.append(f"{hook} already exists and is not Smart Tool's (it has no {MARKER} line)")
|
|
142
|
+
return found
|
|
143
|
+
|
|
144
|
+
|
|
145
|
+
def install_hook(root):
|
|
146
|
+
"""Writes the opt-in `pre-commit` hook of the repository at `root`, after a backup of an older hook of ours.
|
|
147
|
+
|
|
148
|
+
Parameters: root (str) project folder, inside a git repository.
|
|
149
|
+
Returns: dict with `installed`, `changed`, `file` and `backup` (path of the copy of our older hook, None when new).
|
|
150
|
+
Raises: ValueError, naming everything found and changing nothing, when core.hooksPath is set, husky is used or a
|
|
151
|
+
`pre-commit` that is not ours exists.
|
|
152
|
+
"""
|
|
153
|
+
hook = os.path.join(_hooks_dir(root), HOOK_NAME)
|
|
154
|
+
found = _blockers(root, hook)
|
|
155
|
+
if found:
|
|
156
|
+
raise ValueError("The commit hook was not installed and nothing was changed: " + "; ".join(found) + ".")
|
|
157
|
+
os.makedirs(os.path.dirname(hook), exist_ok=True)
|
|
158
|
+
backup = atomic_io.backup_file(hook)
|
|
159
|
+
atomic_io.write_text(hook, _hook_text())
|
|
160
|
+
os.chmod(hook, 0o755)
|
|
161
|
+
return {"installed": True, "changed": True, "file": hook, "backup": backup,
|
|
162
|
+
"note": "Commits in this repository print the review of the staged change to stderr and never wait on it "
|
|
163
|
+
"to succeed."}
|
|
164
|
+
|
|
165
|
+
|
|
166
|
+
def uninstall_hook(root):
|
|
167
|
+
"""Removes our `pre-commit` hook of the repository at `root` and puts back a backed-up hook that is not ours.
|
|
168
|
+
|
|
169
|
+
Parameters: root (str) project folder, inside a git repository.
|
|
170
|
+
Returns: dict with `removed`, `restored` (path of the backup put back, None when there is none) and `file`; a
|
|
171
|
+
repository without a hook answers `removed` false with a `note`.
|
|
172
|
+
Raises: ValueError when the `pre-commit` file is not ours; it stays as it is.
|
|
173
|
+
"""
|
|
174
|
+
hook = os.path.join(_hooks_dir(root), HOOK_NAME)
|
|
175
|
+
if not os.path.isfile(hook):
|
|
176
|
+
return {"removed": False, "restored": None, "file": hook, "note": f"No pre-commit hook at {hook}."}
|
|
177
|
+
if MARKER not in _read(hook):
|
|
178
|
+
raise ValueError(f"{hook} is not Smart Tool's (it has no {MARKER} line) and was left as it is.")
|
|
179
|
+
os.remove(hook)
|
|
180
|
+
copies = sorted(glob.glob(glob.escape(hook + atomic_io.BACKUP_INFIX) + "*"), reverse=True)
|
|
181
|
+
saved = next((copy for copy in copies if MARKER not in _read(copy)), None)
|
|
182
|
+
if saved:
|
|
183
|
+
os.replace(saved, hook)
|
|
184
|
+
return {"removed": True, "restored": saved, "file": hook}
|
package/document_converter.py
CHANGED
|
@@ -114,16 +114,16 @@ def _checked(engine, pages, expected_pages):
|
|
|
114
114
|
return pages
|
|
115
115
|
|
|
116
116
|
|
|
117
|
-
def cache_folder():
|
|
118
|
-
"""The
|
|
119
|
-
here; the folders of other versions are deleted the first time it is asked for in a process.
|
|
117
|
+
def cache_folder(name="convert"):
|
|
118
|
+
"""The folder `name` of this version's cache: `%LOCALAPPDATA%\\SmartTool\\cache\\<version>\\<name>` (the converted
|
|
119
|
+
pages by default), created here; the folders of other versions are deleted the first time it is asked for in a process.
|
|
120
120
|
|
|
121
121
|
Raises: RuntimeError when LOCALAPPDATA is not set."""
|
|
122
122
|
local = os.environ.get("LOCALAPPDATA")
|
|
123
123
|
if not local:
|
|
124
124
|
raise RuntimeError("LOCALAPPDATA is not set: there is no per-user folder for the converter's cache.")
|
|
125
125
|
root = os.path.join(local, "SmartTool", "cache")
|
|
126
|
-
folder = os.path.join(root, version.VERSION,
|
|
126
|
+
folder = os.path.join(root, version.VERSION, name)
|
|
127
127
|
os.makedirs(folder, exist_ok=True)
|
|
128
128
|
if root not in _swept:
|
|
129
129
|
_swept.add(root)
|
package/document_read.py
CHANGED
|
@@ -286,12 +286,13 @@ def _spans(numbers):
|
|
|
286
286
|
return ', '.join(spans)
|
|
287
287
|
|
|
288
288
|
|
|
289
|
-
def header(document):
|
|
289
|
+
def header(document, shown_as=None):
|
|
290
290
|
"""The metadata block that opens every answer: path, type, size, units and the estimated tokens of the whole text.
|
|
291
291
|
|
|
292
|
-
Parameters: document
|
|
292
|
+
Parameters: document; shown_as (what the first line names instead of the document's path, for a file downloaded
|
|
293
|
+
from a URL). Returns: the block as text."""
|
|
293
294
|
total = sum(estimate_tokens(unit.text) for unit in document.units)
|
|
294
|
-
lines = [f'document: {document.path}',
|
|
295
|
+
lines = [f'document: {shown_as or document.path}',
|
|
295
296
|
f'type: {document.kind} | size: {document.size / 1048576:.2f} MiB | '
|
|
296
297
|
f'{UNIT_NAMES[document.kind]}s: {len(document.units)} | est. tokens (all text): {total:,}']
|
|
297
298
|
lines += [f'{key}: {value}' for key, value in document.facts.items()
|
|
@@ -299,18 +300,23 @@ def header(document):
|
|
|
299
300
|
return '\n'.join(lines)
|
|
300
301
|
|
|
301
302
|
|
|
302
|
-
def
|
|
303
|
+
def check_max_chars(max_chars):
|
|
304
|
+
"""Raises ValueError unless max_chars is an integer from 1000 to MAX_CHARS_LIMIT."""
|
|
305
|
+
if isinstance(max_chars, bool) or not isinstance(max_chars, int) or not 1000 <= max_chars <= MAX_CHARS_LIMIT:
|
|
306
|
+
raise ValueError(f'max_chars must be an integer from 1000 to {MAX_CHARS_LIMIT}')
|
|
307
|
+
|
|
308
|
+
|
|
309
|
+
def read(path, range_spec=None, max_chars=DEFAULT_MAX_CHARS, info_only=False, shown_as=None):
|
|
303
310
|
"""The answer of read_document: the metadata block, then the chosen units as markdown, a note naming the next range
|
|
304
311
|
when the text was cut.
|
|
305
312
|
|
|
306
313
|
Parameters: path (see load), range_spec (see parse_range; None reads from the first unit up to max_chars),
|
|
307
314
|
max_chars (cap of the text returned, 1000-MAX_CHARS_LIMIT; a single unit over the cap is cut at a line with a
|
|
308
|
-
notice), info_only (only the metadata block). Returns: the text. Raises: as load and
|
|
309
|
-
max_chars out of bounds."""
|
|
310
|
-
|
|
311
|
-
raise ValueError(f'max_chars must be an integer from 1000 to {MAX_CHARS_LIMIT}')
|
|
315
|
+
notice), info_only (only the metadata block), shown_as (see header). Returns: the text. Raises: as load and
|
|
316
|
+
parse_range, ValueError for a max_chars out of bounds."""
|
|
317
|
+
check_max_chars(max_chars)
|
|
312
318
|
document = load(path)
|
|
313
|
-
head = header(document)
|
|
319
|
+
head = header(document, shown_as)
|
|
314
320
|
if info_only:
|
|
315
321
|
return head
|
|
316
322
|
numbers = parse_range(range_spec, document) if range_spec else list(range(1, len(document.units) + 1))
|
package/document_url.py
ADDED
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
"""read_document by URL: the file is downloaded with the web_fetch download (public addresses only, every redirect hop
|
|
2
|
+
checked, 5 MB cap) into the per-user cache folder and then read by document_read like a local file, so ranges and the
|
|
3
|
+
token estimate work the same. A web page is refused with the pointer to web_fetch. The query string and the
|
|
4
|
+
credentials of a URL are never part of the answer or of a file name."""
|
|
5
|
+
import hashlib
|
|
6
|
+
import os
|
|
7
|
+
import time
|
|
8
|
+
import urllib.parse
|
|
9
|
+
|
|
10
|
+
import atomic_io
|
|
11
|
+
import document_converter
|
|
12
|
+
import document_read
|
|
13
|
+
import web_fetch
|
|
14
|
+
|
|
15
|
+
FOLDER = "download"
|
|
16
|
+
URL_EXTENSIONS = (".pdf", ".docx", ".pptx", ".xlsx", ".xlsm", ".csv", ".epub")
|
|
17
|
+
CONTENT_TYPES = {"application/pdf": ".pdf", "text/csv": ".csv", "application/epub+zip": ".epub",
|
|
18
|
+
"application/vnd.ms-excel.sheet.macroenabled.12": ".xlsm",
|
|
19
|
+
**{kind: "." + extension for kind, extension in web_fetch.OFFICE_TYPES.items()}}
|
|
20
|
+
EXTENSION_FALLBACK_TYPES = web_fetch.GENERIC_BINARY_TYPES + web_fetch.TEXT_TYPES
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
def is_url(value):
|
|
24
|
+
"""Whether `value` is read as a URL: a string that starts with `http://` or `https://`, nothing else."""
|
|
25
|
+
return isinstance(value, str) and value.startswith(("http://", "https://"))
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
def shown(url):
|
|
29
|
+
"""The URL as an answer names it: scheme, host, port and path, without credentials, query or fragment.
|
|
30
|
+
|
|
31
|
+
Raises: ValueError when the URL cannot be parsed."""
|
|
32
|
+
try:
|
|
33
|
+
parts = urllib.parse.urlsplit(url)
|
|
34
|
+
host, port = parts.hostname, parts.port
|
|
35
|
+
except ValueError:
|
|
36
|
+
raise ValueError("read_document could not parse the URL") from None
|
|
37
|
+
if not host:
|
|
38
|
+
raise ValueError("read_document needs a host in the URL")
|
|
39
|
+
host = f"[{host}]" if ":" in host else host
|
|
40
|
+
return f"{parts.scheme}://{host}{f':{port}' if port else ''}{parts.path}"
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
def extension(url):
|
|
44
|
+
"""Lowercase extension of the URL path (`.pdf`), empty when there is none or the URL cannot be parsed; this is what
|
|
45
|
+
the metric records, so the query string never reaches it."""
|
|
46
|
+
try:
|
|
47
|
+
return os.path.splitext(urllib.parse.urlsplit(url).path)[1].lower()
|
|
48
|
+
except ValueError:
|
|
49
|
+
return ""
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
def _document_extension(kind, final_url):
|
|
53
|
+
"""File extension for the downloaded bytes: by content type, or by the extension of the final URL path when the
|
|
54
|
+
server sends a generic or text type. Raises ValueError, naming web_fetch, for a web page, and naming the readable
|
|
55
|
+
types for anything else."""
|
|
56
|
+
name = shown(final_url)
|
|
57
|
+
if "html" in kind:
|
|
58
|
+
raise ValueError(f"{name} is a web page ({kind}), not a document: use web_fetch with this URL and a prompt "
|
|
59
|
+
"to read it.")
|
|
60
|
+
found = CONTENT_TYPES.get(kind)
|
|
61
|
+
if not found and kind in EXTENSION_FALLBACK_TYPES and extension(final_url) in URL_EXTENSIONS:
|
|
62
|
+
found = extension(final_url)
|
|
63
|
+
if not found:
|
|
64
|
+
raise ValueError(f"{name} is {kind}, which is not a document read_document reads "
|
|
65
|
+
f"({', '.join(URL_EXTENSIONS)}); for a web page use web_fetch.")
|
|
66
|
+
return found
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
def _stale_sweep(folder):
|
|
70
|
+
"""Deletes the downloads older than the 24 h web_fetch keeps a page."""
|
|
71
|
+
cutoff = time.time() - web_fetch.PAGE_TTL_S
|
|
72
|
+
for entry in os.scandir(folder):
|
|
73
|
+
if entry.is_file() and entry.stat().st_mtime < cutoff:
|
|
74
|
+
try:
|
|
75
|
+
os.remove(entry.path)
|
|
76
|
+
except OSError:
|
|
77
|
+
continue
|
|
78
|
+
|
|
79
|
+
|
|
80
|
+
def download(url, deadline):
|
|
81
|
+
"""The local path of the file at `url`: the copy downloaded in the last 24 h, else a fresh download kept in
|
|
82
|
+
`<cache>/download` under a name made of a SHA-256 of the URL (never the URL, the query or the file name).
|
|
83
|
+
|
|
84
|
+
Parameters: url (http or https), deadline (time.monotonic() value). Returns: the path. Raises: FetchError for a
|
|
85
|
+
refused address, a failed or oversized download; ValueError for a web page or a type that is not a document;
|
|
86
|
+
TimeoutError past the deadline."""
|
|
87
|
+
folder = document_converter.cache_folder(FOLDER)
|
|
88
|
+
_stale_sweep(folder)
|
|
89
|
+
stem = hashlib.sha256(web_fetch.normalize(url).encode()).hexdigest()[:40]
|
|
90
|
+
for name in os.listdir(folder):
|
|
91
|
+
if name.startswith(stem + "."):
|
|
92
|
+
return os.path.join(folder, name)
|
|
93
|
+
final_url, kind, _charset, body = web_fetch.fetch_http(web_fetch.normalize(url), deadline)
|
|
94
|
+
path = os.path.join(folder, stem + _document_extension(kind, final_url))
|
|
95
|
+
atomic_io.write_secret_bytes(path, body)
|
|
96
|
+
return path
|
|
97
|
+
|
|
98
|
+
|
|
99
|
+
def read(url, range_spec, max_chars, info_only, deadline):
|
|
100
|
+
"""The answer of read_document for an http(s) URL: the file downloaded (or kept from a recent read) and read by
|
|
101
|
+
document_read, with the URL, without its query and credentials, on the first line.
|
|
102
|
+
|
|
103
|
+
Parameters: as document_read.read, with the URL for the path and the deadline of the download. Returns: the text.
|
|
104
|
+
Raises: as download and document_read.read."""
|
|
105
|
+
label = shown(url)
|
|
106
|
+
document_read.check_max_chars(max_chars)
|
|
107
|
+
return document_read.read(download(url, deadline), range_spec, max_chars, info_only, label)
|
package/hook_call.py
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
|
-
"""Client side of every command hook that calls the Smart Tool daemon (PreToolUse, PostToolUse, SubagentStart, Stop
|
|
1
|
+
"""Client side of every command hook that calls the Smart Tool daemon (PreToolUse, PostToolUse, SubagentStart, Stop, and the
|
|
2
|
+
opt-in git pre-commit).
|
|
2
3
|
|
|
3
4
|
The daemon address is read from its run file on every call, never from a port written into a settings file. A refused or
|
|
4
5
|
reset connection (a daemon restarting) is retried with backoff for `RESTART_WINDOW_S`, so a restart produces no message at
|
|
@@ -21,7 +22,7 @@ DOWN_MEMORY_S = 60
|
|
|
21
22
|
CONNECT_TIMEOUT_S = 2
|
|
22
23
|
QUIET_CONNECT_TIMEOUT_S = 0.4
|
|
23
24
|
BACKOFF_S = (0.1, 0.2, 0.4, 0.8, 1.0)
|
|
24
|
-
ROUTE_TIMEOUT_S = {"pretool": 14, "posttool": 59, "subagent-start": 4, "stop": 110}
|
|
25
|
+
ROUTE_TIMEOUT_S = {"pretool": 14, "posttool": 59, "subagent-start": 4, "stop": 110, "commit-check": 55}
|
|
25
26
|
DOWN_MARKER = os.path.join(paths.DATA_DIR, "hook-down")
|
|
26
27
|
LOG_NAME = "daemon.log"
|
|
27
28
|
RETRYABLE = (ConnectionError, http.client.RemoteDisconnected, http.client.BadStatusLine)
|