@allansantos-dev/smart-tool 0.9.21 → 0.9.23
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +36 -0
- package/README.md +10 -4
- package/client_hooks.py +68 -37
- package/code_graph.py +33 -6
- package/code_impact.py +1 -1
- package/dossier.py +130 -0
- package/duplicates.py +18 -1
- package/endpoint_sync.py +10 -6
- package/hook_decision.py +120 -46
- package/hooks/pretool_router.py +2 -1
- package/integration_metrics.py +5 -2
- package/package.json +1 -1
- package/redirect_rule.py +75 -9
- package/setup_ui.py +1 -1
- package/smart_tool_daemon.py +88 -15
- package/tool_redirect.py +103 -0
- package/version.py +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,41 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.9.23 - beta
|
|
4
|
+
|
|
5
|
+
- The hook no longer blocks. A content search over a project folder (the Grep tool on a folder, `grep -r`, `rg`,
|
|
6
|
+
`git grep`, `findstr /s`, `Select-String` over a listing, an inline python/node script that walks a folder) and
|
|
7
|
+
native WebFetch/WebSearch are answered by Smart Tool, and the answer comes back as the output of the call the agent
|
|
8
|
+
made, with no block message: before, the deny pushed agents to retry the same search through Bash and the Smart Tool
|
|
9
|
+
went unused. The query is the search pattern of the call (regex syntax dropped); the project is the registered one
|
|
10
|
+
holding the folder, else the session folder.
|
|
11
|
+
- Claude Code: a new PostToolUse entry (`/hooks/posttool`, 60 s) puts the answer in the tool's own output, in the
|
|
12
|
+
shape that tool returns. A shell search whose answer is ready within 10 s runs as a no-op (`true`, `$null` in
|
|
13
|
+
PowerShell), so a grep with no match no longer ends in exit 1; a later answer replaces the native output when the
|
|
14
|
+
command succeeds. WebFetch and WebSearch still run natively and their result is replaced (measured on Claude Code
|
|
15
|
+
2.1.300: a PreToolUse cannot skip them, and a WebFetch pointed at the daemon fails because http turns into https).
|
|
16
|
+
WebSearch with `allowed_domains`/`blocked_domains` is left alone.
|
|
17
|
+
- Codex: the shell search is rewritten to `cat` a file holding the answer, approved with `allow` (Codex applies a
|
|
18
|
+
rewrite only with it); the daemon lets `allow` through for that command and nothing else.
|
|
19
|
+
- A Smart Tool call that fails or does not answer in time leaves the native call and its output as they were. A deny,
|
|
20
|
+
with the exact call to make, remains only for a search whose pattern cannot be read from the command, and for a
|
|
21
|
+
client that sends no `tool_use_id`.
|
|
22
|
+
- A redirected shell search waits up to 10 s for its answer in the PreToolUse (a warm `smart_search` takes 4 to
|
|
23
|
+
13 s with the reranker), so the PreToolUse entry's timeout goes back from 6 s to 15 s (Codex's thin command 5 s
|
|
24
|
+
to 14 s); every other decision keeps its 4 s budget in the daemon.
|
|
25
|
+
- The setup screen counts a redirect answered this way as `served`. The hook entry of an older version shows as
|
|
26
|
+
outdated until you accept the update (preview and backup first).
|
|
27
|
+
- `smart_search` with `mode=dossier` answers a narrow keyword query over `docs/research`, the backlog, `CLAUDE.md` and
|
|
28
|
+
`AGENTS.md` with ids, short excerpts and a time budget, without an index.
|
|
29
|
+
|
|
30
|
+
## 0.9.22 - beta
|
|
31
|
+
|
|
32
|
+
- The review of an Edit or Write (docstring reminder, copied-function check) is about twice as fast on a warm daemon
|
|
33
|
+
(a whole-file Write of the 130 KB `smart_tool_daemon.py`: 1.7 s to 0.8 s), so it stays within the hook's 4 s on large
|
|
34
|
+
files instead of being skipped: the scan of pending changes runs once per review instead of once per use, the
|
|
35
|
+
functions of each changed file are remembered until the file is written again, and the impact lookup reads the cached
|
|
36
|
+
graph instead of copying it. The first review after the daemon starts still waits for the graph to load and may be
|
|
37
|
+
skipped; the work finishes in the background and the next one is fast.
|
|
38
|
+
|
|
3
39
|
## 0.9.21 - beta
|
|
4
40
|
|
|
5
41
|
- The PreToolUse hook can no longer hold a tool call for long. Its decision has 4 s; past that it answers without one
|
package/README.md
CHANGED
|
@@ -15,7 +15,7 @@ registered as the `playwright` server where this machine lets a browser be drive
|
|
|
15
15
|
An optional hook routes the agent's native Grep/Glob/Read/Bash/WebSearch/WebFetch calls to these tools when they
|
|
16
16
|
are cheaper, or just tells the agent what Smart Tool would do.
|
|
17
17
|
|
|
18
|
-
**Status:** 0.9.
|
|
18
|
+
**Status:** 0.9.23 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
|
|
19
19
|
|
|
20
20
|
## How it works
|
|
21
21
|
|
|
@@ -122,8 +122,14 @@ Redirects) and Maintenance (Setup audit, Startup).
|
|
|
122
122
|
`claude mcp remove playwright -s user` / `codex mcp remove playwright`.
|
|
123
123
|
|
|
124
124
|
The hook behaviors:
|
|
125
|
-
- **Redirect** (default): the
|
|
126
|
-
|
|
125
|
+
- **Redirect** (default): the call is not denied. Smart Tool answers it and the answer comes back as the output
|
|
126
|
+
of the call the agent made: `smart_search` for a search, `web_fetch` for WebFetch, `web_search` for WebSearch
|
|
127
|
+
(without `allowed_domains`/`blocked_domains`). In Claude Code a PostToolUse entry swaps the output (WebFetch and
|
|
128
|
+
WebSearch still run natively, their result is replaced); a shell search whose answer is ready within 10 s runs as a
|
|
129
|
+
no-op instead, so a grep with no match does not end in exit 1. In Codex the shell search is rewritten to print the
|
|
130
|
+
answer from a file. When the Smart Tool call fails or is late, the native call and its output stay as they are; a
|
|
131
|
+
deny, with the exact call to make, is left only for a search whose query cannot be read from the command. The
|
|
132
|
+
decision is a fixed rule, under a millisecond and the same every time: a content search over a project folder (`grep -r`,
|
|
127
133
|
`rg`, `git grep`, the Grep tool on a folder) or over more than 20 listed files goes to `smart_search`, in Bash and
|
|
128
134
|
PowerShell alike and also when it is wrapped (`bash -c`, `powershell -Command`, `timeout`, a subshell), looped
|
|
129
135
|
(`for f in src/*.py`), fed by a listing (`find -exec grep`, `xargs`, `Get-ChildItem | Select-String`) or written
|
|
@@ -131,7 +137,7 @@ Redirects) and Maintenance (Setup audit, Startup).
|
|
|
131
137
|
listings, Glob and commands that change files run. The setup screen counts the redirects and what each session
|
|
132
138
|
did next. With **Block patterns proposed by a
|
|
133
139
|
model** on (default), the scope model reviews, in the background, bulk reads the rule could not measure and
|
|
134
|
-
proposes regex patterns, each checked against the logged calls; a pattern
|
|
140
|
+
proposes regex patterns, each checked against the logged calls; a pattern redirects only after you accept it.
|
|
135
141
|
- **Advise only**: the native tool runs; the agent receives the same advice next to the result and decides.
|
|
136
142
|
- **Off**: no routing.
|
|
137
143
|
|
package/client_hooks.py
CHANGED
|
@@ -15,12 +15,17 @@ AUDIT_MARKER = "session_audit.py"
|
|
|
15
15
|
AUDIT_MATCHER = "startup|resume|clear"
|
|
16
16
|
AUDIT_TIMEOUT_S = 10
|
|
17
17
|
HTTP_PATH = "/hooks/pretool"
|
|
18
|
-
|
|
18
|
+
POST_PATH = "/hooks/posttool"
|
|
19
|
+
# A redirected shell search waits for its answer in the PreToolUse (smart_tool_daemon.HOOK_SHELL_BUDGET_S).
|
|
20
|
+
HTTP_TIMEOUT_S = 15
|
|
21
|
+
# The PostToolUse waits for the Smart Tool answer of a redirected call (web_fetch reads a whole page).
|
|
22
|
+
POST_TIMEOUT_S = 60
|
|
19
23
|
HOME = os.path.expanduser("~")
|
|
20
24
|
PREVIEW_TTL_S = 600
|
|
21
25
|
CLIENTS = {
|
|
22
26
|
"claude": {"label": "Claude Code", "names": {"claude-code"}, "files": [os.path.join(endpoint_sync.claude_dir(), "settings.json")],
|
|
23
|
-
"docs": "https://code.claude.com/docs/en/hooks", "event": "PreToolUse", "matcher": "Grep|Glob|Read|Bash|PowerShell|WebSearch|WebFetch|Edit|MultiEdit|Write|mcp__smart-tool__smart_search", "http": True
|
|
27
|
+
"docs": "https://code.claude.com/docs/en/hooks", "event": "PreToolUse", "matcher": "Grep|Glob|Read|Bash|PowerShell|WebSearch|WebFetch|Edit|MultiEdit|Write|mcp__smart-tool__smart_search", "http": True,
|
|
28
|
+
"post_matcher": "Grep|Bash|PowerShell|WebFetch|WebSearch"},
|
|
24
29
|
"codex": {"label": "Codex", "prefix": "codex", "files": [os.path.join(endpoint_sync.codex_dir(), "hooks.json"), os.path.join(endpoint_sync.codex_dir(), "config.toml")],
|
|
25
30
|
"docs": "https://learn.chatgpt.com/docs/hooks", "event": "PreToolUse", "matcher": "Bash|apply_patch",
|
|
26
31
|
"after_install": "Codex only runs new hooks after the user approves the definition in /hooks."},
|
|
@@ -39,7 +44,18 @@ def identify(client_info):
|
|
|
39
44
|
|
|
40
45
|
|
|
41
46
|
def _our_hook(hook):
|
|
42
|
-
return isinstance(hook, dict) and (MARKER in str(hook.get("command", "")) or
|
|
47
|
+
return isinstance(hook, dict) and (MARKER in str(hook.get("command", "")) or
|
|
48
|
+
any(path in str(hook.get("url", "")) for path in (HTTP_PATH, POST_PATH)))
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
def _entries(client):
|
|
52
|
+
"""{event: (matcher, handler)} of our hook entries for the client: the PreToolUse, plus the PostToolUse that returns
|
|
53
|
+
the Smart Tool answer of a redirected call in Claude Code."""
|
|
54
|
+
spec = CLIENTS[client]
|
|
55
|
+
entries = {spec["event"]: (spec["matcher"], _handler(client))}
|
|
56
|
+
if spec.get("post_matcher"):
|
|
57
|
+
entries["PostToolUse"] = (spec["post_matcher"], _handler(client, POST_PATH, POST_TIMEOUT_S))
|
|
58
|
+
return entries
|
|
43
59
|
|
|
44
60
|
|
|
45
61
|
def _ours(group):
|
|
@@ -79,30 +95,37 @@ def inspect(client):
|
|
|
79
95
|
return {"state": "current", "changes": []}
|
|
80
96
|
data = json.loads(text)
|
|
81
97
|
hooks = data.get("hooks") if isinstance(data, dict) else None
|
|
82
|
-
groups = hooks.get(spec["event"]) if isinstance(hooks, dict) else None
|
|
83
|
-
ours = [group for group in groups if _ours(group)] if isinstance(groups, list) else []
|
|
84
|
-
if not ours:
|
|
85
|
-
return {"state": "outdated", "changes": [f"Smart Tool's entry is not under {spec['event']}."]}
|
|
86
|
-
expected = _handler(client)
|
|
87
|
-
if any(group.get("matcher") == spec["matcher"] and expected in group["hooks"] for group in ours):
|
|
88
|
-
return {"state": "current", "changes": []}
|
|
89
|
-
group = ours[0]
|
|
90
98
|
changes = []
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
found = next(hook for hook in group["hooks"] if _our_hook(hook))
|
|
96
|
-
if _target(found) != _target(expected):
|
|
97
|
-
changes.append(f"It calls {_target(found)} instead of {_target(expected)}.")
|
|
98
|
-
elif found != expected:
|
|
99
|
-
changes.append("Its timeout or type is older than the current one.")
|
|
100
|
-
if not changes:
|
|
101
|
-
changes.append("Its tool list is ordered or written differently from the current one.")
|
|
102
|
-
return {"state": "outdated", "changes": changes}
|
|
99
|
+
for event, (matcher, expected) in _entries(client).items():
|
|
100
|
+
groups = hooks.get(event) if isinstance(hooks, dict) else None
|
|
101
|
+
changes += _entry_changes(event, matcher, expected, groups)
|
|
102
|
+
return {"state": "outdated" if changes else "current", "changes": changes}
|
|
103
103
|
return {"state": "missing", "changes": []}
|
|
104
104
|
|
|
105
105
|
|
|
106
|
+
def _entry_changes(event, matcher, expected, groups):
|
|
107
|
+
"""Sentences saying what an update rewrites in our entry under one event; empty when it is current."""
|
|
108
|
+
ours = [group for group in groups if _ours(group)] if isinstance(groups, list) else []
|
|
109
|
+
if not ours:
|
|
110
|
+
return [f"Smart Tool's entry is not under {event}."]
|
|
111
|
+
if any(group.get("matcher") == matcher and expected in group["hooks"] for group in ours):
|
|
112
|
+
return []
|
|
113
|
+
group = ours[0]
|
|
114
|
+
changes = []
|
|
115
|
+
covered = str(group.get("matcher") or "").split("|")
|
|
116
|
+
left_out = [tool for tool in matcher.split("|") if tool not in covered]
|
|
117
|
+
if left_out:
|
|
118
|
+
changes.append(f"Its {event} entry does not cover " + ", ".join(left_out) + ".")
|
|
119
|
+
found = next(hook for hook in group["hooks"] if _our_hook(hook))
|
|
120
|
+
if _target(found) != _target(expected):
|
|
121
|
+
changes.append(f"It calls {_target(found)} instead of {_target(expected)}.")
|
|
122
|
+
elif found != expected:
|
|
123
|
+
changes.append("Its timeout or type is older than the current one.")
|
|
124
|
+
if not changes:
|
|
125
|
+
changes.append("Its tool list is ordered or written differently from the current one.")
|
|
126
|
+
return changes
|
|
127
|
+
|
|
128
|
+
|
|
106
129
|
def installed(client):
|
|
107
130
|
"""Our hook is present and current; an older entry of ours counts as not installed."""
|
|
108
131
|
return inspect(client)["state"] == "current"
|
|
@@ -160,18 +183,20 @@ def _command(client):
|
|
|
160
183
|
return f"{_shell_safe(sys.executable)} {_shell_safe(HOOK_SCRIPT)} --client {client}"
|
|
161
184
|
|
|
162
185
|
|
|
163
|
-
def _handler(client):
|
|
186
|
+
def _handler(client, route=HTTP_PATH, timeout=HTTP_TIMEOUT_S):
|
|
164
187
|
"""Claude Code posts the payload straight to the warm daemon (`type: "http"`, no process per tool call); other
|
|
165
|
-
clients run the thin command that forwards to the
|
|
188
|
+
clients run the thin command that forwards to the PreToolUse route."""
|
|
166
189
|
if CLIENTS[client].get("http"):
|
|
167
190
|
registry = daemon_launcher.read_registry()
|
|
168
191
|
if not registry or not registry.get("url"):
|
|
169
192
|
raise ValueError("Smart Tool daemon has no registered URL; start the daemon before installing the hook.")
|
|
170
|
-
return {"type": "http", "url": f"{registry['url']}{
|
|
171
|
-
return {"type": "command", "command": _command(client), "timeout":
|
|
193
|
+
return {"type": "http", "url": f"{registry['url']}{route}?client={client}", "timeout": timeout}
|
|
194
|
+
return {"type": "command", "command": _command(client), "timeout": timeout}
|
|
172
195
|
|
|
173
196
|
|
|
174
197
|
def _plan(client_info):
|
|
198
|
+
"""(client, spec, path, data, changes): the client's config with our entries already taken out, and changes
|
|
199
|
+
{event: (groups, entry)}, the entry to append to each event's groups."""
|
|
175
200
|
client = identify(client_info)
|
|
176
201
|
if client is None:
|
|
177
202
|
raise ValueError(f"Unrecognized MCP client ({(client_info or {}).get('name') or 'no clientInfo'}); "
|
|
@@ -187,21 +212,26 @@ def _plan(client_info):
|
|
|
187
212
|
data = {}
|
|
188
213
|
if not isinstance(data, dict) or not isinstance(data.setdefault("hooks", {}), dict):
|
|
189
214
|
raise ValueError(f"{path} does not have the expected format (object with 'hooks'); nothing was changed.")
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
215
|
+
changes = {}
|
|
216
|
+
for event, (matcher, handler) in _entries(client).items():
|
|
217
|
+
groups = data["hooks"].setdefault(event, [])
|
|
218
|
+
if not isinstance(groups, list):
|
|
219
|
+
raise ValueError(f"{path}: hooks.{event} is not a list; nothing was changed.")
|
|
220
|
+
groups[:] = _without_ours(groups)
|
|
221
|
+
changes[event] = (groups, {"matcher": matcher, "hooks": [handler]})
|
|
222
|
+
return client, spec, path, data, changes
|
|
196
223
|
|
|
197
224
|
|
|
198
225
|
def preview(client_info):
|
|
199
|
-
|
|
226
|
+
"""The entries install would write for client_info's client ({event: entry} in `change`, None when current) and
|
|
227
|
+
the token `install` needs."""
|
|
228
|
+
client, spec, path, _data, changes = _plan(client_info)
|
|
200
229
|
if installed(client):
|
|
201
230
|
return {"client": client, "installed": True, "file": path, "change": None}
|
|
202
231
|
token = secrets.token_urlsafe(12)
|
|
203
232
|
_PREVIEWS[token] = {"client": client, "at": time.time()}
|
|
204
|
-
return {"client": client, "installed": False, "file": path,
|
|
233
|
+
return {"client": client, "installed": False, "file": path,
|
|
234
|
+
"change": {event: entry for event, (_groups, entry) in changes.items()},
|
|
205
235
|
"token": token, "expires_s": PREVIEW_TTL_S, "after_install": spec.get("after_install")}
|
|
206
236
|
|
|
207
237
|
|
|
@@ -209,12 +239,13 @@ def install(client_info, token, confirm):
|
|
|
209
239
|
"""Writes the redirect hook for client_info's client with a backup; needs the `preview` token of this session
|
|
210
240
|
(10 min) and confirm=True. Returns {client, installed, changed, file, backup}."""
|
|
211
241
|
planned = _PREVIEWS.pop(token or "", None)
|
|
212
|
-
client, spec, path, data,
|
|
242
|
+
client, spec, path, data, changes = _plan(client_info)
|
|
213
243
|
if confirm is not True or not planned or planned["client"] != client or time.time() - planned["at"] > PREVIEW_TTL_S:
|
|
214
244
|
raise ValueError("Installing requires the install_hook_preview token from this session (valid for 10 min) and confirm=true.")
|
|
215
245
|
if installed(client):
|
|
216
246
|
return {"client": client, "installed": True, "changed": False, "file": path}
|
|
217
|
-
groups.
|
|
247
|
+
for groups, entry in changes.values():
|
|
248
|
+
groups.append(entry)
|
|
218
249
|
backup = _write_checked(path, data, lambda: installed(client), "The hook")
|
|
219
250
|
return {"client": client, "installed": True, "changed": True, "file": path, "backup": backup,
|
|
220
251
|
"after_install": spec.get("after_install")}
|
package/code_graph.py
CHANGED
|
@@ -3,6 +3,7 @@ import ast
|
|
|
3
3
|
import collections
|
|
4
4
|
import copy
|
|
5
5
|
import gzip
|
|
6
|
+
import contextlib
|
|
6
7
|
import contextvars
|
|
7
8
|
import hashlib
|
|
8
9
|
import json
|
|
@@ -504,7 +505,32 @@ def working_tree_changes(root):
|
|
|
504
505
|
return changes
|
|
505
506
|
|
|
506
507
|
|
|
508
|
+
_PENDING_SCOPE=contextvars.ContextVar('pending_scope',default=None)
|
|
509
|
+
|
|
510
|
+
|
|
511
|
+
@contextlib.contextmanager
|
|
512
|
+
def pending_scope():
|
|
513
|
+
"""Inside the block, `pending_changes` of one (root, view) is computed once: an edit review asks for it from several
|
|
514
|
+
places (copy pool, impact graph) and each answer costs a git status, a git ls-files and a stat of every indexed
|
|
515
|
+
file. The memo ends with the block, so the next review reads the working tree again."""
|
|
516
|
+
token=_PENDING_SCOPE.set({})
|
|
517
|
+
try:
|
|
518
|
+
yield
|
|
519
|
+
finally:
|
|
520
|
+
_PENDING_SCOPE.reset(token)
|
|
521
|
+
|
|
522
|
+
|
|
507
523
|
def pending_changes(root,view_path):
|
|
524
|
+
"""`_pending_changes`, memoized while a `pending_scope` is open."""
|
|
525
|
+
memo=_PENDING_SCOPE.get()
|
|
526
|
+
if memo is None:
|
|
527
|
+
return _pending_changes(root,view_path)
|
|
528
|
+
if (root,view_path) not in memo:
|
|
529
|
+
memo[(root,view_path)]=_pending_changes(root,view_path)
|
|
530
|
+
return dict(memo[(root,view_path)])
|
|
531
|
+
|
|
532
|
+
|
|
533
|
+
def _pending_changes(root,view_path):
|
|
508
534
|
"""Files the indexed view does not reflect yet: working-tree changes, indexed files whose size or modification time
|
|
509
535
|
differs on disk, and files tracked by git inside the index scope that were never indexed (commits made after the
|
|
510
536
|
last indexing). Path -> 'deleted' or 'changed'."""
|
|
@@ -537,21 +563,22 @@ def pending_changes(root,view_path):
|
|
|
537
563
|
return changes
|
|
538
564
|
|
|
539
565
|
|
|
540
|
-
def build_current(root,view_id=None):
|
|
566
|
+
def build_current(root,view_id=None,shared=False):
|
|
541
567
|
"""The graph an agent should see while editing: the indexed view plus every file it does not reflect yet (see
|
|
542
568
|
pending_changes and build_overlay) when that view is the current one; a pinned or older view is returned as
|
|
543
|
-
indexed."""
|
|
569
|
+
indexed. shared=True hands back the cached graph without copying it: the caller must only read it."""
|
|
544
570
|
inventory=index_inventory.inspect(root,view_id)
|
|
545
571
|
selected=inventory.get('selected') or {}
|
|
546
572
|
changes=pending_changes(root,selected['path']) if selected.get('current') else {}
|
|
547
|
-
return build_overlay(root,changes,view_id) if changes else build(root,view_id)
|
|
573
|
+
return build_overlay(root,changes,view_id,shared) if changes else build(root,view_id)
|
|
548
574
|
|
|
549
575
|
|
|
550
|
-
def build_overlay(root,changes,view_id=None):
|
|
576
|
+
def build_overlay(root,changes,view_id=None,shared=False):
|
|
551
577
|
"""Graph of the indexed view with the given changed files read from the working tree instead of the snapshot
|
|
552
578
|
(changes: project-relative path -> 'deleted' or any other status), so a caller right after an edit sees new files,
|
|
553
579
|
new imports and current line numbers without reindexing or embeddings. A deleted file keeps its indexed copy so
|
|
554
|
-
the files that imported it stay linked to it. Cached per snapshot and file contents
|
|
580
|
+
the files that imported it stay linked to it. Cached per snapshot and file contents; shared=True returns a new top-level
|
|
581
|
+
dict over the cached lists instead of a deep copy (read-only for the caller)."""
|
|
555
582
|
inventory=index_inventory.inspect(root,view_id)
|
|
556
583
|
if not inventory['selected']:
|
|
557
584
|
return {**inventory,'symbols':[],'dependencies':[],'calls':[],'diagnostics':[],'counts':{},'files':[]}
|
|
@@ -586,6 +613,6 @@ def build_overlay(root,changes,view_id=None):
|
|
|
586
613
|
with _LOCK:
|
|
587
614
|
_CACHE[key]=cached
|
|
588
615
|
while len(_CACHE)>3:_CACHE.popitem(last=False)
|
|
589
|
-
data=copy.deepcopy(cached)
|
|
616
|
+
data=dict(cached) if shared else copy.deepcopy(cached)
|
|
590
617
|
data.update(selected=inventory['selected'],source='indexed_snapshot_with_working_tree',overlaid=[r for r,_h in overlaid])
|
|
591
618
|
return data
|
package/code_impact.py
CHANGED
|
@@ -80,7 +80,7 @@ def impact(root, symbol, view_id=None, depth=3, limit=30):
|
|
|
80
80
|
raise ValueError("Pass symbol: a function, method (Class.method) or class name.")
|
|
81
81
|
if type(depth) is not int or not 1 <= depth <= MAX_DEPTH:
|
|
82
82
|
raise ValueError(f"depth must be an integer from 1 to {MAX_DEPTH}.")
|
|
83
|
-
data = code_graph.build_current(root, view_id)
|
|
83
|
+
data = code_graph.build_current(root, view_id, shared=True)
|
|
84
84
|
found = _matches(data.get("symbols") or [], symbol)
|
|
85
85
|
diagnostics = [d.get("reason") for d in data.get("diagnostics") or [] if d.get("reason")]
|
|
86
86
|
if not found:
|
package/dossier.py
ADDED
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
"""Dossier mode of smart_search: a narrow keyword query over the notes a project keeps about its own intent
|
|
2
|
+
(docs/research, the backlog, CLAUDE.md and AGENTS.md), answered with ids and one short excerpt per hit, inside a time
|
|
3
|
+
budget. It reads straight from disk, uses no index and no model, and never raises on a slow or unreadable file: it
|
|
4
|
+
stops at the deadline and says so in `degraded`/`reasons`."""
|
|
5
|
+
import json
|
|
6
|
+
import os
|
|
7
|
+
import re
|
|
8
|
+
import time
|
|
9
|
+
|
|
10
|
+
DEFAULT_TIMEOUT_MS = 800
|
|
11
|
+
MAX_TIMEOUT_MS = 5000
|
|
12
|
+
DEFAULT_MAX_HITS = 5
|
|
13
|
+
MAX_HITS = 10
|
|
14
|
+
SNIPPET_CHARS = 160
|
|
15
|
+
TITLE_CHARS = 80
|
|
16
|
+
MAX_FILE_BYTES = 512 * 1024
|
|
17
|
+
BACKLOG_FILES = ("docs/BACKLOG.md", "BACKLOG.md")
|
|
18
|
+
INSTRUCTION_FILES = (("CLAUDE.md", "claude_md"), (".claude/CLAUDE.md", "claude_md"), ("AGENTS.md", "agents_md"))
|
|
19
|
+
_TERM = re.compile(r"[\w]{2,}", re.UNICODE)
|
|
20
|
+
_UNIT_START = re.compile(r"#{1,6}\s|[-*]\s")
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
def _terms(query):
|
|
24
|
+
"""Up to 24 distinct casefolded word terms of the query."""
|
|
25
|
+
return list(dict.fromkeys(term.casefold() for term in _TERM.findall(query or "")))[:24]
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
def _sources(root):
|
|
29
|
+
"""`[(rel_path, kind)]` of the notes to read, newest research first so a timeout drops the oldest ones."""
|
|
30
|
+
found = []
|
|
31
|
+
base = os.path.join(root, "docs", "research")
|
|
32
|
+
for folder, dirs, files in os.walk(base):
|
|
33
|
+
dirs.sort()
|
|
34
|
+
found += [os.path.relpath(os.path.join(folder, name), root).replace(os.sep, "/")
|
|
35
|
+
for name in files if name.lower().endswith(".md")]
|
|
36
|
+
sources = [(rel, "research") for rel in sorted(found, reverse=True)]
|
|
37
|
+
sources += [(rel, "backlog") for rel in BACKLOG_FILES if os.path.isfile(os.path.join(root, rel))]
|
|
38
|
+
sources += [(rel, kind) for rel, kind in INSTRUCTION_FILES if os.path.isfile(os.path.join(root, rel))]
|
|
39
|
+
return sources
|
|
40
|
+
|
|
41
|
+
|
|
42
|
+
def _units(lines):
|
|
43
|
+
"""`[(first_line_number, [lines])]`: a unit starts at a heading or a bullet at column 0 and runs to the next one."""
|
|
44
|
+
units, current = [], None
|
|
45
|
+
for number, line in enumerate(lines, 1):
|
|
46
|
+
if _UNIT_START.match(line) or current is None:
|
|
47
|
+
current = (number, [])
|
|
48
|
+
units.append(current)
|
|
49
|
+
current[1].append(line)
|
|
50
|
+
return units
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
def _clean(text):
|
|
54
|
+
return re.sub(r"\s+", " ", re.sub(r"[#*`>]+", "", text)).strip()
|
|
55
|
+
|
|
56
|
+
|
|
57
|
+
def _snippet(unit_lines, terms):
|
|
58
|
+
"""The text around the first term found in the unit body, `SNIPPET_CHARS` long at most."""
|
|
59
|
+
body = _clean(" ".join(unit_lines))
|
|
60
|
+
low = body.casefold()
|
|
61
|
+
at = min((low.find(term) for term in terms if term in low), default=0)
|
|
62
|
+
start = max(0, at - SNIPPET_CHARS // 4)
|
|
63
|
+
piece = body[start:start + SNIPPET_CHARS].strip()
|
|
64
|
+
return ("…" if start else "") + piece + ("…" if start + SNIPPET_CHARS < len(body) else "")
|
|
65
|
+
|
|
66
|
+
|
|
67
|
+
def _score(unit_lines, terms):
|
|
68
|
+
"""Distinct terms in the unit, a term in its first line (the title) counting double."""
|
|
69
|
+
low = " ".join(unit_lines).casefold()
|
|
70
|
+
title = unit_lines[0].casefold()
|
|
71
|
+
return sum((term in low) + (term in title) for term in terms)
|
|
72
|
+
|
|
73
|
+
|
|
74
|
+
def query(root, text, timeout_ms=DEFAULT_TIMEOUT_MS, max_hits=DEFAULT_MAX_HITS):
|
|
75
|
+
"""The dossier answer for `text` over the notes of `root`: `{"mode", "degraded", "reasons", "elapsed_ms",
|
|
76
|
+
"scanned", "hits": [{"id", "source", "title", "snippet"}]}`, best hit first. `id` is `path:line` of the unit's
|
|
77
|
+
first line. `degraded` is true when the deadline cut the scan or a file could not be read; `hits` then holds what
|
|
78
|
+
was found until then."""
|
|
79
|
+
started = time.monotonic()
|
|
80
|
+
deadline = started + timeout_ms / 1000
|
|
81
|
+
terms = _terms(text)
|
|
82
|
+
reasons, scanned, unreadable, scored = [], 0, [], []
|
|
83
|
+
for rel, kind in _sources(root):
|
|
84
|
+
if time.monotonic() > deadline:
|
|
85
|
+
reasons.append(f"timeout after {timeout_ms} ms")
|
|
86
|
+
break
|
|
87
|
+
scanned += 1
|
|
88
|
+
path = os.path.join(root, rel)
|
|
89
|
+
try:
|
|
90
|
+
if os.path.getsize(path) > MAX_FILE_BYTES:
|
|
91
|
+
unreadable.append(f"{rel}: over {MAX_FILE_BYTES} bytes")
|
|
92
|
+
continue
|
|
93
|
+
with open(path, encoding="utf-8", errors="replace") as handle:
|
|
94
|
+
lines = handle.read().splitlines()
|
|
95
|
+
except OSError as exc:
|
|
96
|
+
unreadable.append(f"{rel}: {type(exc).__name__}")
|
|
97
|
+
continue
|
|
98
|
+
for number, unit in _units(lines):
|
|
99
|
+
score = _score(unit, terms)
|
|
100
|
+
if score:
|
|
101
|
+
scored.append((score, rel, kind, number, unit))
|
|
102
|
+
if unreadable:
|
|
103
|
+
reasons.append("not read: " + "; ".join(unreadable[:5]))
|
|
104
|
+
scored.sort(key=lambda item: -item[0])
|
|
105
|
+
hits = [{"id": f"{rel}:{number}", "source": kind, "title": _clean(unit[0])[:TITLE_CHARS],
|
|
106
|
+
"snippet": _snippet(unit, terms)} for _score_value, rel, kind, number, unit in scored[:max_hits]]
|
|
107
|
+
return {"mode": "dossier", "degraded": bool(reasons), "reasons": reasons,
|
|
108
|
+
"elapsed_ms": round((time.monotonic() - started) * 1000), "scanned": scanned, "hits": hits}
|
|
109
|
+
|
|
110
|
+
|
|
111
|
+
def answer(arguments):
|
|
112
|
+
"""The text smart_search returns for `mode=dossier`: the compact JSON of `query`. Raises ValueError for a missing
|
|
113
|
+
folder or text, or a `timeout_ms`/`max_hits` out of range, so the caller sees the error instead of an empty dossier."""
|
|
114
|
+
root = arguments.get("project_root")
|
|
115
|
+
if not isinstance(root, str) or not os.path.isdir(root):
|
|
116
|
+
raise ValueError("smart_search mode=dossier requires 'project_root' pointing to an existing folder")
|
|
117
|
+
text = next((arguments[key] for key in ("query_identifiers", "query_comments", "doc_query")
|
|
118
|
+
if isinstance(arguments.get(key), str) and arguments[key].strip()), "")
|
|
119
|
+
if not _terms(text):
|
|
120
|
+
raise ValueError("smart_search mode=dossier requires 'query_identifiers' (or 'doc_query') with at least one word.")
|
|
121
|
+
timeout_ms = _bounded(arguments, "timeout_ms", DEFAULT_TIMEOUT_MS, 1, MAX_TIMEOUT_MS)
|
|
122
|
+
max_hits = _bounded(arguments, "max_hits", DEFAULT_MAX_HITS, 1, MAX_HITS)
|
|
123
|
+
return json.dumps(query(root, text, timeout_ms, max_hits), ensure_ascii=False, separators=(",", ":"))
|
|
124
|
+
|
|
125
|
+
|
|
126
|
+
def _bounded(arguments, name, default, low, high):
|
|
127
|
+
value = arguments.get(name, default)
|
|
128
|
+
if isinstance(value, bool) or not isinstance(value, int) or not low <= value <= high:
|
|
129
|
+
raise ValueError(f"{name} must be an integer from {low} to {high}.")
|
|
130
|
+
return value
|
package/duplicates.py
CHANGED
|
@@ -199,10 +199,15 @@ def _entry(path, name, start, end, file_lines, ext):
|
|
|
199
199
|
"hash": hashlib.sha1(normalized.encode("utf-8")).hexdigest(), **_features(body, ext, short)}
|
|
200
200
|
|
|
201
201
|
|
|
202
|
+
_WORKING_TREE_FUNCTIONS = {}
|
|
203
|
+
_WORKING_TREE_FUNCTIONS_MAX = 2000
|
|
204
|
+
|
|
205
|
+
|
|
202
206
|
def _working_tree_functions(root, changes, profile, include_tests=False):
|
|
203
207
|
"""Functions of the files the index does not reflect yet, read from disk (production code, plus tests when
|
|
204
208
|
include_tests), with the same identity as indexed ones, so a copy written earlier in the session is caught and
|
|
205
|
-
a function removed since the last indexing is not reported.
|
|
209
|
+
a function removed since the last indexing is not reported. The functions of a file are remembered per
|
|
210
|
+
(modification time, size), so only files written since the last call are read and parsed again."""
|
|
206
211
|
found = []
|
|
207
212
|
kinds = ("code", "test") if include_tests else ("code",)
|
|
208
213
|
for rel, status in changes.items():
|
|
@@ -211,12 +216,20 @@ def _working_tree_functions(root, changes, profile, include_tests=False):
|
|
|
211
216
|
if status == "deleted" or ext not in _LANG or kind not in kinds:
|
|
212
217
|
continue
|
|
213
218
|
try:
|
|
219
|
+
stat = os.stat(os.path.join(root, rel))
|
|
220
|
+
key = (root, rel, stat.st_mtime_ns, stat.st_size, kind)
|
|
221
|
+
with _LOCK:
|
|
222
|
+
cached = _WORKING_TREE_FUNCTIONS.get(key)
|
|
223
|
+
if cached is not None:
|
|
224
|
+
found += cached
|
|
225
|
+
continue
|
|
214
226
|
with open(os.path.join(root, rel), encoding="utf-8") as stream:
|
|
215
227
|
source = stream.read()
|
|
216
228
|
except (OSError, UnicodeDecodeError):
|
|
217
229
|
continue
|
|
218
230
|
if _GENERATED.search(source[:4000]):
|
|
219
231
|
continue
|
|
232
|
+
before = len(found)
|
|
220
233
|
lines = source.splitlines()
|
|
221
234
|
functions = doc_check.functions(rel, source)
|
|
222
235
|
occurrences = {}
|
|
@@ -230,6 +243,10 @@ def _working_tree_functions(root, changes, profile, include_tests=False):
|
|
|
230
243
|
occurrence = occurrences[entry["hash"]] = occurrences.get(entry["hash"], 0) + 1
|
|
231
244
|
found.append({**entry, "kind": kind, "fp": hashlib.sha1(
|
|
232
245
|
f"{rel}\0{entry['hash']}\0{occurrence}".encode("utf-8")).hexdigest()})
|
|
246
|
+
with _LOCK:
|
|
247
|
+
if len(_WORKING_TREE_FUNCTIONS) >= _WORKING_TREE_FUNCTIONS_MAX:
|
|
248
|
+
_WORKING_TREE_FUNCTIONS.pop(next(iter(_WORKING_TREE_FUNCTIONS)))
|
|
249
|
+
_WORKING_TREE_FUNCTIONS[key] = found[before:]
|
|
233
250
|
return found
|
|
234
251
|
|
|
235
252
|
|
package/endpoint_sync.py
CHANGED
|
@@ -15,6 +15,7 @@ import urllib.parse
|
|
|
15
15
|
|
|
16
16
|
SERVER = "smart-tool"
|
|
17
17
|
HOOK_PATH = "/hooks/pretool"
|
|
18
|
+
POST_HOOK_PATH = "/hooks/posttool"
|
|
18
19
|
_CODEX_URL_RE = re.compile(r'(\[mcp_servers\.smart-tool\][^\[]*?\burl\s*=\s*")http://127\.0\.0\.1:\d+(/mcp")')
|
|
19
20
|
|
|
20
21
|
|
|
@@ -58,14 +59,17 @@ def _claude_mcp(text, base):
|
|
|
58
59
|
|
|
59
60
|
|
|
60
61
|
def _claude_hook(text, base):
|
|
62
|
+
"""settings.json text with our PreToolUse and PostToolUse hook URLs moved to base, or None when none changed."""
|
|
61
63
|
data = json.loads(text)
|
|
62
64
|
changed = False
|
|
63
|
-
|
|
64
|
-
for
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
65
|
+
hooks = (data.get("hooks") or {}) if isinstance(data, dict) else {}
|
|
66
|
+
for event, path in (("PreToolUse", HOOK_PATH), ("PostToolUse", POST_HOOK_PATH)):
|
|
67
|
+
groups = (hooks.get(event) or []) if isinstance(hooks, dict) else []
|
|
68
|
+
for group in groups if isinstance(groups, list) else []:
|
|
69
|
+
for hook in (group.get("hooks") or []) if isinstance(group, dict) else []:
|
|
70
|
+
if isinstance(hook, dict) and path in str(hook.get("url", "")):
|
|
71
|
+
new = _rebase(hook["url"], base)
|
|
72
|
+
changed, hook["url"] = changed or new != hook["url"], new
|
|
69
73
|
return json.dumps(data, indent=2, ensure_ascii=False) if changed else None
|
|
70
74
|
|
|
71
75
|
|