@allansantos-dev/smart-tool 0.9.12 → 0.9.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +33 -0
- package/README.md +15 -1
- package/daemon_launcher.py +15 -0
- package/install.py +34 -7
- package/package.json +1 -1
- package/setup_audit.py +828 -0
- package/setup_ui.py +154 -0
- package/smart_tool_daemon.py +15 -2
- package/version.py +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,38 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.9.14 - beta
|
|
4
|
+
|
|
5
|
+
- The setup audit runs on its own: the daemon audits every registered project two minutes after it starts and then
|
|
6
|
+
every 30 minutes, one at a time. MCP servers several projects share are contacted once and a successful result is
|
|
7
|
+
reused for 6 hours (a failure never is, so a server fixed meanwhile shows at once; Run now always contacts them
|
|
8
|
+
again). A project's `.mcp.json` servers are only counted and started when Claude Code approved them
|
|
9
|
+
(`enableAllProjectMcpServers`, `enabledMcpjsonServers`, `disabledMcpjsonServers`): registering a cloned repository
|
|
10
|
+
for search never runs the commands it declares. The summary per project is in
|
|
11
|
+
`~/.smart-tool/data/setup-audit/index.json`; projects no longer registered are dropped from it.
|
|
12
|
+
- **Setup audit** panel on the setup screen: pick a registered project (the one with the most errors opens first) and
|
|
13
|
+
see its last report, scheduled or on request: findings by severity with their documentation link, what is new,
|
|
14
|
+
what was resolved, the tokens each MCP server's tools cost and what was not verified; Run now audits it again.
|
|
15
|
+
Everything taken from config files is shown as text.
|
|
16
|
+
|
|
17
|
+
## 0.9.13 - beta
|
|
18
|
+
|
|
19
|
+
- Setup audit: `project_manage` with `action=setup_audit` and a `project_root` (or `python setup_audit.py --project
|
|
20
|
+
<folder>`) reports what Claude Code and Codex actually load for that folder (settings layers, enabled plugins with
|
|
21
|
+
their hooks, MCP servers and skills, CLAUDE.md and AGENTS.md, Codex `config.toml` and `hooks.json`) and where they
|
|
22
|
+
collide: several PreToolUse hooks deciding on the same tool in parallel, hook scripts that do not exist or discard
|
|
23
|
+
their errors, hooks calling an MCP server that is not configured, hook endpoints down, MCP servers that do not
|
|
24
|
+
start (each is started or called with `initialize` and `tools/list`, as a session would) or offer the same tools
|
|
25
|
+
twice (a gateway re-exposing a server also registered directly), skills sharing a name or over the description
|
|
26
|
+
budget, an AGENTS.md only Codex reads, broken `@imports` and a Codex instruction chain over 32 KiB. Each finding
|
|
27
|
+
links the documentation it relies on; nothing is edited; env and header values never enter the report; what could
|
|
28
|
+
not be checked is listed as not verified, and a second run shows what is new and what was resolved. On the
|
|
29
|
+
author's machine the first run found four hooks deciding on every Bash call in parallel and a gateway offering six
|
|
30
|
+
tools twice.
|
|
31
|
+
- Updating while agents work no longer fails with "Another Smart Tool daemon is still active": a Codex hook running
|
|
32
|
+
during the update started the daemon again from the half-copied folder and held the port. The installer now marks
|
|
33
|
+
the update so hooks and the tray do not start the daemon until it ends, and closes again a daemon a previous
|
|
34
|
+
version's hook brought back meanwhile.
|
|
35
|
+
|
|
3
36
|
## 0.9.12 - beta
|
|
4
37
|
|
|
5
38
|
- Security: `web_fetch` rendering with Moli (0.9.9 to 0.9.11) followed redirects and page requests to private
|
package/README.md
CHANGED
|
@@ -15,7 +15,7 @@ registered as the `playwright` server where this machine lets a browser be drive
|
|
|
15
15
|
An optional hook routes the agent's native Grep/Glob/Read/Bash/WebSearch/WebFetch calls to these tools when they
|
|
16
16
|
are cheaper, or just tells the agent what Smart Tool would do.
|
|
17
17
|
|
|
18
|
-
**Status:** 0.9.
|
|
18
|
+
**Status:** 0.9.14 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
|
|
19
19
|
|
|
20
20
|
## How it works
|
|
21
21
|
|
|
@@ -151,6 +151,20 @@ Open the setup screen from the tray icon ("Open settings") or at `http://127.0.0
|
|
|
151
151
|
likeliest failures first, and the command to run them. Changed files are read from disk, so new files and imports
|
|
152
152
|
count before the project is reindexed. `run_all` is true, with
|
|
153
153
|
the reason, when a configuration or lockfile changed or a changed code file is outside the import graph.
|
|
154
|
+
|
|
155
|
+
`project_manage` with `action=setup_audit` and a `project_root` (or `python setup_audit.py --project <folder>`)
|
|
156
|
+
reports what Claude Code and Codex actually load for that folder (settings layers, enabled plugins, hooks, MCP
|
|
157
|
+
servers, skills, CLAUDE.md and AGENTS.md) and where they collide: several hooks deciding on the same tool in
|
|
158
|
+
parallel, hook scripts that do not exist or hide their errors, hooks calling an MCP server that is not configured,
|
|
159
|
+
MCP servers that do not start or offer the same tools twice (a gateway re-exposing a server also registered
|
|
160
|
+
directly), skills sharing a name or over the description budget, an AGENTS.md only Codex reads, broken `@imports`
|
|
161
|
+
and a Codex instruction chain over 32 KiB. Each finding links the documentation it relies on; nothing is edited,
|
|
162
|
+
env and header values never enter the report, and what could not be checked is listed as not verified. A second
|
|
163
|
+
run shows what is new and what was resolved. The daemon also runs it on its own for every registered project,
|
|
164
|
+
two minutes after it starts and then every 30 minutes (MCP servers shared by several projects are contacted once
|
|
165
|
+
and their result reused for 6 hours; Run now always contacts them again), and the setup screen's **Setup audit**
|
|
166
|
+
panel shows each project's last report: findings by severity, what is new, what was resolved, the tokens each MCP
|
|
167
|
+
server's tools cost and what was not verified.
|
|
154
168
|
4. **Local models** (optional): if the mcp-memory embedding sidecar is installed on the machine, its local embedder
|
|
155
169
|
can serve as a fallback (or preferred) embedding model.
|
|
156
170
|
5. **Web search providers** work without keys on their free public tiers; add your own key for more volume. Set your
|
package/daemon_launcher.py
CHANGED
|
@@ -23,6 +23,8 @@ RUN_DIR = paths.DATA_DIR
|
|
|
23
23
|
REGISTRY_PATH = os.path.join(RUN_DIR, "daemon.json")
|
|
24
24
|
LOCK_PATH = os.path.join(RUN_DIR, "daemon.lock")
|
|
25
25
|
LOCK_STALE_SECONDS = 30
|
|
26
|
+
UPDATE_MARKER = os.path.join(RUN_DIR, "updating")
|
|
27
|
+
UPDATE_STALE_SECONDS = 900
|
|
26
28
|
DEFAULT_PORT = int(os.environ.get("SMART_TOOL_PORT", "8765"))
|
|
27
29
|
DAEMON_SCRIPT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "smart_tool_daemon.py")
|
|
28
30
|
|
|
@@ -106,11 +108,22 @@ def _spawn_daemon():
|
|
|
106
108
|
)
|
|
107
109
|
|
|
108
110
|
|
|
111
|
+
def updating():
|
|
112
|
+
"""Whether the installer is replacing the files (its marker, younger than UPDATE_STALE_SECONDS): a hook or the
|
|
113
|
+
tray starting the daemon then would run half-copied code and hold the port the installer waits for."""
|
|
114
|
+
try:
|
|
115
|
+
return time.time() - os.path.getmtime(UPDATE_MARKER) < UPDATE_STALE_SECONDS
|
|
116
|
+
except OSError:
|
|
117
|
+
return False
|
|
118
|
+
|
|
119
|
+
|
|
109
120
|
def ensure_daemon_running(wait_seconds=10):
|
|
110
121
|
"""Retorna a URL base do daemon (sem `/mcp`), subindo o processo se necessário."""
|
|
111
122
|
info = read_registry()
|
|
112
123
|
if info and health(info["url"]):
|
|
113
124
|
return info["url"]
|
|
125
|
+
if updating():
|
|
126
|
+
raise RuntimeError("Smart Tool is being updated; the daemon starts again when the update finishes.")
|
|
114
127
|
|
|
115
128
|
if not _acquire_lock():
|
|
116
129
|
url = _wait_for_daemon(wait_seconds)
|
|
@@ -145,6 +158,8 @@ def start_daemon_if_down():
|
|
|
145
158
|
processo falha no bind exclusivo da porta e morre sozinho."""
|
|
146
159
|
if daemon_url_if_up():
|
|
147
160
|
return True
|
|
161
|
+
if updating():
|
|
162
|
+
return False
|
|
148
163
|
if _acquire_lock():
|
|
149
164
|
# O lock NÃO é liberado aqui: ele é o freio de taxa. Expirando por idade
|
|
150
165
|
# (`LOCK_STALE_SECONDS`), um daemon que não consegue subir — porta tomada, import
|
package/install.py
CHANGED
|
@@ -93,9 +93,25 @@ def stop_processes(target):
|
|
|
93
93
|
time.sleep(0.5)
|
|
94
94
|
|
|
95
95
|
|
|
96
|
-
|
|
96
|
+
UPDATE_MARKER = Path(paths.DATA_DIR) / "updating"
|
|
97
|
+
|
|
98
|
+
|
|
99
|
+
def mark_updating(on):
|
|
100
|
+
"""While on, hooks and the tray do not start the daemon (daemon_launcher.updating): an agent working during the
|
|
101
|
+
update used to bring it back from the half-copied folder and hold the port."""
|
|
102
|
+
if on:
|
|
103
|
+
UPDATE_MARKER.parent.mkdir(parents=True, exist_ok=True)
|
|
104
|
+
UPDATE_MARKER.write_text(str(os.getpid()), encoding="utf-8")
|
|
105
|
+
else:
|
|
106
|
+
UPDATE_MARKER.unlink(missing_ok=True)
|
|
107
|
+
|
|
108
|
+
|
|
109
|
+
def wait_port_free(target):
|
|
110
|
+
"""Waits for the registered port to go quiet, closing again a daemon of this installation that a hook of the
|
|
111
|
+
previous version (which does not know the marker) brought back meanwhile."""
|
|
97
112
|
deadline = time.monotonic() + 15
|
|
98
113
|
while time.monotonic() < deadline and _daemon_up():
|
|
114
|
+
stop_processes(target)
|
|
99
115
|
time.sleep(0.5)
|
|
100
116
|
if _daemon_up():
|
|
101
117
|
raise RuntimeError("Another Smart Tool daemon is still active on the registered port; close it before updating.")
|
|
@@ -346,9 +362,21 @@ def main():
|
|
|
346
362
|
print(("Updating" if existed else "Installing") + f" Smart Tool at {target} ({len(files)} files from {SOURCE})")
|
|
347
363
|
target.mkdir(parents=True, exist_ok=True)
|
|
348
364
|
remove_other_installs(target)
|
|
365
|
+
mark_updating(True)
|
|
366
|
+
try:
|
|
367
|
+
_update(args, target, files, existed, python, pythonw)
|
|
368
|
+
finally:
|
|
369
|
+
mark_updating(False)
|
|
370
|
+
if not args.no_start:
|
|
371
|
+
enable_browser_control(target, python)
|
|
372
|
+
print("Smart Tool ready." if not args.no_start else "Files installed; daemon not restarted (--no-start).")
|
|
373
|
+
|
|
374
|
+
|
|
375
|
+
def _update(args, target, files, existed, python, pythonw):
|
|
376
|
+
"""Replaces the installed files, dependencies and daemon, restoring the previous version on any failure."""
|
|
349
377
|
stop_processes(target)
|
|
350
378
|
if not args.no_start:
|
|
351
|
-
wait_port_free()
|
|
379
|
+
wait_port_free(target)
|
|
352
380
|
backup = target.parent / f".smart-tool-backup-{time.strftime('%Y%m%d-%H%M%S')}"
|
|
353
381
|
backup.mkdir()
|
|
354
382
|
changed, removed, frozen, deps_touched = [], [], None, False
|
|
@@ -361,16 +389,18 @@ def main():
|
|
|
361
389
|
deps_touched = True
|
|
362
390
|
install_dependencies(target, python)
|
|
363
391
|
if not args.no_start:
|
|
392
|
+
mark_updating(False)
|
|
364
393
|
start(target, python, pythonw)
|
|
365
394
|
verify(target, python)
|
|
366
395
|
except BaseException as exc:
|
|
367
396
|
print(f"Failed: {exc}. Restoring the previous version.", file=sys.stderr)
|
|
368
397
|
problems = []
|
|
369
|
-
steps = [lambda:
|
|
398
|
+
steps = [lambda: mark_updating(True), lambda: stop_processes(target),
|
|
399
|
+
lambda: restore(target, backup, changed, removed)]
|
|
370
400
|
if deps_touched:
|
|
371
401
|
steps.append(lambda: restore_packages(python, frozen, backup))
|
|
372
402
|
if existed and not args.no_start:
|
|
373
|
-
steps
|
|
403
|
+
steps += [lambda: mark_updating(False), lambda: start(target, python, pythonw)]
|
|
374
404
|
for step in steps:
|
|
375
405
|
try:
|
|
376
406
|
step()
|
|
@@ -382,9 +412,6 @@ def main():
|
|
|
382
412
|
shutil.rmtree(backup, ignore_errors=True)
|
|
383
413
|
raise SystemExit(1)
|
|
384
414
|
shutil.rmtree(backup)
|
|
385
|
-
if not args.no_start:
|
|
386
|
-
enable_browser_control(target, python)
|
|
387
|
-
print("Smart Tool ready." if not args.no_start else "Files installed; daemon not restarted (--no-start).")
|
|
388
415
|
|
|
389
416
|
|
|
390
417
|
if __name__ == "__main__":
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@allansantos-dev/smart-tool",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.14",
|
|
4
4
|
"description": "Local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharper tools than their built-in search. Windows.",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"author": "Allan Santos",
|
package/setup_audit.py
ADDED
|
@@ -0,0 +1,828 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Setup audit: what Claude Code and Codex actually load for a project, and where the pieces collide.
|
|
3
|
+
|
|
4
|
+
Collects the effective set of both harnesses (settings layers, enabled plugins, hooks, MCP servers, skills and the
|
|
5
|
+
instruction chain) and runs deterministic checks, each tied to the documented behavior it relies on. Nothing is
|
|
6
|
+
edited. What could not be checked is reported as not verified, never as ok. MCP servers are started (stdio) or
|
|
7
|
+
called (HTTP) the way a session would, with `initialize` and `tools/list`, unless probe=False. Secrets stay out of
|
|
8
|
+
the report: env and header values are never read into it, only their names.
|
|
9
|
+
|
|
10
|
+
Usage: python setup_audit.py [--project DIR] [--no-probe] [--json]
|
|
11
|
+
"""
|
|
12
|
+
import argparse
|
|
13
|
+
import concurrent.futures
|
|
14
|
+
import hashlib
|
|
15
|
+
import json
|
|
16
|
+
import os
|
|
17
|
+
import queue
|
|
18
|
+
import re
|
|
19
|
+
import shutil
|
|
20
|
+
import socket
|
|
21
|
+
import subprocess
|
|
22
|
+
import threading
|
|
23
|
+
import time
|
|
24
|
+
import tomllib
|
|
25
|
+
import urllib.error
|
|
26
|
+
import urllib.parse
|
|
27
|
+
import urllib.request
|
|
28
|
+
|
|
29
|
+
import yaml
|
|
30
|
+
|
|
31
|
+
import atomic_io
|
|
32
|
+
import endpoint_sync
|
|
33
|
+
import paths
|
|
34
|
+
|
|
35
|
+
REPORT_DIR = os.path.join(paths.DATA_DIR, "setup-audit")
|
|
36
|
+
INDEX_NAME = "index.json"
|
|
37
|
+
PROBE_TIMEOUT_S = 15
|
|
38
|
+
SCHEDULE_S = 30 * 60
|
|
39
|
+
FIRST_RUN_DELAY_S = 120
|
|
40
|
+
PROBE_MAX_AGE_S = 6 * 3600
|
|
41
|
+
SKILL_DESCRIPTION_MAX = 1536
|
|
42
|
+
SKILL_BUDGET_CHARS = 8000
|
|
43
|
+
CODEX_PROJECT_DOC_MAX = 32 * 1024
|
|
44
|
+
CLAUDE_TOOLS = ("Bash", "Read", "Grep", "Glob", "LS", "Edit", "MultiEdit", "Write", "NotebookEdit", "WebFetch",
|
|
45
|
+
"WebSearch", "Task")
|
|
46
|
+
CODEX_TOOLS = ("Bash", "apply_patch")
|
|
47
|
+
DECIDING_EVENTS = ("PreToolUse",)
|
|
48
|
+
DOCS = {"claude_hooks": "https://code.claude.com/docs/en/hooks.md", "codex_hooks": "https://learn.chatgpt.com/docs/hooks.md",
|
|
49
|
+
"claude_skills": "https://code.claude.com/docs/en/skills.md",
|
|
50
|
+
"codex_skills": "https://learn.chatgpt.com/docs/build-skills.md",
|
|
51
|
+
"claude_memory": "https://code.claude.com/docs/en/memory.md",
|
|
52
|
+
"codex_agents_md": "https://developers.openai.com/codex/guides/agents-md",
|
|
53
|
+
"claude_mcp": "https://code.claude.com/docs/en/mcp.md", "mcp_lifecycle":
|
|
54
|
+
"https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle"}
|
|
55
|
+
_SCRIPT_RE = re.compile(r"""["']?([^\s"'|&;<>]+\.(?:js|cjs|mjs|py|ps1|sh|bat|cmd|exe))["']?""", re.IGNORECASE)
|
|
56
|
+
_SILENCED_RE = re.compile(r"\|\|\s*(?:true|exit\s+0)\b|2>\s*(?:/dev/null|nul)\b|;\s*exit\s+0\s*$", re.IGNORECASE)
|
|
57
|
+
_IMPORT_RE = re.compile(r"(?<![\w`])@((?:~|\.{1,2})?/?[\w.\-/\\]+\.\w+)")
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def _home(*parts):
|
|
61
|
+
"""Path under the user's home folder."""
|
|
62
|
+
return os.path.join(os.path.expanduser("~"), *parts)
|
|
63
|
+
|
|
64
|
+
|
|
65
|
+
def codex_dir():
|
|
66
|
+
"""Codex's home: CODEX_HOME when set, else ~/.codex."""
|
|
67
|
+
return os.environ.get("CODEX_HOME") or _home(".codex")
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
def _managed_settings():
|
|
71
|
+
"""Where an administrator's managed-settings.json can live on this OS (highest settings layer)."""
|
|
72
|
+
roots = (os.environ.get("ProgramFiles", r"C:\Program Files"), os.environ.get("ProgramData", r"C:\ProgramData"))
|
|
73
|
+
return [os.path.join(root, "ClaudeCode", "managed-settings.json") for root in roots] if os.name == "nt" else \
|
|
74
|
+
["/Library/Application Support/ClaudeCode/managed-settings.json", "/etc/claude-code/managed-settings.json"]
|
|
75
|
+
|
|
76
|
+
|
|
77
|
+
class _Snapshot:
|
|
78
|
+
"""Files read while collecting (path -> sha256) plus what could not be read."""
|
|
79
|
+
|
|
80
|
+
def __init__(self):
|
|
81
|
+
self.files, self.unverified = {}, []
|
|
82
|
+
|
|
83
|
+
def read(self, path):
|
|
84
|
+
try:
|
|
85
|
+
with open(path, "rb") as stream:
|
|
86
|
+
data = stream.read()
|
|
87
|
+
except FileNotFoundError:
|
|
88
|
+
return None
|
|
89
|
+
except OSError as exc:
|
|
90
|
+
self.unverified.append(f"{path}: {exc}")
|
|
91
|
+
return None
|
|
92
|
+
self.files[os.path.abspath(path)] = hashlib.sha256(data).hexdigest()
|
|
93
|
+
return data.decode("utf-8-sig", "replace")
|
|
94
|
+
|
|
95
|
+
def json(self, path):
|
|
96
|
+
text = self.read(path)
|
|
97
|
+
if text is None:
|
|
98
|
+
return None
|
|
99
|
+
try:
|
|
100
|
+
data = json.loads(text)
|
|
101
|
+
except ValueError as exc:
|
|
102
|
+
self.unverified.append(f"{path}: invalid JSON ({exc})")
|
|
103
|
+
return None
|
|
104
|
+
return data if isinstance(data, dict) else None
|
|
105
|
+
|
|
106
|
+
def toml(self, path):
|
|
107
|
+
text = self.read(path)
|
|
108
|
+
if text is None:
|
|
109
|
+
return None
|
|
110
|
+
try:
|
|
111
|
+
return tomllib.loads(text)
|
|
112
|
+
except tomllib.TOMLDecodeError as exc:
|
|
113
|
+
self.unverified.append(f"{path}: invalid TOML ({exc})")
|
|
114
|
+
return None
|
|
115
|
+
|
|
116
|
+
|
|
117
|
+
def _hooks_from(data, source, root=None):
|
|
118
|
+
"""Hook entries of a settings/hooks.json object: one dict per hook with its event, matcher and target."""
|
|
119
|
+
found = []
|
|
120
|
+
for event, groups in ((data or {}).get("hooks") or {}).items():
|
|
121
|
+
for group in groups if isinstance(groups, list) else []:
|
|
122
|
+
for hook in (group.get("hooks") or []) if isinstance(group, dict) else []:
|
|
123
|
+
if isinstance(hook, dict):
|
|
124
|
+
found.append({"event": event, "matcher": group.get("matcher") or "", "type": hook.get("type"),
|
|
125
|
+
"command": hook.get("command"), "url": hook.get("url"),
|
|
126
|
+
"server": hook.get("server"), "tool": hook.get("tool"), "source": source,
|
|
127
|
+
"root": root})
|
|
128
|
+
return found
|
|
129
|
+
|
|
130
|
+
|
|
131
|
+
_VAR_RE = re.compile(r"\$\{([A-Za-z_][A-Za-z0-9_]*)(?::-([^}]*))?\}")
|
|
132
|
+
|
|
133
|
+
|
|
134
|
+
def _expand_vars(value, extra):
|
|
135
|
+
"""${VAR} and ${VAR:-default} as Claude Code expands them in .mcp.json (extra: CLAUDE_PLUGIN_ROOT and such)."""
|
|
136
|
+
if not isinstance(value, str):
|
|
137
|
+
return value
|
|
138
|
+
return _VAR_RE.sub(lambda m: extra.get(m.group(1)) or os.environ.get(m.group(1)) or m.group(2) or m.group(0), value)
|
|
139
|
+
|
|
140
|
+
|
|
141
|
+
def _mcp_entries(servers, scope, source, prefix="", extra=None):
|
|
142
|
+
"""MCP server entries of one config (enabled only); env and header values kept in private fields. prefix names
|
|
143
|
+
a plugin's servers the way Claude Code does (plugin:<plugin>:<server>)."""
|
|
144
|
+
entries, extra = [], extra or {}
|
|
145
|
+
for name, spec in (servers or {}).items():
|
|
146
|
+
if not isinstance(spec, dict) or spec.get("enabled") is False:
|
|
147
|
+
continue
|
|
148
|
+
spec = {k: ([_expand_vars(a, extra) for a in v] if isinstance(v, list) else
|
|
149
|
+
{kk: _expand_vars(vv, extra) for kk, vv in v.items()} if isinstance(v, dict) else
|
|
150
|
+
_expand_vars(v, extra)) for k, v in spec.items()}
|
|
151
|
+
entries.append({"name": prefix + name, "scope": scope, "source": source, "url": spec.get("url"),
|
|
152
|
+
"transport": "http" if spec.get("url") else "stdio", "command": spec.get("command"),
|
|
153
|
+
"args": [str(a) for a in spec.get("args") or []],
|
|
154
|
+
"env_names": sorted((spec.get("env") or {}).keys()),
|
|
155
|
+
"_env": spec.get("env") or {}, "_headers": dict(spec.get("headers") or spec.get("http_headers")
|
|
156
|
+
or {}),
|
|
157
|
+
"_bearer_env": spec.get("bearer_token_env_var"),
|
|
158
|
+
"disabled_tools": list(spec.get("disabled_tools") or [])})
|
|
159
|
+
return entries
|
|
160
|
+
|
|
161
|
+
|
|
162
|
+
def _frontmatter(text):
|
|
163
|
+
"""YAML frontmatter of a SKILL.md as a dict ({} when absent or invalid)."""
|
|
164
|
+
match = re.match(r"^---\s*\n(.*?)\n---", text or "", re.S)
|
|
165
|
+
if not match:
|
|
166
|
+
return {}
|
|
167
|
+
try:
|
|
168
|
+
data = yaml.safe_load(match.group(1))
|
|
169
|
+
except yaml.YAMLError:
|
|
170
|
+
return {}
|
|
171
|
+
return data if isinstance(data, dict) else {}
|
|
172
|
+
|
|
173
|
+
|
|
174
|
+
def _skills(snapshot, folder, source, recursive=False, disabled=()):
|
|
175
|
+
"""Skills (name, description, path) under folder, skipping names turned off in skillOverrides."""
|
|
176
|
+
found = []
|
|
177
|
+
pattern = "**/SKILL.md" if recursive else "*/SKILL.md"
|
|
178
|
+
from pathlib import Path
|
|
179
|
+
for path in sorted(Path(folder).glob(pattern)) if os.path.isdir(folder) else []:
|
|
180
|
+
if path.parent.name in disabled:
|
|
181
|
+
continue
|
|
182
|
+
meta = _frontmatter(snapshot.read(str(path)))
|
|
183
|
+
found.append({"name": str(meta.get("name") or path.parent.name), "path": str(path), "source": source,
|
|
184
|
+
"description": " ".join(str(meta.get("description") or "").split())})
|
|
185
|
+
return found
|
|
186
|
+
|
|
187
|
+
|
|
188
|
+
def _ancestors(project):
|
|
189
|
+
"""Folders from the drive root down to project, in that order."""
|
|
190
|
+
chain, current = [], os.path.abspath(project)
|
|
191
|
+
while True:
|
|
192
|
+
chain.append(current)
|
|
193
|
+
parent = os.path.dirname(current)
|
|
194
|
+
if parent == current:
|
|
195
|
+
return list(reversed(chain))
|
|
196
|
+
current = parent
|
|
197
|
+
|
|
198
|
+
|
|
199
|
+
def _approvals(data, approval):
|
|
200
|
+
"""Folds one settings object's .mcp.json approvals (enableAllProjectMcpServers, enabledMcpjsonServers,
|
|
201
|
+
disabledMcpjsonServers) into approval."""
|
|
202
|
+
approval["all"] = approval["all"] or data.get("enableAllProjectMcpServers") is True
|
|
203
|
+
approval["on"].update(data.get("enabledMcpjsonServers") or [])
|
|
204
|
+
approval["off"].update(data.get("disabledMcpjsonServers") or [])
|
|
205
|
+
|
|
206
|
+
|
|
207
|
+
def collect_claude(project, snapshot):
|
|
208
|
+
"""Claude Code's effective set for project: settings layers, enabled plugins, hooks, MCP, skills, CLAUDE.md."""
|
|
209
|
+
layers = [(path, "managed") for path in _managed_settings()] + [
|
|
210
|
+
(os.path.join(endpoint_sync.claude_dir(), "settings.json"), "user"),
|
|
211
|
+
(os.path.join(project, ".claude", "settings.json"), "project"),
|
|
212
|
+
(os.path.join(project, ".claude", "settings.local.json"), "local")]
|
|
213
|
+
hooks, enabled, overrides, settings, approval = [], {}, {}, [], {"all": False, "on": set(), "off": set()}
|
|
214
|
+
for path, scope in layers:
|
|
215
|
+
data = snapshot.json(path)
|
|
216
|
+
if data is None:
|
|
217
|
+
continue
|
|
218
|
+
settings.append({"scope": scope, "path": path, "disable_all_hooks": bool(data.get("disableAllHooks"))})
|
|
219
|
+
hooks += _hooks_from(data, f"settings:{scope}")
|
|
220
|
+
enabled.update({k: v for k, v in (data.get("enabledPlugins") or {}).items()})
|
|
221
|
+
overrides.update(data.get("skillOverrides") or {})
|
|
222
|
+
_approvals(data, approval)
|
|
223
|
+
disabled_skills = {name for name, value in overrides.items() if value == "off"}
|
|
224
|
+
state = snapshot.json(endpoint_sync.claude_json()) or {}
|
|
225
|
+
mcp = _mcp_entries(state.get("mcpServers"), "user", endpoint_sync.claude_json())
|
|
226
|
+
project_state = (state.get("projects") or {}).get(os.path.abspath(project).replace("\\", "/")) or \
|
|
227
|
+
(state.get("projects") or {}).get(os.path.abspath(project)) or {}
|
|
228
|
+
mcp += _mcp_entries(project_state.get("mcpServers"), "local", endpoint_sync.claude_json())
|
|
229
|
+
_approvals(project_state, approval)
|
|
230
|
+
project_mcp = snapshot.json(os.path.join(project, ".mcp.json")) or {}
|
|
231
|
+
shared = _mcp_entries(project_mcp.get("mcpServers"), "project", os.path.join(project, ".mcp.json"))
|
|
232
|
+
unapproved = [e["name"] for e in shared
|
|
233
|
+
if e["name"] in approval["off"] or not (approval["all"] or e["name"] in approval["on"])]
|
|
234
|
+
mcp += [e for e in shared if e["name"] not in unapproved]
|
|
235
|
+
skills = _skills(snapshot, os.path.join(endpoint_sync.claude_dir(), "skills"), "user", disabled=disabled_skills)
|
|
236
|
+
skills += _skills(snapshot, os.path.join(project, ".claude", "skills"), "project", disabled=disabled_skills)
|
|
237
|
+
plugins = []
|
|
238
|
+
installed = snapshot.json(os.path.join(endpoint_sync.claude_dir(), "plugins", "installed_plugins.json")) or {}
|
|
239
|
+
for plugin_id, installs in (installed.get("plugins") or {}).items():
|
|
240
|
+
if not enabled.get(plugin_id):
|
|
241
|
+
continue
|
|
242
|
+
for install in installs if isinstance(installs, list) else []:
|
|
243
|
+
root = install.get("installPath")
|
|
244
|
+
if install.get("projectPath") and os.path.abspath(install["projectPath"]) != os.path.abspath(project):
|
|
245
|
+
continue
|
|
246
|
+
if not root or not os.path.isdir(root):
|
|
247
|
+
snapshot.unverified.append(f"plugin {plugin_id}: install path missing ({root})")
|
|
248
|
+
continue
|
|
249
|
+
name = plugin_id.split("@")[0]
|
|
250
|
+
plugins.append({"id": plugin_id, "path": root})
|
|
251
|
+
hooks += _hooks_from(snapshot.json(os.path.join(root, "hooks", "hooks.json")), f"plugin:{name}", root)
|
|
252
|
+
plugin_mcp = snapshot.json(os.path.join(root, ".mcp.json")) or {}
|
|
253
|
+
mcp += _mcp_entries(plugin_mcp.get("mcpServers", plugin_mcp), f"plugin:{name}",
|
|
254
|
+
os.path.join(root, ".mcp.json"), f"plugin:{name}:", {"CLAUDE_PLUGIN_ROOT": root})
|
|
255
|
+
skills += [{**s, "name": f"{name}:{s['name']}"} for s in
|
|
256
|
+
_skills(snapshot, os.path.join(root, "skills"), f"plugin:{name}")]
|
|
257
|
+
instructions = []
|
|
258
|
+
for path, kind in [(os.path.join(endpoint_sync.claude_dir(), "CLAUDE.md"), "user")] + [
|
|
259
|
+
(os.path.join(folder, name), "project") for folder in _ancestors(project)
|
|
260
|
+
for name in ("CLAUDE.md", "CLAUDE.local.md", os.path.join(".claude", "CLAUDE.md"))]:
|
|
261
|
+
text = snapshot.read(path)
|
|
262
|
+
if text is not None:
|
|
263
|
+
instructions.append({"path": path, "kind": kind, "bytes": len(text.encode("utf-8")), "_text": text})
|
|
264
|
+
return {"settings": settings, "plugins": plugins, "hooks": hooks, "mcp": mcp, "skills": skills,
|
|
265
|
+
"instructions": instructions, "unapproved_mcp": unapproved}
|
|
266
|
+
|
|
267
|
+
|
|
268
|
+
def collect_codex(project, snapshot):
|
|
269
|
+
"""Codex's effective set for project: config.toml (MCP, hooks), hooks.json, skills, AGENTS.md chain."""
|
|
270
|
+
config = snapshot.toml(os.path.join(codex_dir(), "config.toml")) or {}
|
|
271
|
+
mcp = _mcp_entries(config.get("mcp_servers"), "user", os.path.join(codex_dir(), "config.toml"))
|
|
272
|
+
hooks = _hooks_from(snapshot.json(os.path.join(codex_dir(), "hooks.json")), "hooks.json")
|
|
273
|
+
hooks += _hooks_from(config, "config.toml")
|
|
274
|
+
skills = _skills(snapshot, os.path.join(codex_dir(), "skills"), "user", recursive=True)
|
|
275
|
+
skills += _skills(snapshot, _home(".agents", "skills"), "user-agents")
|
|
276
|
+
skills += _skills(snapshot, os.path.join(project, ".agents", "skills"), "project")
|
|
277
|
+
instructions = []
|
|
278
|
+
for name in ("AGENTS.override.md", "AGENTS.md"):
|
|
279
|
+
text = snapshot.read(os.path.join(codex_dir(), name))
|
|
280
|
+
if text is not None:
|
|
281
|
+
instructions.append({"path": os.path.join(codex_dir(), name), "kind": "user",
|
|
282
|
+
"bytes": len(text.encode("utf-8")), "_text": text})
|
|
283
|
+
break
|
|
284
|
+
git_root = next((folder for folder in reversed(_ancestors(project)) if os.path.exists(os.path.join(folder, ".git"))),
|
|
285
|
+
os.path.abspath(project))
|
|
286
|
+
for folder in _ancestors(project):
|
|
287
|
+
if len(folder) < len(git_root):
|
|
288
|
+
continue
|
|
289
|
+
for name in ("AGENTS.override.md", "AGENTS.md"):
|
|
290
|
+
text = snapshot.read(os.path.join(folder, name))
|
|
291
|
+
if text is not None:
|
|
292
|
+
instructions.append({"path": os.path.join(folder, name), "kind": "project",
|
|
293
|
+
"bytes": len(text.encode("utf-8")), "_text": text})
|
|
294
|
+
break
|
|
295
|
+
return {"hooks": hooks, "mcp": mcp, "skills": skills, "instructions": instructions,
|
|
296
|
+
"plugins_unverified": bool(config.get("plugins"))}
|
|
297
|
+
|
|
298
|
+
|
|
299
|
+
def _finding(check, severity, harness, subject, detail, source, evidence=()):
|
|
300
|
+
"""One finding; its id is stable for the same check, harness and subject (used to diff runs)."""
|
|
301
|
+
key = hashlib.sha1(f"{check}\0{harness}\0{subject}".encode()).hexdigest()[:10]
|
|
302
|
+
return {"id": key, "check": check, "severity": severity, "harness": harness, "subject": subject,
|
|
303
|
+
"detail": detail, "source": source, "evidence": list(evidence)}
|
|
304
|
+
|
|
305
|
+
|
|
306
|
+
def _matches(matcher, tool):
|
|
307
|
+
"""Whether a hook matcher (empty or * for all, else a regex over the tool name) covers tool."""
|
|
308
|
+
if matcher in ("", "*"):
|
|
309
|
+
return True
|
|
310
|
+
try:
|
|
311
|
+
return re.fullmatch(matcher, tool) is not None
|
|
312
|
+
except re.error:
|
|
313
|
+
return matcher == tool
|
|
314
|
+
|
|
315
|
+
|
|
316
|
+
def check_hook_overlap(harness, hooks, tools):
|
|
317
|
+
"""Deciding hooks (PreToolUse) that run on the same tool: they run in parallel; a deny wins, and two rewrites
|
|
318
|
+
(updatedInput replaces the whole input) have no documented order."""
|
|
319
|
+
findings, groups = [], {}
|
|
320
|
+
for event in DECIDING_EVENTS:
|
|
321
|
+
for tool in tools:
|
|
322
|
+
matching = [h for h in hooks if h["event"] == event and _matches(h["matcher"], tool)]
|
|
323
|
+
if len(matching) > 1:
|
|
324
|
+
key = (event, tuple(sorted(f"{h['source']} {h['type']} {h['matcher'] or '*'}" for h in matching)))
|
|
325
|
+
groups.setdefault(key, []).append(tool)
|
|
326
|
+
doc = DOCS["claude_hooks"] if harness == "claude" else DOCS["codex_hooks"]
|
|
327
|
+
rule = ("all run in parallel; deny > defer > ask > allow, and two updatedInput rewrites have no documented order"
|
|
328
|
+
if harness == "claude" else "they run concurrently and any deny wins")
|
|
329
|
+
for (event, members), names in groups.items():
|
|
330
|
+
sources = sorted({m.split(" ")[0] for m in members})
|
|
331
|
+
findings.append(_finding("hook_overlap", "warn", harness, f"{event} on {', '.join(names)}",
|
|
332
|
+
f"{len(members)} hooks decide on the same call ({rule}): " + "; ".join(members)
|
|
333
|
+
+ (". From different sources: check they do not rewrite the same input."
|
|
334
|
+
if len(sources) > 1 else "."), doc))
|
|
335
|
+
return findings
|
|
336
|
+
|
|
337
|
+
|
|
338
|
+
def _expand(token, root):
|
|
339
|
+
"""A hook script path with ${CLAUDE_PLUGIN_ROOT}, ~ and environment variables expanded."""
|
|
340
|
+
token = token.replace("${CLAUDE_PLUGIN_ROOT}", root or "${CLAUDE_PLUGIN_ROOT}")
|
|
341
|
+
return os.path.expandvars(os.path.expanduser(token))
|
|
342
|
+
|
|
343
|
+
|
|
344
|
+
def check_hook_targets(harness, hooks, mcp_names, probe):
|
|
345
|
+
"""Hooks that cannot work as configured: missing script, silenced errors, unknown MCP server, endpoint down."""
|
|
346
|
+
findings = []
|
|
347
|
+
doc = DOCS["claude_hooks"] if harness == "claude" else DOCS["codex_hooks"]
|
|
348
|
+
for hook in hooks:
|
|
349
|
+
where = f"{hook['source']} {hook['event']} {hook['matcher'] or '*'}"
|
|
350
|
+
if hook["type"] == "command" and hook["command"]:
|
|
351
|
+
for token in _SCRIPT_RE.findall(hook["command"]):
|
|
352
|
+
path = _expand(token, hook.get("root"))
|
|
353
|
+
if "$" in path or "%" in path:
|
|
354
|
+
continue
|
|
355
|
+
if not os.path.isabs(path):
|
|
356
|
+
path = os.path.join(hook["root"], path) if hook.get("root") else None
|
|
357
|
+
if path and not os.path.exists(path):
|
|
358
|
+
findings.append(_finding("hook_missing_script", "error", harness, where,
|
|
359
|
+
f"The hook runs {token}, which does not exist.", doc, [hook["command"]]))
|
|
360
|
+
if _SILENCED_RE.search(hook["command"]):
|
|
361
|
+
findings.append(_finding("hook_silenced", "warn", harness, where,
|
|
362
|
+
"The hook discards its errors (|| true / 2>/dev/null): when it breaks, "
|
|
363
|
+
"nothing shows.", doc, [hook["command"]]))
|
|
364
|
+
elif hook["type"] == "mcp_tool" and hook.get("server") and hook["server"] not in mcp_names:
|
|
365
|
+
findings.append(_finding("hook_unknown_server", "error", harness, where,
|
|
366
|
+
f"The hook calls MCP server '{hook['server']}', which is not configured here.",
|
|
367
|
+
doc))
|
|
368
|
+
elif hook["type"] == "http" and hook.get("url") and probe:
|
|
369
|
+
problem = _reachable(hook["url"])
|
|
370
|
+
if problem:
|
|
371
|
+
findings.append(_finding("hook_endpoint_down", "error", harness, where,
|
|
372
|
+
f"The hook posts to {hook['url']}, which does not answer: {problem}", doc))
|
|
373
|
+
return findings
|
|
374
|
+
|
|
375
|
+
|
|
376
|
+
def _reachable(url):
|
|
377
|
+
"""None when a TCP connection to the URL's host and port opens, else the error."""
|
|
378
|
+
parsed = urllib.parse.urlsplit(url)
|
|
379
|
+
try:
|
|
380
|
+
with socket.create_connection((parsed.hostname, parsed.port or (443 if parsed.scheme == "https" else 80)),
|
|
381
|
+
timeout=3):
|
|
382
|
+
return None
|
|
383
|
+
except OSError as exc:
|
|
384
|
+
return str(exc)
|
|
385
|
+
|
|
386
|
+
|
|
387
|
+
def check_skills(harness, skills):
|
|
388
|
+
"""Skill name collisions, descriptions over Claude's per-skill cut, and the total over the list budget."""
|
|
389
|
+
findings, by_name = [], {}
|
|
390
|
+
doc = DOCS["claude_skills"] if harness == "claude" else DOCS["codex_skills"]
|
|
391
|
+
for skill in skills:
|
|
392
|
+
by_name.setdefault(skill["name"], []).append(skill)
|
|
393
|
+
for name, same in by_name.items():
|
|
394
|
+
if len(same) > 1:
|
|
395
|
+
findings.append(_finding("skill_name_collision", "warn", harness, name,
|
|
396
|
+
f"{len(same)} skills share the name '{name}': only one is offered.", doc,
|
|
397
|
+
[s["path"] for s in same]))
|
|
398
|
+
if harness == "claude":
|
|
399
|
+
for skill in skills:
|
|
400
|
+
if len(skill["description"]) > SKILL_DESCRIPTION_MAX:
|
|
401
|
+
findings.append(_finding("skill_description_long", "warn", harness, skill["name"],
|
|
402
|
+
f"Description of {len(skill['description'])} characters; Claude Code cuts "
|
|
403
|
+
f"each one at {SKILL_DESCRIPTION_MAX}.", doc, [skill["path"]]))
|
|
404
|
+
total = sum(len(s["name"]) + len(s["description"]) for s in skills)
|
|
405
|
+
if total > SKILL_BUDGET_CHARS:
|
|
406
|
+
findings.append(_finding("skill_budget", "warn", harness, f"{len(skills)} skills",
|
|
407
|
+
f"Names and descriptions add up to {total:,} characters, over the ~{SKILL_BUDGET_CHARS:,} "
|
|
408
|
+
"the skill list gets (Claude: 1% of a 200k context; Codex: 2% or 8,000 characters):"
|
|
409
|
+
" some skills are left out of what the model sees.", doc))
|
|
410
|
+
return findings
|
|
411
|
+
|
|
412
|
+
|
|
413
|
+
def check_instructions(project, claude, codex):
|
|
414
|
+
"""AGENTS.md that only Codex reads, broken @imports in CLAUDE.md, and a Codex chain over its byte limit."""
|
|
415
|
+
findings = []
|
|
416
|
+
claude_texts = [i for i in claude["instructions"]]
|
|
417
|
+
agents_here = os.path.exists(os.path.join(project, "AGENTS.md"))
|
|
418
|
+
imports_agents = any(re.search(r"(?<![\w`])@\S*AGENTS\.md", i["_text"]) for i in claude_texts)
|
|
419
|
+
if agents_here and not imports_agents:
|
|
420
|
+
findings.append(_finding("agents_md_not_read_by_claude", "warn", "claude", os.path.join(project, "AGENTS.md"),
|
|
421
|
+
"Codex reads AGENTS.md; Claude Code reads only CLAUDE.md (and what it imports with "
|
|
422
|
+
"@AGENTS.md), so the two agents get different instructions here.", DOCS["claude_memory"]))
|
|
423
|
+
for item in claude_texts:
|
|
424
|
+
base = os.path.dirname(item["path"])
|
|
425
|
+
for target in _IMPORT_RE.findall(item["_text"]):
|
|
426
|
+
path = os.path.expanduser(target) if target.startswith("~") else os.path.join(base, target)
|
|
427
|
+
if not os.path.exists(path):
|
|
428
|
+
findings.append(_finding("broken_import", "error", "claude", f"{item['path']} @{target}",
|
|
429
|
+
"Imported file does not exist: that part of the instructions never loads.",
|
|
430
|
+
DOCS["claude_memory"], [item["path"]]))
|
|
431
|
+
total = sum(i["bytes"] for i in codex["instructions"] if i["kind"] == "project")
|
|
432
|
+
if total > CODEX_PROJECT_DOC_MAX:
|
|
433
|
+
findings.append(_finding("codex_agents_truncated", "error", "codex", project,
|
|
434
|
+
f"Project AGENTS.md chain is {total:,} bytes; Codex reads {CODEX_PROJECT_DOC_MAX:,} "
|
|
435
|
+
"(project_doc_max_bytes) and drops the rest.", DOCS["codex_agents_md"],
|
|
436
|
+
[i["path"] for i in codex["instructions"]]))
|
|
437
|
+
return findings
|
|
438
|
+
|
|
439
|
+
|
|
440
|
+
def _rpc_stdio(entry, cwd, timeout):
|
|
441
|
+
"""initialize + tools/list over stdio; {"tools": [...]} or {"error", "kind": missing|down|timeout}."""
|
|
442
|
+
command = entry["command"]
|
|
443
|
+
resolved = command if os.path.isabs(command or "") else shutil.which(command or "")
|
|
444
|
+
if not resolved or not os.path.exists(resolved):
|
|
445
|
+
return {"error": f"command not found: {command}", "kind": "missing"}
|
|
446
|
+
env = {**os.environ, **{k: str(v) for k, v in entry["_env"].items()}}
|
|
447
|
+
flags = getattr(subprocess, "CREATE_NO_WINDOW", 0)
|
|
448
|
+
try:
|
|
449
|
+
proc = subprocess.Popen([resolved, *entry["args"]], cwd=cwd, env=env, stdin=subprocess.PIPE,
|
|
450
|
+
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, encoding="utf-8",
|
|
451
|
+
errors="replace", creationflags=flags)
|
|
452
|
+
except OSError as exc:
|
|
453
|
+
return {"error": f"did not start: {exc}", "kind": "down"}
|
|
454
|
+
lines, errors = queue.Queue(), []
|
|
455
|
+
threading.Thread(target=lambda: [lines.put(l) for l in proc.stdout] + [lines.put(None)], daemon=True).start()
|
|
456
|
+
threading.Thread(target=lambda: errors.extend(proc.stderr), daemon=True).start()
|
|
457
|
+
deadline = time.monotonic() + timeout
|
|
458
|
+
|
|
459
|
+
def call(request_id, method, params):
|
|
460
|
+
proc.stdin.write(json.dumps({"jsonrpc": "2.0", "id": request_id, "method": method, "params": params}) + "\n")
|
|
461
|
+
proc.stdin.flush()
|
|
462
|
+
while True:
|
|
463
|
+
try:
|
|
464
|
+
line = lines.get(timeout=max(0.1, deadline - time.monotonic()))
|
|
465
|
+
except queue.Empty:
|
|
466
|
+
raise TimeoutError(f"no answer to {method} in {timeout}s") from None
|
|
467
|
+
if line is None:
|
|
468
|
+
raise RuntimeError(f"exited: {''.join(errors).strip()[-200:] or 'no output'}")
|
|
469
|
+
try:
|
|
470
|
+
message = json.loads(line)
|
|
471
|
+
except ValueError:
|
|
472
|
+
continue
|
|
473
|
+
if message.get("id") == request_id:
|
|
474
|
+
if "error" in message:
|
|
475
|
+
raise RuntimeError(str(message["error"].get("message"))[:200])
|
|
476
|
+
return message["result"]
|
|
477
|
+
try:
|
|
478
|
+
call(1, "initialize", {"protocolVersion": "2025-06-18", "capabilities": {},
|
|
479
|
+
"clientInfo": {"name": "smart-tool-setup-audit", "version": "1"}})
|
|
480
|
+
proc.stdin.write(json.dumps({"jsonrpc": "2.0", "method": "notifications/initialized"}) + "\n")
|
|
481
|
+
return {"tools": call(2, "tools/list", {}).get("tools") or []}
|
|
482
|
+
except (OSError, RuntimeError, TimeoutError) as exc:
|
|
483
|
+
return {"error": str(exc), "kind": "timeout" if isinstance(exc, TimeoutError) else "down"}
|
|
484
|
+
finally:
|
|
485
|
+
if os.name == "nt":
|
|
486
|
+
subprocess.run(["taskkill", "/PID", str(proc.pid), "/T", "/F"], capture_output=True, creationflags=flags)
|
|
487
|
+
else:
|
|
488
|
+
proc.kill()
|
|
489
|
+
try:
|
|
490
|
+
proc.wait(timeout=10)
|
|
491
|
+
except subprocess.TimeoutExpired:
|
|
492
|
+
pass
|
|
493
|
+
for stream in (proc.stdin, proc.stdout, proc.stderr):
|
|
494
|
+
try:
|
|
495
|
+
stream.close()
|
|
496
|
+
except OSError:
|
|
497
|
+
pass
|
|
498
|
+
|
|
499
|
+
|
|
500
|
+
def _rpc_http(entry, timeout):
|
|
501
|
+
"""initialize + tools/list over streamable HTTP; {"tools": [...]} or {"error", "kind": auth|down}."""
|
|
502
|
+
headers = {"Content-Type": "application/json", "Accept": "application/json, text/event-stream",
|
|
503
|
+
**{k: str(v) for k, v in entry["_headers"].items()}}
|
|
504
|
+
if entry.get("_bearer_env") and os.environ.get(entry["_bearer_env"]):
|
|
505
|
+
headers["Authorization"] = f"Bearer {os.environ[entry['_bearer_env']]}"
|
|
506
|
+
|
|
507
|
+
def post(body):
|
|
508
|
+
request = urllib.request.Request(entry["url"], data=json.dumps(body).encode(), headers=headers, method="POST")
|
|
509
|
+
with urllib.request.urlopen(request, timeout=timeout) as response:
|
|
510
|
+
if response.headers.get("Mcp-Session-Id"):
|
|
511
|
+
headers["Mcp-Session-Id"] = response.headers["Mcp-Session-Id"]
|
|
512
|
+
text = response.read().decode("utf-8", "replace")
|
|
513
|
+
if not text.strip():
|
|
514
|
+
return {}
|
|
515
|
+
if text.lstrip().startswith("{"):
|
|
516
|
+
return json.loads(text)
|
|
517
|
+
data = [line[5:].strip() for line in text.splitlines() if line.startswith("data:")]
|
|
518
|
+
return json.loads(data[-1]) if data else {}
|
|
519
|
+
try:
|
|
520
|
+
post({"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {
|
|
521
|
+
"protocolVersion": "2025-06-18", "capabilities": {},
|
|
522
|
+
"clientInfo": {"name": "smart-tool-setup-audit", "version": "1"}}})
|
|
523
|
+
post({"jsonrpc": "2.0", "method": "notifications/initialized"})
|
|
524
|
+
answer = post({"jsonrpc": "2.0", "id": 2, "method": "tools/list", "params": {}})
|
|
525
|
+
return {"tools": (answer.get("result") or {}).get("tools") or []}
|
|
526
|
+
except urllib.error.HTTPError as exc:
|
|
527
|
+
kind = "auth" if exc.code in (401, 403) else "down"
|
|
528
|
+
return {"error": f"HTTP {exc.code}", "kind": kind}
|
|
529
|
+
except (OSError, ValueError) as exc:
|
|
530
|
+
return {"error": str(getattr(exc, "reason", exc)), "kind": "down"}
|
|
531
|
+
|
|
532
|
+
|
|
533
|
+
_PROBES = {}
|
|
534
|
+
_PROBES_LOCK = threading.Lock()
|
|
535
|
+
|
|
536
|
+
|
|
537
|
+
def _probe_key(entry, cwd):
|
|
538
|
+
"""Identity of a server launch (secrets only as a hash), shared by the projects and harnesses that register it;
|
|
539
|
+
the folder counts for servers a project registers itself and for arguments that may be relative paths."""
|
|
540
|
+
relative = any(not a.startswith("-") and not os.path.isabs(a) and ("/" in a or "\\" in a or "." in a)
|
|
541
|
+
for a in entry["args"])
|
|
542
|
+
per_project = entry["transport"] == "stdio" and (entry["scope"] in ("project", "local") or relative)
|
|
543
|
+
spec = [entry["transport"], entry.get("command"), entry["args"], entry.get("url"), sorted(entry["_env"].items()),
|
|
544
|
+
sorted(entry["_headers"].items()), entry.get("_bearer_env"), cwd if per_project else None]
|
|
545
|
+
return hashlib.sha256(json.dumps(spec, default=str).encode()).hexdigest()
|
|
546
|
+
|
|
547
|
+
|
|
548
|
+
def probe_mcp(entries, cwd, timeout=PROBE_TIMEOUT_S, max_age=0):
|
|
549
|
+
"""Tools of each MCP server, started or called as a session would (in parallel). With max_age, a result of the
|
|
550
|
+
same launch younger than that is reused (failures never are: a server fixed meanwhile shows at once): the periodic
|
|
551
|
+
audit does not start every server of every project each time. Each result carries `probed_at`."""
|
|
552
|
+
def one(entry):
|
|
553
|
+
key = _probe_key(entry, cwd)
|
|
554
|
+
with _PROBES_LOCK:
|
|
555
|
+
cached = _PROBES.get(key)
|
|
556
|
+
if max_age and cached and not cached.get("error") and time.time() - cached["probed_at"] < max_age:
|
|
557
|
+
return cached
|
|
558
|
+
# A server whose config cannot even be used (a URL with an unexpanded variable, a bad port) is a finding.
|
|
559
|
+
try:
|
|
560
|
+
result = _rpc_http(entry, timeout) if entry["transport"] == "http" else _rpc_stdio(entry, cwd, timeout)
|
|
561
|
+
except Exception as exc:
|
|
562
|
+
result = {"error": f"{type(exc).__name__}: {exc}"[:200], "kind": "down"}
|
|
563
|
+
result["probed_at"] = time.time()
|
|
564
|
+
with _PROBES_LOCK:
|
|
565
|
+
_PROBES[key] = result
|
|
566
|
+
return result
|
|
567
|
+
with concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool:
|
|
568
|
+
return list(pool.map(one, entries))
|
|
569
|
+
|
|
570
|
+
|
|
571
|
+
def check_mcp(harness, entries, results):
|
|
572
|
+
"""Unhealthy servers, and tools offered twice (same name, or a proxy re-exposing a directly registered server)."""
|
|
573
|
+
findings, tools_by_server = [], {}
|
|
574
|
+
doc = DOCS["claude_mcp"] if harness == "claude" else DOCS["mcp_lifecycle"]
|
|
575
|
+
for entry, result in zip(entries, results):
|
|
576
|
+
subject = f"{entry['name']} ({entry['scope']})"
|
|
577
|
+
if result is None:
|
|
578
|
+
continue
|
|
579
|
+
if result.get("error"):
|
|
580
|
+
severity = {"missing": "error", "down": "error", "timeout": "warn", "auth": "info"}[result["kind"]]
|
|
581
|
+
text = {"auth": "needs sign-in; its tools were not verified",
|
|
582
|
+
"timeout": "did not answer in time; its tools were not verified"}.get(result["kind"],
|
|
583
|
+
"does not start or answer")
|
|
584
|
+
findings.append(_finding("mcp_unhealthy", severity, harness, subject,
|
|
585
|
+
f"MCP server {text}: {result['error']}", doc, [entry["source"]]))
|
|
586
|
+
continue
|
|
587
|
+
names = [t.get("name") for t in result["tools"] if t.get("name") not in entry["disabled_tools"]]
|
|
588
|
+
tools_by_server[entry["name"]] = set(names)
|
|
589
|
+
chars = sum(len(json.dumps(t)) for t in result["tools"])
|
|
590
|
+
entry["tools"], entry["tool_chars"] = len(names), chars
|
|
591
|
+
for server, names in tools_by_server.items():
|
|
592
|
+
for other, other_names in tools_by_server.items():
|
|
593
|
+
if other == server:
|
|
594
|
+
continue
|
|
595
|
+
same = sorted(n for n in names if n in other_names) if server < other else []
|
|
596
|
+
proxied = sorted(n for n in names if "__" in n and n.split("__", 1)[0] == other
|
|
597
|
+
and n.split("__", 1)[1] in other_names)
|
|
598
|
+
if same:
|
|
599
|
+
findings.append(_finding("mcp_duplicate_tools", "warn", harness, f"{server} and {other}",
|
|
600
|
+
f"{len(same)} tools with the same name on both servers: {', '.join(same[:6])}",
|
|
601
|
+
doc))
|
|
602
|
+
if proxied:
|
|
603
|
+
findings.append(_finding("mcp_duplicate_tools", "warn", harness, f"{server} proxies {other}",
|
|
604
|
+
f"{server} re-exposes {len(proxied)} tools of {other}, which is also "
|
|
605
|
+
f"registered directly: {', '.join(proxied[:6])}. The model sees both and "
|
|
606
|
+
"pays for both descriptions.", doc))
|
|
607
|
+
return findings
|
|
608
|
+
|
|
609
|
+
|
|
610
|
+
def _public(section):
|
|
611
|
+
"""A collected section without private fields (instruction text, env and header values)."""
|
|
612
|
+
out = {}
|
|
613
|
+
for key, value in section.items():
|
|
614
|
+
if isinstance(value, list):
|
|
615
|
+
out[key] = [{k: v for k, v in item.items() if not k.startswith("_")} if isinstance(item, dict) else item
|
|
616
|
+
for item in value]
|
|
617
|
+
else:
|
|
618
|
+
out[key] = value
|
|
619
|
+
return out
|
|
620
|
+
|
|
621
|
+
|
|
622
|
+
def run(project, probe=True, save=True, probe_max_age=0, trigger="manual"):
|
|
623
|
+
"""Collects both harnesses for project, runs every check and returns the report; with save, stores it and
|
|
624
|
+
marks findings new or resolved against the previous report of the same project. probe_max_age reuses recent
|
|
625
|
+
MCP probes (periodic runs); trigger records who asked (manual or schedule)."""
|
|
626
|
+
project = os.path.abspath(project)
|
|
627
|
+
snapshot = _Snapshot()
|
|
628
|
+
claude, codex = collect_claude(project, snapshot), collect_codex(project, snapshot)
|
|
629
|
+
findings = []
|
|
630
|
+
for harness, data, tools in (("claude", claude, CLAUDE_TOOLS), ("codex", codex, CODEX_TOOLS)):
|
|
631
|
+
names = {m["name"] for m in data["mcp"]}
|
|
632
|
+
findings += check_hook_overlap(harness, data["hooks"], tools)
|
|
633
|
+
findings += check_hook_targets(harness, data["hooks"], names, probe)
|
|
634
|
+
findings += check_skills(harness, data["skills"])
|
|
635
|
+
results = (probe_mcp(data["mcp"], project, max_age=probe_max_age) if probe
|
|
636
|
+
else [None] * len(data["mcp"]))
|
|
637
|
+
findings += check_mcp(harness, data["mcp"], results)
|
|
638
|
+
findings += check_instructions(project, claude, codex)
|
|
639
|
+
unverified = list(snapshot.unverified)
|
|
640
|
+
if not probe:
|
|
641
|
+
unverified.append("MCP servers and HTTP hook endpoints were not contacted (probe off)")
|
|
642
|
+
for name in claude["unapproved_mcp"]:
|
|
643
|
+
unverified.append(f".mcp.json server '{name}' is not approved in Claude Code, so it is neither loaded nor "
|
|
644
|
+
"started here")
|
|
645
|
+
if codex.get("plugins_unverified"):
|
|
646
|
+
unverified.append("Codex plugins (their skills and hooks) are not collected yet")
|
|
647
|
+
unverified += ["skill trigger overlap and contradictions between instructions need a model (not run)",
|
|
648
|
+
"the order between two parallel rewrites is undocumented and cannot be checked"]
|
|
649
|
+
key = hashlib.sha256(json.dumps(sorted(snapshot.files.items())).encode()).hexdigest()[:16]
|
|
650
|
+
order = {"error": 0, "warn": 1, "info": 2}
|
|
651
|
+
findings.sort(key=lambda f: (order[f["severity"]], f["harness"], f["check"], f["subject"]))
|
|
652
|
+
report = {"project": project, "checked_at": time.time(), "snapshot_key": key, "probe": probe, "trigger": trigger,
|
|
653
|
+
"summary": {s: sum(f["severity"] == s for f in findings) for s in order},
|
|
654
|
+
"findings": findings, "unverified": unverified,
|
|
655
|
+
"claude": _public(claude), "codex": _public(codex), "files": len(snapshot.files)}
|
|
656
|
+
if save:
|
|
657
|
+
_save_with_diff(report)
|
|
658
|
+
return report
|
|
659
|
+
|
|
660
|
+
|
|
661
|
+
def _report_path(project):
|
|
662
|
+
return os.path.join(REPORT_DIR, hashlib.sha1(os.path.abspath(project).lower().encode()).hexdigest()[:12] + ".json")
|
|
663
|
+
|
|
664
|
+
|
|
665
|
+
def last_report(project):
|
|
666
|
+
"""The last stored report of project, or None when it was never audited."""
|
|
667
|
+
try:
|
|
668
|
+
with open(_report_path(project), encoding="utf-8") as stream:
|
|
669
|
+
return json.load(stream)
|
|
670
|
+
except FileNotFoundError:
|
|
671
|
+
return None
|
|
672
|
+
|
|
673
|
+
|
|
674
|
+
def index():
|
|
675
|
+
"""{project: {summary, new, resolved, checked_at, trigger}} of every audited project (the periodic summary)."""
|
|
676
|
+
try:
|
|
677
|
+
with open(os.path.join(REPORT_DIR, INDEX_NAME), encoding="utf-8") as stream:
|
|
678
|
+
data = json.load(stream)
|
|
679
|
+
except FileNotFoundError:
|
|
680
|
+
return {}
|
|
681
|
+
if not isinstance(data, dict):
|
|
682
|
+
raise ValueError(f"{os.path.join(REPORT_DIR, INDEX_NAME)} is not a setup audit index: fix or delete it.")
|
|
683
|
+
return data
|
|
684
|
+
|
|
685
|
+
|
|
686
|
+
_INDEX_LOCK = threading.Lock()
|
|
687
|
+
|
|
688
|
+
|
|
689
|
+
def index_key(project):
|
|
690
|
+
"""The index key of a folder: the same for any spelling of it on this OS."""
|
|
691
|
+
return os.path.normcase(os.path.abspath(project))
|
|
692
|
+
|
|
693
|
+
|
|
694
|
+
def _write_index(data):
|
|
695
|
+
atomic_io.write_secret_text(os.path.join(REPORT_DIR, INDEX_NAME), json.dumps(data, indent=1, ensure_ascii=False))
|
|
696
|
+
|
|
697
|
+
|
|
698
|
+
def prune_index(projects):
|
|
699
|
+
"""Drops the index entries and reports of folders no longer registered."""
|
|
700
|
+
keep = {index_key(p) for p in projects}
|
|
701
|
+
with _INDEX_LOCK:
|
|
702
|
+
data = index()
|
|
703
|
+
gone = [key for key in data if key not in keep]
|
|
704
|
+
for key in gone:
|
|
705
|
+
data.pop(key)
|
|
706
|
+
try:
|
|
707
|
+
os.remove(_report_path(key))
|
|
708
|
+
except FileNotFoundError:
|
|
709
|
+
pass
|
|
710
|
+
if gone:
|
|
711
|
+
_write_index(data)
|
|
712
|
+
return gone
|
|
713
|
+
|
|
714
|
+
|
|
715
|
+
def _save_with_diff(report):
|
|
716
|
+
"""Stores the report, marks findings new or resolved against the previous one and updates the index, all under
|
|
717
|
+
one lock: the scheduler, Run now and project_manage may audit the same folder at once."""
|
|
718
|
+
with _INDEX_LOCK:
|
|
719
|
+
_save_locked(report)
|
|
720
|
+
|
|
721
|
+
|
|
722
|
+
def _save_locked(report):
|
|
723
|
+
os.makedirs(REPORT_DIR, exist_ok=True)
|
|
724
|
+
last_path = _report_path(report["project"])
|
|
725
|
+
try:
|
|
726
|
+
with open(last_path, encoding="utf-8") as stream:
|
|
727
|
+
previous = json.load(stream)
|
|
728
|
+
except (FileNotFoundError, ValueError):
|
|
729
|
+
previous = None
|
|
730
|
+
if previous:
|
|
731
|
+
before = {f["id"] for f in previous.get("findings", [])}
|
|
732
|
+
now = {f["id"] for f in report["findings"]}
|
|
733
|
+
report["changes"] = {"snapshot_changed": previous.get("snapshot_key") != report["snapshot_key"],
|
|
734
|
+
"new": sorted(now - before), "resolved": [f for f in previous["findings"]
|
|
735
|
+
if f["id"] not in now]}
|
|
736
|
+
for finding in report["findings"]:
|
|
737
|
+
finding["new"] = finding["id"] not in before
|
|
738
|
+
atomic_io.write_secret_text(last_path, json.dumps(report, indent=1, ensure_ascii=False))
|
|
739
|
+
changes = report.get("changes") or {}
|
|
740
|
+
data = index()
|
|
741
|
+
data[index_key(report["project"])] = {"summary": report["summary"], "new": len(changes.get("new", [])),
|
|
742
|
+
"resolved": len(changes.get("resolved", [])),
|
|
743
|
+
"checked_at": report["checked_at"], "trigger": report["trigger"],
|
|
744
|
+
"snapshot_key": report["snapshot_key"]}
|
|
745
|
+
_write_index(data)
|
|
746
|
+
|
|
747
|
+
|
|
748
|
+
class Scheduler:
|
|
749
|
+
"""Periodic mode: audits every registered project whose folder exists, FIRST_RUN_DELAY_S after start and then
|
|
750
|
+
every SCHEDULE_S, one project at a time, reusing MCP probes younger than PROBE_MAX_AGE_S. projects() returns the
|
|
751
|
+
folders; log gets one line per run and every failure (a failing project never stops the others)."""
|
|
752
|
+
|
|
753
|
+
def __init__(self, projects, log, interval=SCHEDULE_S, first_delay=FIRST_RUN_DELAY_S):
|
|
754
|
+
self._projects, self._log, self._interval, self._first_delay = projects, log, interval, first_delay
|
|
755
|
+
self._stop = threading.Event()
|
|
756
|
+
self._thread = None
|
|
757
|
+
|
|
758
|
+
def start(self):
|
|
759
|
+
self._thread = threading.Thread(target=self._loop, name="setup-audit", daemon=True)
|
|
760
|
+
self._thread.start()
|
|
761
|
+
|
|
762
|
+
def stop(self):
|
|
763
|
+
self._stop.set()
|
|
764
|
+
|
|
765
|
+
def run_once(self):
|
|
766
|
+
"""Audits each project once; returns {project: summary or error text}."""
|
|
767
|
+
outcome, projects = {}, list(self._projects())
|
|
768
|
+
prune_index(projects)
|
|
769
|
+
for root in projects:
|
|
770
|
+
if self._stop.is_set():
|
|
771
|
+
break
|
|
772
|
+
if not os.path.isdir(root):
|
|
773
|
+
continue
|
|
774
|
+
try:
|
|
775
|
+
report = run(root, probe_max_age=PROBE_MAX_AGE_S, trigger="schedule")
|
|
776
|
+
outcome[root] = report["summary"]
|
|
777
|
+
except Exception as exc:
|
|
778
|
+
outcome[root] = f"{type(exc).__name__}: {exc}"[:200]
|
|
779
|
+
self._log(f"setup audit failed for {root}: {outcome[root]}")
|
|
780
|
+
errors = sum(v["error"] for v in outcome.values() if isinstance(v, dict))
|
|
781
|
+
self._log(f"setup audit: {len(outcome)} project(s), {errors} error finding(s)")
|
|
782
|
+
return outcome
|
|
783
|
+
|
|
784
|
+
def _loop(self):
|
|
785
|
+
if self._stop.wait(self._first_delay):
|
|
786
|
+
return
|
|
787
|
+
while True:
|
|
788
|
+
self.run_once()
|
|
789
|
+
if self._stop.wait(self._interval):
|
|
790
|
+
return
|
|
791
|
+
|
|
792
|
+
|
|
793
|
+
def markdown(report):
|
|
794
|
+
"""The report as Markdown for people: findings by severity, what changed, what was not verified."""
|
|
795
|
+
s = report["summary"]
|
|
796
|
+
lines = [f"# Setup audit: {report['project']}", "",
|
|
797
|
+
f"{s['error']} errors, {s['warn']} warnings, {s['info']} notes. Snapshot {report['snapshot_key']} "
|
|
798
|
+
f"({report['files']} files read).", ""]
|
|
799
|
+
changes = report.get("changes")
|
|
800
|
+
if changes:
|
|
801
|
+
lines += [f"Since the last run: {len(changes['new'])} new, {len(changes['resolved'])} resolved"
|
|
802
|
+
+ (" (configuration changed)." if changes["snapshot_changed"] else "."), ""]
|
|
803
|
+
for finding in report["findings"]:
|
|
804
|
+
mark = {"error": "ERROR", "warn": "WARN", "info": "NOTE"}[finding["severity"]]
|
|
805
|
+
lines.append(f"- **{mark}** [{finding['harness']}] {finding['subject']}: {finding['detail']} "
|
|
806
|
+
f"([source]({finding['source']}))" + (" *(new)*" if finding.get("new") else ""))
|
|
807
|
+
for side in ("claude", "codex"):
|
|
808
|
+
servers = [m for m in report[side]["mcp"] if "tools" in m]
|
|
809
|
+
if servers:
|
|
810
|
+
lines += ["", f"MCP tools offered to {side}: " + ", ".join(
|
|
811
|
+
f"{m['name']} {m['tools']} (~{m['tool_chars'] // 4:,} tokens)" for m in servers)]
|
|
812
|
+
lines += ["", "Not verified:"] + [f"- {item}" for item in report["unverified"]]
|
|
813
|
+
return "\n".join(lines)
|
|
814
|
+
|
|
815
|
+
|
|
816
|
+
def main():
|
|
817
|
+
"""CLI: prints the Markdown report (or JSON with --json)."""
|
|
818
|
+
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
|
819
|
+
parser.add_argument("--project", default=os.getcwd())
|
|
820
|
+
parser.add_argument("--no-probe", action="store_true")
|
|
821
|
+
parser.add_argument("--json", action="store_true")
|
|
822
|
+
args = parser.parse_args()
|
|
823
|
+
report = run(args.project, probe=not args.no_probe)
|
|
824
|
+
print(json.dumps(report, indent=1, ensure_ascii=False) if args.json else markdown(report))
|
|
825
|
+
|
|
826
|
+
|
|
827
|
+
if __name__ == "__main__":
|
|
828
|
+
main()
|
package/setup_ui.py
CHANGED
|
@@ -29,7 +29,9 @@ import gateway
|
|
|
29
29
|
import model_catalog
|
|
30
30
|
import model_defaults
|
|
31
31
|
import local_embedder
|
|
32
|
+
import project_store
|
|
32
33
|
import secret_store
|
|
34
|
+
import setup_audit
|
|
33
35
|
import version
|
|
34
36
|
import web_search_external
|
|
35
37
|
|
|
@@ -251,6 +253,30 @@ def handle_integration_save(body):
|
|
|
251
253
|
raise SetupError("Invalid action: use hook_mode, register_mcp, browser_control, preview or install.")
|
|
252
254
|
|
|
253
255
|
|
|
256
|
+
def handle_audit(body):
|
|
257
|
+
"""Setup audit panel: `index` lists the registered projects with their last audit summary; `report` returns the
|
|
258
|
+
last stored report of one of them; `run` audits it now (MCP servers contacted unless probe is false). Only
|
|
259
|
+
registered project folders are accepted."""
|
|
260
|
+
if not isinstance(body, dict):
|
|
261
|
+
raise SetupError("Invalid request.")
|
|
262
|
+
roots = sorted({p["root"] for p in project_store.all_projects() if p.get("root")}, key=str.casefold)
|
|
263
|
+
try:
|
|
264
|
+
if body.get("action") == "index":
|
|
265
|
+
stored = setup_audit.index()
|
|
266
|
+
found = {root: stored[setup_audit.index_key(root)] for root in roots if setup_audit.index_key(root) in stored}
|
|
267
|
+
return {"projects": roots, "index": found, "schedule_s": setup_audit.SCHEDULE_S}
|
|
268
|
+
root = body.get("project_root")
|
|
269
|
+
if root not in roots:
|
|
270
|
+
raise SetupError("Choose one of the registered projects.")
|
|
271
|
+
if body.get("action") == "report":
|
|
272
|
+
return {"report": setup_audit.last_report(root)}
|
|
273
|
+
if body.get("action") == "run":
|
|
274
|
+
return {"report": setup_audit.run(root, probe=body.get("probe") is not False)}
|
|
275
|
+
except (OSError, ValueError) as exc:
|
|
276
|
+
raise SetupError(str(exc))
|
|
277
|
+
raise SetupError("Invalid action: use index, report or run.")
|
|
278
|
+
|
|
279
|
+
|
|
254
280
|
def handle_providers_get():
|
|
255
281
|
return {"providers": {
|
|
256
282
|
name: {**web_search_external.PROVIDERS[name], "status": config.provider_key_status(name)}
|
|
@@ -498,6 +524,15 @@ _PAGE_TEMPLATE = """<!doctype html>
|
|
|
498
524
|
.pattern-actions button { padding: 4px 10px; font-size: .78rem; }
|
|
499
525
|
.pattern-pager { justify-content: space-between; }
|
|
500
526
|
.pattern-blocking > summary { cursor: pointer; font: 600 .9rem var(--display); }
|
|
527
|
+
.audit-list { list-style: none; padding: 0; margin: 10px 0; }
|
|
528
|
+
.audit-row { padding: 10px 0; border-bottom: 1px solid var(--line); }
|
|
529
|
+
.audit-head { display: flex; gap: 8px; align-items: baseline; flex-wrap: wrap; }
|
|
530
|
+
.audit-sev { font: 600 .72rem var(--display); letter-spacing: .06em; text-transform: uppercase; }
|
|
531
|
+
.audit-error { color: var(--bad); }
|
|
532
|
+
.audit-warn { color: var(--warn); }
|
|
533
|
+
.audit-info { color: var(--muted); }
|
|
534
|
+
.audit-new { font: 600 .72rem var(--display); color: var(--signal); }
|
|
535
|
+
.audit-detail { font-size: .84rem; margin: 4px 0 0; overflow-wrap: anywhere; }
|
|
501
536
|
|
|
502
537
|
body:not(.ready) .station[data-live="1"] .wire::after { transform: scaleX(0); }
|
|
503
538
|
@media (max-width: 900px) { .panels { grid-template-columns: minmax(0, 1fr); } }
|
|
@@ -659,6 +694,17 @@ _PAGE_TEMPLATE = """<!doctype html>
|
|
|
659
694
|
<div id="agents-list"></div>
|
|
660
695
|
</section>
|
|
661
696
|
|
|
697
|
+
<section id="audit-section">
|
|
698
|
+
<h2>Setup audit</h2>
|
|
699
|
+
<p class="lede">What Claude Code and Codex load for a project, and where they collide: hooks deciding on the same tool at once, hooks or MCP servers that do not work, tools offered twice, skills and instructions one agent never sees. It runs on its own every 30 minutes for the registered projects; nothing is changed.</p>
|
|
700
|
+
<div class="field"><label for="sel-audit-project">Project</label>
|
|
701
|
+
<select id="sel-audit-project"></select>
|
|
702
|
+
<span class="hint" id="audit-hint"></span></div>
|
|
703
|
+
<div class="actions"><button type="button" id="btn-audit-run">Run now</button></div>
|
|
704
|
+
<div class="msg" id="audit-msg" role="status"></div>
|
|
705
|
+
<div id="audit-report"></div>
|
|
706
|
+
</section>
|
|
707
|
+
|
|
662
708
|
<section id="providers-section">
|
|
663
709
|
<h2>Web search providers</h2>
|
|
664
710
|
<p class="lede">All of them work without a key, on each provider's public free tier. Add your own key only if you need more volume: it is tested with a real search before being saved and is protected by DPAPI on this Windows machine.</p>
|
|
@@ -1226,6 +1272,114 @@ $("btn-save-hook-mode").addEventListener("click", async () => {
|
|
|
1226
1272
|
}
|
|
1227
1273
|
});
|
|
1228
1274
|
|
|
1275
|
+
const AUDIT_SEVERITY = { error: "Error", warn: "Warning", info: "Note" };
|
|
1276
|
+
let auditProjects = [];
|
|
1277
|
+
function auditWhen(seconds) {
|
|
1278
|
+
return seconds ? new Date(seconds * 1000).toLocaleString() : "never";
|
|
1279
|
+
}
|
|
1280
|
+
function renderAudit(report) {
|
|
1281
|
+
const box = $("audit-report");
|
|
1282
|
+
box.replaceChildren();
|
|
1283
|
+
if (!report) {
|
|
1284
|
+
box.appendChild(patternEl("p", "hint", "Not audited yet: the next scheduled run covers it, or press Run now."));
|
|
1285
|
+
return;
|
|
1286
|
+
}
|
|
1287
|
+
const s = report.summary;
|
|
1288
|
+
box.appendChild(patternEl("p", "", s.error + " error(s), " + s.warn + " warning(s), " + s.info + " note(s) · "
|
|
1289
|
+
+ (report.trigger === "schedule" ? "scheduled run" : "run on request") + " at " + auditWhen(report.checked_at)));
|
|
1290
|
+
const changes = report.changes;
|
|
1291
|
+
if (changes) box.appendChild(patternEl("p", "hint", "Since the previous run: " + changes.new.length + " new, "
|
|
1292
|
+
+ changes.resolved.length + " resolved" + (changes.snapshot_changed ? " (configuration changed)." : ".")));
|
|
1293
|
+
const list = patternEl("ul", "audit-list");
|
|
1294
|
+
for (const finding of report.findings) {
|
|
1295
|
+
const row = patternEl("li", "audit-row");
|
|
1296
|
+
const head = patternEl("div", "audit-head");
|
|
1297
|
+
head.append(patternEl("span", "audit-sev audit-" + finding.severity, AUDIT_SEVERITY[finding.severity] || finding.severity),
|
|
1298
|
+
patternEl("span", "pattern-tool", finding.harness), patternEl("strong", "", finding.subject));
|
|
1299
|
+
if (finding.new) head.appendChild(patternEl("span", "audit-new", "new"));
|
|
1300
|
+
const detail = patternEl("p", "audit-detail", finding.detail + " ");
|
|
1301
|
+
if ((finding.source || "").startsWith("https://")) {
|
|
1302
|
+
const link = patternEl("a", "", "documentation");
|
|
1303
|
+
link.setAttribute("href", finding.source);
|
|
1304
|
+
link.setAttribute("target", "_blank");
|
|
1305
|
+
link.setAttribute("rel", "noopener noreferrer");
|
|
1306
|
+
detail.appendChild(link);
|
|
1307
|
+
}
|
|
1308
|
+
row.append(head, detail);
|
|
1309
|
+
list.appendChild(row);
|
|
1310
|
+
}
|
|
1311
|
+
if (report.findings.length) box.appendChild(list);
|
|
1312
|
+
else box.appendChild(patternEl("p", "msg ok", "No collision or broken piece found."));
|
|
1313
|
+
if (changes && changes.resolved.length) {
|
|
1314
|
+
const resolved = patternEl("details", "");
|
|
1315
|
+
resolved.appendChild(patternEl("summary", "", "Resolved since the previous run (" + changes.resolved.length + ")"));
|
|
1316
|
+
const items = patternEl("ul", "audit-list");
|
|
1317
|
+
for (const finding of changes.resolved) items.appendChild(patternEl("li", "audit-detail", finding.subject + ": " + finding.detail));
|
|
1318
|
+
resolved.appendChild(items);
|
|
1319
|
+
box.appendChild(resolved);
|
|
1320
|
+
}
|
|
1321
|
+
const costs = [];
|
|
1322
|
+
for (const side of ["claude", "codex"]) {
|
|
1323
|
+
const servers = ((report[side] || {}).mcp || []).filter((m) => m.tools !== undefined);
|
|
1324
|
+
if (servers.length) costs.push(side + ": " + servers.map((m) => m.name + " " + m.tools + " tools (~"
|
|
1325
|
+
+ Math.round(m.tool_chars / 4).toLocaleString() + " tokens)").join(", "));
|
|
1326
|
+
}
|
|
1327
|
+
if (costs.length) box.appendChild(patternEl("p", "hint", "MCP tools offered — " + costs.join(" · ")));
|
|
1328
|
+
const unverified = patternEl("details", "");
|
|
1329
|
+
unverified.appendChild(patternEl("summary", "", "Not verified (" + report.unverified.length + ")"));
|
|
1330
|
+
const items = patternEl("ul", "audit-list");
|
|
1331
|
+
for (const item of report.unverified) items.appendChild(patternEl("li", "audit-detail", item));
|
|
1332
|
+
unverified.appendChild(items);
|
|
1333
|
+
box.appendChild(unverified);
|
|
1334
|
+
}
|
|
1335
|
+
async function showAudit(root) {
|
|
1336
|
+
try {
|
|
1337
|
+
renderAudit((await post("/setup/audit", { action: "report", project_root: root })).report);
|
|
1338
|
+
} catch (error) {
|
|
1339
|
+
setMsg("audit-msg", "error", error.message);
|
|
1340
|
+
}
|
|
1341
|
+
}
|
|
1342
|
+
async function loadAudit() {
|
|
1343
|
+
try {
|
|
1344
|
+
const data = await post("/setup/audit", { action: "index" });
|
|
1345
|
+
auditProjects = data.projects;
|
|
1346
|
+
const select = $("sel-audit-project");
|
|
1347
|
+
select.replaceChildren();
|
|
1348
|
+
for (const root of data.projects) {
|
|
1349
|
+
const entry = data.index[root];
|
|
1350
|
+
const label = root + (entry ? " — " + entry.summary.error + " error(s), " + entry.summary.warn + " warning(s)" : " — not audited yet");
|
|
1351
|
+
const option = patternEl("option", "", label);
|
|
1352
|
+
option.value = root;
|
|
1353
|
+
select.appendChild(option);
|
|
1354
|
+
}
|
|
1355
|
+
$("audit-hint").textContent = data.projects.length ? "Runs every " + Math.round(data.schedule_s / 60) + " minutes." : "Register a project to audit it.";
|
|
1356
|
+
$("btn-audit-run").disabled = !data.projects.length;
|
|
1357
|
+
if (data.projects.length) {
|
|
1358
|
+
const worst = data.projects.slice().sort((a, b) => ((data.index[b] || {summary: {error: -1}}).summary.error)
|
|
1359
|
+
- ((data.index[a] || {summary: {error: -1}}).summary.error))[0];
|
|
1360
|
+
select.value = worst;
|
|
1361
|
+
await showAudit(worst);
|
|
1362
|
+
}
|
|
1363
|
+
} catch (error) {
|
|
1364
|
+
setMsg("audit-msg", "error", "Setup audit unavailable: " + error.message);
|
|
1365
|
+
}
|
|
1366
|
+
}
|
|
1367
|
+
$("sel-audit-project").addEventListener("change", () => showAudit($("sel-audit-project").value));
|
|
1368
|
+
$("btn-audit-run").addEventListener("click", async () => {
|
|
1369
|
+
const button = $("btn-audit-run");
|
|
1370
|
+
button.disabled = true;
|
|
1371
|
+
setMsg("audit-msg", "", "Auditing; MCP servers are being contacted…");
|
|
1372
|
+
try {
|
|
1373
|
+
renderAudit((await post("/setup/audit", { action: "run", project_root: $("sel-audit-project").value })).report);
|
|
1374
|
+
setMsg("audit-msg", "ok", "Audit finished.");
|
|
1375
|
+
} catch (error) {
|
|
1376
|
+
setMsg("audit-msg", "error", error.message);
|
|
1377
|
+
} finally {
|
|
1378
|
+
button.disabled = false;
|
|
1379
|
+
}
|
|
1380
|
+
});
|
|
1381
|
+
loadAudit();
|
|
1382
|
+
|
|
1229
1383
|
const providerLabels = { keyless: ["warn", "No key · free tier"], configured: ["ok", "Key configured"], unreadable: ["bad", "Key unreadable in this Windows profile"] };
|
|
1230
1384
|
function providerCard(name, info) {
|
|
1231
1385
|
const card = document.createElement("div");
|
package/smart_tool_daemon.py
CHANGED
|
@@ -59,6 +59,7 @@ import project_monitor
|
|
|
59
59
|
import research_metrics
|
|
60
60
|
import research_cache
|
|
61
61
|
import router
|
|
62
|
+
import setup_audit
|
|
62
63
|
import setup_ui
|
|
63
64
|
import site_memory
|
|
64
65
|
import web_search_adapters
|
|
@@ -326,10 +327,10 @@ SERVER_INSTRUCTIONS = (
|
|
|
326
327
|
TOOLS = [
|
|
327
328
|
{
|
|
328
329
|
"name": "project_manage",
|
|
329
|
-
"description": "Manages folders and indexing. Every action takes project_root (the folder) or project_id. list gives a short line per project; status shows storage, scope and jobs. graph with symbol (function, Class.method, class name, or path::name when the name repeats) answers what an edit touches: definition, callers with line, calls, tests that reach it through static calls (depth hops, default 3) and files importing it; use it before changing a function to know what to update. affected_tests answers which tests to run after editing: every test file importing a changed file (git diff against base, default HEAD, untracked included) plus changed tests, likeliest failures first, with the command to run them and run_all when config or unanalyzed code changed. graph without symbol reads the indexed view: coverage, references and static calls in Java, Angular, Python and JS/TS/JSX/TSX; includes HTML/CSS/Markdown references and TXT/DOCX coverage; with file_path, narrows to that file's symbols, imports and calls (lists capped by limit). inspect lists the files of a view_id; with file_path, returns only that file's chunks. usage shows consumption and reuse; docs lists public functions without a docstring (Python docstring, JSDoc, Javadoc; private, nested and override functions excluded) with coverage percent, excluding tests unless include_tests=true, for documenting a project that started without it; duplicates lists duplicated functions (identical, near-identical and semantic bodies, min_similarity default 0.90; below that most pairs measured were false positives), excluding tests unless include_tests=true; fix real duplicates; only dismiss with duplicates_dismiss (finding_id, reason=false_positive|intentional, note) what is not a duplicate or is a copy kept on purpose: the finding stays hidden until the code changes, and duplicates_restore reopens it; integration shows the MCP client, hook and tool usage metrics; install_hook_preview shows the change that would install the hook redirecting Grep/Read to smart_search and WebSearch/WebFetch to web_search/web_fetch in this session's client and returns a plan_id; install_hook writes it with that plan_id and confirm=true, with a backup, only after the user approves the change; search_limits saves the project's default top_k per block ({code, test, doc}, each 1-20, total at most 20); storage shows disk usage and views; pin_view protects a view; cleanup_preview and cleanup_commit remove selected views after confirmation; compare compares two stored views, without checkout or embeddings. register adds a folder; preview checks the scope; index/rebuild update; pause/cancel keep checkpoints; resume continues; policy picks on_search/eager; scope adjusts the scope; remove deletes only the local index. Jobs via smart_search_result.",
|
|
330
|
+
"description": "Manages folders and indexing. Every action takes project_root (the folder) or project_id. list gives a short line per project; status shows storage, scope and jobs. graph with symbol (function, Class.method, class name, or path::name when the name repeats) answers what an edit touches: definition, callers with line, calls, tests that reach it through static calls (depth hops, default 3) and files importing it; use it before changing a function to know what to update. affected_tests answers which tests to run after editing: every test file importing a changed file (git diff against base, default HEAD, untracked included) plus changed tests, likeliest failures first, with the command to run them and run_all when config or unanalyzed code changed. graph without symbol reads the indexed view: coverage, references and static calls in Java, Angular, Python and JS/TS/JSX/TSX; includes HTML/CSS/Markdown references and TXT/DOCX coverage; with file_path, narrows to that file's symbols, imports and calls (lists capped by limit). inspect lists the files of a view_id; with file_path, returns only that file's chunks. usage shows consumption and reuse; docs lists public functions without a docstring (Python docstring, JSDoc, Javadoc; private, nested and override functions excluded) with coverage percent, excluding tests unless include_tests=true, for documenting a project that started without it; duplicates lists duplicated functions (identical, near-identical and semantic bodies, min_similarity default 0.90; below that most pairs measured were false positives), excluding tests unless include_tests=true; fix real duplicates; only dismiss with duplicates_dismiss (finding_id, reason=false_positive|intentional, note) what is not a duplicate or is a copy kept on purpose: the finding stays hidden until the code changes, and duplicates_restore reopens it; integration shows the MCP client, hook and tool usage metrics; install_hook_preview shows the change that would install the hook redirecting Grep/Read to smart_search and WebSearch/WebFetch to web_search/web_fetch in this session's client and returns a plan_id; install_hook writes it with that plan_id and confirm=true, with a backup, only after the user approves the change; search_limits saves the project's default top_k per block ({code, test, doc}, each 1-20, total at most 20); storage shows disk usage and views; pin_view protects a view; cleanup_preview and cleanup_commit remove selected views after confirmation; compare compares two stored views, without checkout or embeddings. register adds a folder; preview checks the scope; index/rebuild update; pause/cancel keep checkpoints; resume continues; policy picks on_search/eager; scope adjusts the scope; remove deletes only the local index. setup_audit (project_root) reports what Claude Code and Codex actually load for that folder (settings layers, plugins, hooks, MCP servers, skills, CLAUDE.md/AGENTS.md) and where they collide: hooks deciding on the same tool in parallel, missing hook scripts, silenced hooks, MCP servers down or offering duplicate tools, skill name and budget problems, AGENTS.md that only Codex reads, broken @imports; what could not be checked is listed as not verified. Jobs via smart_search_result.",
|
|
330
331
|
"inputSchema": {
|
|
331
332
|
"type": "object", "properties": {
|
|
332
|
-
"action": {"type": "string", "enum": ["list", "status", "register", "preview", "index", "rebuild", "pause", "cancel", "resume", "watch", "scope", "remove", "relocate", "probe", "policy", "inspect", "graph", "usage", "storage", "pin_view", "cleanup_preview", "cleanup_commit", "compare", "search_limits", "integration", "install_hook_preview", "install_hook", "duplicates", "duplicates_dismiss", "duplicates_restore", "docs", "affected_tests"]},
|
|
333
|
+
"action": {"type": "string", "enum": ["list", "status", "register", "preview", "index", "rebuild", "pause", "cancel", "resume", "watch", "scope", "remove", "relocate", "probe", "policy", "inspect", "graph", "usage", "storage", "pin_view", "cleanup_preview", "cleanup_commit", "compare", "search_limits", "integration", "install_hook_preview", "install_hook", "duplicates", "duplicates_dismiss", "duplicates_restore", "docs", "affected_tests", "setup_audit"]},
|
|
333
334
|
"finding_id": {"type": "string"},
|
|
334
335
|
"reason": {"type": "string", "enum": ["false_positive", "intentional"]},
|
|
335
336
|
"note": {"type": "string"},
|
|
@@ -348,6 +349,7 @@ TOOLS = [
|
|
|
348
349
|
"left_view":{"type":"string"},"right_view":{"type":"string"},"relations":{"type":"boolean"},
|
|
349
350
|
"update_mode": {"type": "string", "enum": ["on_search", "eager"]},
|
|
350
351
|
"view_id": {"type": "string"}, "file_path": {"type": "string"},
|
|
352
|
+
"probe": {"type": "boolean", "default": True, "description": "setup_audit: start/call each MCP server (initialize + tools/list); false reads configs only"},
|
|
351
353
|
"base": {"type": "string", "default": "HEAD", "description": "affected_tests: git revision the changes are compared with (HEAD = uncommitted work; main or origin/main for a branch)"},
|
|
352
354
|
"symbol": {"type": "string", "description": "graph: function, Class.method, class name or path::name to get its callers, calls, tests and importers"},
|
|
353
355
|
"depth": {"type": "integer", "minimum": 1, "maximum": 4, "default": 3, "description": "graph with symbol: call hops searched for tests"},
|
|
@@ -1487,6 +1489,13 @@ def _project_summary(project):
|
|
|
1487
1489
|
def _project_action(arguments):
|
|
1488
1490
|
if not isinstance(arguments, dict):
|
|
1489
1491
|
raise ValueError("Pass the action in a JSON object.")
|
|
1492
|
+
if arguments.get("action") == "setup_audit":
|
|
1493
|
+
root = arguments.get("project_root")
|
|
1494
|
+
if not isinstance(root, str) or not os.path.isdir(root):
|
|
1495
|
+
raise ValueError("setup_audit needs project_root: the folder whose Claude Code and Codex setup to audit.")
|
|
1496
|
+
report = setup_audit.run(root, probe=arguments.get("probe", True) is not False)
|
|
1497
|
+
return {"summary": report["summary"], "snapshot_key": report["snapshot_key"],
|
|
1498
|
+
"report": setup_audit.markdown(report)}
|
|
1490
1499
|
arguments = _with_project_id(arguments)
|
|
1491
1500
|
action = arguments.get("action", "list")
|
|
1492
1501
|
if action == "probe":
|
|
@@ -3464,6 +3473,7 @@ _SETUP_SAVES = {
|
|
|
3464
3473
|
"/setup/gateway": setup_ui.handle_gateway_save,
|
|
3465
3474
|
"/setup/autostart": setup_ui.handle_autostart_action,
|
|
3466
3475
|
"/setup/contact": setup_ui.handle_contact_save,
|
|
3476
|
+
"/setup/audit": setup_ui.handle_audit,
|
|
3467
3477
|
}
|
|
3468
3478
|
|
|
3469
3479
|
|
|
@@ -3780,9 +3790,12 @@ def main():
|
|
|
3780
3790
|
_log(f"project monitor started; {recovered} interrupted job(s) recovered")
|
|
3781
3791
|
except Exception as exc:
|
|
3782
3792
|
_log(f"project monitor unavailable: {_project_error(exc)}")
|
|
3793
|
+
audit = setup_audit.Scheduler(lambda: [p["root"] for p in project_store.all_projects() if p.get("root")], _log)
|
|
3794
|
+
audit.start()
|
|
3783
3795
|
try:
|
|
3784
3796
|
server.serve_forever()
|
|
3785
3797
|
finally:
|
|
3798
|
+
audit.stop()
|
|
3786
3799
|
if _PROJECT_MONITOR:
|
|
3787
3800
|
_PROJECT_MONITOR.stop()
|
|
3788
3801
|
with _JOBS_LOCK:
|
package/version.py
CHANGED