@allansantos-dev/smart-tool 0.9.53 → 0.9.55
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +40 -0
- package/README.md +18 -17
- package/client_hooks.py +2 -1
- package/disk_cleanup.py +119 -0
- package/hooks/session_audit.py +2 -2
- package/index_scope.py +25 -4
- package/indexer.py +108 -31
- package/install.py +19 -0
- package/locales/pt-BR.json +3 -9
- package/package.json +1 -1
- package/project_store.py +40 -0
- package/setup_audit.py +21 -77
- package/setup_ui.py +5 -6
- package/smart_tool_daemon.py +65 -32
- package/tray.py +1 -7
- package/version.py +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,45 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.9.55 - beta
|
|
4
|
+
|
|
5
|
+
- The agent learns about the problems of the project it works on, and the user is not notified about them. The first
|
|
6
|
+
Smart Tool call of a session in a project (`smart_search`, `smart_scout` or `project_manage` with that
|
|
7
|
+
`project_root`) carries, after its answer, a short note: the project, its setup audit error and warning counts, the
|
|
8
|
+
checks that found them and the `project_manage action=setup_audit mode=last` call that shows each one with its fix.
|
|
9
|
+
Once per session and project (a subfolder counts as its project); a clean or missing report adds nothing; a stored
|
|
10
|
+
report that cannot be read is told with its cause; nothing is fixed on its own. The tray no longer raises a Windows
|
|
11
|
+
notification per project for new audit findings, and the session-start hook now carries only the update notices
|
|
12
|
+
and the redirect proposals, so the note is not given twice (`tests/test_session_audit.py`,
|
|
13
|
+
`tests/test_tray_notices.py`).
|
|
14
|
+
- A push to `main` no longer holds the only self-hosted runner waiting for npm to list a version the tag's Release
|
|
15
|
+
run has not published yet: the Pages workflow checks npm once on a push and skips publishing when the version is not
|
|
16
|
+
listed (the Release run republishes the page), and waits only when dispatched (`tests/test_pages.py`).
|
|
17
|
+
|
|
18
|
+
## 0.9.54 - beta
|
|
19
|
+
|
|
20
|
+
- A folder whose subfolders are projects is not indexed as one project. When it has no `.git` of its own and two or
|
|
21
|
+
more subfolders are registered projects or Git repositories, `smart_search`, `smart_scout` and `project_manage
|
|
22
|
+
action=register` fail naming those subfolders and asking for one of them as `project_root`; such a folder already
|
|
23
|
+
registered is unregistered and its index removed when the daemon starts. A project registered inside another is left
|
|
24
|
+
out of the outer one's scope, so the same files are no longer indexed twice; at its next update the outer index drops
|
|
25
|
+
their chunks from the current Git view and from the other views kept for that folder, compacting each view left
|
|
26
|
+
mostly free (`tests/test_disk_waste.py`).
|
|
27
|
+
- A folder of the indexing scope that was deleted no longer fails every indexing attempt of the project: it is left out
|
|
28
|
+
and the rest of the scope is indexed. A folder that cannot be listed fails naming it and the reason
|
|
29
|
+
(`tests/test_disk_waste.py`).
|
|
30
|
+
- A file whose date or size does not fit SQLite's 64-bit integer (a date past the year 2262) no longer fails every
|
|
31
|
+
indexing attempt with `OverflowError`: it is skipped by name with a `daemon.log` line saying how to index it, and the
|
|
32
|
+
rest of the folder is indexed. An index left with the journal of a write the daemon did not finish is rolled back on
|
|
33
|
+
the next read, with a log line, instead of failing as read-only on every attempt (`tests/test_disk_waste.py`).
|
|
34
|
+
- Disk left behind. A replaced index generation is no longer kept as `.previous.sqlite3` (nothing read it). An index
|
|
35
|
+
that lost at least 8 MB and a quarter of its pages to removed files is compacted (FTS optimize and `VACUUM`) after the
|
|
36
|
+
update. When the daemon starts it removes write temporaries orphaned by a crash, generations under construction
|
|
37
|
+
untouched for 7 days, the old `.previous` copies, the index files of folders that are not registered and saved graphs
|
|
38
|
+
of views that no longer exist; a registered folder it cannot list is logged and the rest of the cleanup goes on. The
|
|
39
|
+
installer removes `SmartTool.bak-*` and `.smart-tool-backup-*` copies next to the installation after the new version
|
|
40
|
+
is installed. `project_manage action=remove` reports the files and bytes it removed
|
|
41
|
+
(`tests/test_disk_waste.py`, `tests/test_graph_cache.py`).
|
|
42
|
+
|
|
3
43
|
## 0.9.53 - beta
|
|
4
44
|
|
|
5
45
|
- An update started in the first minutes after a release finds it. The update screen and the manual command in the
|
package/README.md
CHANGED
|
@@ -17,7 +17,7 @@ registered as the `playwright` server where this machine lets a browser be drive
|
|
|
17
17
|
An optional hook routes the agent's native Grep/Glob/Read/Bash/WebSearch/WebFetch calls to these tools when they
|
|
18
18
|
are cheaper, or just tells the agent what Smart Tool would do.
|
|
19
19
|
|
|
20
|
-
**Status:** 0.9.
|
|
20
|
+
**Status:** 0.9.55 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
|
|
21
21
|
|
|
22
22
|
## How it works
|
|
23
23
|
|
|
@@ -83,7 +83,7 @@ gateway such as Ollama, nothing about your code leaves the machine.
|
|
|
83
83
|
## Install
|
|
84
84
|
|
|
85
85
|
```powershell
|
|
86
|
-
npx @allansantos-dev/smart-tool@0.9.
|
|
86
|
+
npx @allansantos-dev/smart-tool@0.9.55 install
|
|
87
87
|
```
|
|
88
88
|
|
|
89
89
|
Updates install themselves, on by default: the tray checks npm when it starts (at logon) and every 24 h, and when there
|
|
@@ -253,21 +253,22 @@ Redirects) and Maintenance (Setup audit, Startup).
|
|
|
253
253
|
two minutes after it starts and then every 30 minutes (MCP servers shared by several projects are contacted once
|
|
254
254
|
and their result reused for 6 hours; Run now always contacts them again), and the setup screen's **Setup audit**
|
|
255
255
|
panel shows each project's last report: findings by severity, what is new, what was resolved, the tokens each MCP
|
|
256
|
-
server's tools cost and what was not verified.
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
256
|
+
server's tools cost and what was not verified. It never raises a Windows notification: a project's problems are
|
|
257
|
+
told to the agent working in it.
|
|
258
|
+
|
|
259
|
+
The agent hears about it without asking for it. The first Smart Tool call of a session in a project
|
|
260
|
+
(`smart_search`, `smart_scout` or `project_manage` with that `project_root`) carries, after its answer, a short
|
|
261
|
+
note for the agent only: the project, its error and warning counts, the checks that found them and the call that
|
|
262
|
+
shows each one with its fix (`project_manage action=setup_audit mode=last project_root="<project>"`), with the rule
|
|
263
|
+
that a fix is applied only when you ask. It comes once per session and project (a subfolder counts as its project),
|
|
264
|
+
a project with a clean report or no report adds nothing, and nothing is fixed on its own. `mode=last` returns the
|
|
265
|
+
stored report for the folder (or the nearest audited parent) without auditing again, and fails with what to do
|
|
266
|
+
when none is stored. Optionally, **Report at session start** on the Setup audit page (Claude Code or Codex) starts
|
|
267
|
+
every new session with the update notices and the redirect proposals waiting for a decision: it previews the exact
|
|
268
|
+
SessionStart entry, adds it to `~/.claude/settings.json` or `~/.codex/hooks.json` only after you confirm (backup
|
|
269
|
+
first), and **Turn off** removes only that entry. Codex runs a new hook only after you approve it in `/hooks`;
|
|
270
|
+
until then the page shows "waiting for your approval". It is off by default. The hook script
|
|
271
|
+
(`hooks/session_audit.py`) only asks the running service and prints nothing when the service is down.
|
|
271
272
|
4. **Local models** (optional): if the mcp-memory embedding sidecar is installed on the machine, its local embedder
|
|
272
273
|
can serve as a fallback (or preferred) embedding model.
|
|
273
274
|
5. **Web search providers** work without keys on their free public tiers; add your own key for more volume. Set your
|
package/client_hooks.py
CHANGED
|
@@ -372,7 +372,8 @@ def audit_state(client="claude"):
|
|
|
372
372
|
|
|
373
373
|
|
|
374
374
|
def audit_hook_installed(client):
|
|
375
|
-
"""The session-start
|
|
375
|
+
"""The session-start hook is on and running (Codex: approved in /hooks), so a search need not repeat the redirect
|
|
376
|
+
proposals it already carries.
|
|
376
377
|
An unreadable config file counts as off here; `audit_state` raises for it on the setup screen."""
|
|
377
378
|
if client not in AUDIT_CLIENTS:
|
|
378
379
|
return False
|
package/disk_cleanup.py
ADDED
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
"""Removes what Smart Tool left on disk that no registered project needs.
|
|
2
|
+
|
|
3
|
+
Runs once when the daemon starts, before the project monitor: the registry of projects (a folder of projects is
|
|
4
|
+
never one of them), write temporaries orphaned by a crash, generations under construction that stopped advancing,
|
|
5
|
+
the `.previous` copies older versions kept of a replaced generation, index files of projects no longer registered,
|
|
6
|
+
and saved code-graph analyses of index views that no longer exist. Source files are never touched."""
|
|
7
|
+
import os
|
|
8
|
+
import re
|
|
9
|
+
import time
|
|
10
|
+
|
|
11
|
+
import atomic_io
|
|
12
|
+
import indexer
|
|
13
|
+
import project_identity
|
|
14
|
+
import project_store
|
|
15
|
+
|
|
16
|
+
# A generation under construction resumes where it stopped; one untouched this long belongs to a model or scope that
|
|
17
|
+
# is no longer used, and a new one reuses the cached vectors anyway.
|
|
18
|
+
STALE_BUILD_S = 7 * 86400
|
|
19
|
+
_STATE_FILE = re.compile(r"([0-9a-f]{16})(?:\.v-[0-9a-f]{16})?(?:\.previous|\.building-[0-9a-f]+|\.vectors)?"
|
|
20
|
+
r"(?:\.sqlite3(?:-journal|-wal|-shm)?|\.scope\.json)")
|
|
21
|
+
_BUILDING = re.compile(r"[0-9a-f]{16}(?:\.v-[0-9a-f]{16})?\.building-[0-9a-f]+\.sqlite3")
|
|
22
|
+
_PREVIOUS = re.compile(r"[0-9a-f]{16}(?:\.v-[0-9a-f]{16})?\.previous\.sqlite3")
|
|
23
|
+
_TEMPORARY = re.compile(r".+" + re.escape(atomic_io.TMP_SUFFIX) + r"[0-9a-f]{32}")
|
|
24
|
+
|
|
25
|
+
|
|
26
|
+
def _remove(path, report):
|
|
27
|
+
"""Deletes one state file and adds its name and size to report."""
|
|
28
|
+
size = os.path.getsize(path)
|
|
29
|
+
os.remove(project_identity.state_path(path, indexer.INDEX_DIR))
|
|
30
|
+
report["files"].append(os.path.basename(path))
|
|
31
|
+
report["bytes"] += size
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
def remove_project_files(root, report=None):
|
|
35
|
+
"""Deletes every index, vector cache, scope and saved graph of the project at root (all Git views, the current and
|
|
36
|
+
legacy ids); returns the report {files, bytes}. Raises RuntimeError when another process holds one open."""
|
|
37
|
+
report = report if report is not None else {"files": [], "bytes": 0}
|
|
38
|
+
keys = {project_identity.project_id(root), *project_identity.legacy_ids(root)}
|
|
39
|
+
for name in sorted(os.listdir(indexer.INDEX_DIR)):
|
|
40
|
+
match = _STATE_FILE.fullmatch(name)
|
|
41
|
+
if not match or match.group(1) not in keys:
|
|
42
|
+
continue
|
|
43
|
+
path = os.path.join(indexer.INDEX_DIR, name)
|
|
44
|
+
try:
|
|
45
|
+
_remove(path, report)
|
|
46
|
+
except PermissionError as exc:
|
|
47
|
+
raise RuntimeError(f"{name} is open in another process. Close the external reader and try again.") from exc
|
|
48
|
+
graph = indexer.graph_cache_path(path)
|
|
49
|
+
if name.endswith(".sqlite3") and os.path.isfile(graph):
|
|
50
|
+
report["bytes"] += os.path.getsize(graph)
|
|
51
|
+
os.remove(graph)
|
|
52
|
+
indexer.invalidate_cache(root)
|
|
53
|
+
return report
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
def sweep(extra_roots, log=None):
|
|
57
|
+
"""Removes the waste described in the module docstring and returns {containers, files, bytes, errors}.
|
|
58
|
+
|
|
59
|
+
extra_roots are folders that own a vector cache without being projects (the web_fetch cache). A file that
|
|
60
|
+
cannot be removed, or a registered folder that cannot be listed, is reported in errors and logged, and the sweep
|
|
61
|
+
goes on with the rest: the daemon starts its project monitor right after, so the sweep must not stop it."""
|
|
62
|
+
log = log or (lambda message: None)
|
|
63
|
+
report = {"containers": [], "files": [], "bytes": 0, "errors": []}
|
|
64
|
+
for project in project_store.all_projects():
|
|
65
|
+
if not os.path.isdir(project["root"]):
|
|
66
|
+
continue
|
|
67
|
+
try:
|
|
68
|
+
refused = project_store.container_error(project["root"])
|
|
69
|
+
except OSError as exc:
|
|
70
|
+
report["errors"].append(f"{project['root']}: {exc}")
|
|
71
|
+
log(f"disk cleanup: {project['root']} not checked as a folder of projects: {exc}")
|
|
72
|
+
continue
|
|
73
|
+
if refused:
|
|
74
|
+
try:
|
|
75
|
+
remove_project_files(project["root"], report)
|
|
76
|
+
except RuntimeError as exc:
|
|
77
|
+
report["errors"].append(str(exc))
|
|
78
|
+
log(f"disk cleanup: {project['root']} kept: {exc}")
|
|
79
|
+
continue
|
|
80
|
+
project_store.forget(project["id"])
|
|
81
|
+
report["containers"].append(project["root"])
|
|
82
|
+
log(f"disk cleanup: unregistered {refused}")
|
|
83
|
+
owners = [p["root"] for p in project_store.all_projects()] + list(extra_roots)
|
|
84
|
+
keep = {key for root in owners for key in (project_identity.project_id(root), *project_identity.legacy_ids(root))}
|
|
85
|
+
now = time.time()
|
|
86
|
+
for name in sorted(os.listdir(indexer.INDEX_DIR)):
|
|
87
|
+
path = os.path.join(indexer.INDEX_DIR, name)
|
|
88
|
+
match = _STATE_FILE.fullmatch(name)
|
|
89
|
+
try:
|
|
90
|
+
if not os.path.isfile(path):
|
|
91
|
+
continue
|
|
92
|
+
age = now - os.path.getmtime(path)
|
|
93
|
+
if _TEMPORARY.fullmatch(name):
|
|
94
|
+
waste = age > atomic_io.ORPHAN_AGE_S
|
|
95
|
+
elif match:
|
|
96
|
+
waste = (match.group(1) not in keep or bool(_PREVIOUS.fullmatch(name)) or
|
|
97
|
+
bool(_BUILDING.fullmatch(name)) and age > STALE_BUILD_S)
|
|
98
|
+
else:
|
|
99
|
+
waste = False
|
|
100
|
+
if waste:
|
|
101
|
+
_remove(path, report)
|
|
102
|
+
except OSError as exc:
|
|
103
|
+
report["errors"].append(f"{name}: {exc}")
|
|
104
|
+
graphs = os.path.join(indexer.INDEX_DIR, "graph-cache")
|
|
105
|
+
if os.path.isdir(graphs):
|
|
106
|
+
for name in sorted(os.listdir(graphs)):
|
|
107
|
+
view = os.path.join(indexer.INDEX_DIR, name.removesuffix(".json.gz") + ".sqlite3")
|
|
108
|
+
if name.endswith(".json.gz") and not os.path.isfile(view):
|
|
109
|
+
path = os.path.join(graphs, name)
|
|
110
|
+
try:
|
|
111
|
+
report["bytes"] += os.path.getsize(path)
|
|
112
|
+
os.remove(path)
|
|
113
|
+
report["files"].append("graph-cache/" + name)
|
|
114
|
+
except OSError as exc:
|
|
115
|
+
report["errors"].append(f"graph-cache/{name}: {exc}")
|
|
116
|
+
if report["files"] or report["errors"]:
|
|
117
|
+
log(f"disk cleanup: removed {len(report['files'])} file(s), {report['bytes']} bytes; "
|
|
118
|
+
f"{len(report['errors'])} error(s): {'; '.join(report['errors'][:5])}")
|
|
119
|
+
return report
|
package/hooks/session_audit.py
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
#!/usr/bin/env python3
|
|
2
2
|
"""SessionStart hook of Claude Code and Codex (opt-in, from the setup screen): forwards the payload to the Smart Tool daemon
|
|
3
|
-
(POST /hooks/session-audit) and prints its answer
|
|
4
|
-
context. Never starts the daemon or an audit: daemon down or any error means empty output, so a session start is
|
|
3
|
+
(POST /hooks/session-audit) and prints its answer: the update notices and the redirect proposals waiting for a decision
|
|
4
|
+
as additional context. Never starts the daemon or an audit: daemon down or any error means empty output, so a session start is
|
|
5
5
|
never delayed or blocked.
|
|
6
6
|
"""
|
|
7
7
|
import json
|
package/index_scope.py
CHANGED
|
@@ -16,6 +16,7 @@ import time
|
|
|
16
16
|
|
|
17
17
|
import atomic_io
|
|
18
18
|
import project_identity
|
|
19
|
+
import project_store
|
|
19
20
|
import index_views
|
|
20
21
|
|
|
21
22
|
import model_client
|
|
@@ -532,6 +533,18 @@ def _exclude_matcher(exclude):
|
|
|
532
533
|
return matches
|
|
533
534
|
|
|
534
535
|
|
|
536
|
+
def _nested_matcher(root):
|
|
537
|
+
"""Predicate "is this inside a registered project nested under root?", by exact path prefix: that project is
|
|
538
|
+
indexed on its own, so the folder holding it leaves it out. A folder elsewhere with the same name stays in."""
|
|
539
|
+
nested = tuple(path.lower() for path in project_store.nested_projects(root)) if root else ()
|
|
540
|
+
|
|
541
|
+
def inside(rel_path):
|
|
542
|
+
target = rel_path.replace("\\", "/").strip("/").lower()
|
|
543
|
+
return any(target == path or target.startswith(path + "/") for path in nested)
|
|
544
|
+
|
|
545
|
+
return inside
|
|
546
|
+
|
|
547
|
+
|
|
535
548
|
def scope_filter(scope, root=None):
|
|
536
549
|
"""Predicate `keep(rel_path)`: whether resolve_included_files would list rel_path (a file) or walk it (a folder) —
|
|
537
550
|
under an include base, outside the exclusions, outside hidden folders. Built once so a burst of file events does not
|
|
@@ -540,11 +553,12 @@ def scope_filter(scope, root=None):
|
|
|
540
553
|
matcher = _exclude_matcher(set(scope.get("exclude") or []) | set(scope.get("user_exclude") or []) |
|
|
541
554
|
set(mechanical_excludes(root) if root else ()) | _ALWAYS_EXCLUDE | _ALWAYS_EXCLUDE_FILES)
|
|
542
555
|
whole = "" in bases
|
|
556
|
+
nested = _nested_matcher(root)
|
|
543
557
|
|
|
544
558
|
def keep(rel_path):
|
|
545
559
|
rel_path = rel_path.replace("\\", "/")
|
|
546
560
|
parts = rel_path.split("/")
|
|
547
|
-
if matcher(rel_path) or is_hidden(rel_path):
|
|
561
|
+
if matcher(rel_path) or is_hidden(rel_path) or nested(rel_path):
|
|
548
562
|
return False
|
|
549
563
|
if whole or len(parts) == 1:
|
|
550
564
|
return True
|
|
@@ -563,11 +577,16 @@ def resolve_included_files(root, scope, cancel_check=None):
|
|
|
563
577
|
if not valid_scope(scope):
|
|
564
578
|
raise ValueError("Invalid scope; request a new project analysis.")
|
|
565
579
|
include, _dropped = normalize_include(scope.get("include"))
|
|
566
|
-
|
|
567
|
-
|
|
580
|
+
excluded = _exclude_matcher(set(scope.get("exclude") or []) | set(scope.get("user_exclude") or []) |
|
|
581
|
+
set(mechanical_excludes(root)) | _ALWAYS_EXCLUDE | _ALWAYS_EXCLUDE_FILES)
|
|
582
|
+
nested = _nested_matcher(root)
|
|
583
|
+
|
|
584
|
+
def matcher(rel_path):
|
|
585
|
+
return excluded(rel_path) or nested(rel_path)
|
|
586
|
+
|
|
568
587
|
files = []
|
|
569
588
|
def walk_error(error):
|
|
570
|
-
raise OSError("Could not list
|
|
589
|
+
raise OSError(f"Could not list {error.filename} ({error.strerror}); try again when it is accessible.") from error
|
|
571
590
|
if "" not in include:
|
|
572
591
|
files.extend(_root_level_files(root, matcher))
|
|
573
592
|
for base in include:
|
|
@@ -577,6 +596,8 @@ def resolve_included_files(root, scope, cancel_check=None):
|
|
|
577
596
|
# projeto e mandar conteúdo de fora pro endpoint de embeddings.
|
|
578
597
|
if not _within_root(root, base_full):
|
|
579
598
|
continue
|
|
599
|
+
if not os.path.lexists(base_full):
|
|
600
|
+
continue
|
|
580
601
|
if os.path.isfile(base_full):
|
|
581
602
|
rel = os.path.relpath(base_full, root).replace(os.sep, "/")
|
|
582
603
|
if not matcher(rel) and not is_hidden(rel):
|
package/indexer.py
CHANGED
|
@@ -14,6 +14,7 @@ import os
|
|
|
14
14
|
import paths
|
|
15
15
|
import re
|
|
16
16
|
import sqlite3
|
|
17
|
+
import sys
|
|
17
18
|
import threading
|
|
18
19
|
import time
|
|
19
20
|
import uuid
|
|
@@ -21,6 +22,8 @@ from pathlib import Path
|
|
|
21
22
|
|
|
22
23
|
import atomic_io
|
|
23
24
|
import project_identity
|
|
25
|
+
import index_scope
|
|
26
|
+
import project_store
|
|
24
27
|
import index_views
|
|
25
28
|
import embedding_cache
|
|
26
29
|
import document_text
|
|
@@ -87,6 +90,23 @@ SKIP_GRAPHIC = "large svg graphic"
|
|
|
87
90
|
SKIP_BINARY = "binary"
|
|
88
91
|
SKIP_MINIFIED = "minified"
|
|
89
92
|
SKIP_UNREADABLE = "unreadable"
|
|
93
|
+
SKIP_OUT_OF_RANGE = "date or size outside what the index stores"
|
|
94
|
+
# SQLite INTEGER is a signed 64-bit value: a file dated past 2262 (or before 1677) has an st_mtime_ns beyond it.
|
|
95
|
+
_SQLITE_INTEGER = range(-2**63, 2**63)
|
|
96
|
+
# A removal that leaves this much free space in an index file is followed by VACUUM: with secure_delete the freed
|
|
97
|
+
# pages are zeroed but the file keeps its size until compacted.
|
|
98
|
+
COMPACT_MIN_FREE_BYTES = 8 * 1024 * 1024
|
|
99
|
+
COMPACT_MIN_FREE_SHARE = 0.25
|
|
100
|
+
# Where notices about a skipped file or a recovered index go; the daemon points this at daemon.log.
|
|
101
|
+
NOTICE = None
|
|
102
|
+
|
|
103
|
+
|
|
104
|
+
def _notice(message):
|
|
105
|
+
"""Sends one diagnostic line to NOTICE (daemon.log when the daemon runs), or to stderr without a daemon."""
|
|
106
|
+
if NOTICE is None:
|
|
107
|
+
print(message, file=sys.stderr)
|
|
108
|
+
else:
|
|
109
|
+
NOTICE(message)
|
|
90
110
|
|
|
91
111
|
|
|
92
112
|
LEXICAL_MODEL = "lexical"
|
|
@@ -207,20 +227,48 @@ def existing_db_path(root):
|
|
|
207
227
|
|
|
208
228
|
|
|
209
229
|
def _readonly(path):
|
|
230
|
+
"""Read-only connection to an index. A rollback journal left by a crashed writer (a hot journal) cannot be rolled
|
|
231
|
+
back by a read-only connection, which then fails every read with "attempt to write a readonly database": the
|
|
232
|
+
journal is rolled back once through a read-write connection, logged, and the read-only open is retried."""
|
|
233
|
+
conn = sqlite3.connect(Path(path).absolute().as_uri() + "?mode=ro", uri=True, timeout=10)
|
|
234
|
+
try:
|
|
235
|
+
conn.execute("PRAGMA schema_version").fetchone()
|
|
236
|
+
return conn
|
|
237
|
+
except sqlite3.OperationalError as exc:
|
|
238
|
+
conn.close()
|
|
239
|
+
journal = str(path) + "-journal"
|
|
240
|
+
if "readonly" not in str(exc) or not os.path.isfile(journal):
|
|
241
|
+
raise
|
|
242
|
+
size = os.path.getsize(journal)
|
|
243
|
+
writer = sqlite3.connect(path, timeout=10)
|
|
244
|
+
try:
|
|
245
|
+
writer.execute("PRAGMA schema_version").fetchone()
|
|
246
|
+
finally:
|
|
247
|
+
writer.close()
|
|
248
|
+
_notice(f"index {os.path.basename(str(path))}: rolled back the {size}-byte journal a crashed write left behind")
|
|
210
249
|
return sqlite3.connect(Path(path).absolute().as_uri() + "?mode=ro", uri=True, timeout=10)
|
|
211
250
|
|
|
212
251
|
|
|
213
252
|
def _backup(source, target):
|
|
253
|
+
"""Consistent copy of the index at source into target, through a temporary file removed if the copy fails."""
|
|
214
254
|
target = project_identity.state_path(target, INDEX_DIR)
|
|
215
255
|
temp = atomic_io.tmp_path_for(target)
|
|
216
|
-
src, dst = _readonly(source), sqlite3.connect(temp)
|
|
217
256
|
try:
|
|
218
|
-
src
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
257
|
+
src = _readonly(source)
|
|
258
|
+
try:
|
|
259
|
+
dst = sqlite3.connect(temp)
|
|
260
|
+
try:
|
|
261
|
+
src.backup(dst)
|
|
262
|
+
dst.commit()
|
|
263
|
+
finally:
|
|
264
|
+
dst.close()
|
|
265
|
+
finally:
|
|
266
|
+
src.close()
|
|
267
|
+
atomic_io.atomic_replace(temp, target)
|
|
268
|
+
except BaseException:
|
|
269
|
+
if os.path.exists(temp):
|
|
270
|
+
os.remove(temp)
|
|
271
|
+
raise
|
|
224
272
|
|
|
225
273
|
|
|
226
274
|
def migrate_legacy(root):
|
|
@@ -513,6 +561,8 @@ def _stat_candidate(root, rel):
|
|
|
513
561
|
return None, SKIP_TOO_LARGE
|
|
514
562
|
if name.endswith(".svg") and stat.st_size > MAX_SVG_INDEXABLE_BYTES:
|
|
515
563
|
return None, SKIP_GRAPHIC
|
|
564
|
+
if stat.st_mtime_ns not in _SQLITE_INTEGER or stat.st_size not in _SQLITE_INTEGER:
|
|
565
|
+
return None, SKIP_OUT_OF_RANGE
|
|
516
566
|
return (stat.st_mtime_ns, stat.st_size), None
|
|
517
567
|
|
|
518
568
|
|
|
@@ -620,6 +670,10 @@ def _reindex(root, included_paths, embed_batch, chunk_lines=CHUNK_LINES, on_prog
|
|
|
620
670
|
stat, reason = _stat_candidate(root, rel)
|
|
621
671
|
if stat is None:
|
|
622
672
|
stats["skipped"][reason] = stats["skipped"].get(reason, 0) + 1
|
|
673
|
+
if reason == SKIP_OUT_OF_RANGE:
|
|
674
|
+
stats.setdefault("out_of_range", []).append(rel)
|
|
675
|
+
_notice(f"index {root}: skipped {rel}: its date or size is outside the 64-bit range SQLite stores; "
|
|
676
|
+
"fix the file's date to index it")
|
|
623
677
|
continue
|
|
624
678
|
observed[rel] = stat
|
|
625
679
|
force_file = any(rel.casefold() == p or rel.casefold().startswith(p + "/") for p in forced)
|
|
@@ -724,6 +778,8 @@ def _reindex(root, included_paths, embed_batch, chunk_lines=CHUNK_LINES, on_prog
|
|
|
724
778
|
if building or stats["added"] or stats["changed"] or stats["removed"]:
|
|
725
779
|
_set_metadata(conn, {"updated_at": time.time()})
|
|
726
780
|
conn.commit()
|
|
781
|
+
if stats["removed"] and not building:
|
|
782
|
+
_compact(conn)
|
|
727
783
|
stats.update(generation=generation, dimensions=dimensions,
|
|
728
784
|
embedded_chunks=cache.misses, reused_chunks=cache.hits,
|
|
729
785
|
embedded_characters=cache.characters, view=index_views.public(view),
|
|
@@ -738,16 +794,59 @@ def _reindex(root, included_paths, embed_batch, chunk_lines=CHUNK_LINES, on_prog
|
|
|
738
794
|
cache.close()
|
|
739
795
|
check()
|
|
740
796
|
if building and not stats["failed"]:
|
|
741
|
-
if os.path.isfile(target):
|
|
742
|
-
_backup(target, target[:-8] + ".previous.sqlite3")
|
|
743
797
|
atomic_io.atomic_replace(work_path, project_identity.state_path(target, INDEX_DIR))
|
|
744
798
|
stats["ready"] = not bool(stats["failed"])
|
|
745
799
|
stats["path"] = target if stats["ready"] else work_path
|
|
746
800
|
if building or total or stats["removed"] or full_check:
|
|
747
801
|
invalidate_cache(root)
|
|
802
|
+
_trim_other_views(root, target)
|
|
748
803
|
return stats
|
|
749
804
|
|
|
750
805
|
|
|
806
|
+
def _trim_other_views(root, target):
|
|
807
|
+
"""Drops, from the project's other Git views, the files of projects registered inside root (their own index holds
|
|
808
|
+
them; target already left them out through its scope), then compacts each view left mostly free. A view another
|
|
809
|
+
process holds is logged and trimmed at the next update. Runs on every update, changed files or not, so views written
|
|
810
|
+
before the inner project was registered are trimmed too; reading the paths is an index scan per view."""
|
|
811
|
+
if not project_store.nested_projects(root):
|
|
812
|
+
return
|
|
813
|
+
nested = index_scope._nested_matcher(root)
|
|
814
|
+
key = project_identity.project_id(root)
|
|
815
|
+
for name in sorted(os.listdir(INDEX_DIR)):
|
|
816
|
+
path = os.path.join(INDEX_DIR, name)
|
|
817
|
+
if (not name.startswith(key + ".v-") or not name.endswith(".sqlite3") or ".building-" in name
|
|
818
|
+
or ".previous." in name or os.path.normcase(path) == os.path.normcase(target)):
|
|
819
|
+
continue
|
|
820
|
+
conn = sqlite3.connect(path, timeout=10)
|
|
821
|
+
try:
|
|
822
|
+
conn.execute("PRAGMA secure_delete=ON")
|
|
823
|
+
states = conn.execute("SELECT 1 FROM sqlite_master WHERE name='file_state'").fetchone()
|
|
824
|
+
for (rel,) in conn.execute("SELECT path FROM manifest UNION SELECT DISTINCT path FROM chunks").fetchall():
|
|
825
|
+
if nested(rel):
|
|
826
|
+
remove_file(conn, rel)
|
|
827
|
+
if states:
|
|
828
|
+
conn.execute("DELETE FROM file_state WHERE path = ?", (rel,))
|
|
829
|
+
conn.commit()
|
|
830
|
+
_compact(conn)
|
|
831
|
+
except sqlite3.OperationalError as exc:
|
|
832
|
+
_notice(f"index {name}: other view not trimmed now ({exc}); it is trimmed at the next update")
|
|
833
|
+
finally:
|
|
834
|
+
conn.close()
|
|
835
|
+
|
|
836
|
+
|
|
837
|
+
def _compact(conn):
|
|
838
|
+
"""VACUUM when the removals left at least COMPACT_MIN_FREE_BYTES and COMPACT_MIN_FREE_SHARE of the file free."""
|
|
839
|
+
page = conn.execute("PRAGMA page_size").fetchone()[0]
|
|
840
|
+
free = conn.execute("PRAGMA freelist_count").fetchone()[0] * page
|
|
841
|
+
total = conn.execute("PRAGMA page_count").fetchone()[0] * page
|
|
842
|
+
if free >= COMPACT_MIN_FREE_BYTES and free >= COMPACT_MIN_FREE_SHARE * total:
|
|
843
|
+
if conn.execute("SELECT 1 FROM sqlite_master WHERE name='chunks_fts'").fetchone():
|
|
844
|
+
# FTS5 records a deletion as a tombstone; merging the segments drops the deleted text for good.
|
|
845
|
+
conn.execute("INSERT INTO chunks_fts(chunks_fts) VALUES('optimize')")
|
|
846
|
+
conn.commit()
|
|
847
|
+
conn.execute("VACUUM")
|
|
848
|
+
|
|
849
|
+
|
|
751
850
|
def _vector_rows(root, path):
|
|
752
851
|
stat = os.stat(path)
|
|
753
852
|
key = (project_identity.canonical_root(root), path, stat.st_mtime_ns, stat.st_size)
|
|
@@ -967,28 +1066,6 @@ def graph_cache_path(view_path):
|
|
|
967
1066
|
os.path.basename(view_path).removesuffix(".sqlite3") + ".json.gz")
|
|
968
1067
|
|
|
969
1068
|
|
|
970
|
-
def remove_index(root):
|
|
971
|
-
"""Remove somente arquivos de índice da raiz conhecida, nunca arquivos do projeto."""
|
|
972
|
-
import glob
|
|
973
|
-
removed = []
|
|
974
|
-
for key in set([project_identity.project_id(root), *project_identity.legacy_ids(root)]):
|
|
975
|
-
pattern = re.compile(re.escape(key) + r"(?:\.v-[a-f0-9]{16})?(?:\.previous|\.building-[a-f0-9]+|\.vectors)?\.sqlite3$")
|
|
976
|
-
for path in glob.glob(os.path.join(INDEX_DIR, key + "*.sqlite3")):
|
|
977
|
-
if pattern.fullmatch(os.path.basename(path)):
|
|
978
|
-
safe = project_identity.state_path(path, INDEX_DIR)
|
|
979
|
-
try:
|
|
980
|
-
os.remove(safe)
|
|
981
|
-
except PermissionError as exc:
|
|
982
|
-
raise RuntimeError("The index is open in another process. Close the external reader and try removing it again.") from exc
|
|
983
|
-
removed.append(safe)
|
|
984
|
-
try:
|
|
985
|
-
os.remove(graph_cache_path(safe))
|
|
986
|
-
except FileNotFoundError:
|
|
987
|
-
pass
|
|
988
|
-
invalidate_cache(root)
|
|
989
|
-
return removed
|
|
990
|
-
|
|
991
|
-
|
|
992
1069
|
def copy_project_indexes(root, destination):
|
|
993
1070
|
"""Copia todas as visões e cache ao reassociar uma raiz, preservando a origem."""
|
|
994
1071
|
import glob
|
package/install.py
CHANGED
|
@@ -382,9 +382,28 @@ def main():
|
|
|
382
382
|
mark_updating(False)
|
|
383
383
|
if not args.no_start:
|
|
384
384
|
enable_browser_control(target, python)
|
|
385
|
+
for folder in prune_backups(target):
|
|
386
|
+
print(f"Removed the old copy {folder}.")
|
|
385
387
|
print("Smart Tool ready." if not args.no_start else "Files installed; daemon not restarted (--no-start).")
|
|
386
388
|
|
|
387
389
|
|
|
390
|
+
def prune_backups(target):
|
|
391
|
+
"""After a successful install, deletes the copies of older versions beside target: `.smart-tool-backup-*` (kept by
|
|
392
|
+
an incomplete rollback) and `<target name>.bak-*` (copies made by hand before an update). Returns the removed
|
|
393
|
+
paths; a copy that cannot be removed is reported on stderr and kept."""
|
|
394
|
+
removed = []
|
|
395
|
+
for folder in sorted(target.parent.iterdir()):
|
|
396
|
+
if not folder.is_dir() or not (folder.name.startswith(".smart-tool-backup-") or
|
|
397
|
+
folder.name.startswith(target.name + ".bak-")):
|
|
398
|
+
continue
|
|
399
|
+
try:
|
|
400
|
+
shutil.rmtree(folder)
|
|
401
|
+
removed.append(str(folder))
|
|
402
|
+
except OSError as exc:
|
|
403
|
+
print(f"Could not remove the old copy {folder}: {exc}", file=sys.stderr)
|
|
404
|
+
return removed
|
|
405
|
+
|
|
406
|
+
|
|
388
407
|
def _update(args, target, files, existed, python, pythonw):
|
|
389
408
|
"""Replaces the installed files, dependencies and daemon, restoring the previous version on any failure."""
|
|
390
409
|
stop_processes(target)
|
package/locales/pt-BR.json
CHANGED
|
@@ -77,8 +77,8 @@
|
|
|
77
77
|
"Client": "Cliente",
|
|
78
78
|
"Claude Code": "Claude Code",
|
|
79
79
|
"Codex": "Codex",
|
|
80
|
-
"Off by default. When on, every new session starts with
|
|
81
|
-
", written after a backup, and turning it off removes only that entry. Codex runs a new hook only after you approve it in /hooks.
|
|
80
|
+
"Off by default. When on, every new session starts with the Smart Tool update notices (also shown to you) and the redirect proposals waiting for a decision. It adds a SessionStart entry to": "Desligado por padrão. Quando ligado, cada nova sessão começa com os avisos de atualização do Smart Tool (mostrados também a você) e as propostas de redirecionamento que esperam uma decisão. Ela adiciona uma entrada SessionStart em",
|
|
81
|
+
", written after a backup, and turning it off removes only that entry. Codex runs a new hook only after you approve it in /hooks. Either way, the agent learns about a project's setup audit on its first Smart Tool call there, once per session.": ", gravada depois de um backup; ao desligar, só essa entrada é removida. O Codex só executa um hook novo depois que você o aprova em /hooks. Com ou sem ele, o agente fica sabendo da auditoria de um projeto na primeira chamada ao Smart Tool ali, uma vez por sessão.",
|
|
82
82
|
"Turn on": "Ligar",
|
|
83
83
|
"Turn off": "Desligar",
|
|
84
84
|
"Write it": "Gravar",
|
|
@@ -679,15 +679,12 @@
|
|
|
679
679
|
"{0} subagent(s) list tools but none of mcp__{1}__*: {2}. Add mcp__{3}__smart_search and mcp__{4}__smart_search_result to their `tools`, or drop the `tools` line to inherit every tool.": "{0} subagente(s) listam ferramentas, mas nenhuma de mcp__{1}__*: {2}. Adicione mcp__{3}__smart_search e mcp__{4}__smart_search_result ao `tools` deles, ou remova a linha `tools` para herdar todas as ferramentas.",
|
|
680
680
|
"MCP servers and HTTP hook endpoints were not contacted (probe off)": "Os servidores MCP e os endpoints de hook HTTP não foram contatados (sondagem desligada)",
|
|
681
681
|
"{0} is not a setup audit index: fix or delete it.": "{0} não é um índice de auditoria da configuração: corrija ou apague.",
|
|
682
|
-
"Setup audit: 1 new problem in the agent configuration": "Auditoria da configuração: 1 problema novo na configuração dos agentes",
|
|
683
|
-
"Setup audit: {count} new problems in the agent configuration": "Auditoria da configuração: {count} problemas novos na configuração dos agentes",
|
|
684
682
|
"setup audit: {0} project(s), {1} error finding(s)": "auditoria da configuração: {0} projeto(s), {1} erro(s) encontrado(s)",
|
|
685
683
|
"Not verified:": "Não verificado:",
|
|
686
684
|
"Names and descriptions add up to {0} characters, over the ~{1} the skill list gets (Claude: 1% of a 200k context; Codex: 2% or 8,000 characters): some skills are left out of what the model sees.": "Os nomes e as descrições somam {0} caracteres, acima dos ~{1} que a lista de skills recebe (Claude: 1% de um contexto de 200k; Codex: 2% ou 8.000 caracteres): algumas skills ficam de fora do que o modelo vê.",
|
|
687
685
|
"Codex reads AGENTS.md; Claude Code reads only CLAUDE.md (and what it imports with @AGENTS.md), so the two agents get different instructions here.": "O Codex lê o AGENTS.md; o Claude Code lê apenas o CLAUDE.md (e o que ele importa com @AGENTS.md), então os dois agentes recebem instruções diferentes aqui.",
|
|
688
686
|
"Project AGENTS.md chain is {0} bytes; Codex reads {1} (project_doc_max_bytes) and drops the rest.": "A cadeia de AGENTS.md do projeto tem {0} bytes; o Codex lê {1} (project_doc_max_bytes) e descarta o resto.",
|
|
689
687
|
"does not start or answer": "não inicia nem responde",
|
|
690
|
-
"and {count} more": "e mais {count}",
|
|
691
688
|
"Since the last run: {0} new, {1} resolved": "Desde a última execução: {0} novos, {1} resolvidos",
|
|
692
689
|
"{0}: invalid JSON ({1})": "{0}: JSON inválido ({1})",
|
|
693
690
|
"{0}: invalid TOML ({1})": "{0}: TOML inválido ({1})",
|
|
@@ -728,7 +725,6 @@
|
|
|
728
725
|
"Don't show again": "Não mostrar de novo",
|
|
729
726
|
"Switch model": "Trocar de modelo",
|
|
730
727
|
"Mute 7 days": "Silenciar por 7 dias",
|
|
731
|
-
"Open the audit": "Abrir a auditoria",
|
|
732
728
|
"Projects and indexing…": "Projetos e indexação…",
|
|
733
729
|
"Index a folder…": "Indexar uma pasta…",
|
|
734
730
|
"Open settings": "Abrir configurações",
|
|
@@ -875,7 +871,6 @@
|
|
|
875
871
|
"{0}{1} · {2} files": "{0}{1} · {2} arquivos",
|
|
876
872
|
"({0} sites)": "({0} sites)",
|
|
877
873
|
"{0} skills": "{0} skills",
|
|
878
|
-
", {0} new": ", {0} novos",
|
|
879
874
|
"{0} {1} (~{2} tokens)": "{0} {1} (~{2} tokens)",
|
|
880
875
|
"Checking the browsers…": "Verificando os navegadores…",
|
|
881
876
|
"View log": "Ver o log",
|
|
@@ -889,6 +884,7 @@
|
|
|
889
884
|
"{0} holds a setup audit of {1} from an older Smart Tool, whose findings have no fix: run project_manage action=setup_audit mode=run to replace it.": "{0} guarda uma auditoria da configuração de {1} feita por um Smart Tool mais antigo, cujos achados não trazem a correção: rode project_manage action=setup_audit mode=run para substituí-la."
|
|
890
885
|
},
|
|
891
886
|
"ignore": [
|
|
887
|
+
"and {0} more",
|
|
892
888
|
"saved msg",
|
|
893
889
|
"more row-more",
|
|
894
890
|
"callout info",
|
|
@@ -932,12 +928,10 @@
|
|
|
932
928
|
"remove the skill folders you do not use",
|
|
933
929
|
"Sign in: run /mcp in Claude Code and pick {0}.",
|
|
934
930
|
"Sign in: run `codex mcp login {0}`.",
|
|
935
|
-
"in the Claude Code and Codex configuration. Top items, each with its fix:",
|
|
936
931
|
"Read these hooks in {0}: when two of them rewrite or deny the same {1} call, narrow one matcher so they stop sharing it; when each does something different (one logs, one rewrites), nothing needs to change.",
|
|
937
932
|
"Bring names and descriptions under ~{0} characters: {1}, or shorten the longest descriptions ({2}).",
|
|
938
933
|
"Add a line `@AGENTS.md` to {0} (create the file with that line if it does not exist).",
|
|
939
934
|
"Shorten the project AGENTS.md files to under {0} bytes in total, or set project_doc_max_bytes = {1} (or more) in {2}.",
|
|
940
|
-
"Smart Tool setup audit of {0} ({1} min ago): {2} errors, {3} warnings",
|
|
941
935
|
"Rename the `name` in the frontmatter of all but one of these, or remove the copies you do not use: {0}.",
|
|
942
936
|
"Reinstall or update the {0} plugin, which is missing {1}.",
|
|
943
937
|
"Restore {0}, or delete this {1} hook from {2}.",
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@allansantos-dev/smart-tool",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.55",
|
|
4
4
|
"description": "Local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharper tools than their built-in search. Windows.",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"author": "Allan Santos",
|
package/project_store.py
CHANGED
|
@@ -38,10 +38,50 @@ def _db():
|
|
|
38
38
|
db.close()
|
|
39
39
|
|
|
40
40
|
|
|
41
|
+
CONTAINER_MIN_PROJECTS = 2
|
|
42
|
+
|
|
43
|
+
|
|
44
|
+
def project_subfolders(root):
|
|
45
|
+
"""Names of the immediate subfolders of root that are projects: registered, or holding a `.git` folder or file."""
|
|
46
|
+
registered = {project_identity.canonical_root(p["root"]) for p in all_projects()}
|
|
47
|
+
names = []
|
|
48
|
+
with os.scandir(root) as entries:
|
|
49
|
+
for entry in entries:
|
|
50
|
+
if entry.name.startswith(".") or not entry.is_dir():
|
|
51
|
+
continue
|
|
52
|
+
if (project_identity.canonical_root(entry.path) in registered or
|
|
53
|
+
os.path.exists(os.path.join(entry.path, ".git"))):
|
|
54
|
+
names.append(entry.name)
|
|
55
|
+
return sorted(names)
|
|
56
|
+
|
|
57
|
+
|
|
58
|
+
def container_error(root):
|
|
59
|
+
"""Why root cannot be one project, or None: a folder without its own `.git` whose subfolders are at least
|
|
60
|
+
CONTAINER_MIN_PROJECTS projects holds projects (each indexed on its own); indexing it would store them twice."""
|
|
61
|
+
if os.path.exists(os.path.join(root, ".git")):
|
|
62
|
+
return None
|
|
63
|
+
names = project_subfolders(root)
|
|
64
|
+
if len(names) < CONTAINER_MIN_PROJECTS:
|
|
65
|
+
return None
|
|
66
|
+
shown = ", ".join(names[:8]) + (", ..." if len(names) > 8 else "")
|
|
67
|
+
return (f"{root} is a folder of {len(names)} projects ({shown}), not one project: "
|
|
68
|
+
"pass the folder of one of them as project_root.")
|
|
69
|
+
|
|
70
|
+
|
|
71
|
+
def nested_projects(root):
|
|
72
|
+
"""Paths, relative to root with forward slashes, of the registered projects strictly inside root."""
|
|
73
|
+
base = project_identity.canonical_root(root)
|
|
74
|
+
inner = (project_identity.canonical_root(p["root"]) for p in all_projects())
|
|
75
|
+
return sorted(os.path.relpath(path, base).replace(os.sep, "/") for path in inner if path.startswith(base + os.sep))
|
|
76
|
+
|
|
77
|
+
|
|
41
78
|
def register(root, watch=None):
|
|
42
79
|
display = project_identity.display_root(root)
|
|
43
80
|
if not os.path.isdir(display):
|
|
44
81
|
raise ValueError("The project folder does not exist or is not accessible.")
|
|
82
|
+
refused = container_error(display)
|
|
83
|
+
if refused:
|
|
84
|
+
raise ValueError(refused)
|
|
45
85
|
key = project_identity.project_id(display)
|
|
46
86
|
with _LOCK, _db() as db:
|
|
47
87
|
row = db.execute("SELECT data FROM projects WHERE id=?", (key,)).fetchone()
|
package/setup_audit.py
CHANGED
|
@@ -31,7 +31,6 @@ import yaml
|
|
|
31
31
|
|
|
32
32
|
import atomic_io
|
|
33
33
|
import endpoint_sync
|
|
34
|
-
import i18n
|
|
35
34
|
import paths
|
|
36
35
|
|
|
37
36
|
REPORT_DIR = os.path.join(paths.DATA_DIR, "setup-audit")
|
|
@@ -43,9 +42,7 @@ PROBE_MAX_AGE_S = 6 * 3600
|
|
|
43
42
|
SKILL_DESCRIPTION_MAX = 1536
|
|
44
43
|
SKILL_BUDGET_CHARS = 8000
|
|
45
44
|
CODEX_PROJECT_DOC_MAX = 32 * 1024
|
|
46
|
-
|
|
47
|
-
NOTICE_ACTION_MAX = 360
|
|
48
|
-
REVIEW_FIRST = ("hook_overlap", "skill_budget")
|
|
45
|
+
NOTICE_CHECKS = 3
|
|
49
46
|
SMART_TOOL_SERVER = "smart-tool"
|
|
50
47
|
CLAUDE_TOOLS = ("Bash", "Read", "Grep", "Glob", "LS", "Edit", "MultiEdit", "Write", "NotebookEdit", "WebFetch",
|
|
51
48
|
"WebSearch", "Task")
|
|
@@ -865,39 +862,28 @@ def stored_for(path):
|
|
|
865
862
|
folder = parent
|
|
866
863
|
|
|
867
864
|
|
|
868
|
-
def
|
|
869
|
-
"""
|
|
870
|
-
|
|
871
|
-
|
|
872
|
-
|
|
873
|
-
for position, f in enumerate(report["findings"]) if f["severity"] in order]
|
|
874
|
-
return [row[-1] for row in sorted(ranked, key=lambda row: row[:4])[:count]]
|
|
875
|
-
|
|
876
|
-
|
|
877
|
-
def session_notice(path):
|
|
878
|
-
"""Context for an agent session about the stored audit of path (nothing is run): the project, the counts, the top
|
|
879
|
-
NOTICE_ITEMS findings with their fix and the exact call that lists all of them. "" when there is no stored report
|
|
880
|
-
or it holds no error or warning. Each fix changes the user's agent setup, so the agent is told to apply one when
|
|
881
|
-
the user agrees, never on its own. Stays far below the ~2,500 tokens Codex spills to disk."""
|
|
865
|
+
def agent_notice(path):
|
|
866
|
+
"""(project, text) of the short note an agent gets on its first Smart Tool call in path during a session: the
|
|
867
|
+
audited project, its error and warning counts, up to NOTICE_CHECKS of their checks (errors first) and the call
|
|
868
|
+
that shows each one with its fix (nothing is run). None when no stored report covers path or it holds no error or
|
|
869
|
+
warning. Each fix changes the user's agent setup, so the agent is told to apply one only when the user asks."""
|
|
882
870
|
report = stored_for(path)
|
|
883
871
|
if report is None:
|
|
884
|
-
return
|
|
885
|
-
|
|
886
|
-
if not
|
|
887
|
-
return
|
|
888
|
-
|
|
889
|
-
|
|
890
|
-
|
|
891
|
-
|
|
892
|
-
|
|
893
|
-
for number, finding in enumerate(top_findings(report), 1):
|
|
894
|
-
lines.append(f"{number}. [{finding['harness']}] {finding['severity']} {finding['check']}: "
|
|
895
|
-
f"{finding['subject'][:120]}. Fix: {finding['action'][:NOTICE_ACTION_MAX]}")
|
|
872
|
+
return None
|
|
873
|
+
errors, warnings = report["summary"]["error"], report["summary"]["warn"]
|
|
874
|
+
if not errors and not warnings:
|
|
875
|
+
return None
|
|
876
|
+
ranked = sorted((f for f in report["findings"] if f["severity"] in ("error", "warn")),
|
|
877
|
+
key=lambda f: f["severity"] != "error")
|
|
878
|
+
checks = list(dict.fromkeys(f["check"] for f in ranked))
|
|
879
|
+
named = ", ".join(checks[:NOTICE_CHECKS]) + (f" and {len(checks) - NOTICE_CHECKS} more"
|
|
880
|
+
if len(checks) > NOTICE_CHECKS else "")
|
|
896
881
|
root = report["project"].replace("\\", "/")
|
|
897
|
-
|
|
898
|
-
|
|
899
|
-
|
|
900
|
-
|
|
882
|
+
text = (f"Smart Tool: the project {root} has {errors} error{'' if errors == 1 else 's'} and {warnings} "
|
|
883
|
+
f"warning{'' if warnings == 1 else 's'} in its agent setup ({named}). To see each one with its fix, call "
|
|
884
|
+
f'project_manage action=setup_audit mode=last project_root="{root}". This note is for you; fix one only '
|
|
885
|
+
"when the user asks.")
|
|
886
|
+
return report["project"], text
|
|
901
887
|
|
|
902
888
|
|
|
903
889
|
def index():
|
|
@@ -970,53 +956,11 @@ def _save_locked(report):
|
|
|
970
956
|
key = index_key(report["project"])
|
|
971
957
|
entry = {"project": report["project"], "summary": report["summary"], "new": len(changes.get("new", [])),
|
|
972
958
|
"resolved": len(changes.get("resolved", [])), "checked_at": report["checked_at"],
|
|
973
|
-
"trigger": report["trigger"], "snapshot_key": report["snapshot_key"]
|
|
974
|
-
"pending": _pending_notice(data.get(key), report)}
|
|
959
|
+
"trigger": report["trigger"], "snapshot_key": report["snapshot_key"]}
|
|
975
960
|
data[key] = entry
|
|
976
961
|
_write_index(data)
|
|
977
962
|
|
|
978
963
|
|
|
979
|
-
def _pending_notice(previous, report):
|
|
980
|
-
"""The new errors and warnings of a scheduled run not yet told to the user, added to those a former run left
|
|
981
|
-
unannounced (by finding id). A manual run adds none: the user is looking at its result."""
|
|
982
|
-
pending = list((previous or {}).get("pending") or [])
|
|
983
|
-
if report["trigger"] == "schedule":
|
|
984
|
-
known = {item["id"] for item in pending}
|
|
985
|
-
pending += [{"id": f["id"], "severity": f["severity"], "check": f["check"], "subject": f["subject"]}
|
|
986
|
-
for f in report["findings"]
|
|
987
|
-
if f.get("new") and f["severity"] in ("error", "warn") and f["id"] not in known]
|
|
988
|
-
return pending
|
|
989
|
-
|
|
990
|
-
|
|
991
|
-
def notices_due():
|
|
992
|
-
"""[{project, errors, warnings, items}] for the projects whose scheduled audits found new errors or warnings since
|
|
993
|
-
the last call; marks them announced, so each finding is told once (the tray toast)."""
|
|
994
|
-
with _INDEX_LOCK:
|
|
995
|
-
data = index()
|
|
996
|
-
due = []
|
|
997
|
-
for entry in data.values():
|
|
998
|
-
pending = entry.get("pending") or []
|
|
999
|
-
if pending:
|
|
1000
|
-
due.append({"project": entry["project"], "errors": sum(i["severity"] == "error" for i in pending),
|
|
1001
|
-
"warnings": sum(i["severity"] == "warn" for i in pending), "items": pending})
|
|
1002
|
-
entry["pending"] = []
|
|
1003
|
-
if due:
|
|
1004
|
-
_write_index(data)
|
|
1005
|
-
return due
|
|
1006
|
-
|
|
1007
|
-
|
|
1008
|
-
def toast_text(row):
|
|
1009
|
-
"""(title, lines) of the toast for one `notices_due` row, in the interface language."""
|
|
1010
|
-
total = row["errors"] + row["warnings"]
|
|
1011
|
-
title = (i18n.t("Setup audit: 1 new problem in the agent configuration") if total == 1
|
|
1012
|
-
else i18n.t("Setup audit: {count} new problems in the agent configuration", count=total))
|
|
1013
|
-
lines = [os.path.basename(row["project"].rstrip("\\/")) or row["project"]]
|
|
1014
|
-
lines += [f"{item['severity']}: {item['check']} - {item['subject']}"[:120] for item in row["items"][:3]]
|
|
1015
|
-
if len(row["items"]) > 3:
|
|
1016
|
-
lines.append(i18n.t("and {count} more", count=len(row["items"]) - 3))
|
|
1017
|
-
return title, lines
|
|
1018
|
-
|
|
1019
|
-
|
|
1020
964
|
class Scheduler:
|
|
1021
965
|
"""Periodic mode: audits every registered project whose folder exists, FIRST_RUN_DELAY_S after start and then
|
|
1022
966
|
every SCHEDULE_S, one project at a time, reusing MCP probes younger than PROBE_MAX_AGE_S. projects() returns the
|
package/setup_ui.py
CHANGED
|
@@ -203,13 +203,12 @@ def handle_health_get():
|
|
|
203
203
|
|
|
204
204
|
|
|
205
205
|
def handle_notices():
|
|
206
|
-
"""What the tray should show now: the model offer (once a day until answered)
|
|
207
|
-
(once a day each)
|
|
208
|
-
|
|
206
|
+
"""What the tray should show now: the model offer (once a day until answered) and models over the failure limit
|
|
207
|
+
(once a day each). Fetching marks them announced, so each toast is shown once per reminder period. A project's
|
|
208
|
+
setup audit is told to the agent working there (first Smart Tool call of a session), never as a toast."""
|
|
209
209
|
upgrade = _checked_upgrade()
|
|
210
210
|
offer = model_upgrade.due() if upgrade["state"] == "pending" else None
|
|
211
|
-
return {"upgrade": offer, "failing": model_health.alerts_due(), "
|
|
212
|
-
"check_error": upgrade["check_error"]}
|
|
211
|
+
return {"upgrade": offer, "failing": model_health.alerts_due(), "check_error": upgrade["check_error"]}
|
|
213
212
|
|
|
214
213
|
|
|
215
214
|
def handle_local_models_get():
|
|
@@ -828,7 +827,7 @@ __I18N__
|
|
|
828
827
|
<div class="card">
|
|
829
828
|
<div class="card-head"><h3>Report at session start</h3><span class="pill" id="audit-hook-pill" data-tone="busy">Checking</span></div>
|
|
830
829
|
<div class="field"><label for="sel-audit-hook-client">Client</label><select id="sel-audit-hook-client"><option value="claude">Claude Code</option><option value="codex">Codex</option></select></div>
|
|
831
|
-
<p class="lede">Off by default. When on, every new session starts with
|
|
830
|
+
<p class="lede">Off by default. When on, every new session starts with the Smart Tool update notices (also shown to you) and the redirect proposals waiting for a decision. It adds a SessionStart entry to <span class="mono" id="audit-hook-file"></span>, written after a backup, and turning it off removes only that entry. Codex runs a new hook only after you approve it in /hooks. Either way, the agent learns about a project's setup audit on its first Smart Tool call there, once per session.</p>
|
|
832
831
|
<div class="actions"><button type="button" class="primary" id="btn-audit-hook-on" hidden>Turn on</button><button type="button" id="btn-audit-hook-off" hidden>Turn off</button></div>
|
|
833
832
|
<div id="audit-hook-confirm" hidden>
|
|
834
833
|
<p class="hint" id="audit-hook-where"></p>
|
package/smart_tool_daemon.py
CHANGED
|
@@ -42,6 +42,7 @@ import install_runtime
|
|
|
42
42
|
import integration_metrics
|
|
43
43
|
import node_locator
|
|
44
44
|
import config
|
|
45
|
+
import disk_cleanup
|
|
45
46
|
import freshness
|
|
46
47
|
import model_client
|
|
47
48
|
import index_scope
|
|
@@ -845,6 +846,8 @@ def _log(message):
|
|
|
845
846
|
pass
|
|
846
847
|
|
|
847
848
|
|
|
849
|
+
indexer.NOTICE = _log
|
|
850
|
+
|
|
848
851
|
_WEB_WARNINGS = contextvars.ContextVar("web_warnings", default=None)
|
|
849
852
|
_WEB_TRACE = contextvars.ContextVar("web_trace", default=None)
|
|
850
853
|
|
|
@@ -1624,30 +1627,62 @@ def _mark_notice_delivered(job_id):
|
|
|
1624
1627
|
|
|
1625
1628
|
|
|
1626
1629
|
def _pending_notices(session, client, project_root):
|
|
1627
|
-
"""[(key, text)] the session has not been told yet: the hook state (once), the
|
|
1628
|
-
|
|
1629
|
-
|
|
1630
|
-
|
|
1630
|
+
"""[(key, text)] the session has not been told yet: the hook state (once), the update notices (once each) and the
|
|
1631
|
+
redirect proposals waiting for a decision (once, not when the SessionStart hook already reports them at the start
|
|
1632
|
+
of the session). The setup audit of the folder is not here: `_project_notice` adds it to the first Smart Tool
|
|
1633
|
+
call of the session in that project, whatever the tool."""
|
|
1631
1634
|
delivered = session.get("notices_delivered") or set()
|
|
1632
1635
|
pending = []
|
|
1633
1636
|
if session["hook_notice"] and "hook" not in delivered:
|
|
1634
1637
|
pending.append(("hook", session["hook_notice"]))
|
|
1635
1638
|
pending += [(key, f"{text} Tell the user.") for key, text in _update_notices() if key not in delivered]
|
|
1636
|
-
|
|
1637
|
-
|
|
1638
|
-
try:
|
|
1639
|
-
text = setup_audit.session_notice(project_root) if key not in delivered else ""
|
|
1640
|
-
except Exception as exc:
|
|
1641
|
-
_log(f"setup audit notice not read: {type(exc).__name__}: {exc}")
|
|
1642
|
-
text = ""
|
|
1643
|
-
if text:
|
|
1644
|
-
pending.append((key, text))
|
|
1645
|
-
proposals = _proposals_notice() if "proposals" not in delivered else ""
|
|
1639
|
+
if "proposals" not in delivered and not client_hooks.audit_hook_installed(client):
|
|
1640
|
+
proposals = _proposals_notice()
|
|
1646
1641
|
if proposals:
|
|
1647
1642
|
pending.append(("proposals", proposals))
|
|
1648
1643
|
return pending
|
|
1649
1644
|
|
|
1650
1645
|
|
|
1646
|
+
_PROJECT_NOTICE_TOOLS = frozenset({"smart_search", "smart_scout", "project_manage"})
|
|
1647
|
+
_NOTICE_LOCK = threading.Lock()
|
|
1648
|
+
|
|
1649
|
+
|
|
1650
|
+
def _notice_root(arguments):
|
|
1651
|
+
"""The folder a tool call is about: arguments["project_root"], or the root of the registered arguments["project_id"];
|
|
1652
|
+
None when the call names neither or the id is not registered."""
|
|
1653
|
+
if not isinstance(arguments, dict):
|
|
1654
|
+
return None
|
|
1655
|
+
if isinstance(arguments.get("project_root"), str):
|
|
1656
|
+
return arguments["project_root"]
|
|
1657
|
+
project = project_store.get(arguments["project_id"]) if isinstance(arguments.get("project_id"), str) else None
|
|
1658
|
+
return project["root"] if project else None
|
|
1659
|
+
|
|
1660
|
+
|
|
1661
|
+
def _project_notice(name, arguments):
|
|
1662
|
+
"""The setup audit note of the project the call names (project_root or project_id) when this is the first call of
|
|
1663
|
+
the MCP session there (keyed by the audited project, so a subfolder does not repeat it), or "". Marks it told:
|
|
1664
|
+
once per session and project, in the tool answer only (the user gets no notification for it). A setup_audit call
|
|
1665
|
+
marks it told without the note, since its own answer already holds the findings. A stored report that cannot be
|
|
1666
|
+
read is told the same way, with its cause, instead of being skipped."""
|
|
1667
|
+
session, _client = _session_client()
|
|
1668
|
+
root = _notice_root(arguments)
|
|
1669
|
+
if session is None or name not in _PROJECT_NOTICE_TOOLS or not isinstance(root, str) or not os.path.isdir(root):
|
|
1670
|
+
return ""
|
|
1671
|
+
try:
|
|
1672
|
+
found = setup_audit.agent_notice(root)
|
|
1673
|
+
except (OSError, ValueError, KeyError) as exc:
|
|
1674
|
+
found = (root, f"Smart Tool could not read the setup audit of {root.replace(os.sep, '/')}: {exc}")
|
|
1675
|
+
if found is None:
|
|
1676
|
+
return ""
|
|
1677
|
+
key = "audit:" + setup_audit.index_key(found[0])
|
|
1678
|
+
with _NOTICE_LOCK:
|
|
1679
|
+
told = session.setdefault("notices_delivered", set())
|
|
1680
|
+
if key in told:
|
|
1681
|
+
return ""
|
|
1682
|
+
told.add(key)
|
|
1683
|
+
return "" if arguments.get("action") == "setup_audit" else found[1]
|
|
1684
|
+
|
|
1685
|
+
|
|
1651
1686
|
def _proposals_notice():
|
|
1652
1687
|
"""One line for the agent when redirect proposals wait for a decision, "" when none; an unreadable store is
|
|
1653
1688
|
logged, not raised, so neither a search nor a session start fails over it."""
|
|
@@ -2318,14 +2353,10 @@ def _project_action_locked(arguments):
|
|
|
2318
2353
|
if active:
|
|
2319
2354
|
raise ValueError("Cancel the jobs and wait for them to stop before removing the index.")
|
|
2320
2355
|
project_store.update(key, watch=False, paused=True, enabled=False)
|
|
2321
|
-
removed =
|
|
2322
|
-
import glob
|
|
2323
|
-
for scope_id in set([key, *project_identity.legacy_ids(project["root"])]):
|
|
2324
|
-
for path in glob.glob(os.path.join(index_scope.SCOPE_DIR, scope_id + '*.scope.json')):
|
|
2325
|
-
if re.fullmatch(re.escape(scope_id) + r'(?:\.v-[a-f0-9]{16})?\.scope\.json', os.path.basename(path)):
|
|
2326
|
-
os.remove(project_identity.state_path(path, index_scope.SCOPE_DIR))
|
|
2356
|
+
removed = disk_cleanup.remove_project_files(project["root"])
|
|
2327
2357
|
project_store.forget(key)
|
|
2328
|
-
return {"removed_index_files": len(removed), "
|
|
2358
|
+
return {"removed_index_files": len(removed["files"]), "removed_bytes": removed["bytes"],
|
|
2359
|
+
"source_files_preserved": True}
|
|
2329
2360
|
elif action == "relocate":
|
|
2330
2361
|
if active:
|
|
2331
2362
|
raise ValueError("Wait for the jobs to stop before relocating the folder.")
|
|
@@ -4049,18 +4080,16 @@ def _pretool_hook(handler):
|
|
|
4049
4080
|
|
|
4050
4081
|
|
|
4051
4082
|
def _session_audit_hook(handler):
|
|
4052
|
-
"""SessionStart output
|
|
4053
|
-
additionalContext,
|
|
4054
|
-
|
|
4083
|
+
"""SessionStart output of the session-start hook: the update notices both as systemMessage (shown to the user) and
|
|
4084
|
+
as additionalContext, plus the redirect proposals waiting for a decision as context; `{}` when there is nothing to
|
|
4085
|
+
say. The setup audit of a project reaches the agent on its first Smart Tool call there (`_project_notice`), so it
|
|
4086
|
+
is told once per session and project."""
|
|
4055
4087
|
try:
|
|
4056
|
-
|
|
4057
|
-
|
|
4058
|
-
|
|
4059
|
-
|
|
4060
|
-
|
|
4061
|
-
text = ""
|
|
4062
|
-
updates = " ".join(notice for _key, notice in _update_notices())
|
|
4063
|
-
context = "\n".join(part for part in (updates, text, _proposals_notice()) if part)
|
|
4088
|
+
handler._read_body(HOOK_MAX_BODY_BYTES)
|
|
4089
|
+
except ValueError as exc:
|
|
4090
|
+
_log(f"SessionStart hook body refused: {exc}")
|
|
4091
|
+
updates =" ".join(notice for _key, notice in _update_notices())
|
|
4092
|
+
context = "\n".join(part for part in (updates, _proposals_notice()) if part)
|
|
4064
4093
|
if not context:
|
|
4065
4094
|
return {}
|
|
4066
4095
|
output = {"hookSpecificOutput": {"hookEventName": "SessionStart", "additionalContext": context}}
|
|
@@ -4145,6 +4174,9 @@ def _dispatch_tool_call(params, session_id=None):
|
|
|
4145
4174
|
warning = model_health.warning_text(failures, setup_ui.setup_origin())
|
|
4146
4175
|
if warning:
|
|
4147
4176
|
result["content"][0]["text"] += _sanitize_text(warning)
|
|
4177
|
+
notice = _project_notice(name, arguments)
|
|
4178
|
+
if notice:
|
|
4179
|
+
result["content"][0]["text"] += "\n\n" + _sanitize_text(notice)
|
|
4148
4180
|
return result
|
|
4149
4181
|
finally:
|
|
4150
4182
|
if who is not None:
|
|
@@ -4569,6 +4601,7 @@ def main():
|
|
|
4569
4601
|
threading.Thread(target=_heal_browser_control, daemon=True).start()
|
|
4570
4602
|
try:
|
|
4571
4603
|
recovered = project_store.recover_interrupted()
|
|
4604
|
+
disk_cleanup.sweep([WEB_FETCH_VECTOR_ROOT], _log)
|
|
4572
4605
|
project_store.import_known_legacy(indexer.INDEX_DIR)
|
|
4573
4606
|
_PROJECT_MONITOR = project_monitor.ProjectMonitor(_enqueue_monitored_project)
|
|
4574
4607
|
_PROJECT_MONITOR.start()
|
package/tray.py
CHANGED
|
@@ -26,7 +26,6 @@ import config
|
|
|
26
26
|
import daemon_launcher
|
|
27
27
|
import i18n
|
|
28
28
|
import paths
|
|
29
|
-
import setup_audit
|
|
30
29
|
import toast_notice
|
|
31
30
|
import update_check
|
|
32
31
|
import version
|
|
@@ -156,8 +155,7 @@ def _toast(title, lines, buttons, tag):
|
|
|
156
155
|
|
|
157
156
|
|
|
158
157
|
def _show_notices(icon, notices):
|
|
159
|
-
"""Shows a toast for each pending notice: the new-default-models offer
|
|
160
|
-
project whose scheduled setup audit found new errors or warnings."""
|
|
158
|
+
"""Shows a toast for each pending notice: the new-default-models offer and each model over the failure limit."""
|
|
161
159
|
offer = notices.get("upgrade")
|
|
162
160
|
if offer:
|
|
163
161
|
lines = [f"{role}: {change['current']} -> {change['new']}" for role, change in offer["changes"].items()]
|
|
@@ -175,10 +173,6 @@ def _show_notices(icon, notices):
|
|
|
175
173
|
[(i18n.t("Switch model"), toast_notice.action_uri("open-setup", section="models")),
|
|
176
174
|
(i18n.t("Mute 7 days"), toast_notice.action_uri("mute-model", model=row["model"]))],
|
|
177
175
|
"model-failing:" + row["model"])
|
|
178
|
-
for row in notices.get("audit") or []:
|
|
179
|
-
title, lines = setup_audit.toast_text(row)
|
|
180
|
-
_toast(title, lines, [(i18n.t("Open the audit"), toast_notice.action_uri("open-setup", section="audit"))],
|
|
181
|
-
"setup-audit:" + row["project"])
|
|
182
176
|
|
|
183
177
|
|
|
184
178
|
def _log(text):
|
package/version.py
CHANGED