@allansantos-dev/smart-tool 0.9.17 → 0.9.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +30 -0
- package/README.md +8 -4
- package/agents/smart-scout.md +13 -0
- package/hook_decision.py +2 -4
- package/indexer.py +77 -4
- package/package.json +1 -1
- package/project_store.py +9 -2
- package/smart_tool_daemon.py +201 -26
- package/tray.py +37 -9
- package/update_check.py +102 -11
- package/version.py +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,35 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.9.19 - beta
|
|
4
|
+
|
|
5
|
+
- A search no longer waits for the index to catch up. It answers from the existing index at once, searches the files
|
|
6
|
+
edited since the last index by keyword straight from disk, and says "Indexing in progress: N file(s) changed..." in
|
|
7
|
+
the answer while the update runs in the background. An index that does not exist yet, or was built by another model,
|
|
8
|
+
is still built first.
|
|
9
|
+
- A search waits up to 25 s for its own result before answering with a `job_id` (it was 3 s, below the 3.3 s a warm
|
|
10
|
+
search takes at best, so nearly every call needed a second `smart_search_result`).
|
|
11
|
+
- The scope and the language profile are no longer asked of the model inside a search that has an index: the saved ones
|
|
12
|
+
are used and refreshed in the background.
|
|
13
|
+
- New folders are registered watched and eager: the index follows edits by itself. A PreToolUse Edit/Write of an eager
|
|
14
|
+
folder without a running watcher queues one debounced update with the edited paths. Folders already registered keep
|
|
15
|
+
their policy.
|
|
16
|
+
- A `top_k` whose total is above 20 is scaled down in proportion instead of rejecting the call (the rejection cost the
|
|
17
|
+
agent a turn on its first search).
|
|
18
|
+
- A search made right after an edit keeps the just-edited chunk in the answer: the reranker can no longer push it out.
|
|
19
|
+
- An edit whose path comes in the Windows 8.3 short form (`ALLANS~1`-style folder names) is matched to its registered
|
|
20
|
+
project; before, the project lookup compared it with the long form and found nothing.
|
|
21
|
+
- The daemon log prints the seconds spent in each phase of a job (`phases=queued:0.02,auth:0.01,...`).
|
|
22
|
+
|
|
23
|
+
## 0.9.18 - beta
|
|
24
|
+
|
|
25
|
+
- Updates are no longer applied only at logon. The tray checks npm at start and every 24 h and installs a newer
|
|
26
|
+
version as soon as it sees it (on by default); a version that failed to install is tried again 24 h later instead of
|
|
27
|
+
never. `SMART_TOOL_AUTO_UPDATE=off` turns automatic updates off, like the tray switch.
|
|
28
|
+
- Nothing about updates is hidden. The tray notifies when a newer version is available, when it starts updating, when
|
|
29
|
+
Smart Tool was updated (from which version to which) and when an update failed. Agent sessions get the same line once
|
|
30
|
+
on their first `smart_search`, with the link to the changes, and the SessionStart report (when on) shows it to you
|
|
31
|
+
at session start.
|
|
32
|
+
|
|
3
33
|
## 0.9.17 - beta
|
|
4
34
|
|
|
5
35
|
- The agent learns the setup audit result without asking for it. On its first `smart_search` of a session, once per
|
package/README.md
CHANGED
|
@@ -15,7 +15,7 @@ registered as the `playwright` server where this machine lets a browser be drive
|
|
|
15
15
|
An optional hook routes the agent's native Grep/Glob/Read/Bash/WebSearch/WebFetch calls to these tools when they
|
|
16
16
|
are cheaper, or just tells the agent what Smart Tool would do.
|
|
17
17
|
|
|
18
|
-
**Status:** 0.9.
|
|
18
|
+
**Status:** 0.9.19 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
|
|
19
19
|
|
|
20
20
|
## How it works
|
|
21
21
|
|
|
@@ -76,9 +76,13 @@ gateway such as Ollama, nothing about your code leaves the machine.
|
|
|
76
76
|
npx @allansantos-dev/smart-tool@latest install
|
|
77
77
|
```
|
|
78
78
|
|
|
79
|
-
Updates install themselves:
|
|
80
|
-
a
|
|
81
|
-
|
|
79
|
+
Updates install themselves, on by default: the tray checks npm when it starts (at logon) and every 24 h, and when there
|
|
80
|
+
is a newer version it runs this same command in a visible console (once per version; a failed version is tried again
|
|
81
|
+
after 24 h, with the previous version kept). You are told each time: a Windows notification from the tray (newer version,
|
|
82
|
+
updating, updated, update failed), and agent sessions get the same line on their first `smart_search` (and at session
|
|
83
|
+
start when the SessionStart report is on, shown to you too) with the link to the changes. The tray menu has "Update to
|
|
84
|
+
X" (or "Check for updates") on demand and an "Update automatically" switch; `SMART_TOOL_AUTO_UPDATE=off` also turns
|
|
85
|
+
automatic updates off, and the newer-version notice then carries the command to run. Running the command by hand also updates; `~/.smart-tool` is kept. It downloads [uv](https://docs.astral.sh/uv/) (checksum
|
|
82
86
|
verified) and a private Python 3.12 into `~/.smart-tool/runtime`, with nothing added to `PATH` or the registry, then
|
|
83
87
|
copies Smart Tool to `%LOCALAPPDATA%\Programs\SmartTool` with its own virtual environment,
|
|
84
88
|
installs the web runtime, registers a per-user Scheduled Task that starts the tray at logon and starts the daemon on
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: smart-scout
|
|
3
|
+
description: Optional helper for the Smart Tool. Use it only for a task that chains several Smart Tool calls (search, then graph, then affected_tests) or while the index is still loading. A plain search is faster called directly.
|
|
4
|
+
model: haiku
|
|
5
|
+
tools: mcp__smart-tool__smart_search, mcp__smart-tool__smart_search_result, mcp__smart-tool__project_manage, Read
|
|
6
|
+
---
|
|
7
|
+
You run Smart Tool calls for another session and return only the result.
|
|
8
|
+
|
|
9
|
+
Use smart_search with project_root equal to the working directory (forward slashes), query_identifiers in English identifier terms, query_comments in the language of the comments, and top_k {"code": 6, "test": 2, "doc": 2}. When it answers pending, call smart_search_result with the job_id until it finishes. project_manage action=graph (symbol) and action=affected_tests are yours to chain when the task asks for them. Open at most 3 files with Read and make at most 8 tool calls in total.
|
|
10
|
+
|
|
11
|
+
Reply ONLY with a list of up to 8 items, each `path:first_line-last_line: reason in up to 15 words`, no code excerpts and no introduction.
|
|
12
|
+
|
|
13
|
+
If nothing relevant came back after two different queries, reply exactly `EMPTY: <the two queries you tried>`. Never invent a path. Never use Grep, shell search or Read of folders to compensate: an empty answer is a valid answer, and the session that called you decides whether to retry with smart_search or to report that the code was not found.
|
package/hook_decision.py
CHANGED
|
@@ -171,10 +171,8 @@ def advise(reason):
|
|
|
171
171
|
|
|
172
172
|
|
|
173
173
|
def _project_of(path):
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
if target.startswith(os.path.normcase(os.path.abspath(p["root"])) + os.sep)]
|
|
177
|
-
return max(roots, key=len) if roots else None
|
|
174
|
+
project = project_store.containing(path)
|
|
175
|
+
return project["root"] if project else None
|
|
178
176
|
|
|
179
177
|
|
|
180
178
|
def _impact_line(root, rel, name):
|
package/indexer.py
CHANGED
|
@@ -779,9 +779,14 @@ def _vector_rows(root, path):
|
|
|
779
779
|
return rows, meta, norms
|
|
780
780
|
|
|
781
781
|
|
|
782
|
+
def _query_terms(query):
|
|
783
|
+
"""Up to 24 distinct word terms of the query, in order."""
|
|
784
|
+
return list(dict.fromkeys(re.findall(r"[\w]{2,}", query or "", re.UNICODE)))[:24]
|
|
785
|
+
|
|
786
|
+
|
|
782
787
|
def _lexical_scored(path, query, rows, limit, offset=0):
|
|
783
788
|
"""(chunk_id, score) best first; higher score is a better lexical match."""
|
|
784
|
-
terms =
|
|
789
|
+
terms = _query_terms(query)
|
|
785
790
|
if not terms:
|
|
786
791
|
return []
|
|
787
792
|
expression = " OR ".join('"' + term.replace('"', '""') + '"' for term in terms)
|
|
@@ -808,11 +813,72 @@ def _min_max(pairs):
|
|
|
808
813
|
return {key: (value - low) / (high - low) if high > low else 1.0 for key, value in pairs}
|
|
809
814
|
|
|
810
815
|
|
|
811
|
-
def
|
|
812
|
-
"""
|
|
816
|
+
def stale_files(root, included_paths):
|
|
817
|
+
"""`(changed, removed)` against the indexed state: `changed` lists in-scope files whose `(mtime_ns, size)` differs
|
|
818
|
+
from the stored one (or that were never indexed), `removed` lists indexed files that left the scope. It only
|
|
819
|
+
stats files, so it is the cheap answer to "what would a reindex have to touch"."""
|
|
820
|
+
snapshot = manifest_snapshot(root)
|
|
821
|
+
changed, removed = [], set(snapshot)
|
|
822
|
+
for rel in sorted(set(included_paths)):
|
|
823
|
+
stat, _reason = _stat_candidate(root, rel)
|
|
824
|
+
if stat is None:
|
|
825
|
+
continue
|
|
826
|
+
removed.discard(rel)
|
|
827
|
+
if snapshot.get(rel) != stat:
|
|
828
|
+
changed.append(rel)
|
|
829
|
+
return changed, sorted(removed)
|
|
830
|
+
|
|
831
|
+
|
|
832
|
+
def overlay_chunks(root, rel_paths):
|
|
833
|
+
"""`[(path, start_line, end_line, text)]` read from disk right now for the given files, chunked like the index.
|
|
834
|
+
Files that cannot be read consistently are skipped: the overlay only complements the index, it never fails a
|
|
835
|
+
search."""
|
|
836
|
+
chunks = []
|
|
837
|
+
for rel in rel_paths:
|
|
838
|
+
data, encoding, _stat = _stable_read(root, rel)
|
|
839
|
+
if data is None:
|
|
840
|
+
continue
|
|
841
|
+
pieces = chunk_file(root, rel, CHUNK_LINES, data=data, encoding=encoding)
|
|
842
|
+
chunks.extend((rel, start, end, text) for start, end, text in pieces or [])
|
|
843
|
+
return chunks
|
|
844
|
+
|
|
845
|
+
|
|
846
|
+
def _overlay_scored(chunks, query, alpha):
|
|
847
|
+
"""`[(score, path, start, end, text)]` for overlay chunks matching the query terms by keyword. A chunk that holds
|
|
848
|
+
every term scores 1.0, the top of the fused index scores (`search` ranks it first on a tie): the edited file is read
|
|
849
|
+
from disk, so it is the current answer for an exact match. A partial match stays on the 0-1 scale of the lexical
|
|
850
|
+
share of the fused score."""
|
|
851
|
+
terms = _query_terms(query)
|
|
852
|
+
scored = []
|
|
853
|
+
for path, start, end, text in chunks:
|
|
854
|
+
haystack = (path + " " + text).casefold()
|
|
855
|
+
matched = sum(term.casefold() in haystack for term in terms)
|
|
856
|
+
if matched:
|
|
857
|
+
score = 1.0 if matched == len(terms) else (1 - alpha) * matched / len(terms)
|
|
858
|
+
scored.append((score, path, start, end, text))
|
|
859
|
+
return scored
|
|
860
|
+
|
|
861
|
+
|
|
862
|
+
def overlay_exact_keys(overlay, query):
|
|
863
|
+
"""`{(path, start, end)}` of the overlay chunks that hold every query term. A reranker must not push them below
|
|
864
|
+
indexed chunks: they come from the file as it is on disk now."""
|
|
865
|
+
if not overlay or not overlay["chunks"] or not query:
|
|
866
|
+
return set()
|
|
867
|
+
return {(path, start, end) for score, path, start, end, _text in _overlay_scored(overlay["chunks"], query, 0.0)
|
|
868
|
+
if score == 1.0}
|
|
869
|
+
|
|
870
|
+
|
|
871
|
+
def search(root, query_vector=None, top_k=20, query=None, model_id=None, include=None, overlay=None):
|
|
872
|
+
"""Vector plus lexical retrieval by fusion of relative scores, without network. `overlay` is
|
|
873
|
+
`{"drop": paths, "chunks": overlay_chunks(...)}`: the indexed chunks of the `drop` paths (stale or removed
|
|
874
|
+
files) are ignored and the overlay chunks, read from disk, take part by keyword instead."""
|
|
813
875
|
path = existing_db_path(root)
|
|
814
876
|
if not path:
|
|
815
877
|
return []
|
|
878
|
+
kind_filter = include
|
|
879
|
+
if overlay and overlay["drop"]:
|
|
880
|
+
dropped = set(overlay["drop"])
|
|
881
|
+
include = lambda rel: rel not in dropped and (kind_filter is None or kind_filter(rel))
|
|
816
882
|
rows, meta, norms = _vector_rows(root, path)
|
|
817
883
|
if include is not None:
|
|
818
884
|
rows = [row for row in rows if include(row[1])]
|
|
@@ -854,7 +920,14 @@ def search(root, query_vector=None, top_k=20, query=None, model_id=None, include
|
|
|
854
920
|
else:
|
|
855
921
|
scores, ids = vector_scores, vector_ids
|
|
856
922
|
by_id = {row[0]: row for row in rows}
|
|
857
|
-
|
|
923
|
+
found = [(scores[chunk_id], *by_id[chunk_id][1:5]) for chunk_id in ids[:top_k]]
|
|
924
|
+
if overlay and overlay["chunks"]:
|
|
925
|
+
extra = _overlay_scored([chunk for chunk in overlay["chunks"] if kind_filter is None or kind_filter(chunk[0])],
|
|
926
|
+
query, FUSION_ALPHA if query_vector is not None else 0.0)
|
|
927
|
+
found = [item for _score, item in sorted(
|
|
928
|
+
[((item[0], 1), item) for item in extra] + [((item[0], 0), item) for item in found],
|
|
929
|
+
key=lambda pair: pair[0], reverse=True)][:top_k]
|
|
930
|
+
return found
|
|
858
931
|
|
|
859
932
|
|
|
860
933
|
def graph_cache_path(view_path):
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@allansantos-dev/smart-tool",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.19",
|
|
4
4
|
"description": "Local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharper tools than their built-in search. Windows.",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"author": "Allan Santos",
|
package/project_store.py
CHANGED
|
@@ -40,8 +40,8 @@ def register(root, watch=None):
|
|
|
40
40
|
row = db.execute("SELECT data FROM projects WHERE id=?", (key,)).fetchone()
|
|
41
41
|
data = json.loads(row[0]) if row else {
|
|
42
42
|
"id": key, "root": display, "name": os.path.basename(display) or display,
|
|
43
|
-
"status": "registered", "watch":
|
|
44
|
-
"update_mode": "
|
|
43
|
+
"status": "registered", "watch": True, "paused": False, "enabled": False,
|
|
44
|
+
"update_mode": "eager",
|
|
45
45
|
"dirty_seq": 0, "indexed_seq": 0, "created_at": time.time(),
|
|
46
46
|
"last_indexed": None, "last_checked": None, "last_error": "",
|
|
47
47
|
"last_job_id": None, "watch_backend": "off", "next_retry_at": 0,
|
|
@@ -60,6 +60,13 @@ def get(project_id):
|
|
|
60
60
|
return json.loads(row[0]) if row else None
|
|
61
61
|
|
|
62
62
|
|
|
63
|
+
def containing(path):
|
|
64
|
+
"""The registered project whose folder holds `path` (the deepest one when projects nest), or `None`."""
|
|
65
|
+
target = project_identity.canonical_root(path)
|
|
66
|
+
inside = [p for p in all_projects() if target.startswith(project_identity.canonical_root(p["root"]) + os.sep)]
|
|
67
|
+
return max(inside, key=lambda p: len(p["root"])) if inside else None
|
|
68
|
+
|
|
69
|
+
|
|
63
70
|
def all_projects():
|
|
64
71
|
with _LOCK, _db() as db:
|
|
65
72
|
rows = db.execute("SELECT data FROM projects ORDER BY root COLLATE NOCASE").fetchall()
|
package/smart_tool_daemon.py
CHANGED
|
@@ -68,6 +68,7 @@ import local_embedder
|
|
|
68
68
|
import web_search_health
|
|
69
69
|
import web_fetch
|
|
70
70
|
import version
|
|
71
|
+
import update_check
|
|
71
72
|
import gateway
|
|
72
73
|
import hook_decision
|
|
73
74
|
import endpoint_sync
|
|
@@ -171,7 +172,11 @@ _capabilities_cache = None
|
|
|
171
172
|
_jobs = {}
|
|
172
173
|
_JOBS_LOCK = threading.Lock()
|
|
173
174
|
_MAX_JOBS_RETAINED = 200
|
|
174
|
-
|
|
175
|
+
# A warm search never took less than 3.3 s at its fastest 5% and the median is 8.6 s: waiting less answers pending to
|
|
176
|
+
# nearly every call and costs the agent extra smart_search_result round trips. Codex gives an MCP call 60 s by default.
|
|
177
|
+
SMART_SEARCH_SYNC_WAIT_S = 25
|
|
178
|
+
OVERLAY_MAX_FILES = 40
|
|
179
|
+
EDIT_INDEX_DELAY_S = 8
|
|
175
180
|
# smart_search_result waits for the job this long before answering pending: answering at once had an agent poll the
|
|
176
181
|
# same job 7 times in 16 s and then give up on Smart Tool (claude -p session, 2026-10-09; job durations median 6 s,
|
|
177
182
|
# p75 10.7 s).
|
|
@@ -231,6 +236,8 @@ def _update_job(job_id, **fields):
|
|
|
231
236
|
current = _jobs.get(job_id)
|
|
232
237
|
if current is not None and current.get("status") == "pending":
|
|
233
238
|
updated = {**current, **fields, "updated_at": time.time()}
|
|
239
|
+
if fields.get("phase") and fields["phase"] != current.get("phase"):
|
|
240
|
+
updated["phases"] = [*current.get("phases", []), [fields["phase"], time.time()]]
|
|
234
241
|
_jobs[job_id] = updated
|
|
235
242
|
if (fields.get("phase") != current.get("phase") or
|
|
236
243
|
time.monotonic() - _PERSIST_LAST.get(job_id, 0) > .5 or fields.get("cancel_requested")):
|
|
@@ -240,6 +247,20 @@ def _update_job(job_id, **fields):
|
|
|
240
247
|
_persist_project_job(job_id, persistent)
|
|
241
248
|
|
|
242
249
|
|
|
250
|
+
def _phase_timings(job):
|
|
251
|
+
""""phase:seconds" for each phase of a job in the order it ran; the last phase lasts until the job finished."""
|
|
252
|
+
marks = [("queued", job.get("created_at") or 0), *[tuple(mark) for mark in job.get("phases", [])]]
|
|
253
|
+
end = job.get("finished_at") or time.time()
|
|
254
|
+
spans = {}
|
|
255
|
+
for (name, start), following in zip(marks, [*marks[1:], ("", end)]):
|
|
256
|
+
spans[name] = spans.get(name, 0.0) + max(following[1] - start, 0.0)
|
|
257
|
+
return ",".join(f"{name}:{seconds:.2f}" for name, seconds in spans.items())
|
|
258
|
+
|
|
259
|
+
|
|
260
|
+
_EDIT_PENDING = {}
|
|
261
|
+
_EDIT_LOCK = threading.Lock()
|
|
262
|
+
|
|
263
|
+
|
|
243
264
|
def _project_lock(root):
|
|
244
265
|
key = project_identity.canonical_root(root)
|
|
245
266
|
with _PROJECT_LOCKS_LOCK:
|
|
@@ -321,7 +342,10 @@ SERVER_INSTRUCTIONS = (
|
|
|
321
342
|
"with query). Never retry the same search or download through Bash, python, node, PowerShell, curl or wget: that "
|
|
322
343
|
"bypasses the rule the user set. Reading one known file, or searching inside one known file, is never blocked. "
|
|
323
344
|
"Before changing a function, project_manage action=graph with symbol lists its callers and tests; after editing, "
|
|
324
|
-
"project_manage action=affected_tests lists the tests to run."
|
|
345
|
+
"project_manage action=affected_tests lists the tests to run. When smart_search returns nothing, retry smart_search with "
|
|
346
|
+
"other terms; do not switch to Grep, Read of folders or shell search. A helper agent (smart-scout, a cheap model) is "
|
|
347
|
+
"optional: use it only for a task that chains several Smart Tool calls or while the index is still loading; a plain "
|
|
348
|
+
"search is faster called directly. A helper that returns empty means retry with the tool or report not found."
|
|
325
349
|
)
|
|
326
350
|
|
|
327
351
|
TOOLS = [
|
|
@@ -376,7 +400,10 @@ TOOLS = [
|
|
|
376
400
|
"query_comments with the same request in the comments' language; when the profile shows two "
|
|
377
401
|
"languages both are required, or explain in single_language_reason. Put documentation searches in "
|
|
378
402
|
"doc_query. Code and test results list the likeliest functions as name:line in symbols. Tune top_k per block for the project (project_manage search_limits). Fast searches return results directly. If work takes longer than a few seconds, "
|
|
379
|
-
"a pending job_id is returned; call smart_search_result with that ID until done."
|
|
403
|
+
"a pending job_id is returned; call smart_search_result with that ID until done. A warning that starts "
|
|
404
|
+
"with 'Indexing in progress' means the files named there changed after the last index: they were "
|
|
405
|
+
"searched by keyword from disk and the rest of the answer is current; repeat the search later for their "
|
|
406
|
+
"semantic ranking."
|
|
380
407
|
),
|
|
381
408
|
"inputSchema": {
|
|
382
409
|
"type": "object",
|
|
@@ -813,6 +840,110 @@ def _agent_scope_request(root, scope, problem):
|
|
|
813
840
|
"(always included unless listed in exclude).")
|
|
814
841
|
|
|
815
842
|
|
|
843
|
+
def _profile_due(scope, scope_model):
|
|
844
|
+
"""Whether the saved scope lacks a current language profile and the model may be asked for one now, that is, a
|
|
845
|
+
scope model is configured and the pause after a failed attempt has passed."""
|
|
846
|
+
failed_at = (scope.get("profile_error") or {}).get("at", 0)
|
|
847
|
+
return bool(scope_model and (scope.get("profile") or {}).get("version") != index_profile.PROFILE_VERSION
|
|
848
|
+
and time.time() - failed_at >= index_profile.RETRY_AFTER_S)
|
|
849
|
+
|
|
850
|
+
|
|
851
|
+
def _index_lag(root, model, cfg, problem, job_id):
|
|
852
|
+
"""What a search can serve without waiting for the index. `None` when the existing index cannot be used as it is
|
|
853
|
+
(missing, built by another model, chunker or view, no saved scope, or a scope that needs the model while it is
|
|
854
|
+
unavailable): the caller then updates it before searching. Otherwise `{"changed", "removed", "refresh"}` with the
|
|
855
|
+
in-scope files that differ from the indexed state, the indexed files that left the scope, and whether a
|
|
856
|
+
background update is due (stale files, or a scope or profile the model has to refresh)."""
|
|
857
|
+
project = project_store.get(project_identity.project_id(root)) or {}
|
|
858
|
+
info = indexer.info(root)
|
|
859
|
+
view = index_views.describe(root)
|
|
860
|
+
scope = index_scope.load_scope(root)
|
|
861
|
+
if (not info.get("chunks") or info.get("model_id") != model or not index_scope.valid_scope(scope)
|
|
862
|
+
or info.get("chunker_version") != indexer.CHUNKER_VERSION
|
|
863
|
+
or info.get("schema_version") != indexer.SCHEMA_VERSION
|
|
864
|
+
or info.get("chunk_lines") != indexer.CHUNK_LINES
|
|
865
|
+
or (view.get("git") and info.get("view_id") != view.get("view_id"))
|
|
866
|
+
or project.get("scope_dirty")):
|
|
867
|
+
return None
|
|
868
|
+
scope_model = cfg.get("scope_model") or cfg.get("router_model")
|
|
869
|
+
rescan = index_scope.needs_rescan(root, scope)
|
|
870
|
+
if rescan and (not scope_model or problem):
|
|
871
|
+
return None
|
|
872
|
+
_update_job(job_id, phase="listing")
|
|
873
|
+
changed, removed = indexer.stale_files(root, index_scope.resolve_included_files(root, scope))
|
|
874
|
+
return {"changed": changed, "removed": removed,
|
|
875
|
+
"refresh": bool(changed or removed or rescan or _profile_due(scope, scope_model))}
|
|
876
|
+
|
|
877
|
+
|
|
878
|
+
def _lag_overlay(root, lag):
|
|
879
|
+
"""`(overlay, warning)` for `indexer.search`: the stale files read from disk, searched by keyword in place of their
|
|
880
|
+
outdated chunks, and the notice telling the agent that indexing is in progress. `(None, "")` when the index is
|
|
881
|
+
current."""
|
|
882
|
+
stale = lag["changed"] + lag["removed"]
|
|
883
|
+
if not stale:
|
|
884
|
+
return None, ""
|
|
885
|
+
shown = lag["changed"][:OVERLAY_MAX_FILES]
|
|
886
|
+
skipped = len(lag["changed"]) - len(shown)
|
|
887
|
+
names = ", ".join(stale[:3]) + (f" (+{len(stale) - 3})" if len(stale) > 3 else "")
|
|
888
|
+
warning = (f"Indexing in progress: {len(stale)} file(s) changed after the last index ({names}). Results come from "
|
|
889
|
+
"the existing index; those files were searched by keyword straight from disk, without semantic "
|
|
890
|
+
"ranking" + (f", and {skipped} of them were not searched" if skipped else "") +
|
|
891
|
+
". The index updates in the background: repeat the search later for the updated ranking.")
|
|
892
|
+
return {"drop": stale, "chunks": indexer.overlay_chunks(root, shown)}, warning
|
|
893
|
+
|
|
894
|
+
|
|
895
|
+
def _refresh_in_background(root, lag):
|
|
896
|
+
"""Queues the index update that the search did not wait for. Returns the notice to show when it could not be
|
|
897
|
+
queued, `""` when it was queued or none was due."""
|
|
898
|
+
if not lag["refresh"]:
|
|
899
|
+
return ""
|
|
900
|
+
try:
|
|
901
|
+
_submit_project_job("project_index", {"project_root": root, "_origin": "search"})
|
|
902
|
+
except RuntimeError as exc:
|
|
903
|
+
return f"The background index update could not start ({exc}); the next search tries again."
|
|
904
|
+
return ""
|
|
905
|
+
|
|
906
|
+
|
|
907
|
+
def _note_edit(payload):
|
|
908
|
+
"""Queues a debounced background index update for a file an agent is about to write, in an eager project whose
|
|
909
|
+
watcher is not running. The PreToolUse hook sees the edit before it lands, so the update waits
|
|
910
|
+
`EDIT_INDEX_DELAY_S` and groups the edits of a burst into one job with `changed_paths`."""
|
|
911
|
+
if not isinstance(payload, dict) or payload.get("tool_name") not in ("Edit", "MultiEdit", "Write"):
|
|
912
|
+
return
|
|
913
|
+
target = (payload.get("tool_input") or {}).get("file_path")
|
|
914
|
+
if not isinstance(target, str) or not os.path.isabs(target):
|
|
915
|
+
return
|
|
916
|
+
project = project_store.containing(target)
|
|
917
|
+
if (not project or not project.get("enabled") or project.get("paused") or project.get("update_mode") != "eager"
|
|
918
|
+
or project.get("watch_backend", "off") != "off"):
|
|
919
|
+
return
|
|
920
|
+
rel = os.path.relpath(project_identity.display_root(target), project["root"]).replace(os.sep, "/")
|
|
921
|
+
with _EDIT_LOCK:
|
|
922
|
+
pending = _EDIT_PENDING.get(project["id"])
|
|
923
|
+
if pending is not None:
|
|
924
|
+
pending.add(rel)
|
|
925
|
+
return
|
|
926
|
+
_EDIT_PENDING[project["id"]] = {rel}
|
|
927
|
+
timer = threading.Timer(EDIT_INDEX_DELAY_S, _flush_edits, args=(project["id"],))
|
|
928
|
+
timer.daemon = True
|
|
929
|
+
timer.start()
|
|
930
|
+
|
|
931
|
+
|
|
932
|
+
def _flush_edits(project_id):
|
|
933
|
+
"""Submits the index update for the files `_note_edit` collected; a failure is logged because it runs on a timer
|
|
934
|
+
thread nobody waits for."""
|
|
935
|
+
with _EDIT_LOCK:
|
|
936
|
+
paths = _EDIT_PENDING.pop(project_id, set())
|
|
937
|
+
project = project_store.get(project_id)
|
|
938
|
+
if not paths or not project or project.get("paused") or not project.get("enabled"):
|
|
939
|
+
return
|
|
940
|
+
try:
|
|
941
|
+
_submit_project_job("project_index", {"project_root": project["root"], "changed_paths": sorted(paths),
|
|
942
|
+
"_origin": "edit"})
|
|
943
|
+
except Exception as exc:
|
|
944
|
+
_log(f"edit index update not queued: {type(exc).__name__}: {exc}")
|
|
945
|
+
|
|
946
|
+
|
|
816
947
|
def _prepare_project_index(arguments, job_id, cfg, token, query_vector=None, preview=False, model=None):
|
|
817
948
|
root = arguments["project_root"]
|
|
818
949
|
project = project_store.get(project_identity.project_id(root)) or {}
|
|
@@ -834,9 +965,7 @@ def _prepare_project_index(arguments, job_id, cfg, token, query_vector=None, pre
|
|
|
834
965
|
deadline=_job_deadline(job_id), cancel_check=checkpoint)
|
|
835
966
|
_update_job(job_id, phase="listing")
|
|
836
967
|
included = index_scope.resolve_included_files(root, scope, cancel_check=checkpoint)
|
|
837
|
-
|
|
838
|
-
if ((scope.get("profile") or {}).get("version") != index_profile.PROFILE_VERSION
|
|
839
|
-
and time.time() - failed_at >= index_profile.RETRY_AFTER_S and scope_model):
|
|
968
|
+
if _profile_due(scope, scope_model):
|
|
840
969
|
_update_job(job_id, phase="profile")
|
|
841
970
|
try:
|
|
842
971
|
profile = index_profile.decide(root, included, scope_model, deadline=_job_deadline(job_id))
|
|
@@ -917,6 +1046,8 @@ def _grouped_yaml(profile, blocks, warning):
|
|
|
917
1046
|
|
|
918
1047
|
LOCAL_EMBEDDER_DOWN_WARNING = ("The mcp-memory local embedder is configured but not responding: index and search are lexical only "
|
|
919
1048
|
"until it is back (check the mcp-memory embedding service).")
|
|
1049
|
+
EMPTY_RESULT_NOTICE = ("No matches. Do not fall back to Grep, Read of folders or shell search: retry smart_search with other "
|
|
1050
|
+
"identifier terms, another group, or a wider top_k, and if it stays empty report that the code was not found.")
|
|
920
1051
|
LEXICAL_WARNING = ("No embedding model: index and search are lexical only (exact words, no synonyms or cross-language matches). "
|
|
921
1052
|
"Configure an embedding_model for semantic search.")
|
|
922
1053
|
|
|
@@ -938,6 +1069,7 @@ def _perform_smart_search_isolated(arguments, job_id):
|
|
|
938
1069
|
model = local_embedder.resolve(cfg.get("embedding_model"), indexer.info(root).get("model_id"))
|
|
939
1070
|
warnings = []
|
|
940
1071
|
vectors = {}
|
|
1072
|
+
overlay = None
|
|
941
1073
|
search_model = model
|
|
942
1074
|
if problem:
|
|
943
1075
|
model_client.OFFLINE.set(problem)
|
|
@@ -974,18 +1106,23 @@ def _perform_smart_search_isolated(arguments, job_id):
|
|
|
974
1106
|
warnings.append("Embedding unavailable; lexical results from the last ready index. The update was not confirmed. " + _project_error(exc))
|
|
975
1107
|
search_model = None
|
|
976
1108
|
if vectors:
|
|
977
|
-
|
|
978
|
-
|
|
979
|
-
|
|
980
|
-
|
|
981
|
-
|
|
982
|
-
|
|
983
|
-
|
|
1109
|
+
lag = _index_lag(root, model, cfg, problem, job_id)
|
|
1110
|
+
if lag is not None:
|
|
1111
|
+
overlay, notice = _lag_overlay(root, lag)
|
|
1112
|
+
warnings += [text for text in (notice, _refresh_in_background(root, lag)) if text]
|
|
1113
|
+
else:
|
|
1114
|
+
try:
|
|
1115
|
+
_prepare_project_index(arguments, job_id, cfg, token, query_vector=next(iter(vectors.values())),
|
|
1116
|
+
model=model)
|
|
1117
|
+
except ProjectJobCancelled:
|
|
984
1118
|
raise
|
|
985
|
-
|
|
986
|
-
|
|
987
|
-
|
|
988
|
-
|
|
1119
|
+
except Exception as exc:
|
|
1120
|
+
if not indexer.info(root).get("chunks"):
|
|
1121
|
+
raise
|
|
1122
|
+
warnings.append("Index update failed; results from the last ready index. " + _project_error(exc))
|
|
1123
|
+
search_model = None
|
|
1124
|
+
if indexer.info(root).get("model_id") != model:
|
|
1125
|
+
vectors = {}
|
|
989
1126
|
if vectors and not cfg.get("rerank_model"):
|
|
990
1127
|
warnings.append("Rerank not configured: hybrid lexical/vector order, less precise at the top. Configure a rerank_model.")
|
|
991
1128
|
_job_checkpoint(job_id)
|
|
@@ -1014,7 +1151,7 @@ def _perform_smart_search_isolated(arguments, job_id):
|
|
|
1014
1151
|
query_terms = _symbol_terms(" ".join(queries[group] for group in groups if group != "doc"))
|
|
1015
1152
|
_update_job(job_id, phase="searching")
|
|
1016
1153
|
found = {group: indexer.search(root, vectors.get(queries[group]), top_k=max(limits[group] * 4, candidate_pool),
|
|
1017
|
-
query=queries[group], model_id=search_model,
|
|
1154
|
+
query=queries[group], model_id=search_model, overlay=overlay,
|
|
1018
1155
|
include=lambda path, group=group: index_profile.kind(path, profile) == group)
|
|
1019
1156
|
for group in groups}
|
|
1020
1157
|
|
|
@@ -1034,7 +1171,12 @@ def _perform_smart_search_isolated(arguments, job_id):
|
|
|
1034
1171
|
ordered = [(float(item["relevance_score"]), "rerank", candidates[item["index"]][1:]) for item in reranked]
|
|
1035
1172
|
except Exception as exc:
|
|
1036
1173
|
problem = f"Rerank unavailable for {group}; hybrid lexical/vector order kept. " + _project_error(exc)
|
|
1037
|
-
|
|
1174
|
+
exact = indexer.overlay_exact_keys(overlay, query)
|
|
1175
|
+
if exact:
|
|
1176
|
+
pinned = [(c[0], "overlay", c[1:]) for c in candidates if tuple(c[1:4]) in exact][:2]
|
|
1177
|
+
pinned_keys = {tuple(item[2][:3]) for item in pinned}
|
|
1178
|
+
ordered = pinned + [item for item in ordered if tuple(item[2][:3]) not in pinned_keys]
|
|
1179
|
+
rows =[{"path": path, "lines": f"{start}-{end}", "score": round(score, 6), "snippet": text[:500],
|
|
1038
1180
|
"ranking": ranking, "location_kind": document_text.location_kind(path),
|
|
1039
1181
|
**({"symbols": _chunk_symbols(symbol_index, path, start, end, text, query_terms)}
|
|
1040
1182
|
if symbol_index and group != "doc" else {})}
|
|
@@ -1049,6 +1191,8 @@ def _perform_smart_search_isolated(arguments, job_id):
|
|
|
1049
1191
|
if problem:
|
|
1050
1192
|
warnings.append(problem)
|
|
1051
1193
|
_job_checkpoint(job_id)
|
|
1194
|
+
if not any(blocks.values()):
|
|
1195
|
+
notes.append(EMPTY_RESULT_NOTICE)
|
|
1052
1196
|
warning = " ".join(warnings)
|
|
1053
1197
|
if warning:
|
|
1054
1198
|
_update_job(job_id, warning=warning)
|
|
@@ -1119,7 +1263,8 @@ def _run_project_job_in_view(job_id, arguments, kind):
|
|
|
1119
1263
|
if updated:
|
|
1120
1264
|
fields.update(indexed_seq=captured_seq, last_indexed=time.time())
|
|
1121
1265
|
project_store.update(project_id, **fields)
|
|
1122
|
-
_log(f"project_job {job_id} {project_id} {kind} done elapsed={time.time()-job['created_at']:.2f}s"
|
|
1266
|
+
_log(f"project_job {job_id} {project_id} {kind} done elapsed={time.time()-job['created_at']:.2f}s "
|
|
1267
|
+
f"phases={_phase_timings(final)}")
|
|
1123
1268
|
return result
|
|
1124
1269
|
except Exception as exc:
|
|
1125
1270
|
with _JOBS_LOCK:
|
|
@@ -1214,8 +1359,13 @@ def _search_limits(value, groups, saved=None):
|
|
|
1214
1359
|
for group in groups:
|
|
1215
1360
|
raw = value if not isinstance(value, dict) else value.get(group, (saved or {}).get(group, SEARCH_LIMIT_DEFAULT))
|
|
1216
1361
|
limits[group] = _limit_value(raw, f"top_k.{group}")
|
|
1217
|
-
|
|
1218
|
-
|
|
1362
|
+
total = sum(limits.values())
|
|
1363
|
+
if total > SEARCH_LIMIT_TOTAL:
|
|
1364
|
+
# A first call that asks for more than the total is scaled down in proportion (a rejection costs the agent a turn).
|
|
1365
|
+
scaled = {group: max(1, limit * SEARCH_LIMIT_TOTAL // total) for group, limit in limits.items()}
|
|
1366
|
+
while sum(scaled.values()) > SEARCH_LIMIT_TOTAL:
|
|
1367
|
+
scaled[max(scaled, key=scaled.get)] -= 1
|
|
1368
|
+
limits = scaled
|
|
1219
1369
|
return limits
|
|
1220
1370
|
|
|
1221
1371
|
|
|
@@ -1313,6 +1463,7 @@ def _pending_notices(session, client, project_root):
|
|
|
1313
1463
|
pending = []
|
|
1314
1464
|
if session["hook_notice"] and "hook" not in delivered:
|
|
1315
1465
|
pending.append(("hook", session["hook_notice"]))
|
|
1466
|
+
pending += [(key, f"{text} Tell the user.") for key, text in _update_notices() if key not in delivered]
|
|
1316
1467
|
key = "audit:" + setup_audit.index_key(project_root)
|
|
1317
1468
|
if key not in delivered and not client_hooks.audit_hook_installed(client):
|
|
1318
1469
|
text = setup_audit.session_notice(project_root)
|
|
@@ -1321,6 +1472,16 @@ def _pending_notices(session, client, project_root):
|
|
|
1321
1472
|
return pending
|
|
1322
1473
|
|
|
1323
1474
|
|
|
1475
|
+
def _update_notices():
|
|
1476
|
+
"""Update notices (updated, failed, newer version on npm); an unreadable state file is logged, not raised, so a
|
|
1477
|
+
search never fails over it."""
|
|
1478
|
+
try:
|
|
1479
|
+
return update_check.notices()
|
|
1480
|
+
except Exception as exc:
|
|
1481
|
+
_log(f"update notices unavailable: {type(exc).__name__}: {exc}")
|
|
1482
|
+
return []
|
|
1483
|
+
|
|
1484
|
+
|
|
1324
1485
|
def _smart_search_arguments(arguments):
|
|
1325
1486
|
if not isinstance(arguments, dict):
|
|
1326
1487
|
raise ValueError("smart_search requires its arguments as a JSON object")
|
|
@@ -3376,6 +3537,10 @@ def _pretool_hook(handler):
|
|
|
3376
3537
|
client = (urllib.parse.parse_qs(urllib.parse.urlsplit(handler.path).query).get("client") or [""])[0]
|
|
3377
3538
|
try:
|
|
3378
3539
|
payload = json.loads(handler._read_body(HOOK_MAX_BODY_BYTES) or b"{}")
|
|
3540
|
+
try:
|
|
3541
|
+
_note_edit(payload)
|
|
3542
|
+
except Exception as exc:
|
|
3543
|
+
_log(f"edit index update skipped: {type(exc).__name__}: {exc}")
|
|
3379
3544
|
output = hook_decision.decide(payload, client or None, _web_route)
|
|
3380
3545
|
except Exception as exc:
|
|
3381
3546
|
_log(f"PreToolUse hook without a decision: {type(exc).__name__}: {exc}")
|
|
@@ -3388,17 +3553,23 @@ def _pretool_hook(handler):
|
|
|
3388
3553
|
|
|
3389
3554
|
def _session_audit_hook(handler):
|
|
3390
3555
|
"""SessionStart output for the audit report hook: the stored audit notice of the session's folder as
|
|
3391
|
-
additionalContext,
|
|
3556
|
+
additionalContext, and the update notices both as systemMessage (shown to the user) and as context; `{}` when
|
|
3557
|
+
there is nothing to say. Never runs an audit."""
|
|
3392
3558
|
try:
|
|
3393
3559
|
payload = json.loads(handler._read_body(HOOK_MAX_BODY_BYTES) or b"{}")
|
|
3394
3560
|
cwd = payload.get("cwd")
|
|
3395
3561
|
text = setup_audit.session_notice(cwd) if isinstance(cwd, str) and os.path.isdir(cwd) else ""
|
|
3396
3562
|
except Exception as exc:
|
|
3397
3563
|
_log(f"SessionStart audit hook without a report: {type(exc).__name__}: {exc}")
|
|
3564
|
+
text = ""
|
|
3565
|
+
updates = " ".join(notice for _key, notice in _update_notices())
|
|
3566
|
+
context = " ".join(part for part in (updates, text) if part)
|
|
3567
|
+
if not context:
|
|
3398
3568
|
return {}
|
|
3399
|
-
|
|
3400
|
-
|
|
3401
|
-
|
|
3569
|
+
output = {"hookSpecificOutput": {"hookEventName": "SessionStart", "additionalContext": context}}
|
|
3570
|
+
if updates:
|
|
3571
|
+
output["systemMessage"] = updates
|
|
3572
|
+
return output
|
|
3402
3573
|
|
|
3403
3574
|
|
|
3404
3575
|
_TOOL_HANDLERS = {
|
|
@@ -3823,6 +3994,10 @@ def main():
|
|
|
3823
3994
|
server = _bind_server()
|
|
3824
3995
|
_BOUND_PORT = server.server_address[1]
|
|
3825
3996
|
threading.Thread(target=_probe_capabilities, daemon=True).start()
|
|
3997
|
+
try:
|
|
3998
|
+
update_check.note_running_version()
|
|
3999
|
+
except Exception as exc:
|
|
4000
|
+
_log(f"running version not recorded: {type(exc).__name__}: {exc}")
|
|
3826
4001
|
_write_registry(_BOUND_PORT)
|
|
3827
4002
|
atexit.register(_remove_registry)
|
|
3828
4003
|
try:
|
package/tray.py
CHANGED
|
@@ -111,24 +111,48 @@ def _poll_loop(icon):
|
|
|
111
111
|
time.sleep(POLL_INTERVAL_S)
|
|
112
112
|
|
|
113
113
|
|
|
114
|
+
def _announce_start(icon):
|
|
115
|
+
"""Tells the user, once per start, that Smart Tool was just updated or that the last automatic update failed."""
|
|
116
|
+
updated = update_check.note_running_version()
|
|
117
|
+
if updated:
|
|
118
|
+
icon.notify(f"Smart Tool updated from {updated['from']} to {updated['to']}.", "Smart Tool")
|
|
119
|
+
failed = update_check.failed_update()
|
|
120
|
+
if failed:
|
|
121
|
+
icon.notify(f"The update to {failed} failed; Smart Tool {version.VERSION} is still installed. "
|
|
122
|
+
"It is tried again in 24 h, or now from this menu.", "Smart Tool")
|
|
123
|
+
|
|
124
|
+
|
|
114
125
|
def _update_loop(icon):
|
|
115
|
-
"""Checks npm at start and every 24 h.
|
|
116
|
-
|
|
117
|
-
|
|
126
|
+
"""Checks npm at start and every 24 h. A newer version is installed right away when automatic updates are on (once
|
|
127
|
+
per version, again 24 h after a failed try); otherwise the user is told and the menu offers it."""
|
|
128
|
+
try:
|
|
129
|
+
_announce_start(icon)
|
|
130
|
+
except Exception as exc:
|
|
131
|
+
icon.notify(f"Could not read the update state: {exc}", "Smart Tool")
|
|
118
132
|
while True:
|
|
119
133
|
try:
|
|
120
134
|
latest = update_check.newer_version()
|
|
121
135
|
except Exception:
|
|
122
136
|
latest = None
|
|
123
|
-
|
|
137
|
+
else:
|
|
138
|
+
try:
|
|
139
|
+
update_check.record_check(latest)
|
|
140
|
+
except Exception as exc:
|
|
141
|
+
icon.notify(f"Could not save the update check: {exc}", "Smart Tool")
|
|
142
|
+
if latest:
|
|
143
|
+
known = latest == _state["update"]
|
|
124
144
|
_state["update"] = latest
|
|
125
145
|
icon.update_menu()
|
|
126
|
-
|
|
127
|
-
|
|
146
|
+
try:
|
|
147
|
+
automatic = update_check.automatic_enabled() and update_check.should_auto_update(latest)
|
|
148
|
+
except Exception as exc:
|
|
149
|
+
automatic = False
|
|
150
|
+
icon.notify(f"Automatic update skipped: {exc}", "Smart Tool")
|
|
151
|
+
if automatic:
|
|
152
|
+
icon.notify(f"Updating Smart Tool from {version.VERSION} to {latest}...", "Smart Tool")
|
|
128
153
|
update_check.run_update(latest, automatic=True)
|
|
129
|
-
|
|
154
|
+
elif not known:
|
|
130
155
|
icon.notify(f"Smart Tool {latest} is available: use \"Update to {latest}\" in this menu.", "Smart Tool")
|
|
131
|
-
first = False
|
|
132
156
|
time.sleep(UPDATE_CHECK_INTERVAL_S)
|
|
133
157
|
|
|
134
158
|
|
|
@@ -162,12 +186,16 @@ def _update_now(icon, item):
|
|
|
162
186
|
|
|
163
187
|
def _auto_update_checked(item):
|
|
164
188
|
try:
|
|
165
|
-
return
|
|
189
|
+
return update_check.automatic_enabled()
|
|
166
190
|
except Exception:
|
|
167
191
|
return False
|
|
168
192
|
|
|
169
193
|
|
|
170
194
|
def _toggle_auto_update(icon, item):
|
|
195
|
+
if update_check.env_opted_out():
|
|
196
|
+
icon.notify(f"{update_check.ENV_OPT_OUT}=off is set: automatic updates stay off until it is removed.",
|
|
197
|
+
"Smart Tool")
|
|
198
|
+
return
|
|
171
199
|
try:
|
|
172
200
|
config.update_config(lambda cfg: cfg.__setitem__("auto_update", not config.auto_update(cfg)))
|
|
173
201
|
except Exception as exc:
|
package/update_check.py
CHANGED
|
@@ -1,5 +1,10 @@
|
|
|
1
|
-
"""Newer Smart Tool release on npm and the update itself: the tray
|
|
2
|
-
|
|
1
|
+
"""Newer Smart Tool release on npm and the update itself: the tray checks npm at start and every 24 h and, when
|
|
2
|
+
automatic updates are on (the default), runs the same install command a user would, in a visible console. A failed
|
|
3
|
+
version is tried again after 24 h. What happened (newer version available, updated, update failed) is told to the
|
|
4
|
+
user by the tray and to agent sessions by the daemon, from the state file, without contacting npm again.
|
|
5
|
+
|
|
6
|
+
Automatic updates are turned off by the tray switch (config `auto_update`) or by SMART_TOOL_AUTO_UPDATE=off; the
|
|
7
|
+
newer-version notice still shows then, with the command to update by hand."""
|
|
3
8
|
import json
|
|
4
9
|
import os
|
|
5
10
|
import subprocess
|
|
@@ -7,13 +12,18 @@ import time
|
|
|
7
12
|
import urllib.request
|
|
8
13
|
|
|
9
14
|
import atomic_io
|
|
15
|
+
import config
|
|
10
16
|
import paths
|
|
11
17
|
import version
|
|
12
18
|
|
|
13
19
|
PACKAGE = "@allansantos-dev/smart-tool"
|
|
14
20
|
REGISTRY_URL = "https://registry.npmjs.org/@allansantos-dev%2Fsmart-tool/latest"
|
|
15
21
|
UPDATE_COMMAND = f"npx --yes {PACKAGE}@latest install"
|
|
22
|
+
CHANGELOG_URL = "https://github.com/AllanSantos-DV/smart-tool-dist/blob/main/CHANGELOG.md"
|
|
16
23
|
STATE_PATH = os.path.join(paths.DATA_DIR, "update-state.json")
|
|
24
|
+
ENV_OPT_OUT = "SMART_TOOL_AUTO_UPDATE"
|
|
25
|
+
RETRY_AFTER_S = 24 * 3600
|
|
26
|
+
UPDATED_NOTICE_S = 24 * 3600
|
|
17
27
|
|
|
18
28
|
|
|
19
29
|
def _parts(value):
|
|
@@ -29,27 +39,108 @@ def newer_version(timeout=10):
|
|
|
29
39
|
return latest if _parts(latest) > _parts(version.VERSION) else None
|
|
30
40
|
|
|
31
41
|
|
|
32
|
-
def
|
|
33
|
-
"""
|
|
42
|
+
def env_opted_out():
|
|
43
|
+
"""SMART_TOOL_AUTO_UPDATE set to off, 0, false or no."""
|
|
44
|
+
return os.environ.get(ENV_OPT_OUT, "").strip().lower() in ("off", "0", "false", "no")
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
def automatic_enabled(cfg=None):
|
|
48
|
+
"""Automatic updates are on: the tray switch (config `auto_update`, on by default) and no environment opt-out."""
|
|
49
|
+
return not env_opted_out() and config.auto_update(cfg)
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
def _state():
|
|
34
53
|
try:
|
|
35
54
|
with open(STATE_PATH, encoding="utf-8") as stream:
|
|
36
|
-
return json.load(stream)
|
|
55
|
+
return json.load(stream)
|
|
37
56
|
except FileNotFoundError:
|
|
57
|
+
return {}
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def _save(state):
|
|
61
|
+
os.makedirs(os.path.dirname(STATE_PATH), exist_ok=True)
|
|
62
|
+
atomic_io.write_secret_text(STATE_PATH, json.dumps(state))
|
|
63
|
+
|
|
64
|
+
|
|
65
|
+
def attempted():
|
|
66
|
+
"""Version the last automatic update went for, or None."""
|
|
67
|
+
return _state().get("attempted")
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
def should_auto_update(latest, now=None):
|
|
71
|
+
"""An automatic update runs once per version and again 24 h after a failed try: when it fails, the installer
|
|
72
|
+
restores the previous version and restarts the tray, which would otherwise try the same version on every start."""
|
|
73
|
+
if not latest:
|
|
74
|
+
return False
|
|
75
|
+
state = _state()
|
|
76
|
+
if state.get("attempted") != latest:
|
|
77
|
+
return True
|
|
78
|
+
return (now or time.time()) - state.get("at", 0) >= RETRY_AFTER_S
|
|
79
|
+
|
|
80
|
+
|
|
81
|
+
def record_check(latest):
|
|
82
|
+
"""Stores the result of an npm check (the newer version, or None) for the notices read without network."""
|
|
83
|
+
state = _state()
|
|
84
|
+
state.update(latest=latest, checked_at=time.time())
|
|
85
|
+
_save(state)
|
|
86
|
+
|
|
87
|
+
|
|
88
|
+
def note_running_version(now=None):
|
|
89
|
+
"""Called when the tray and the daemon start: a version different from the one that ran last is recorded as an
|
|
90
|
+
update (from, to, at) for the updated notice; the very first run records nothing. Returns the update record when
|
|
91
|
+
this call found it, else None."""
|
|
92
|
+
state = _state()
|
|
93
|
+
running = state.get("running")
|
|
94
|
+
if running == version.VERSION:
|
|
38
95
|
return None
|
|
96
|
+
state["running"] = version.VERSION
|
|
97
|
+
found = None
|
|
98
|
+
if running:
|
|
99
|
+
found = state["updated"] = {"from": running, "to": version.VERSION, "at": now or time.time()}
|
|
100
|
+
_save(state)
|
|
101
|
+
return found
|
|
102
|
+
|
|
103
|
+
|
|
104
|
+
def failed_update():
|
|
105
|
+
"""The version an automatic update went for and did not install (the previous version is still running), else
|
|
106
|
+
None."""
|
|
107
|
+
target = _state().get("attempted")
|
|
108
|
+
return target if target and _parts(target) > _parts(version.VERSION) else None
|
|
39
109
|
|
|
40
110
|
|
|
41
|
-
def
|
|
42
|
-
"""
|
|
43
|
-
|
|
44
|
-
|
|
111
|
+
def notices(now=None):
|
|
112
|
+
"""[(key, text)] about updates, for the user and the agent: Smart Tool updated in the last 24 h, the last
|
|
113
|
+
automatic update failed, or a newer version is on npm. Reads only the state file."""
|
|
114
|
+
now = now or time.time()
|
|
115
|
+
state = _state()
|
|
116
|
+
found = []
|
|
117
|
+
updated = state.get("updated") or {}
|
|
118
|
+
if updated.get("to") == version.VERSION and now - updated.get("at", 0) < UPDATED_NOTICE_S:
|
|
119
|
+
found.append((f"updated:{version.VERSION}",
|
|
120
|
+
f"Smart Tool was updated from {updated['from']} to {version.VERSION}. Changes: {CHANGELOG_URL}"))
|
|
121
|
+
failed = failed_update()
|
|
122
|
+
latest = state.get("latest")
|
|
123
|
+
if failed:
|
|
124
|
+
retry = ("the tray tries again 24 h after the failed try" if automatic_enabled()
|
|
125
|
+
else "automatic updates are now off")
|
|
126
|
+
found.append((f"failed:{failed}",
|
|
127
|
+
f"The automatic update of Smart Tool to {failed} failed; {version.VERSION} is still installed "
|
|
128
|
+
f"and {retry}. To update now, run: {UPDATE_COMMAND}"))
|
|
129
|
+
elif latest and _parts(latest) > _parts(version.VERSION):
|
|
130
|
+
how = ("the tray installs it automatically" if automatic_enabled()
|
|
131
|
+
else f"automatic updates are off (tray switch or {ENV_OPT_OUT}=off); to update, run: {UPDATE_COMMAND}")
|
|
132
|
+
found.append((f"available:{latest}",
|
|
133
|
+
f"Smart Tool {latest} is available on npm (installed: {version.VERSION}); {how}."))
|
|
134
|
+
return found
|
|
45
135
|
|
|
46
136
|
|
|
47
137
|
def run_update(latest, automatic):
|
|
48
138
|
"""Starts the install command in its own visible console, detached from the tray (the installer stops and restarts
|
|
49
139
|
the tray). The console closes on success and stays open with the error on failure."""
|
|
50
140
|
if automatic:
|
|
51
|
-
|
|
52
|
-
|
|
141
|
+
state = _state()
|
|
142
|
+
state.update(attempted=latest, at=time.time())
|
|
143
|
+
_save(state)
|
|
53
144
|
flags = getattr(subprocess, "CREATE_NEW_CONSOLE", 0) | getattr(subprocess, "CREATE_NEW_PROCESS_GROUP", 0)
|
|
54
145
|
command = f'title Smart Tool update to {latest} && {UPDATE_COMMAND} || pause'
|
|
55
146
|
subprocess.Popen(["cmd.exe", "/c", command], creationflags=flags, close_fds=True,
|
package/version.py
CHANGED