@allansantos-dev/smart-tool 0.9.71 → 0.9.73

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,32 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.9.73 - beta
4
+
5
+ - Once a day (first pass 10 minutes after the daemon starts) the daemon removes, in each registered project without a
6
+ running job, the index views that are not pinned and were not updated for 30 days (`project_storage.auto_cleanup`). It
7
+ goes through the manual cleanup's own preview and commit, so every protection of `cleanup_commit` applies; a refused
8
+ project is kept and logged. A removed view is rebuilt from the cached vectors when its branch is indexed again. Tests:
9
+ `tests/test_storage_auto_cleanup.py`.
10
+ - The local embedder and the gateway models listings go through the shared model protections (circuit, in-flight cap,
11
+ `Retry-After`, the model health table), like the gateway calls. Tests: `tests/test_local_embedder.py`,
12
+ `tests/test_model_health.py`.
13
+ - The usage meter counts calls per purpose by hour (`usage_hourly`), so the 24 h window of the setup screen covers the
14
+ last 24 hours instead of following the UTC day.
15
+ - Product page: larger body text, 44 px touch targets, a labelled tool group, a favicon file, and the product video
16
+ rebuilt with the default voice and a window frame without an address bar. `release/pages.py` no longer publishes hidden
17
+ folders of `site/` (tool state such as `.token-guard`). The release notes section lists the three latest versions it
18
+ claims (it listed twelve), and `release/pages.check_site` refuses to publish otherwise. Tests:
19
+ `tests/test_site_quality.py`, `tests/test_pages.py`.
20
+
21
+ ## 0.9.72 - beta
22
+
23
+ - A Smart Tool hook entry that is absent from the client's config is no longer reported `missing` when the Skill Manager holds
24
+ an item of Smart Tool back: `client_hooks.inspect` asks `skill-engine query --owner smart-tool --json` (through the `cli.json`
25
+ pointer in `SKILL_ENGINE_HOME` or `~/.skill-engine`) and answers `waiting_approval` for a quarantined or held item and
26
+ `update_held` for an approved copy whose new version waits; the settings page shows "Waiting for approval" and "Update held",
27
+ and the notice to the agent says there is nothing to install. Without the Manager, or with items active or absent, the
28
+ state stays `missing`; a failed query shows as the hook being unreadable. Tests: `tests/test_hook_manager_state.py`.
29
+
3
30
  ## 0.9.71 - beta
4
31
 
5
32
  - `npx @allansantos-dev/smart-tool@0.9.71 uninstall [--target DIR] [--keep-data]` (`python uninstall.py`): silent uninstall that
package/README.md CHANGED
@@ -17,7 +17,7 @@ registered as the `playwright` server where this machine lets a browser be drive
17
17
  An optional hook routes the agent's native Grep/Glob/Read/Bash/WebSearch/WebFetch calls to these tools when they
18
18
  are cheaper, or just tells the agent what Smart Tool would do.
19
19
 
20
- **Status:** 0.9.71 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
20
+ **Status:** 0.9.73 beta. Windows only. What changed in each version: [CHANGELOG.md](CHANGELOG.md).
21
21
 
22
22
  ## How it works
23
23
 
@@ -83,7 +83,7 @@ gateway such as Ollama, nothing about your code leaves the machine.
83
83
  ## Install
84
84
 
85
85
  ```powershell
86
- npx @allansantos-dev/smart-tool@0.9.71 install
86
+ npx @allansantos-dev/smart-tool@0.9.73 install
87
87
  ```
88
88
 
89
89
  Updates install themselves, on by default: the tray checks npm when it starts (at logon) and every 24 h, and when there
@@ -382,7 +382,7 @@ fails to load is listed in the setup screen with its error and does not affect t
382
382
  ## Uninstall
383
383
 
384
384
  ```powershell
385
- npx @allansantos-dev/smart-tool@0.9.71 uninstall [--target DIR] [--keep-data]
385
+ npx @allansantos-dev/smart-tool@0.9.73 uninstall [--target DIR] [--keep-data]
386
386
  ```
387
387
 
388
388
  It asks nothing and opens no window. It stops the tray and the daemon, removes the logon task, the hook entries of Smart
package/client_hooks.py CHANGED
@@ -3,6 +3,7 @@ import json
3
3
  import os
4
4
  import secrets
5
5
  import shutil
6
+ import subprocess
6
7
  import sys
7
8
  import time
8
9
  import tomllib
@@ -23,6 +24,9 @@ AUDIT_TIMEOUT_S = 10
23
24
  AUDIT_CLIENTS = ("claude", "codex")
24
25
  AUDIT_ON = "current"
25
26
  AUDIT_WAITING = "waiting_approval"
27
+ OWNER = "smart-tool"
28
+ MANAGER_TIMEOUT_S = 5
29
+ MANAGER_HELD = ("quarantined", "held")
26
30
  HTTP_PATH = "/hooks/pretool"
27
31
  POST_PATH = "/hooks/posttool"
28
32
  # A redirected shell search waits for its answer in the PreToolUse (smart_tool_daemon.HOOK_SHELL_BUDGET_S).
@@ -89,10 +93,47 @@ def _target(handler):
89
93
  return handler.get("url") or handler.get("command") or ""
90
94
 
91
95
 
92
- def inspect(client, trust=True):
96
+ def manager_items(owner=OWNER):
97
+ """Items the Skill Manager holds for `owner` (`skill-engine query --owner <owner> --json`), found through the
98
+ cli.json pointer in its home (SKILL_ENGINE_HOME or ~/.skill-engine). [] when the Manager is not installed;
99
+ an unreadable pointer or a failed query raises."""
100
+ pointer = os.path.join(os.environ.get("SKILL_ENGINE_HOME") or os.path.join(HOME, ".skill-engine"), "cli.json")
101
+ try:
102
+ with open(pointer, encoding="utf-8") as stream:
103
+ cli = json.load(stream)
104
+ except FileNotFoundError:
105
+ return []
106
+ flags = subprocess.CREATE_NO_WINDOW if os.name == "nt" else 0
107
+ result = subprocess.run([cli["node"], cli["cli"], "query", "--owner", owner, "--json"], capture_output=True,
108
+ stdin=subprocess.DEVNULL, text=True, encoding="utf-8", errors="replace",
109
+ timeout=MANAGER_TIMEOUT_S, creationflags=flags)
110
+ if result.returncode != 0:
111
+ raise OSError(f"skill-engine query exited {result.returncode}: {result.stderr.strip()[:200]}")
112
+ return json.loads(result.stdout).get("items") or []
113
+
114
+
115
+ def _missing_state():
116
+ """`missing`, unless the Skill Manager holds something of ours back: `waiting_approval` for a quarantined or held
117
+ item, `update_held` for an approved copy whose new version waits."""
118
+ items = manager_items()
119
+ waiting = [item for item in items if item.get("state") in MANAGER_HELD]
120
+ if waiting:
121
+ return {"state": "waiting_approval", "changes": [f"{item.get('id')}: {item.get('reason') or item['state']}. "
122
+ f"Approve with `{item.get('approve') or 'skill-engine dashboard'}`."
123
+ for item in waiting]}
124
+ pending = [item for item in items if item.get("pendingUpdate")]
125
+ if pending:
126
+ return {"state": "update_held", "changes": [f"{item.get('id')}: a new version waits for approval "
127
+ f"(`{item.get('approve') or 'skill-engine dashboard'}`)." for item in pending]}
128
+ return {"state": "missing", "changes": []}
129
+
130
+
131
+ def inspect(client, trust=True, manager=True):
93
132
  """Our hook in the client's config: `current`, `missing`, or `outdated` (an older entry of ours) with `changes`,
94
133
  plain sentences saying what an update rewrites (tools the old matcher leaves out, an old address or timeout).
95
- In Codex a current entry that config.toml does not trust is outdated too, unless trust=False."""
134
+ In Codex a current entry that config.toml does not trust is outdated too, unless trust=False. Where the entry is
135
+ absent and the Skill Manager holds an item of ours, `waiting_approval` or `update_held` (manager=False skips that
136
+ query)."""
96
137
  spec = CLIENTS[client]
97
138
  for path in spec["files"]:
98
139
  try:
@@ -113,7 +154,7 @@ def inspect(client, trust=True):
113
154
  if client == "codex" and trust and not changes and not _codex_trusted(event, groups, _our_hook):
114
155
  changes.append(f"Codex has not approved its {event} entry; an update records the approval.")
115
156
  return {"state": "outdated" if changes else "current", "changes": changes}
116
- return {"state": "missing", "changes": []}
157
+ return _missing_state() if manager else {"state": "missing", "changes": []}
117
158
 
118
159
 
119
160
  def _entry_changes(event, matcher, expected, groups):
@@ -153,7 +194,7 @@ def status(client_info):
153
194
  spec = CLIENTS[client]
154
195
  try:
155
196
  found, error = inspect(client), None
156
- except (OSError, ValueError) as exc:
197
+ except (OSError, ValueError, subprocess.SubprocessError) as exc:
157
198
  found, error = {"state": None, "changes": []}, f"{type(exc).__name__}: {exc}"[:200]
158
199
  return {"client": client, "name": client_info.get("name"),
159
200
  "installed": None if found["state"] is None else found["state"] == "current", "state": found["state"],
@@ -172,6 +213,10 @@ def notice(state):
172
213
  return None
173
214
  action = spec.get("blocked") or ("If the user asks, use project_manage action=install_hook_preview, show the "
174
215
  "change and confirm with action=install_hook (makes a backup).")
216
+ if state.get("state") in ("waiting_approval", "update_held"):
217
+ label = "waiting for approval" if state["state"] == "waiting_approval" else "update held"
218
+ return (f"Smart Tool hook {label} in the Skill Manager ({spec['label']}): {' '.join(state['changes'])} "
219
+ "Nothing to install.")
175
220
  if state.get("state") == "outdated":
176
221
  return (f"Smart Tool hook outdated in {spec['label']} ({spec['files'][0]}): {' '.join(state['changes'])} "
177
222
  f"{action} Docs: {spec['docs']}")
@@ -311,7 +356,7 @@ def remove(client):
311
356
  del hooks[event]
312
357
  if not moved:
313
358
  return {"client": client, "changed": False, "file": path, "backup": None}
314
- backup = _write_checked(path, data, lambda: inspect(client, trust=False)["state"] == "missing", "The removal")
359
+ backup = _write_checked(path, data, lambda: inspect(client, trust=False, manager=False)["state"] == "missing", "The removal")
315
360
  if client == "codex":
316
361
  for event, (before, kept) in moved.items():
317
362
  record_codex_trust(_codex_trust(event, before, kept))
package/local_embedder.py CHANGED
@@ -9,10 +9,13 @@ import json
9
9
  import os
10
10
  import threading
11
11
  import time
12
+ import urllib.error
12
13
  import urllib.parse
13
14
 
14
15
  import config
15
16
  import indexer
17
+ import model_breaker
18
+ import model_health
16
19
 
17
20
  PREFIX = "local:"
18
21
  PROPERTIES = os.path.join(os.path.expanduser("~"), ".mcp-memory", "embedding.properties")
@@ -60,7 +63,7 @@ def _request(conn, method, path, body=None):
60
63
  response = conn.getresponse()
61
64
  data = response.read()
62
65
  if response.status >= 400:
63
- raise ConnectionError(f"local embedder answered HTTP {response.status}")
66
+ raise urllib.error.HTTPError(f"{conn.host}:{conn.port}{path}", response.status, response.reason, response.headers, None)
64
67
  return json.loads(data)
65
68
 
66
69
 
@@ -123,6 +126,13 @@ def _left(deadline):
123
126
  return min(REQUEST_TIMEOUT_S, left)
124
127
 
125
128
 
129
+ def _retry_delay(failure, attempt):
130
+ """Seconds to wait after a failed attempt: the `Retry-After` of the answer (whole seconds, at most the pause of the
131
+ circuit) when the sidecar sent one, else the attempt number."""
132
+ value = str(getattr(failure, "headers", None) and failure.headers.get("Retry-After") or "")
133
+ return min(int(value), model_breaker.PAUSE_S) if value.isdigit() else attempt
134
+
135
+
126
136
  def embed(model, texts, deadline=None, cancel_check=None, on_retry=None):
127
137
  base = url()
128
138
  if not base:
@@ -144,21 +154,28 @@ def embed(model, texts, deadline=None, cancel_check=None, on_retry=None):
144
154
  for attempt in range(1, MAX_ATTEMPTS + 1):
145
155
  if cancel_check:
146
156
  cancel_check()
157
+ model_breaker.guard(model)
147
158
  conn.timeout = _left(deadline)
148
159
  try:
149
- got = _request(conn, "POST", "/embed", {"inputs": part})
150
- break
160
+ with model_breaker.slot(model):
161
+ got = _request(conn, "POST", "/embed", {"inputs": part})
151
162
  except (OSError, http.client.HTTPException) as exc:
163
+ model_breaker.note(model, exc)
164
+ model_health.record("/v1/embeddings", {"model": model}, exc)
152
165
  conn.close()
153
166
  conn = _connection(base, _left(deadline))
154
167
  if attempt == MAX_ATTEMPTS:
155
168
  raise RuntimeError(f"Local embedder failed after {MAX_ATTEMPTS} attempts: {exc}") from None
156
169
  if on_retry:
157
170
  on_retry(attempt + 1, len(part))
158
- delay = attempt
171
+ delay = _retry_delay(exc, attempt)
159
172
  if deadline is not None and time.monotonic() + delay >= deadline:
160
173
  raise TimeoutError("Not enough time left to retry the local embedder.") from None
161
174
  time.sleep(delay)
175
+ else:
176
+ model_breaker.note(model)
177
+ model_health.record("/v1/embeddings", {"model": model})
178
+ break
162
179
  if not isinstance(got, list) or len(got) != len(part):
163
180
  raise indexer.IndexVectorError("Local embedder returned a vector count different from the input.")
164
181
  for vector in got:
@@ -113,6 +113,16 @@
113
113
  "The redirect hook was written by an older Smart Tool.": "O hook de redirecionamento foi gravado por uma versão mais antiga do Smart Tool.",
114
114
  "No hook": "Sem hook",
115
115
  "Install hook": "Instalar hook",
116
+ "Waiting for approval": "Aguardando aprovação",
117
+ "The Skill Manager holds a Smart Tool item until you approve it.": "O Skill Manager retém um item do Smart Tool até você aprovar.",
118
+ "Update held": "Atualização retida",
119
+ "A new Smart Tool version waits for approval in the Skill Manager.": "Uma nova versão do Smart Tool aguarda aprovação no Skill Manager.",
120
+ "Smart Tool hook {0} in the Skill Manager ({1}): {2} Nothing to install.": "Hook do Smart Tool {0} no Skill Manager ({1}): {2} Não há nada a instalar.",
121
+ "waiting for approval": "aguardando aprovação",
122
+ "update held": "atualização retida",
123
+ "{0}: {1}. Approve with `{2}`.": "{0}: {1}. Aprove com `{2}`.",
124
+ "{0}: a new version waits for approval (`{1}`).": "{0}: uma nova versão aguarda aprovação (`{1}`).",
125
+ "skill-engine dashboard": "skill-engine dashboard",
116
126
  "Connected, but its folder searches and web lookups are not redirected to Smart Tool.": "Conectado, mas as buscas em pastas e as consultas na web dele não são redirecionadas para o Smart Tool.",
117
127
  "Connected": "Conectado",
118
128
  "Connection and redirect hook are up to date.": "A conexão e o hook de redirecionamento estão atualizados.",
package/model_health.py CHANGED
@@ -28,7 +28,7 @@ ERROR_CHARS = 300
28
28
  SAMPLE_CHARS = 160
29
29
  GROUP_ROWS = 20
30
30
  PURPOSES = ("index_embedding", "query_embedding", "rerank", "embedding_probe", "scope")
31
- _PATHS = ("/v1/chat/completions", "/v1/embeddings", "/v1/rerank")
31
+ _PATHS = ("/v1/chat/completions", "/v1/embeddings", "/v1/rerank", "/v1/models", "/v1/model/info")
32
32
  _PURPOSE_ROLE = {"scope": "scope_model", "scout": "scout_model", "docs": "docs_model", "rerank": "rerank_model",
33
33
  "index_embedding": "embedding_model", "query_embedding": "embedding_model"}
34
34
  _TOOL = contextvars.ContextVar("model_health_tool", default=None)
@@ -112,7 +112,7 @@ def record(path, body, failure=None):
112
112
  if path not in _PATHS:
113
113
  return
114
114
  try:
115
- model = str((body or {}).get("model", "")) if isinstance(body, dict) else ""
115
+ model = (str(body.get("model", "")) if isinstance(body, dict) else "") or path
116
116
  role = role_of(model, purpose=usage_meter.purpose())
117
117
  now = time.time()
118
118
  text = error_text(failure) if failure is not None else None
package/package.json CHANGED
@@ -1,35 +1,35 @@
1
- {
2
- "name": "@allansantos-dev/smart-tool",
3
- "version": "0.9.71",
4
- "description": "Local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharper tools than their built-in search. Windows.",
5
- "license": "Apache-2.0",
6
- "author": "Smart Tool contributors",
7
- "homepage": "https://allansantos-dv.github.io/smart-tool/",
8
- "repository": {
9
- "type": "git",
10
- "url": "git+https://github.com/AllanSantos-DV/smart-tool-dist.git"
11
- },
12
- "keywords": [
13
- "mcp",
14
- "claude-code",
15
- "codex",
16
- "semantic-search",
17
- "web-search",
18
- "coding-agent"
19
- ],
20
- "bin": {
21
- "smart-tool": "bin/smart-tool.mjs"
22
- },
23
- "os": [
24
- "win32"
25
- ],
26
- "cpu": [
27
- "x64"
28
- ],
29
- "engines": {
30
- "node": ">=18"
31
- },
32
- "publishConfig": {
33
- "access": "public"
34
- }
35
- }
1
+ {
2
+ "name": "@allansantos-dev/smart-tool",
3
+ "version": "0.9.73",
4
+ "description": "Local MCP server that gives coding agents (Claude Code, Codex) cheaper, sharper tools than their built-in search. Windows.",
5
+ "license": "Apache-2.0",
6
+ "author": "Smart Tool contributors",
7
+ "homepage": "https://allansantos-dv.github.io/smart-tool/",
8
+ "repository": {
9
+ "type": "git",
10
+ "url": "git+https://github.com/AllanSantos-DV/smart-tool-dist.git"
11
+ },
12
+ "keywords": [
13
+ "mcp",
14
+ "claude-code",
15
+ "codex",
16
+ "semantic-search",
17
+ "web-search",
18
+ "coding-agent"
19
+ ],
20
+ "bin": {
21
+ "smart-tool": "bin/smart-tool.mjs"
22
+ },
23
+ "os": [
24
+ "win32"
25
+ ],
26
+ "cpu": [
27
+ "x64"
28
+ ],
29
+ "engines": {
30
+ "node": ">=18"
31
+ },
32
+ "publishConfig": {
33
+ "access": "public"
34
+ }
35
+ }
@@ -19,6 +19,8 @@ import project_identity
19
19
  import project_store
20
20
 
21
21
  _PLANS={}
22
+ # A view of a branch nobody has indexed for this long is rebuilt from the cached vectors on demand.
23
+ STALE_VIEW_S=30*86400
22
24
 
23
25
 
24
26
  def _assets(root):
@@ -116,6 +118,21 @@ def preview(project,storage_ids,clear_cache=True):
116
118
  return result
117
119
 
118
120
 
121
+ def stale_view_ids(project,now=None):
122
+ """Storage ids of the views of `project` that are not protected and were last updated over STALE_VIEW_S ago."""
123
+ now=time.time() if now is None else now
124
+ return [v['storage_id'] for v in inventory(project)['views']
125
+ if not v['protected'] and v.get('updated_at') and now-v['updated_at']>STALE_VIEW_S]
126
+
127
+
128
+ def auto_cleanup(project,now=None):
129
+ """Removes the stale views of `project` through the same preview and commit as the manual cleanup, so what goes is
130
+ exactly what the preview lists and every protection of the commit applies. None when no view is stale."""
131
+ ids=stale_view_ids(project,now)
132
+ if not ids:return None
133
+ return commit(project,preview(project,ids,True)['plan_id'])
134
+
135
+
119
136
  def commit(project,plan_id):
120
137
  plan=_PLANS.pop(plan_id,None)
121
138
  if not plan or plan['project_id']!=project['id'] or plan['expires']<time.time():
package/setup_ui.py CHANGED
@@ -945,6 +945,10 @@ function agentState(item) {
945
945
  text: "Could not read " + item.file + ": " + item.hook_error };
946
946
  if (item.hook_state === "outdated") return { tone: "warn", label: "Hook outdated", action: "Update hook", hook: true,
947
947
  text: "The redirect hook was written by an older Smart Tool. " + item.hook_changes.join(" ") };
948
+ if (item.hook_state === "waiting_approval") return { tone: "warn", label: "Waiting for approval", action: "Open",
949
+ text: "The Skill Manager holds a Smart Tool item until you approve it. " + item.hook_changes.join(" ") };
950
+ if (item.hook_state === "update_held") return { tone: "warn", label: "Update held", action: "Open",
951
+ text: "A new Smart Tool version waits for approval in the Skill Manager. " + item.hook_changes.join(" ") };
948
952
  if (item.hook_state === "missing") return { tone: "warn", label: "No hook", action: "Install hook", hook: true,
949
953
  text: "Connected, but its folder searches and web lookups are not redirected to Smart Tool." };
950
954
  return { tone: "ok", label: "Connected", action: "Open", text: "Connection and redirect hook are up to date." };
@@ -44,6 +44,7 @@ import integration_metrics
44
44
  import node_locator
45
45
  import config
46
46
  import disk_cleanup
47
+ import project_storage
47
48
  import freshness
48
49
  import model_client
49
50
  import index_scope
@@ -2379,12 +2380,10 @@ def _project_action_locked(arguments):
2379
2380
  if action=='usage':
2380
2381
  return usage_meter.summary(key,arguments.get('days',30),arguments.get('view_id'))
2381
2382
  if action=='storage':
2382
- import project_storage
2383
2383
  return project_storage.inventory(project)
2384
2384
  active = _active_project_jobs(key)
2385
2385
  if action in ('pin_view','cleanup_preview','cleanup_commit'):
2386
2386
  if active:raise ValueError('Wait for this project\'s jobs to finish before changing storage.')
2387
- import project_storage
2388
2387
  if action=='pin_view':return project_storage.pin(project,arguments.get('storage_id'),arguments.get('pinned'))
2389
2388
  if action=='cleanup_preview':return project_storage.preview(project,arguments.get('storage_ids'),arguments.get('clear_cache',True))
2390
2389
  if arguments.get('confirm') is not True:raise ValueError('Confirm the cleanup preview before applying it.')
@@ -4769,6 +4768,39 @@ def _heal_browser_control():
4769
4768
  _log(f"browser control not refreshed: {type(exc).__name__}: {exc}")
4770
4769
 
4771
4770
 
4771
+ STORAGE_CLEANUP_FIRST_S = 600
4772
+ STORAGE_CLEANUP_EVERY_S = 24 * 3600
4773
+
4774
+
4775
+ def _storage_cleanup_pass():
4776
+ """Removes the stale views of every registered project that has no job running; returns how many views went. A
4777
+ project whose cleanup is refused (the branch changed, an index is open elsewhere) is kept and logged."""
4778
+ removed = 0
4779
+ for project in project_store.all_projects():
4780
+ if _active_project_jobs(project["id"]):
4781
+ continue
4782
+ try:
4783
+ result = project_storage.auto_cleanup(project)
4784
+ except (ValueError, RuntimeError, OSError) as exc:
4785
+ _log(f"storage cleanup: project {project['id']} kept: {exc}")
4786
+ continue
4787
+ if result:
4788
+ removed += result["removed_views"]
4789
+ _log(f"storage cleanup: {project['root']}: removed {result['removed_views']} view(s), "
4790
+ f"{result['removed_index_bytes']} bytes")
4791
+ return removed
4792
+
4793
+
4794
+ def _storage_cleanup_loop():
4795
+ time.sleep(STORAGE_CLEANUP_FIRST_S)
4796
+ while True:
4797
+ try:
4798
+ _storage_cleanup_pass()
4799
+ except Exception as exc:
4800
+ _log(f"storage cleanup failed: {type(exc).__name__}: {exc}")
4801
+ time.sleep(STORAGE_CLEANUP_EVERY_S)
4802
+
4803
+
4772
4804
  def main():
4773
4805
  global _PROJECT_MONITOR
4774
4806
  # O bind vem antes de qualquer I/O de rede: enquanto a porta não abre, o
@@ -4799,6 +4831,7 @@ def main():
4799
4831
  project_store.import_known_legacy(indexer.INDEX_DIR)
4800
4832
  _PROJECT_MONITOR = project_monitor.ProjectMonitor(_enqueue_monitored_project)
4801
4833
  _PROJECT_MONITOR.start()
4834
+ threading.Thread(target=_storage_cleanup_loop, daemon=True).start()
4802
4835
  _log(f"project monitor started; {recovered} interrupted job(s) recovered")
4803
4836
  except Exception as exc:
4804
4837
  _log(f"project monitor unavailable: {_project_error(exc)}")
package/usage_meter.py CHANGED
@@ -35,6 +35,7 @@ def phase(name):
35
35
 
36
36
  def _tables(db):
37
37
  db.execute('CREATE TABLE IF NOT EXISTS usage_daily(day TEXT,project_id TEXT,view_id TEXT,view_label TEXT,purpose TEXT,model TEXT,attempts INTEGER,successes INTEGER,errors INTEGER,input_items INTEGER,input_characters INTEGER,prompt_tokens INTEGER,output_tokens INTEGER,token_reports INTEGER,reported_usd REAL,price_reports INTEGER,elapsed_s REAL,PRIMARY KEY(day,project_id,view_id,purpose,model))')
38
+ db.execute('CREATE TABLE IF NOT EXISTS usage_hourly(hour INTEGER,purpose TEXT,attempts INTEGER,errors INTEGER,PRIMARY KEY(hour,purpose))')
38
39
  db.execute('CREATE TABLE IF NOT EXISTS usage_index_jobs(job_id TEXT PRIMARY KEY,project_id TEXT,view_id TEXT,view_label TEXT,at REAL,kind TEXT,embedded_chunks INTEGER,reused_chunks INTEGER,unchanged_files INTEGER,embedded_characters INTEGER)')
39
40
  db.execute('CREATE TABLE IF NOT EXISTS usage_imports(project_id TEXT PRIMARY KEY)')
40
41
 
@@ -80,6 +81,9 @@ def _record(path,body,response=None,headers=None,error=None,elapsed=0):
80
81
  fields=['attempts','successes','errors','input_items','input_characters','prompt_tokens','output_tokens','token_reports','reported_usd','price_reports','elapsed_s']
81
82
  update=','.join(f'{name}={name}+excluded.{name}' for name in fields)
82
83
  db.execute('INSERT INTO usage_daily VALUES('+','.join('?' for _ in values)+') ON CONFLICT(day,project_id,view_id,purpose,model) DO UPDATE SET '+update,values)
84
+ hour=int(time.time()//3600)
85
+ db.execute('INSERT INTO usage_hourly VALUES(?,?,1,?) ON CONFLICT(hour,purpose) DO UPDATE SET attempts=attempts+1,errors=errors+excluded.errors',(hour,current['purpose'],int(error is not None)))
86
+ db.execute('DELETE FROM usage_hourly WHERE hour<?',(hour-366*24,))
83
87
  cutoff=(datetime.datetime.now(datetime.timezone.utc)-datetime.timedelta(days=366)).date().isoformat()
84
88
  db.execute('DELETE FROM usage_daily WHERE day<?',(cutoff,))
85
89
  except Exception:
@@ -103,14 +107,13 @@ def _record_job(job_id,payload):
103
107
 
104
108
 
105
109
  def calls_by_purpose(since,purposes):
106
- """Attempts and errors of every project per purpose in `purposes`, from the UTC day of the epoch time `since` on
107
- (the table keeps days, not hours). Purposes with no call are left out."""
108
- day=datetime.datetime.fromtimestamp(since,datetime.timezone.utc).date().isoformat()
110
+ """Attempts and errors of every project per purpose in `purposes`, from the hour of the epoch time `since` on.
111
+ Purposes with no call are left out."""
109
112
  marks=','.join('?' for _ in purposes)
110
113
  with project_store._LOCK,project_store._db() as db:
111
114
  _tables(db)
112
- rows=db.execute('SELECT purpose,SUM(attempts),SUM(errors) FROM usage_daily WHERE day>=? AND purpose IN ('+marks+') GROUP BY purpose',
113
- (day,*purposes)).fetchall()
115
+ rows=db.execute('SELECT purpose,SUM(attempts),SUM(errors) FROM usage_hourly WHERE hour>=? AND purpose IN ('+marks+') GROUP BY purpose',
116
+ (int(since//3600),*purposes)).fetchall()
114
117
  found={purpose:{'purpose':purpose,'attempts':attempts,'errors':errors} for purpose,attempts,errors in rows}
115
118
  return [found[purpose] for purpose in purposes if purpose in found]
116
119
 
package/version.py CHANGED
@@ -1,7 +1,7 @@
1
1
  """Single source of the Smart Tool version and of the User-Agent sent to public APIs."""
2
2
  import re
3
3
 
4
- VERSION = "0.9.71"
4
+ VERSION = "0.9.73"
5
5
  _CONTACT_RE = re.compile(r"^(?:[^\s@()<>;]+@[^\s@()<>;]+\.[^\s@()<>;]+|https?://[^\s()<>;]+)$")
6
6
 
7
7